2024-05-04T02:59:08.237 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-04T02:59:08.237 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688625 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} -- /home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688625/orig.config.yaml 2024-05-04T02:59:08.295 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-04T02:59:08.389 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688625 branch: wip-pdonnell-testing-20240503.163550-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} email: pdonnell@redhat.com first_in_suite: false job_id: '7688625' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240503.163550-debug ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: bd4e00a39d766328b458e4c953f4d722fed96a31 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: bd4e00a39d766328b458e4c953f4d722fed96a31 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240503.163550-debug sha1: bd4e00a39d766328b458e4c953f4d722fed96a31 owner: scheduled_pdonnell@teuthology priority: 65 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1842 sha1: bd4e00a39d766328b458e4c953f4d722fed96a31 sleep_before_teardown: 0 subset: 478/512 suite: fs suite_branch: wip-pdonnell-testing-20240503.163550-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bd4e00a39d766328b458e4c953f4d722fed96a31/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bd4e00a39d766328b458e4c953f4d722fed96a31 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi001.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLG7OrXN4RBuKohy251Ae6lNCwQv4sas+rxUsY819IKTJDGPUpp2B8124bPjwsG/UvXzt9AXhBd6Xrb9gO8AKvQ= smithi072.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIDNK6jH7y9PaXr4k/N7a78+ZQ+cMQFpMz3bwAYVL/QwGgNaY2q3cfa1ddXu/Bbc7VqubRZ7mMDMJX7Kl7RqPPg= smithi077.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI8q7TRnSgmqEs3E3P6FY5VwesbB0x8FF4DiAjt7/hEzuzwNA48wfaX/10bUFUDsWGB/kudMxd6wh8G4uMQXy6A= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-03_22:48:16 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-04T02:59:08.389 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bd4e00a39d766328b458e4c953f4d722fed96a31/qa; will attempt to use it 2024-05-04T02:59:08.390 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bd4e00a39d766328b458e4c953f4d722fed96a31/qa/tasks 2024-05-04T02:59:08.390 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-04T02:59:08.391 INFO:teuthology.task.internal:Checking packages... 2024-05-04T02:59:08.415 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bd4e00a39d766328b458e4c953f4d722fed96a31' 2024-05-04T02:59:08.415 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-04T02:59:08.415 INFO:teuthology.packaging:ref: None 2024-05-04T02:59:08.415 INFO:teuthology.packaging:tag: None 2024-05-04T02:59:08.415 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240503.163550-debug 2024-05-04T02:59:08.415 INFO:teuthology.packaging:sha1: bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-04T02:59:08.415 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240503.163550-debug 2024-05-04T02:59:08.559 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3522.gbd4e00a3 2024-05-04T02:59:08.560 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-04T02:59:08.567 INFO:teuthology.task.internal:no buildpackages task found 2024-05-04T02:59:08.567 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-04T02:59:08.594 INFO:teuthology.task.internal:Saving configuration 2024-05-04T02:59:08.617 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-04T02:59:08.623 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-04T02:59:08.650 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi001.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688625', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-04 02:53:43.662612', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLG7OrXN4RBuKohy251Ae6lNCwQv4sas+rxUsY819IKTJDGPUpp2B8124bPjwsG/UvXzt9AXhBd6Xrb9gO8AKvQ='} 2024-05-04T02:59:08.674 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi072.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688625', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-04 02:53:43.666257', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIDNK6jH7y9PaXr4k/N7a78+ZQ+cMQFpMz3bwAYVL/QwGgNaY2q3cfa1ddXu/Bbc7VqubRZ7mMDMJX7Kl7RqPPg='} 2024-05-04T02:59:08.697 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi077.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688625', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-04 02:53:43.664584', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI8q7TRnSgmqEs3E3P6FY5VwesbB0x8FF4DiAjt7/hEzuzwNA48wfaX/10bUFUDsWGB/kudMxd6wh8G4uMQXy6A='} 2024-05-04T02:59:08.697 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-04T02:59:08.703 INFO:teuthology.task.internal:roles: ubuntu@smithi001.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-04T02:59:08.703 INFO:teuthology.task.internal:roles: ubuntu@smithi072.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-04T02:59:08.703 INFO:teuthology.task.internal:roles: ubuntu@smithi077.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-04T02:59:08.703 INFO:teuthology.run_tasks:Running task console_log... 2024-05-04T02:59:08.825 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f96297420d0>, signals=[15]) 2024-05-04T02:59:08.826 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-04T02:59:08.832 INFO:teuthology.task.internal:Opening connections... 2024-05-04T02:59:08.833 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi001.front.sepia.ceph.com 2024-05-04T02:59:08.835 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-04T02:59:08.916 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi072.front.sepia.ceph.com 2024-05-04T02:59:08.917 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-04T02:59:09.032 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi077.front.sepia.ceph.com 2024-05-04T02:59:09.032 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-04T02:59:09.107 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-04T02:59:09.114 DEBUG:teuthology.orchestra.run.smithi001:> uname -m 2024-05-04T02:59:09.133 INFO:teuthology.orchestra.run.smithi001.stdout:x86_64 2024-05-04T02:59:09.133 DEBUG:teuthology.orchestra.run.smithi001:> cat /etc/os-release 2024-05-04T02:59:09.191 INFO:teuthology.orchestra.run.smithi001.stdout:NAME="CentOS Stream" 2024-05-04T02:59:09.191 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION="9" 2024-05-04T02:59:09.191 INFO:teuthology.orchestra.run.smithi001.stdout:ID="centos" 2024-05-04T02:59:09.191 INFO:teuthology.orchestra.run.smithi001.stdout:ID_LIKE="rhel fedora" 2024-05-04T02:59:09.191 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION_ID="9" 2024-05-04T02:59:09.191 INFO:teuthology.orchestra.run.smithi001.stdout:PLATFORM_ID="platform:el9" 2024-05-04T02:59:09.191 INFO:teuthology.orchestra.run.smithi001.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-04T02:59:09.191 INFO:teuthology.orchestra.run.smithi001.stdout:ANSI_COLOR="0;31" 2024-05-04T02:59:09.191 INFO:teuthology.orchestra.run.smithi001.stdout:LOGO="fedora-logo-icon" 2024-05-04T02:59:09.191 INFO:teuthology.orchestra.run.smithi001.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-04T02:59:09.192 INFO:teuthology.orchestra.run.smithi001.stdout:HOME_URL="https://centos.org/" 2024-05-04T02:59:09.192 INFO:teuthology.orchestra.run.smithi001.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-04T02:59:09.192 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-04T02:59:09.192 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-04T02:59:09.192 INFO:teuthology.lock.ops:Updating smithi001.front.sepia.ceph.com on lock server 2024-05-04T02:59:09.214 DEBUG:teuthology.orchestra.run.smithi072:> uname -m 2024-05-04T02:59:09.231 INFO:teuthology.orchestra.run.smithi072.stdout:x86_64 2024-05-04T02:59:09.231 DEBUG:teuthology.orchestra.run.smithi072:> cat /etc/os-release 2024-05-04T02:59:09.289 INFO:teuthology.orchestra.run.smithi072.stdout:NAME="CentOS Stream" 2024-05-04T02:59:09.289 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION="9" 2024-05-04T02:59:09.290 INFO:teuthology.orchestra.run.smithi072.stdout:ID="centos" 2024-05-04T02:59:09.290 INFO:teuthology.orchestra.run.smithi072.stdout:ID_LIKE="rhel fedora" 2024-05-04T02:59:09.290 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_ID="9" 2024-05-04T02:59:09.290 INFO:teuthology.orchestra.run.smithi072.stdout:PLATFORM_ID="platform:el9" 2024-05-04T02:59:09.291 INFO:teuthology.orchestra.run.smithi072.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-04T02:59:09.291 INFO:teuthology.orchestra.run.smithi072.stdout:ANSI_COLOR="0;31" 2024-05-04T02:59:09.291 INFO:teuthology.orchestra.run.smithi072.stdout:LOGO="fedora-logo-icon" 2024-05-04T02:59:09.291 INFO:teuthology.orchestra.run.smithi072.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-04T02:59:09.291 INFO:teuthology.orchestra.run.smithi072.stdout:HOME_URL="https://centos.org/" 2024-05-04T02:59:09.291 INFO:teuthology.orchestra.run.smithi072.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-04T02:59:09.291 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-04T02:59:09.291 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-04T02:59:09.291 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2024-05-04T02:59:09.314 DEBUG:teuthology.orchestra.run.smithi077:> uname -m 2024-05-04T02:59:09.332 INFO:teuthology.orchestra.run.smithi077.stdout:x86_64 2024-05-04T02:59:09.332 DEBUG:teuthology.orchestra.run.smithi077:> cat /etc/os-release 2024-05-04T02:59:09.390 INFO:teuthology.orchestra.run.smithi077.stdout:NAME="CentOS Stream" 2024-05-04T02:59:09.390 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION="9" 2024-05-04T02:59:09.390 INFO:teuthology.orchestra.run.smithi077.stdout:ID="centos" 2024-05-04T02:59:09.390 INFO:teuthology.orchestra.run.smithi077.stdout:ID_LIKE="rhel fedora" 2024-05-04T02:59:09.390 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_ID="9" 2024-05-04T02:59:09.390 INFO:teuthology.orchestra.run.smithi077.stdout:PLATFORM_ID="platform:el9" 2024-05-04T02:59:09.390 INFO:teuthology.orchestra.run.smithi077.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-04T02:59:09.390 INFO:teuthology.orchestra.run.smithi077.stdout:ANSI_COLOR="0;31" 2024-05-04T02:59:09.390 INFO:teuthology.orchestra.run.smithi077.stdout:LOGO="fedora-logo-icon" 2024-05-04T02:59:09.390 INFO:teuthology.orchestra.run.smithi077.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-04T02:59:09.391 INFO:teuthology.orchestra.run.smithi077.stdout:HOME_URL="https://centos.org/" 2024-05-04T02:59:09.391 INFO:teuthology.orchestra.run.smithi077.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-04T02:59:09.391 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-04T02:59:09.391 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-04T02:59:09.391 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server 2024-05-04T02:59:09.413 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-04T02:59:09.425 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-04T02:59:09.445 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-04T02:59:09.445 DEBUG:teuthology.orchestra.run.smithi001:> test '!' -e /home/ubuntu/cephtest 2024-05-04T02:59:09.447 DEBUG:teuthology.orchestra.run.smithi072:> test '!' -e /home/ubuntu/cephtest 2024-05-04T02:59:09.450 DEBUG:teuthology.orchestra.run.smithi077:> test '!' -e /home/ubuntu/cephtest 2024-05-04T02:59:09.467 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-04T02:59:09.473 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-04T02:59:09.473 DEBUG:teuthology.orchestra.run.smithi001:> test -z $(ls -A /var/lib/ceph) 2024-05-04T02:59:09.505 DEBUG:teuthology.orchestra.run.smithi072:> test -z $(ls -A /var/lib/ceph) 2024-05-04T02:59:09.507 DEBUG:teuthology.orchestra.run.smithi077:> test -z $(ls -A /var/lib/ceph) 2024-05-04T02:59:09.604 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-04T02:59:09.682 INFO:teuthology.run_tasks:Running task kernel... 2024-05-04T02:59:09.698 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-04T02:59:09.698 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-04T02:59:09.698 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-04T02:59:09.698 DEBUG:teuthology.orchestra.run.smithi001:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-04T02:59:09.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T02:59:09.716 DEBUG:teuthology.orchestra.run.smithi001:> uname -r 2024-05-04T02:59:09.775 INFO:teuthology.orchestra.run.smithi001.stdout:5.14.0-437.el9.x86_64 2024-05-04T02:59:09.775 INFO:teuthology.task.kernel:Running kernel on smithi001: 5.14.0-437.el9.x86_64 2024-05-04T02:59:09.775 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum install -y kernel 2024-05-04T02:59:12.822 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - BaseOS 5.8 MB/s | 8.0 MB 00:01 2024-05-04T02:59:15.106 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-05-04T02:59:19.298 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-05-04T02:59:20.924 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-05-04T02:59:22.775 INFO:teuthology.orchestra.run.smithi001.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-05-04T02:59:28.058 INFO:teuthology.orchestra.run.smithi001.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-05-04T02:59:29.982 INFO:teuthology.orchestra.run.smithi001.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-04T02:59:29.982 INFO:teuthology.orchestra.run.smithi001.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-04T02:59:30.082 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2024-05-04T02:59:30.083 INFO:teuthology.orchestra.run.smithi001.stdout:Nothing to do. 2024-05-04T02:59:30.083 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2024-05-04T02:59:30.198 DEBUG:teuthology.orchestra.run.smithi001:> echo no | sudo yum reinstall kernel || true 2024-05-04T02:59:30.721 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:00:02 ago on Sat 04 May 2024 02:59:28 AM UTC. 2024-05-04T02:59:30.844 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2024-05-04T02:59:30.857 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2024-05-04T02:59:30.857 INFO:teuthology.orchestra.run.smithi001.stdout: Package Architecture Version Repository Size 2024-05-04T02:59:30.857 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2024-05-04T02:59:30.857 INFO:teuthology.orchestra.run.smithi001.stdout:Reinstalling: 2024-05-04T02:59:30.857 INFO:teuthology.orchestra.run.smithi001.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-05-04T02:59:30.858 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T02:59:30.858 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2024-05-04T02:59:30.858 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2024-05-04T02:59:30.858 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T02:59:30.858 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 255 k 2024-05-04T02:59:30.858 INFO:teuthology.orchestra.run.smithi001.stdout:Installed size: 0 2024-05-04T02:59:30.858 INFO:teuthology.orchestra.run.smithi001.stderr:Operation aborted. 2024-05-04T02:59:30.928 INFO:teuthology.orchestra.run.smithi001.stdout:Is this ok [y/N]: 2024-05-04T02:59:30.928 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum reinstall -y kernel || true 2024-05-04T02:59:31.447 INFO:teuthology.orchestra.run.smithi001.stdout:Last metadata expiration check: 0:00:03 ago on Sat 04 May 2024 02:59:28 AM UTC. 2024-05-04T02:59:31.569 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2024-05-04T02:59:31.570 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2024-05-04T02:59:31.570 INFO:teuthology.orchestra.run.smithi001.stdout: Package Architecture Version Repository Size 2024-05-04T02:59:31.570 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2024-05-04T02:59:31.570 INFO:teuthology.orchestra.run.smithi001.stdout:Reinstalling: 2024-05-04T02:59:31.571 INFO:teuthology.orchestra.run.smithi001.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-05-04T02:59:31.571 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T02:59:31.571 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2024-05-04T02:59:31.571 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2024-05-04T02:59:31.571 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T02:59:31.571 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 255 k 2024-05-04T02:59:31.571 INFO:teuthology.orchestra.run.smithi001.stdout:Installed size: 0 2024-05-04T02:59:31.571 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2024-05-04T02:59:31.785 INFO:teuthology.orchestra.run.smithi001.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.5 MB/s | 255 kB 00:00 2024-05-04T02:59:31.786 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2024-05-04T02:59:31.786 INFO:teuthology.orchestra.run.smithi001.stdout:Total 1.2 MB/s | 255 kB 00:00 2024-05-04T02:59:31.807 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2024-05-04T02:59:31.989 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2024-05-04T02:59:31.989 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2024-05-04T02:59:31.994 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2024-05-04T02:59:31.994 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2024-05-04T02:59:32.122 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2024-05-04T02:59:32.178 INFO:teuthology.orchestra.run.smithi001.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-05-04T02:59:32.179 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-05-04T02:59:32.824 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-05-04T02:59:32.824 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-05-04T02:59:33.672 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-05-04T02:59:33.672 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T02:59:33.672 INFO:teuthology.orchestra.run.smithi001.stdout:Reinstalled: 2024-05-04T02:59:33.672 INFO:teuthology.orchestra.run.smithi001.stdout: kernel-5.14.0-437.el9.x86_64 2024-05-04T02:59:33.672 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T02:59:33.673 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2024-05-04T02:59:33.840 DEBUG:teuthology.orchestra.run.smithi001:> rpm -q kernel | sort -rV | head -n 1 2024-05-04T02:59:33.889 INFO:teuthology.orchestra.run.smithi001.stdout:kernel-5.14.0-437.el9.x86_64 2024-05-04T02:59:33.889 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-05-04T02:59:33.890 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-05-04T02:59:33.890 DEBUG:teuthology.parallel:result is None 2024-05-04T02:59:33.890 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-04T02:59:33.919 INFO:teuthology.task.internal:Creating test directory... 2024-05-04T02:59:33.920 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-04T02:59:33.922 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-04T02:59:33.924 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-04T02:59:33.942 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-04T02:59:33.954 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-04T02:59:33.986 INFO:teuthology.task.internal:Creating archive directory... 2024-05-04T02:59:33.986 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-04T02:59:33.989 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-04T02:59:33.992 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-04T02:59:34.103 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-04T02:59:34.116 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-04T02:59:34.116 DEBUG:teuthology.orchestra.run.smithi072:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-04T02:59:34.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T02:59:34.136 DEBUG:teuthology.orchestra.run.smithi077:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-04T02:59:34.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T02:59:34.156 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-04T02:59:34.163 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-04T02:59:34.184 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-04T02:59:34.206 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-04T02:59:34.217 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-04T02:59:34.225 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-04T02:59:34.246 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-04T02:59:34.267 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-04T02:59:34.267 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-04T02:59:34.269 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-04T02:59:34.289 INFO:teuthology.task.internal:Configuring sudo... 2024-05-04T02:59:34.290 DEBUG:teuthology.orchestra.run.smithi001:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-04T02:59:34.294 DEBUG:teuthology.orchestra.run.smithi072:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-04T02:59:34.315 DEBUG:teuthology.orchestra.run.smithi077:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-04T02:59:34.347 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-04T02:59:34.378 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-04T02:59:34.379 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-04T02:59:34.384 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-04T02:59:34.388 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-04T02:59:34.407 DEBUG:teuthology.orchestra.run.smithi001:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-04T02:59:34.490 DEBUG:teuthology.orchestra.run.smithi001:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-04T02:59:34.539 DEBUG:teuthology.orchestra.run.smithi001:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-04T02:59:34.624 DEBUG:teuthology.orchestra.run.smithi001:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-04T02:59:34.662 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-04T02:59:34.662 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-04T02:59:34.726 DEBUG:teuthology.orchestra.run.smithi072:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-04T02:59:34.768 DEBUG:teuthology.orchestra.run.smithi072:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-04T02:59:34.820 DEBUG:teuthology.orchestra.run.smithi072:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-04T02:59:34.904 DEBUG:teuthology.orchestra.run.smithi072:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-04T02:59:34.947 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-04T02:59:34.947 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-04T02:59:35.012 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-04T02:59:35.056 DEBUG:teuthology.orchestra.run.smithi077:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-04T02:59:35.110 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-04T02:59:35.192 DEBUG:teuthology.orchestra.run.smithi077:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-04T02:59:35.236 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-04T02:59:35.236 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-04T02:59:35.302 DEBUG:teuthology.orchestra.run.smithi001:> sudo service rsyslog restart 2024-05-04T02:59:35.304 DEBUG:teuthology.orchestra.run.smithi072:> sudo service rsyslog restart 2024-05-04T02:59:35.306 DEBUG:teuthology.orchestra.run.smithi077:> sudo service rsyslog restart 2024-05-04T02:59:35.335 INFO:teuthology.orchestra.run.smithi001.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-04T02:59:35.341 INFO:teuthology.orchestra.run.smithi072.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-04T02:59:35.385 INFO:teuthology.orchestra.run.smithi077.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-04T02:59:35.777 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-04T02:59:35.787 INFO:teuthology.task.internal:Starting timer... 2024-05-04T02:59:35.787 INFO:teuthology.run_tasks:Running task pcp... 2024-05-04T02:59:35.837 INFO:teuthology.run_tasks:Running task selinux... 2024-05-04T02:59:35.860 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-04T02:59:35.860 DEBUG:teuthology.orchestra.run.smithi001:> sudo service auditd rotate 2024-05-04T02:59:35.923 INFO:teuthology.orchestra.run.smithi001.stdout:Rotating logs: 2024-05-04T02:59:35.924 DEBUG:teuthology.orchestra.run.smithi072:> sudo service auditd rotate 2024-05-04T02:59:35.992 INFO:teuthology.orchestra.run.smithi072.stdout:Rotating logs: 2024-05-04T02:59:35.994 DEBUG:teuthology.orchestra.run.smithi077:> sudo service auditd rotate 2024-05-04T02:59:36.060 INFO:teuthology.orchestra.run.smithi077.stdout:Rotating logs: 2024-05-04T02:59:36.061 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-04T02:59:36.062 DEBUG:teuthology.orchestra.run.smithi001:> /usr/sbin/getenforce 2024-05-04T02:59:36.081 INFO:teuthology.orchestra.run.smithi001.stdout:Permissive 2024-05-04T02:59:36.082 DEBUG:teuthology.orchestra.run.smithi072:> /usr/sbin/getenforce 2024-05-04T02:59:36.107 INFO:teuthology.orchestra.run.smithi072.stdout:Permissive 2024-05-04T02:59:36.107 DEBUG:teuthology.orchestra.run.smithi077:> /usr/sbin/getenforce 2024-05-04T02:59:36.126 INFO:teuthology.orchestra.run.smithi077.stdout:Permissive 2024-05-04T02:59:36.126 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi001.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi072.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi077.front.sepia.ceph.com': 'permissive'} 2024-05-04T02:59:36.126 DEBUG:teuthology.orchestra.run.smithi001:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-04T02:59:36.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T02:59:36.154 DEBUG:teuthology.orchestra.run.smithi072:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-04T02:59:36.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T02:59:36.183 DEBUG:teuthology.orchestra.run.smithi077:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-04T02:59:36.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T02:59:36.210 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-04T02:59:36.210 DEBUG:teuthology.orchestra.run.smithi001:> sudo /usr/sbin/setenforce permissive 2024-05-04T02:59:36.243 DEBUG:teuthology.orchestra.run.smithi072:> sudo /usr/sbin/setenforce permissive 2024-05-04T02:59:36.273 DEBUG:teuthology.orchestra.run.smithi077:> sudo /usr/sbin/setenforce permissive 2024-05-04T02:59:36.302 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-04T02:59:36.312 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-04T02:59:36.328 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-04T02:59:36.330 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi001.front.sepia.ceph.com,smithi072.front.sepia.ceph.com,smithi077.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-04T03:04:08.795 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi001.front.sepia.ceph.com'), Remote(name='ubuntu@smithi072.front.sepia.ceph.com'), Remote(name='ubuntu@smithi077.front.sepia.ceph.com')] 2024-05-04T03:04:08.797 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-05-04T03:04:08.798 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-04T03:04:08.875 DEBUG:teuthology.orchestra.run.smithi001:> true 2024-05-04T03:04:08.958 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-05-04T03:04:08.959 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-05-04T03:04:08.959 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-04T03:04:09.033 DEBUG:teuthology.orchestra.run.smithi072:> true 2024-05-04T03:04:09.113 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-05-04T03:04:09.113 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-05-04T03:04:09.114 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-04T03:04:09.187 DEBUG:teuthology.orchestra.run.smithi077:> true 2024-05-04T03:04:09.266 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-05-04T03:04:09.266 INFO:teuthology.run_tasks:Running task clock... 2024-05-04T03:04:09.276 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-04T03:04:09.276 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-04T03:04:09.277 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-04T03:04:09.279 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-04T03:04:09.279 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-04T03:04:09.281 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-04T03:04:09.281 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-04T03:04:09.307 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-04T03:04:09.309 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-04T03:04:09.320 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-04T03:04:09.324 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-04T03:04:09.335 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-04T03:04:09.342 INFO:teuthology.orchestra.run.smithi072.stderr:sudo: ntpd: command not found 2024-05-04T03:04:09.349 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-04T03:04:09.350 INFO:teuthology.orchestra.run.smithi001.stderr:sudo: ntpd: command not found 2024-05-04T03:04:09.353 INFO:teuthology.orchestra.run.smithi072.stdout:506 Cannot talk to daemon 2024-05-04T03:04:09.362 INFO:teuthology.orchestra.run.smithi001.stdout:506 Cannot talk to daemon 2024-05-04T03:04:09.365 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-04T03:04:09.372 INFO:teuthology.orchestra.run.smithi077.stderr:sudo: ntpd: command not found 2024-05-04T03:04:09.376 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-04T03:04:09.378 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-04T03:04:09.383 INFO:teuthology.orchestra.run.smithi077.stdout:506 Cannot talk to daemon 2024-05-04T03:04:09.390 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-04T03:04:09.396 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-04T03:04:09.409 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-04T03:04:09.433 INFO:teuthology.orchestra.run.smithi072.stderr:bash: line 1: ntpq: command not found 2024-05-04T03:04:09.436 INFO:teuthology.orchestra.run.smithi072.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-04T03:04:09.436 INFO:teuthology.orchestra.run.smithi072.stdout:=============================================================================== 2024-05-04T03:04:09.436 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T03:04:09.436 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T03:04:09.436 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T03:04:09.436 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T03:04:09.447 INFO:teuthology.orchestra.run.smithi001.stderr:bash: line 1: ntpq: command not found 2024-05-04T03:04:09.451 INFO:teuthology.orchestra.run.smithi001.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-04T03:04:09.451 INFO:teuthology.orchestra.run.smithi001.stdout:=============================================================================== 2024-05-04T03:04:09.451 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T03:04:09.451 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T03:04:09.451 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T03:04:09.451 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T03:04:09.463 INFO:teuthology.orchestra.run.smithi077.stderr:bash: line 1: ntpq: command not found 2024-05-04T03:04:09.467 INFO:teuthology.orchestra.run.smithi077.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-04T03:04:09.467 INFO:teuthology.orchestra.run.smithi077.stdout:=============================================================================== 2024-05-04T03:04:09.467 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T03:04:09.467 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T03:04:09.467 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T03:04:09.467 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T03:04:09.468 INFO:teuthology.run_tasks:Running task install... 2024-05-04T03:04:09.476 DEBUG:teuthology.task.install:project ceph 2024-05-04T03:04:09.476 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bd4e00a39d766328b458e4c953f4d722fed96a31'}} 2024-05-04T03:04:09.476 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bd4e00a39d766328b458e4c953f4d722fed96a31'} 2024-05-04T03:04:09.478 INFO:teuthology.task.install:Using flavor: default 2024-05-04T03:04:09.484 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-04T03:04:09.484 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-04T03:04:09.485 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bd4e00a39d766328b458e4c953f4d722fed96a31', 'tag': None, 'wait_for_package': False} 2024-05-04T03:04:09.485 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-04T03:04:09.487 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bd4e00a39d766328b458e4c953f4d722fed96a31', 'tag': None, 'wait_for_package': False} 2024-05-04T03:04:09.487 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-04T03:04:09.488 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bd4e00a39d766328b458e4c953f4d722fed96a31', 'tag': None, 'wait_for_package': False} 2024-05-04T03:04:09.488 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-04T03:04:09.634 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/ 2024-05-04T03:04:09.635 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3522.gbd4e00a3 2024-05-04T03:04:09.711 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/ 2024-05-04T03:04:09.712 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3522.gbd4e00a3 2024-05-04T03:04:09.721 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/ 2024-05-04T03:04:09.721 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3522.gbd4e00a3 2024-05-04T03:04:09.808 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-04T03:04:09.809 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-04T03:04:09.809 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-04T03:04:09.821 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-04T03:04:09.821 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-04T03:04:09.822 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-04T03:04:09.830 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-04T03:04:09.830 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-04T03:04:09.830 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-04T03:04:09.835 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-04T03:04:09.835 DEBUG:teuthology.orchestra.run.smithi001:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bd4e00a39d766328b458e4c953f4d722fed96a31/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-04T03:04:09.846 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-04T03:04:09.846 DEBUG:teuthology.orchestra.run.smithi072:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bd4e00a39d766328b458e4c953f4d722fed96a31/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-04T03:04:09.855 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-04T03:04:09.855 DEBUG:teuthology.orchestra.run.smithi077:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bd4e00a39d766328b458e4c953f4d722fed96a31/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-04T03:04:09.902 DEBUG:teuthology.orchestra.run.smithi001:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-04T03:04:09.912 DEBUG:teuthology.orchestra.run.smithi072:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-04T03:04:09.919 DEBUG:teuthology.orchestra.run.smithi077:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-04T03:04:09.980 DEBUG:teuthology.orchestra.run.smithi001:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-04T03:04:09.985 DEBUG:teuthology.orchestra.run.smithi072:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-04T03:04:09.991 DEBUG:teuthology.orchestra.run.smithi077:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-04T03:04:10.045 INFO:teuthology.orchestra.run.smithi001.stdout:check_obsoletes = 1 2024-05-04T03:04:10.046 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum clean all 2024-05-04T03:04:10.047 INFO:teuthology.orchestra.run.smithi072.stdout:check_obsoletes = 1 2024-05-04T03:04:10.049 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum clean all 2024-05-04T03:04:10.053 INFO:teuthology.orchestra.run.smithi077.stdout:check_obsoletes = 1 2024-05-04T03:04:10.055 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum clean all 2024-05-04T03:04:10.274 INFO:teuthology.orchestra.run.smithi072.stdout:50 files removed 2024-05-04T03:04:10.281 INFO:teuthology.orchestra.run.smithi077.stdout:50 files removed 2024-05-04T03:04:10.282 INFO:teuthology.orchestra.run.smithi001.stdout:50 files removed 2024-05-04T03:04:10.302 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-04T03:04:10.308 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-04T03:04:10.309 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-04T03:04:10.922 INFO:teuthology.orchestra.run.smithi077.stdout:ceph packages for x86_64 236 kB/s | 84 kB 00:00 2024-05-04T03:04:10.972 INFO:teuthology.orchestra.run.smithi001.stdout:ceph packages for x86_64 214 kB/s | 84 kB 00:00 2024-05-04T03:04:10.976 INFO:teuthology.orchestra.run.smithi072.stdout:ceph packages for x86_64 202 kB/s | 84 kB 00:00 2024-05-04T03:04:11.298 INFO:teuthology.orchestra.run.smithi001.stdout:ceph noarch packages 42 kB/s | 12 kB 00:00 2024-05-04T03:04:11.302 INFO:teuthology.orchestra.run.smithi072.stdout:ceph noarch packages 42 kB/s | 12 kB 00:00 2024-05-04T03:04:11.348 INFO:teuthology.orchestra.run.smithi077.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-05-04T03:04:11.673 INFO:teuthology.orchestra.run.smithi001.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-05-04T03:04:11.714 INFO:teuthology.orchestra.run.smithi072.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-05-04T03:04:11.764 INFO:teuthology.orchestra.run.smithi077.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-05-04T03:04:12.232 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-05-04T03:04:12.306 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-05-04T03:04:13.365 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.0 MB 00:01 2024-05-04T03:04:13.949 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - AppStream 34 MB/s | 19 MB 00:00 2024-05-04T03:04:14.175 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-05-04T03:04:18.184 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-05-04T03:04:18.261 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-05-04T03:04:19.577 INFO:teuthology.orchestra.run.smithi001.stdout:CentOS Stream 9 - Extras packages 96 kB/s | 16 kB 00:00 2024-05-04T03:04:19.966 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-05-04T03:04:20.928 INFO:teuthology.orchestra.run.smithi001.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-05-04T03:04:21.176 INFO:teuthology.orchestra.run.smithi072.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-05-04T03:04:26.253 INFO:teuthology.orchestra.run.smithi072.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-04T03:04:26.303 INFO:teuthology.orchestra.run.smithi001.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-05-04T03:04:28.399 INFO:teuthology.orchestra.run.smithi072.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-04T03:04:28.399 INFO:teuthology.orchestra.run.smithi072.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-04T03:04:28.410 INFO:teuthology.orchestra.run.smithi072.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-04T03:04:28.411 INFO:teuthology.orchestra.run.smithi072.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-04T03:04:28.412 INFO:teuthology.orchestra.run.smithi072.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-04T03:04:28.413 INFO:teuthology.orchestra.run.smithi072.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-04T03:04:28.414 INFO:teuthology.orchestra.run.smithi072.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-04T03:04:28.416 INFO:teuthology.orchestra.run.smithi072.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-04T03:04:28.417 INFO:teuthology.orchestra.run.smithi072.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-04T03:04:28.478 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-04T03:04:28.507 INFO:teuthology.orchestra.run.smithi072.stdout:======================================================================================= 2024-05-04T03:04:28.507 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2024-05-04T03:04:28.507 INFO:teuthology.orchestra.run.smithi072.stdout:======================================================================================= 2024-05-04T03:04:28.507 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-05-04T03:04:28.508 INFO:teuthology.orchestra.run.smithi072.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-04T03:04:28.508 INFO:teuthology.orchestra.run.smithi072.stdout: ceph x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 6.6 k 2024-05-04T03:04:28.508 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-base x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 5.4 M 2024-05-04T03:04:28.508 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-fuse x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 863 k 2024-05-04T03:04:28.508 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 147 k 2024-05-04T03:04:28.508 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 1.5 M 2024-05-04T03:04:28.508 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 142 k 2024-05-04T03:04:28.508 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 3.5 M 2024-05-04T03:04:28.508 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 7.4 M 2024-05-04T03:04:28.508 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-rook noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 49 k 2024-05-04T03:04:28.508 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-radosgw x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 10 M 2024-05-04T03:04:28.508 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-test x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 43 M 2024-05-04T03:04:28.509 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-volume noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 273 k 2024-05-04T03:04:28.509 INFO:teuthology.orchestra.run.smithi072.stdout: cephadm noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 763 k 2024-05-04T03:04:28.509 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-mirror x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 224 k 2024-05-04T03:04:28.509 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-shell x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 27 k 2024-05-04T03:04:28.509 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-top noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 18 k 2024-05-04T03:04:28.509 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-04T03:04:28.509 INFO:teuthology.orchestra.run.smithi072.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-04T03:04:28.509 INFO:teuthology.orchestra.run.smithi072.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-04T03:04:28.509 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs-devel x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 32 k 2024-05-04T03:04:28.509 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs2 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 756 k 2024-05-04T03:04:28.509 INFO:teuthology.orchestra.run.smithi072.stdout: librados-devel x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 127 k 2024-05-04T03:04:28.509 INFO:teuthology.orchestra.run.smithi072.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-04T03:04:28.509 INFO:teuthology.orchestra.run.smithi072.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-04T03:04:28.509 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-04T03:04:28.510 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-04T03:04:28.510 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-04T03:04:28.510 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-04T03:04:28.510 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cephfs x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 162 k 2024-05-04T03:04:28.510 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rados x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 321 k 2024-05-04T03:04:28.511 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rbd x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 298 k 2024-05-04T03:04:28.511 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rgw x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 100 k 2024-05-04T03:04:28.511 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-fuse x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 85 k 2024-05-04T03:04:28.511 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-mirror x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 3.0 M 2024-05-04T03:04:28.511 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-nbd x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 170 k 2024-05-04T03:04:28.511 INFO:teuthology.orchestra.run.smithi072.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-04T03:04:28.512 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrading: 2024-05-04T03:04:28.512 INFO:teuthology.orchestra.run.smithi072.stdout: librados2 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 3.3 M 2024-05-04T03:04:28.512 INFO:teuthology.orchestra.run.smithi072.stdout: librbd1 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 3.1 M 2024-05-04T03:04:28.512 INFO:teuthology.orchestra.run.smithi072.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-04T03:04:28.512 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-04T03:04:28.512 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2024-05-04T03:04:28.512 INFO:teuthology.orchestra.run.smithi072.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-04T03:04:28.512 INFO:teuthology.orchestra.run.smithi072.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-04T03:04:28.513 INFO:teuthology.orchestra.run.smithi072.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-04T03:04:28.513 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-common x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 20 M 2024-05-04T03:04:28.513 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 34 k 2024-05-04T03:04:28.513 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mds x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 2.3 M 2024-05-04T03:04:28.513 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 274 k 2024-05-04T03:04:28.513 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mon x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 4.9 M 2024-05-04T03:04:28.513 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-osd x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 18 M 2024-05-04T03:04:28.513 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 16 k 2024-05-04T03:04:28.513 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-selinux x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 25 k 2024-05-04T03:04:28.513 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-04T03:04:28.513 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-04T03:04:28.513 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-04T03:04:28.513 INFO:teuthology.orchestra.run.smithi072.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-04T03:04:28.513 INFO:teuthology.orchestra.run.smithi072.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-04T03:04:28.514 INFO:teuthology.orchestra.run.smithi072.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-04T03:04:28.514 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-04T03:04:28.514 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-04T03:04:28.514 INFO:teuthology.orchestra.run.smithi072.stdout: libcephsqlite x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 158 k 2024-05-04T03:04:28.514 INFO:teuthology.orchestra.run.smithi072.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-04T03:04:28.514 INFO:teuthology.orchestra.run.smithi072.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-04T03:04:28.514 INFO:teuthology.orchestra.run.smithi072.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-04T03:04:28.514 INFO:teuthology.orchestra.run.smithi072.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-04T03:04:28.514 INFO:teuthology.orchestra.run.smithi072.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-04T03:04:28.514 INFO:teuthology.orchestra.run.smithi072.stdout: libradosstriper1 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 497 k 2024-05-04T03:04:28.514 INFO:teuthology.orchestra.run.smithi072.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-04T03:04:28.515 INFO:teuthology.orchestra.run.smithi072.stdout: librgw2 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 5.2 M 2024-05-04T03:04:28.515 INFO:teuthology.orchestra.run.smithi072.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-04T03:04:28.515 INFO:teuthology.orchestra.run.smithi072.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-04T03:04:28.515 INFO:teuthology.orchestra.run.smithi072.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-04T03:04:28.515 INFO:teuthology.orchestra.run.smithi072.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-04T03:04:28.515 INFO:teuthology.orchestra.run.smithi072.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-04T03:04:28.515 INFO:teuthology.orchestra.run.smithi072.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-04T03:04:28.515 INFO:teuthology.orchestra.run.smithi072.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-04T03:04:28.515 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-04T03:04:28.515 INFO:teuthology.orchestra.run.smithi072.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-04T03:04:28.516 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-04T03:04:28.516 INFO:teuthology.orchestra.run.smithi072.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-04T03:04:28.516 INFO:teuthology.orchestra.run.smithi072.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-04T03:04:28.516 INFO:teuthology.orchestra.run.smithi072.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-04T03:04:28.516 INFO:teuthology.orchestra.run.smithi072.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-04T03:04:28.516 INFO:teuthology.orchestra.run.smithi072.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-04T03:04:28.516 INFO:teuthology.orchestra.run.smithi072.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-04T03:04:28.516 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-04T03:04:28.516 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-argparse x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 45 k 2024-05-04T03:04:28.516 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-common x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 135 k 2024-05-04T03:04:28.517 INFO:teuthology.orchestra.run.smithi072.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-04T03:04:28.517 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-04T03:04:28.517 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-04T03:04:28.517 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-04T03:04:28.517 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-04T03:04:28.517 INFO:teuthology.orchestra.run.smithi072.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-04T03:04:28.517 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-04T03:04:28.517 INFO:teuthology.orchestra.run.smithi072.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-04T03:04:28.517 INFO:teuthology.orchestra.run.smithi072.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-04T03:04:28.517 INFO:teuthology.orchestra.run.smithi072.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-04T03:04:28.517 INFO:teuthology.orchestra.run.smithi072.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-04T03:04:28.517 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-04T03:04:28.517 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-04T03:04:28.517 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-04T03:04:28.518 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-04T03:04:28.518 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-04T03:04:28.518 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-04T03:04:28.518 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-04T03:04:28.518 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-04T03:04:28.518 INFO:teuthology.orchestra.run.smithi072.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-04T03:04:28.518 INFO:teuthology.orchestra.run.smithi072.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-04T03:04:28.518 INFO:teuthology.orchestra.run.smithi072.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-04T03:04:28.518 INFO:teuthology.orchestra.run.smithi072.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-04T03:04:28.518 INFO:teuthology.orchestra.run.smithi072.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-04T03:04:28.518 INFO:teuthology.orchestra.run.smithi072.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-04T03:04:28.518 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-04T03:04:28.518 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-04T03:04:28.519 INFO:teuthology.orchestra.run.smithi072.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-04T03:04:28.519 INFO:teuthology.orchestra.run.smithi072.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-04T03:04:28.519 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-04T03:04:28.519 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-04T03:04:28.519 INFO:teuthology.orchestra.run.smithi072.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-04T03:04:28.519 INFO:teuthology.orchestra.run.smithi072.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-04T03:04:28.519 INFO:teuthology.orchestra.run.smithi072.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-04T03:04:28.519 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-04T03:04:28.519 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-04T03:04:28.519 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-04T03:04:28.519 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-04T03:04:28.519 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-04T03:04:28.519 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-04T03:04:28.519 INFO:teuthology.orchestra.run.smithi072.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-04T03:04:28.519 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-04T03:04:28.520 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-04T03:04:28.520 INFO:teuthology.orchestra.run.smithi072.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-04T03:04:28.520 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-04T03:04:28.520 INFO:teuthology.orchestra.run.smithi072.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-04T03:04:28.520 INFO:teuthology.orchestra.run.smithi072.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-04T03:04:28.520 INFO:teuthology.orchestra.run.smithi072.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-04T03:04:28.520 INFO:teuthology.orchestra.run.smithi072.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-04T03:04:28.520 INFO:teuthology.orchestra.run.smithi072.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-04T03:04:28.520 INFO:teuthology.orchestra.run.smithi072.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-04T03:04:28.520 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-04T03:04:28.520 INFO:teuthology.orchestra.run.smithi072.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-04T03:04:28.521 INFO:teuthology.orchestra.run.smithi072.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-04T03:04:28.521 INFO:teuthology.orchestra.run.smithi072.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-04T03:04:28.521 INFO:teuthology.orchestra.run.smithi072.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-04T03:04:28.521 INFO:teuthology.orchestra.run.smithi072.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-04T03:04:28.521 INFO:teuthology.orchestra.run.smithi072.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-04T03:04:28.521 INFO:teuthology.orchestra.run.smithi072.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-04T03:04:28.522 INFO:teuthology.orchestra.run.smithi072.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-04T03:04:28.522 INFO:teuthology.orchestra.run.smithi072.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-04T03:04:28.522 INFO:teuthology.orchestra.run.smithi072.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-04T03:04:28.522 INFO:teuthology.orchestra.run.smithi072.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-04T03:04:28.522 INFO:teuthology.orchestra.run.smithi072.stdout:Installing weak dependencies: 2024-05-04T03:04:28.522 INFO:teuthology.orchestra.run.smithi072.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-04T03:04:28.522 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-04T03:04:28.522 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-05-04T03:04:28.522 INFO:teuthology.orchestra.run.smithi072.stdout:======================================================================================= 2024-05-04T03:04:28.522 INFO:teuthology.orchestra.run.smithi072.stdout:Install 148 Packages 2024-05-04T03:04:28.522 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrade 4 Packages 2024-05-04T03:04:28.522 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-04T03:04:28.523 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 214 M 2024-05-04T03:04:28.523 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-05-04T03:04:28.539 INFO:teuthology.orchestra.run.smithi001.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-04T03:04:28.539 INFO:teuthology.orchestra.run.smithi001.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-04T03:04:28.553 INFO:teuthology.orchestra.run.smithi001.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-04T03:04:28.553 INFO:teuthology.orchestra.run.smithi001.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-04T03:04:28.553 INFO:teuthology.orchestra.run.smithi001.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-04T03:04:28.554 INFO:teuthology.orchestra.run.smithi001.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-04T03:04:28.555 INFO:teuthology.orchestra.run.smithi001.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-04T03:04:28.558 INFO:teuthology.orchestra.run.smithi001.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-04T03:04:28.559 INFO:teuthology.orchestra.run.smithi001.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-04T03:04:28.624 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2024-05-04T03:04:28.633 INFO:teuthology.orchestra.run.smithi001.stdout:======================================================================================= 2024-05-04T03:04:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: Package Arch Version Repository Size 2024-05-04T03:04:28.633 INFO:teuthology.orchestra.run.smithi001.stdout:======================================================================================= 2024-05-04T03:04:28.633 INFO:teuthology.orchestra.run.smithi001.stdout:Installing: 2024-05-04T03:04:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-04T03:04:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: ceph x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 6.6 k 2024-05-04T03:04:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-base x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 5.4 M 2024-05-04T03:04:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-fuse x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 863 k 2024-05-04T03:04:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 147 k 2024-05-04T03:04:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 1.5 M 2024-05-04T03:04:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 142 k 2024-05-04T03:04:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 3.5 M 2024-05-04T03:04:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 7.4 M 2024-05-04T03:04:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-rook noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 49 k 2024-05-04T03:04:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-radosgw x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 10 M 2024-05-04T03:04:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-test x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 43 M 2024-05-04T03:04:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-volume noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 273 k 2024-05-04T03:04:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: cephadm noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 763 k 2024-05-04T03:04:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-mirror x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 224 k 2024-05-04T03:04:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-shell x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 27 k 2024-05-04T03:04:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-top noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 18 k 2024-05-04T03:04:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-04T03:04:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-04T03:04:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-04T03:04:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs-devel x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 32 k 2024-05-04T03:04:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs2 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 756 k 2024-05-04T03:04:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: librados-devel x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 127 k 2024-05-04T03:04:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-04T03:04:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-04T03:04:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-04T03:04:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-04T03:04:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-04T03:04:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-04T03:04:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cephfs x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 162 k 2024-05-04T03:04:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rados x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 321 k 2024-05-04T03:04:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rbd x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 298 k 2024-05-04T03:04:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rgw x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 100 k 2024-05-04T03:04:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-fuse x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 85 k 2024-05-04T03:04:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-mirror x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 3.0 M 2024-05-04T03:04:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-nbd x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 170 k 2024-05-04T03:04:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-04T03:04:28.636 INFO:teuthology.orchestra.run.smithi001.stdout:Upgrading: 2024-05-04T03:04:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: librados2 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 3.3 M 2024-05-04T03:04:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: librbd1 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 3.1 M 2024-05-04T03:04:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-04T03:04:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-04T03:04:28.637 INFO:teuthology.orchestra.run.smithi001.stdout:Installing dependencies: 2024-05-04T03:04:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-04T03:04:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-04T03:04:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-04T03:04:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-common x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 20 M 2024-05-04T03:04:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 34 k 2024-05-04T03:04:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mds x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 2.3 M 2024-05-04T03:04:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 274 k 2024-05-04T03:04:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mon x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 4.9 M 2024-05-04T03:04:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-osd x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 18 M 2024-05-04T03:04:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 16 k 2024-05-04T03:04:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-selinux x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 25 k 2024-05-04T03:04:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-04T03:04:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-04T03:04:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-04T03:04:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-04T03:04:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-04T03:04:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-04T03:04:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-04T03:04:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-04T03:04:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: libcephsqlite x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 158 k 2024-05-04T03:04:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-04T03:04:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-04T03:04:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-04T03:04:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-04T03:04:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-04T03:04:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: libradosstriper1 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 497 k 2024-05-04T03:04:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-04T03:04:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: librgw2 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 5.2 M 2024-05-04T03:04:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-04T03:04:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-04T03:04:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-04T03:04:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-04T03:04:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-04T03:04:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-04T03:04:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-04T03:04:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-04T03:04:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-04T03:04:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-04T03:04:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-04T03:04:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-04T03:04:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-04T03:04:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-04T03:04:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-04T03:04:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-04T03:04:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-04T03:04:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-argparse x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 45 k 2024-05-04T03:04:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-common x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 135 k 2024-05-04T03:04:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-04T03:04:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-04T03:04:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-04T03:04:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-04T03:04:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-04T03:04:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-04T03:04:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-04T03:04:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-04T03:04:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-04T03:04:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-04T03:04:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-04T03:04:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-04T03:04:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-04T03:04:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-04T03:04:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-04T03:04:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-04T03:04:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-04T03:04:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-04T03:04:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-04T03:04:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-04T03:04:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-04T03:04:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-04T03:04:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-04T03:04:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-04T03:04:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-04T03:04:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-04T03:04:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-04T03:04:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-04T03:04:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-04T03:04:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-04T03:04:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-04T03:04:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-04T03:04:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-04T03:04:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-04T03:04:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-04T03:04:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-04T03:04:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-04T03:04:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-04T03:04:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-04T03:04:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-04T03:04:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-04T03:04:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-04T03:04:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-04T03:04:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-04T03:04:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-04T03:04:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-04T03:04:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-04T03:04:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-04T03:04:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-04T03:04:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-04T03:04:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-04T03:04:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-04T03:04:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-04T03:04:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-04T03:04:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-04T03:04:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-04T03:04:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-04T03:04:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-04T03:04:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-04T03:04:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-04T03:04:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-04T03:04:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-04T03:04:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-04T03:04:28.647 INFO:teuthology.orchestra.run.smithi001.stdout:Installing weak dependencies: 2024-05-04T03:04:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-04T03:04:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:04:28.647 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2024-05-04T03:04:28.647 INFO:teuthology.orchestra.run.smithi001.stdout:======================================================================================= 2024-05-04T03:04:28.647 INFO:teuthology.orchestra.run.smithi001.stdout:Install 148 Packages 2024-05-04T03:04:28.648 INFO:teuthology.orchestra.run.smithi001.stdout:Upgrade 4 Packages 2024-05-04T03:04:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:04:28.648 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 214 M 2024-05-04T03:04:28.649 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2024-05-04T03:04:29.363 INFO:teuthology.orchestra.run.smithi001.stdout:(1/152): ceph-19.0.0-3522.gbd4e00a3.el9.x86_64. 29 kB/s | 6.6 kB 00:00 2024-05-04T03:04:29.440 INFO:teuthology.orchestra.run.smithi072.stdout:(1/152): ceph-19.0.0-3522.gbd4e00a3.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-05-04T03:04:29.841 INFO:teuthology.orchestra.run.smithi072.stdout:(2/152): ceph-fuse-19.0.0-3522.gbd4e00a3.el9.x8 2.1 MB/s | 863 kB 00:00 2024-05-04T03:04:29.888 INFO:teuthology.orchestra.run.smithi001.stdout:(2/152): ceph-fuse-19.0.0-3522.gbd4e00a3.el9.x8 1.6 MB/s | 863 kB 00:00 2024-05-04T03:04:29.916 INFO:teuthology.orchestra.run.smithi072.stdout:(3/152): ceph-immutable-object-cache-19.0.0-352 1.9 MB/s | 147 kB 00:00 2024-05-04T03:04:29.980 INFO:teuthology.orchestra.run.smithi001.stdout:(3/152): ceph-immutable-object-cache-19.0.0-352 1.6 MB/s | 147 kB 00:00 2024-05-04T03:04:30.222 INFO:teuthology.orchestra.run.smithi001.stdout:(4/152): ceph-base-19.0.0-3522.gbd4e00a3.el9.x8 5.0 MB/s | 5.4 MB 00:01 2024-05-04T03:04:30.497 INFO:teuthology.orchestra.run.smithi001.stdout:(5/152): ceph-mgr-19.0.0-3522.gbd4e00a3.el9.x86 5.5 MB/s | 1.5 MB 00:00 2024-05-04T03:04:30.573 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:16 2024-05-04T03:04:30.625 INFO:teuthology.orchestra.run.smithi072.stdout:(4/152): ceph-mds-19.0.0-3522.gbd4e00a3.el9.x86 3.2 MB/s | 2.3 MB 00:00 2024-05-04T03:04:30.814 INFO:teuthology.orchestra.run.smithi001.stdout:(6/152): ceph-common-19.0.0-3522.gbd4e00a3.el9. 12 MB/s | 20 MB 00:01 2024-05-04T03:04:30.818 INFO:teuthology.orchestra.run.smithi072.stdout:(5/152): ceph-mgr-19.0.0-3522.gbd4e00a3.el9.x86 7.8 MB/s | 1.5 MB 00:00 2024-05-04T03:04:30.890 INFO:teuthology.orchestra.run.smithi001.stdout:(7/152): ceph-mds-19.0.0-3522.gbd4e00a3.el9.x86 2.5 MB/s | 2.3 MB 00:00 2024-05-04T03:04:31.372 INFO:teuthology.orchestra.run.smithi001.stdout:(8/152): ceph-mon-19.0.0-3522.gbd4e00a3.el9.x86 5.6 MB/s | 4.9 MB 00:00 2024-05-04T03:04:31.457 INFO:teuthology.orchestra.run.smithi001.stdout:(9/152): ceph-selinux-19.0.0-3522.gbd4e00a3.el9 297 kB/s | 25 kB 00:00 2024-05-04T03:04:31.776 INFO:teuthology.orchestra.run.smithi072.stdout:(6/152): ceph-mon-19.0.0-3522.gbd4e00a3.el9.x86 5.1 MB/s | 4.9 MB 00:00 2024-05-04T03:04:31.885 INFO:teuthology.orchestra.run.smithi072.stdout:(7/152): ceph-base-19.0.0-3522.gbd4e00a3.el9.x8 2.1 MB/s | 5.4 MB 00:02 2024-05-04T03:04:32.590 INFO:teuthology.orchestra.run.smithi001.stdout:(10/152): ceph-osd-19.0.0-3522.gbd4e00a3.el9.x8 10 MB/s | 18 MB 00:01 2024-05-04T03:04:32.676 INFO:teuthology.orchestra.run.smithi001.stdout:(11/152): cephfs-mirror-19.0.0-3522.gbd4e00a3.e 2.6 MB/s | 224 kB 00:00 2024-05-04T03:04:32.749 INFO:teuthology.orchestra.run.smithi001.stdout:(12/152): cephfs-shell-19.0.0-3522.gbd4e00a3.el 371 kB/s | 27 kB 00:00 2024-05-04T03:04:32.816 INFO:teuthology.orchestra.run.smithi001.stdout:(13/152): libcephfs-devel-19.0.0-3522.gbd4e00a3 475 kB/s | 32 kB 00:00 2024-05-04T03:04:32.983 INFO:teuthology.orchestra.run.smithi001.stdout:(14/152): libcephfs2-19.0.0-3522.gbd4e00a3.el9. 4.4 MB/s | 756 kB 00:00 2024-05-04T03:04:33.067 INFO:teuthology.orchestra.run.smithi001.stdout:(15/152): libcephsqlite-19.0.0-3522.gbd4e00a3.e 1.9 MB/s | 158 kB 00:00 2024-05-04T03:04:33.150 INFO:teuthology.orchestra.run.smithi001.stdout:(16/152): librados-devel-19.0.0-3522.gbd4e00a3. 1.5 MB/s | 127 kB 00:00 2024-05-04T03:04:33.268 INFO:teuthology.orchestra.run.smithi001.stdout:(17/152): libradosstriper1-19.0.0-3522.gbd4e00a 4.2 MB/s | 497 kB 00:00 2024-05-04T03:04:33.444 INFO:teuthology.orchestra.run.smithi072.stdout:(8/152): ceph-radosgw-19.0.0-3522.gbd4e00a3.el9 6.5 MB/s | 10 MB 00:01 2024-05-04T03:04:33.553 INFO:teuthology.orchestra.run.smithi072.stdout:(9/152): ceph-selinux-19.0.0-3522.gbd4e00a3.el9 232 kB/s | 25 kB 00:00 2024-05-04T03:04:33.835 INFO:teuthology.orchestra.run.smithi001.stdout:(18/152): librgw2-19.0.0-3522.gbd4e00a3.el9.x86 9.1 MB/s | 5.2 MB 00:00 2024-05-04T03:04:33.902 INFO:teuthology.orchestra.run.smithi001.stdout:(19/152): python3-ceph-argparse-19.0.0-3522.gbd 680 kB/s | 45 kB 00:00 2024-05-04T03:04:33.977 INFO:teuthology.orchestra.run.smithi001.stdout:(20/152): python3-ceph-common-19.0.0-3522.gbd4e 1.8 MB/s | 135 kB 00:00 2024-05-04T03:04:34.052 INFO:teuthology.orchestra.run.smithi001.stdout:(21/152): python3-cephfs-19.0.0-3522.gbd4e00a3. 2.1 MB/s | 162 kB 00:00 2024-05-04T03:04:34.136 INFO:teuthology.orchestra.run.smithi001.stdout:(22/152): python3-rados-19.0.0-3522.gbd4e00a3.e 3.8 MB/s | 321 kB 00:00 2024-05-04T03:04:34.219 INFO:teuthology.orchestra.run.smithi001.stdout:(23/152): python3-rbd-19.0.0-3522.gbd4e00a3.el9 3.5 MB/s | 298 kB 00:00 2024-05-04T03:04:34.336 INFO:teuthology.orchestra.run.smithi001.stdout:(24/152): ceph-radosgw-19.0.0-3522.gbd4e00a3.el 2.9 MB/s | 10 MB 00:03 2024-05-04T03:04:34.361 INFO:teuthology.orchestra.run.smithi001.stdout:(25/152): python3-rgw-19.0.0-3522.gbd4e00a3.el9 701 kB/s | 100 kB 00:00 2024-05-04T03:04:34.420 INFO:teuthology.orchestra.run.smithi001.stdout:(26/152): rbd-fuse-19.0.0-3522.gbd4e00a3.el9.x8 1.0 MB/s | 85 kB 00:00 2024-05-04T03:04:34.529 INFO:teuthology.orchestra.run.smithi001.stdout:(27/152): rbd-nbd-19.0.0-3522.gbd4e00a3.el9.x86 1.5 MB/s | 170 kB 00:00 2024-05-04T03:04:34.621 INFO:teuthology.orchestra.run.smithi001.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3522.g 373 kB/s | 34 kB 00:00 2024-05-04T03:04:34.721 INFO:teuthology.orchestra.run.smithi001.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3522.gbd4e00a 1.4 MB/s | 142 kB 00:00 2024-05-04T03:04:34.830 INFO:teuthology.orchestra.run.smithi001.stdout:(30/152): rbd-mirror-19.0.0-3522.gbd4e00a3.el9. 6.5 MB/s | 3.0 MB 00:00 2024-05-04T03:04:35.153 INFO:teuthology.orchestra.run.smithi072.stdout:(10/152): ceph-osd-19.0.0-3522.gbd4e00a3.el9.x8 5.3 MB/s | 18 MB 00:03 2024-05-04T03:04:35.304 INFO:teuthology.orchestra.run.smithi072.stdout:(11/152): cephfs-mirror-19.0.0-3522.gbd4e00a3.e 1.5 MB/s | 224 kB 00:00 2024-05-04T03:04:35.388 INFO:teuthology.orchestra.run.smithi072.stdout:(12/152): cephfs-shell-19.0.0-3522.gbd4e00a3.el 324 kB/s | 27 kB 00:00 2024-05-04T03:04:35.473 INFO:teuthology.orchestra.run.smithi072.stdout:(13/152): libcephfs-devel-19.0.0-3522.gbd4e00a3 372 kB/s | 32 kB 00:00 2024-05-04T03:04:35.732 INFO:teuthology.orchestra.run.smithi072.stdout:(14/152): libcephfs2-19.0.0-3522.gbd4e00a3.el9. 2.9 MB/s | 756 kB 00:00 2024-05-04T03:04:35.755 INFO:teuthology.orchestra.run.smithi001.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 8.0 MB/s | 7.4 MB 00:00 2024-05-04T03:04:35.808 INFO:teuthology.orchestra.run.smithi072.stdout:(15/152): libcephsqlite-19.0.0-3522.gbd4e00a3.e 2.1 MB/s | 158 kB 00:00 2024-05-04T03:04:35.838 INFO:teuthology.orchestra.run.smithi001.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3522.gbd 3.2 MB/s | 274 kB 00:00 2024-05-04T03:04:35.892 INFO:teuthology.orchestra.run.smithi072.stdout:(16/152): librados-devel-19.0.0-3522.gbd4e00a3. 1.5 MB/s | 127 kB 00:00 2024-05-04T03:04:35.914 INFO:teuthology.orchestra.run.smithi001.stdout:(33/152): ceph-mgr-rook-19.0.0-3522.gbd4e00a3.e 659 kB/s | 49 kB 00:00 2024-05-04T03:04:35.997 INFO:teuthology.orchestra.run.smithi001.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3522.gb 197 kB/s | 16 kB 00:00 2024-05-04T03:04:36.042 INFO:teuthology.orchestra.run.smithi072.stdout:(17/152): libradosstriper1-19.0.0-3522.gbd4e00a 3.2 MB/s | 497 kB 00:00 2024-05-04T03:04:36.081 INFO:teuthology.orchestra.run.smithi001.stdout:(35/152): ceph-volume-19.0.0-3522.gbd4e00a3.el9 3.2 MB/s | 273 kB 00:00 2024-05-04T03:04:36.140 INFO:teuthology.orchestra.run.smithi001.stdout:(36/152): ceph-mgr-dashboard-19.0.0-3522.gbd4e0 2.5 MB/s | 3.5 MB 00:01 2024-05-04T03:04:36.249 INFO:teuthology.orchestra.run.smithi001.stdout:(37/152): cephadm-19.0.0-3522.gbd4e00a3.el9.noa 4.5 MB/s | 763 kB 00:00 2024-05-04T03:04:36.265 INFO:teuthology.orchestra.run.smithi001.stdout:(38/152): cephfs-top-19.0.0-3522.gbd4e00a3.el9. 146 kB/s | 18 kB 00:00 2024-05-04T03:04:36.382 INFO:teuthology.orchestra.run.smithi001.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 601 kB/s | 80 kB 00:00 2024-05-04T03:04:36.449 INFO:teuthology.orchestra.run.smithi001.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 4.3 MB/s | 803 kB 00:00 2024-05-04T03:04:36.475 INFO:teuthology.orchestra.run.smithi001.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-05-04T03:04:36.500 INFO:teuthology.orchestra.run.smithi001.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-05-04T03:04:36.542 INFO:teuthology.orchestra.run.smithi001.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-05-04T03:04:36.576 INFO:teuthology.orchestra.run.smithi001.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-05-04T03:04:36.601 INFO:teuthology.orchestra.run.smithi001.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-05-04T03:04:36.626 INFO:teuthology.orchestra.run.smithi001.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-05-04T03:04:36.651 INFO:teuthology.orchestra.run.smithi001.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-05-04T03:04:36.893 INFO:teuthology.orchestra.run.smithi001.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 689 kB 00:00 2024-05-04T03:04:37.010 INFO:teuthology.orchestra.run.smithi001.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 943 kB 00:00 2024-05-04T03:04:37.035 INFO:teuthology.orchestra.run.smithi001.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 753 kB/s | 107 kB 00:00 2024-05-04T03:04:37.061 INFO:teuthology.orchestra.run.smithi001.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 428 kB/s | 21 kB 00:00 2024-05-04T03:04:37.111 INFO:teuthology.orchestra.run.smithi001.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 656 kB/s | 33 kB 00:00 2024-05-04T03:04:37.161 INFO:teuthology.orchestra.run.smithi001.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-05-04T03:04:37.167 INFO:teuthology.orchestra.run.smithi072.stdout:(18/152): librgw2-19.0.0-3522.gbd4e00a3.el9.x86 4.6 MB/s | 5.2 MB 00:01 2024-05-04T03:04:37.212 INFO:teuthology.orchestra.run.smithi001.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-05-04T03:04:37.224 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - CRB 2.0 MB/s | 6.2 MB 00:03 2024-05-04T03:04:37.243 INFO:teuthology.orchestra.run.smithi072.stdout:(19/152): python3-ceph-argparse-19.0.0-3522.gbd 602 kB/s | 45 kB 00:00 2024-05-04T03:04:37.262 INFO:teuthology.orchestra.run.smithi001.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-05-04T03:04:37.345 INFO:teuthology.orchestra.run.smithi001.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-05-04T03:04:37.360 INFO:teuthology.orchestra.run.smithi072.stdout:(20/152): python3-ceph-common-19.0.0-3522.gbd4e 1.1 MB/s | 135 kB 00:00 2024-05-04T03:04:37.396 INFO:teuthology.orchestra.run.smithi001.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-05-04T03:04:37.453 INFO:teuthology.orchestra.run.smithi072.stdout:(21/152): python3-cephfs-19.0.0-3522.gbd4e00a3. 1.7 MB/s | 162 kB 00:00 2024-05-04T03:04:37.578 INFO:teuthology.orchestra.run.smithi072.stdout:(22/152): python3-rados-19.0.0-3522.gbd4e00a3.e 2.5 MB/s | 321 kB 00:00 2024-05-04T03:04:37.579 INFO:teuthology.orchestra.run.smithi001.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.5 MB/s | 662 kB 00:00 2024-05-04T03:04:37.645 INFO:teuthology.orchestra.run.smithi072.stdout:(23/152): python3-rbd-19.0.0-3522.gbd4e00a3.el9 4.3 MB/s | 298 kB 00:00 2024-05-04T03:04:37.738 INFO:teuthology.orchestra.run.smithi072.stdout:(24/152): python3-rgw-19.0.0-3522.gbd4e00a3.el9 1.1 MB/s | 100 kB 00:00 2024-05-04T03:04:37.755 INFO:teuthology.orchestra.run.smithi001.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 3.3 MB/s | 582 kB 00:00 2024-05-04T03:04:37.805 INFO:teuthology.orchestra.run.smithi001.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 509 kB/s | 25 kB 00:00 2024-05-04T03:04:37.830 INFO:teuthology.orchestra.run.smithi072.stdout:(25/152): rbd-fuse-19.0.0-3522.gbd4e00a3.el9.x8 931 kB/s | 85 kB 00:00 2024-05-04T03:04:37.864 INFO:teuthology.orchestra.run.smithi001.stdout:(61/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.0 MB/s | 3.0 MB 00:00 2024-05-04T03:04:37.889 INFO:teuthology.orchestra.run.smithi001.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 568 kB/s | 47 kB 00:00 2024-05-04T03:04:37.948 INFO:teuthology.orchestra.run.smithi001.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-05-04T03:04:37.973 INFO:teuthology.orchestra.run.smithi001.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-05-04T03:04:37.998 INFO:teuthology.orchestra.run.smithi001.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 702 kB/s | 35 kB 00:00 2024-05-04T03:04:38.257 INFO:teuthology.orchestra.run.smithi072.stdout:(26/152): ceph-common-19.0.0-3522.gbd4e00a3.el9 2.2 MB/s | 20 MB 00:08 2024-05-04T03:04:38.798 INFO:teuthology.orchestra.run.smithi001.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 5.5 MB/s | 4.4 MB 00:00 2024-05-04T03:04:38.890 INFO:teuthology.orchestra.run.smithi001.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 5.2 MB/s | 4.7 MB 00:00 2024-05-04T03:04:38.916 INFO:teuthology.orchestra.run.smithi072.stdout:(27/152): ceph-test-19.0.0-3522.gbd4e00a3.el9.x 8.1 MB/s | 43 MB 00:05 2024-05-04T03:04:38.941 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 16 kB 00:00 2024-05-04T03:04:38.950 INFO:teuthology.orchestra.run.smithi072.stdout:(28/152): rbd-nbd-19.0.0-3522.gbd4e00a3.el9.x86 245 kB/s | 170 kB 00:00 2024-05-04T03:04:39.034 INFO:teuthology.orchestra.run.smithi072.stdout:(29/152): rbd-mirror-19.0.0-3522.gbd4e00a3.el9. 2.5 MB/s | 3.0 MB 00:01 2024-05-04T03:04:39.059 INFO:teuthology.orchestra.run.smithi072.stdout:(30/152): ceph-grafana-dashboards-19.0.0-3522.g 240 kB/s | 34 kB 00:00 2024-05-04T03:04:39.118 INFO:teuthology.orchestra.run.smithi072.stdout:(31/152): ceph-mgr-cephadm-19.0.0-3522.gbd4e00a 847 kB/s | 142 kB 00:00 2024-05-04T03:04:39.157 INFO:teuthology.orchestra.run.smithi001.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 4.4 MB/s | 1.6 MB 00:00 2024-05-04T03:04:39.191 INFO:teuthology.orchestra.run.smithi001.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 3.0 MB/s | 911 kB 00:00 2024-05-04T03:04:39.319 INFO:teuthology.orchestra.run.smithi072.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3522.gbd 1.3 MB/s | 274 kB 00:00 2024-05-04T03:04:39.324 INFO:teuthology.orchestra.run.smithi001.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 798 kB/s | 133 kB 00:00 2024-05-04T03:04:39.427 INFO:teuthology.orchestra.run.smithi072.stdout:(33/152): ceph-mgr-rook-19.0.0-3522.gbd4e00a3.e 454 kB/s | 49 kB 00:00 2024-05-04T03:04:39.528 INFO:teuthology.orchestra.run.smithi072.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3522.gb 164 kB/s | 16 kB 00:00 2024-05-04T03:04:39.591 INFO:teuthology.orchestra.run.smithi001.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 3.8 MB/s | 1.0 MB 00:00 2024-05-04T03:04:39.650 INFO:teuthology.orchestra.run.smithi001.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-05-04T03:04:39.737 INFO:teuthology.orchestra.run.smithi072.stdout:(35/152): ceph-volume-19.0.0-3522.gbd4e00a3.el9 1.3 MB/s | 273 kB 00:00 2024-05-04T03:04:40.137 INFO:teuthology.orchestra.run.smithi072.stdout:(36/152): cephadm-19.0.0-3522.gbd4e00a3.el9.noa 1.9 MB/s | 763 kB 00:00 2024-05-04T03:04:40.238 INFO:teuthology.orchestra.run.smithi072.stdout:(37/152): cephfs-top-19.0.0-3522.gbd4e00a3.el9. 182 kB/s | 18 kB 00:00 2024-05-04T03:04:40.375 INFO:teuthology.orchestra.run.smithi001.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 4.9 MB/s | 5.8 MB 00:01 2024-05-04T03:04:40.482 INFO:teuthology.orchestra.run.smithi001.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-05-04T03:04:40.497 INFO:teuthology.orchestra.run.smithi072.stdout:(38/152): ceph-mgr-dashboard-19.0.0-3522.gbd4e0 2.4 MB/s | 3.5 MB 00:01 2024-05-04T03:04:40.559 INFO:teuthology.orchestra.run.smithi001.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-05-04T03:04:40.601 INFO:teuthology.orchestra.run.smithi001.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 617 kB/s | 26 kB 00:00 2024-05-04T03:04:40.673 INFO:teuthology.orchestra.run.smithi072.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 185 kB/s | 80 kB 00:00 2024-05-04T03:04:40.985 INFO:teuthology.orchestra.run.smithi001.stdout:(77/152): ceph-test-19.0.0-3522.gbd4e00a3.el9.x 4.5 MB/s | 43 MB 00:09 2024-05-04T03:04:41.002 INFO:teuthology.orchestra.run.smithi001.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 48 kB/s | 19 kB 00:00 2024-05-04T03:04:41.044 INFO:teuthology.orchestra.run.smithi001.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 838 kB/s | 35 kB 00:00 2024-05-04T03:04:41.161 INFO:teuthology.orchestra.run.smithi001.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 983 kB/s | 172 kB 00:00 2024-05-04T03:04:41.312 INFO:teuthology.orchestra.run.smithi077.stdout:Extra Packages for Enterprise Linux 9.3 MB/s | 21 MB 00:02 2024-05-04T03:04:41.348 INFO:teuthology.orchestra.run.smithi072.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 289 kB/s | 195 kB 00:00 2024-05-04T03:04:41.403 INFO:teuthology.orchestra.run.smithi001.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 3.4 MB/s | 6.0 MB 00:01 2024-05-04T03:04:41.428 INFO:teuthology.orchestra.run.smithi001.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 939 kB/s | 250 kB 00:00 2024-05-04T03:04:41.479 INFO:teuthology.orchestra.run.smithi001.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-04T03:04:41.503 INFO:teuthology.orchestra.run.smithi001.stdout:(84/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-05-04T03:04:41.537 INFO:teuthology.orchestra.run.smithi001.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 720 kB/s | 42 kB 00:00 2024-05-04T03:04:41.587 INFO:teuthology.orchestra.run.smithi001.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 269 kB 00:00 2024-05-04T03:04:41.621 INFO:teuthology.orchestra.run.smithi001.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-05-04T03:04:41.671 INFO:teuthology.orchestra.run.smithi001.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-05-04T03:04:41.697 INFO:teuthology.orchestra.run.smithi001.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 678 kB/s | 51 kB 00:00 2024-05-04T03:04:41.722 INFO:teuthology.orchestra.run.smithi001.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-04T03:04:41.772 INFO:teuthology.orchestra.run.smithi001.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 834 kB/s | 42 kB 00:00 2024-05-04T03:04:41.831 INFO:teuthology.orchestra.run.smithi001.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 759 kB/s | 44 kB 00:00 2024-05-04T03:04:41.858 INFO:teuthology.orchestra.run.smithi072.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 496 kB/s | 253 kB 00:00 2024-05-04T03:04:41.914 INFO:teuthology.orchestra.run.smithi001.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-05-04T03:04:41.965 INFO:teuthology.orchestra.run.smithi001.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 719 kB/s | 36 kB 00:00 2024-05-04T03:04:42.023 INFO:teuthology.orchestra.run.smithi001.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 816 kB/s | 47 kB 00:00 2024-05-04T03:04:42.084 INFO:teuthology.orchestra.run.smithi072.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 506 kB/s | 803 kB 00:01 2024-05-04T03:04:42.107 INFO:teuthology.orchestra.run.smithi001.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 305 kB 00:00 2024-05-04T03:04:42.158 INFO:teuthology.orchestra.run.smithi001.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-04T03:04:42.249 INFO:teuthology.orchestra.run.smithi001.stdout:(98/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.2 MB/s | 5.1 MB 00:01 2024-05-04T03:04:42.275 INFO:teuthology.orchestra.run.smithi001.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 802 kB/s | 94 kB 00:00 2024-05-04T03:04:42.300 INFO:teuthology.orchestra.run.smithi001.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-04T03:04:42.326 INFO:teuthology.orchestra.run.smithi001.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-05-04T03:04:42.392 INFO:teuthology.orchestra.run.smithi001.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-05-04T03:04:42.534 INFO:teuthology.orchestra.run.smithi001.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 4.0 MB/s | 863 kB 00:00 2024-05-04T03:04:42.585 INFO:teuthology.orchestra.run.smithi072.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 213 kB/s | 106 kB 00:00 2024-05-04T03:04:42.668 INFO:teuthology.orchestra.run.smithi001.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-05-04T03:04:42.710 INFO:teuthology.orchestra.run.smithi001.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 468 kB/s | 19 kB 00:00 2024-05-04T03:04:42.735 INFO:teuthology.orchestra.run.smithi001.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.5 MB/s | 309 kB 00:00 2024-05-04T03:04:42.835 INFO:teuthology.orchestra.run.smithi001.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-05-04T03:04:42.877 INFO:teuthology.orchestra.run.smithi001.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-05-04T03:04:42.919 INFO:teuthology.orchestra.run.smithi001.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-05-04T03:04:42.978 INFO:teuthology.orchestra.run.smithi001.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-05-04T03:04:43.010 INFO:teuthology.orchestra.run.smithi072.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 318 kB/s | 135 kB 00:00 2024-05-04T03:04:43.295 INFO:teuthology.orchestra.run.smithi001.stdout:(111/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.6 MB/s | 839 kB 00:00 2024-05-04T03:04:43.510 INFO:teuthology.orchestra.run.smithi072.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 250 kB/s | 125 kB 00:00 2024-05-04T03:04:43.537 INFO:teuthology.orchestra.run.smithi001.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 2.2 MB/s | 548 kB 00:00 2024-05-04T03:04:43.570 INFO:teuthology.orchestra.run.smithi001.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-05-04T03:04:43.662 INFO:teuthology.orchestra.run.smithi001.stdout:(114/152): libarrow-9.0.0-11.el9.x86_64.rpm 4.6 MB/s | 4.4 MB 00:00 2024-05-04T03:04:43.696 INFO:teuthology.orchestra.run.smithi001.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 452 kB/s | 15 kB 00:00 2024-05-04T03:04:43.723 INFO:teuthology.orchestra.run.smithi001.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 214 kB/s | 32 kB 00:00 2024-05-04T03:04:43.747 INFO:teuthology.orchestra.run.smithi001.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-05-04T03:04:43.803 INFO:teuthology.orchestra.run.smithi072.stdout:(46/152): ceph-mgr-diskprediction-local-19.0.0- 1.6 MB/s | 7.4 MB 00:04 2024-05-04T03:04:43.813 INFO:teuthology.orchestra.run.smithi001.stdout:(118/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.5 MB/s | 303 kB 00:00 2024-05-04T03:04:43.847 INFO:teuthology.orchestra.run.smithi001.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 1.0 MB/s | 35 kB 00:00 2024-05-04T03:04:43.872 INFO:teuthology.orchestra.run.smithi001.stdout:(120/152): python3-cherrypy-18.6.1-2.el9.noarch 2.3 MB/s | 358 kB 00:00 2024-05-04T03:04:43.920 INFO:teuthology.orchestra.run.smithi072.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 609 kB/s | 1.2 MB 00:02 2024-05-04T03:04:44.031 INFO:teuthology.orchestra.run.smithi001.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.1 MB/s | 206 kB 00:00 2024-05-04T03:04:44.123 INFO:teuthology.orchestra.run.smithi001.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-05-04T03:04:44.165 INFO:teuthology.orchestra.run.smithi001.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 256 kB/s | 11 kB 00:00 2024-05-04T03:04:44.170 INFO:teuthology.orchestra.run.smithi072.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 325 kB/s | 215 kB 00:00 2024-05-04T03:04:44.232 INFO:teuthology.orchestra.run.smithi001.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 266 kB/s | 18 kB 00:00 2024-05-04T03:04:44.282 INFO:teuthology.orchestra.run.smithi001.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-05-04T03:04:44.324 INFO:teuthology.orchestra.run.smithi001.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 466 kB/s | 19 kB 00:00 2024-05-04T03:04:44.366 INFO:teuthology.orchestra.run.smithi001.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 472 kB/s | 20 kB 00:00 2024-05-04T03:04:44.513 INFO:teuthology.orchestra.run.smithi072.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 971 kB/s | 689 kB 00:00 2024-05-04T03:04:44.558 INFO:teuthology.orchestra.run.smithi001.stdout:(128/152): python3-grpcio-1.46.7-10.el9.x86_64. 3.0 MB/s | 2.0 MB 00:00 2024-05-04T03:04:44.596 INFO:teuthology.orchestra.run.smithi072.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 251 kB/s | 107 kB 00:00 2024-05-04T03:04:44.600 INFO:teuthology.orchestra.run.smithi001.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-05-04T03:04:44.622 INFO:teuthology.orchestra.run.smithi072.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 197 kB/s | 21 kB 00:00 2024-05-04T03:04:44.642 INFO:teuthology.orchestra.run.smithi001.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 3.7 MB/s | 1.0 MB 00:00 2024-05-04T03:04:44.668 INFO:teuthology.orchestra.run.smithi001.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.1 MB/s | 79 kB 00:00 2024-05-04T03:04:44.701 INFO:teuthology.orchestra.run.smithi001.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 985 kB/s | 58 kB 00:00 2024-05-04T03:04:44.731 INFO:teuthology.orchestra.run.smithi072.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-05-04T03:04:44.743 INFO:teuthology.orchestra.run.smithi001.stdout:(133/152): python3-portend-3.1.0-2.el9.noarch.r 396 kB/s | 16 kB 00:00 2024-05-04T03:04:44.785 INFO:teuthology.orchestra.run.smithi001.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-05-04T03:04:44.790 INFO:teuthology.orchestra.run.smithi072.stdout:(53/152): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-05-04T03:04:44.810 INFO:teuthology.orchestra.run.smithi001.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-05-04T03:04:44.899 INFO:teuthology.orchestra.run.smithi072.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 1.0 MB/s | 313 kB 00:00 2024-05-04T03:04:44.924 INFO:teuthology.orchestra.run.smithi072.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 136 kB/s | 18 kB 00:00 2024-05-04T03:04:44.986 INFO:teuthology.orchestra.run.smithi001.stdout:(136/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 4.9 MB/s | 16 MB 00:03 2024-05-04T03:04:45.011 INFO:teuthology.orchestra.run.smithi001.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 125 kB/s | 28 kB 00:00 2024-05-04T03:04:45.035 INFO:teuthology.orchestra.run.smithi072.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 101 kB/s | 11 kB 00:00 2024-05-04T03:04:45.036 INFO:teuthology.orchestra.run.smithi001.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 136 kB/s | 31 kB 00:00 2024-05-04T03:04:45.060 INFO:teuthology.orchestra.run.smithi072.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-05-04T03:04:45.079 INFO:teuthology.orchestra.run.smithi001.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-05-04T03:04:45.103 INFO:teuthology.orchestra.run.smithi001.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 538 kB/s | 36 kB 00:00 2024-05-04T03:04:45.129 INFO:teuthology.orchestra.run.smithi001.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-05-04T03:04:45.136 INFO:teuthology.orchestra.run.smithi072.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 449 kB/s | 45 kB 00:00 2024-05-04T03:04:45.154 INFO:teuthology.orchestra.run.smithi001.stdout:(142/152): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2024-05-04T03:04:45.179 INFO:teuthology.orchestra.run.smithi001.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-05-04T03:04:45.246 INFO:teuthology.orchestra.run.smithi001.stdout:(144/152): python3-zc-lockfile-2.0-10.el9.noarc 299 kB/s | 20 kB 00:00 2024-05-04T03:04:45.261 INFO:teuthology.orchestra.run.smithi072.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.6 MB/s | 3.0 MB 00:00 2024-05-04T03:04:45.295 INFO:teuthology.orchestra.run.smithi072.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 2.8 MB/s | 662 kB 00:00 2024-05-04T03:04:45.305 INFO:teuthology.orchestra.run.smithi001.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-05-04T03:04:45.330 INFO:teuthology.orchestra.run.smithi001.stdout:(146/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.0 MB/s | 230 kB 00:00 2024-05-04T03:04:45.362 INFO:teuthology.orchestra.run.smithi072.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 253 kB/s | 25 kB 00:00 2024-05-04T03:04:45.389 INFO:teuthology.orchestra.run.smithi001.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-05-04T03:04:45.396 INFO:teuthology.orchestra.run.smithi072.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 471 kB/s | 47 kB 00:00 2024-05-04T03:04:45.447 INFO:teuthology.orchestra.run.smithi072.stdout:(63/152): libtool-2.4.6-45.el9.x86_64.rpm 1.8 MB/s | 582 kB 00:00 2024-05-04T03:04:45.447 INFO:teuthology.orchestra.run.smithi001.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-04T03:04:45.481 INFO:teuthology.orchestra.run.smithi072.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-05-04T03:04:45.515 INFO:teuthology.orchestra.run.smithi072.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-05-04T03:04:45.549 INFO:teuthology.orchestra.run.smithi072.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 348 kB/s | 35 kB 00:00 2024-05-04T03:04:45.606 INFO:teuthology.orchestra.run.smithi001.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 8.5 MB/s | 1.3 MB 00:00 2024-05-04T03:04:45.765 INFO:teuthology.orchestra.run.smithi072.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-05-04T03:04:45.773 INFO:teuthology.orchestra.run.smithi001.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-05-04T03:04:45.891 INFO:teuthology.orchestra.run.smithi072.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 7.1 MB/s | 911 kB 00:00 2024-05-04T03:04:46.000 INFO:teuthology.orchestra.run.smithi072.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-05-04T03:04:46.115 INFO:teuthology.orchestra.run.smithi001.stdout:(151/152): librbd1-19.0.0-3522.gbd4e00a3.el9.x8 4.3 MB/s | 3.1 MB 00:00 2024-05-04T03:04:46.190 INFO:teuthology.orchestra.run.smithi001.stdout:(152/152): librados2-19.0.0-3522.gbd4e00a3.el9. 3.9 MB/s | 3.3 MB 00:00 2024-05-04T03:04:46.193 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2024-05-04T03:04:46.193 INFO:teuthology.orchestra.run.smithi001.stdout:Total 12 MB/s | 214 MB 00:17 2024-05-04T03:04:46.285 INFO:teuthology.orchestra.run.smithi077.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-05-04T03:04:47.186 INFO:teuthology.orchestra.run.smithi072.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 2.6 MB/s | 4.4 MB 00:01 2024-05-04T03:04:47.305 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2024-05-04T03:04:47.465 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2024-05-04T03:04:47.465 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2024-05-04T03:04:48.345 INFO:teuthology.orchestra.run.smithi072.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 1.7 MB/s | 4.7 MB 00:02 2024-05-04T03:04:48.394 INFO:teuthology.orchestra.run.smithi077.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-04T03:04:48.395 INFO:teuthology.orchestra.run.smithi077.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-04T03:04:48.405 INFO:teuthology.orchestra.run.smithi077.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-04T03:04:48.406 INFO:teuthology.orchestra.run.smithi077.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-04T03:04:48.407 INFO:teuthology.orchestra.run.smithi077.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-04T03:04:48.408 INFO:teuthology.orchestra.run.smithi077.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-04T03:04:48.409 INFO:teuthology.orchestra.run.smithi077.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-04T03:04:48.412 INFO:teuthology.orchestra.run.smithi077.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-04T03:04:48.412 INFO:teuthology.orchestra.run.smithi077.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-04T03:04:48.472 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-05-04T03:04:48.492 INFO:teuthology.orchestra.run.smithi077.stdout:======================================================================================= 2024-05-04T03:04:48.492 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2024-05-04T03:04:48.492 INFO:teuthology.orchestra.run.smithi077.stdout:======================================================================================= 2024-05-04T03:04:48.493 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-05-04T03:04:48.493 INFO:teuthology.orchestra.run.smithi077.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-04T03:04:48.493 INFO:teuthology.orchestra.run.smithi077.stdout: ceph x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 6.6 k 2024-05-04T03:04:48.493 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-base x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 5.4 M 2024-05-04T03:04:48.493 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-fuse x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 863 k 2024-05-04T03:04:48.493 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 147 k 2024-05-04T03:04:48.493 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 1.5 M 2024-05-04T03:04:48.493 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 142 k 2024-05-04T03:04:48.493 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 3.5 M 2024-05-04T03:04:48.493 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 7.4 M 2024-05-04T03:04:48.493 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-rook noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 49 k 2024-05-04T03:04:48.493 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-radosgw x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 10 M 2024-05-04T03:04:48.493 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-test x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 43 M 2024-05-04T03:04:48.493 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-volume noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 273 k 2024-05-04T03:04:48.494 INFO:teuthology.orchestra.run.smithi077.stdout: cephadm noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 763 k 2024-05-04T03:04:48.494 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-mirror x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 224 k 2024-05-04T03:04:48.494 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-shell x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 27 k 2024-05-04T03:04:48.494 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-top noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 18 k 2024-05-04T03:04:48.494 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-04T03:04:48.494 INFO:teuthology.orchestra.run.smithi077.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-04T03:04:48.494 INFO:teuthology.orchestra.run.smithi077.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-04T03:04:48.494 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs-devel x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 32 k 2024-05-04T03:04:48.494 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs2 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 756 k 2024-05-04T03:04:48.494 INFO:teuthology.orchestra.run.smithi077.stdout: librados-devel x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 127 k 2024-05-04T03:04:48.494 INFO:teuthology.orchestra.run.smithi077.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-04T03:04:48.494 INFO:teuthology.orchestra.run.smithi077.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-04T03:04:48.494 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-04T03:04:48.495 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-04T03:04:48.495 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-04T03:04:48.495 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-04T03:04:48.495 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cephfs x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 162 k 2024-05-04T03:04:48.495 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rados x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 321 k 2024-05-04T03:04:48.495 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rbd x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 298 k 2024-05-04T03:04:48.495 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rgw x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 100 k 2024-05-04T03:04:48.495 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-fuse x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 85 k 2024-05-04T03:04:48.495 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-mirror x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 3.0 M 2024-05-04T03:04:48.495 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-nbd x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 170 k 2024-05-04T03:04:48.495 INFO:teuthology.orchestra.run.smithi077.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-04T03:04:48.495 INFO:teuthology.orchestra.run.smithi077.stdout:Upgrading: 2024-05-04T03:04:48.495 INFO:teuthology.orchestra.run.smithi077.stdout: librados2 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 3.3 M 2024-05-04T03:04:48.496 INFO:teuthology.orchestra.run.smithi077.stdout: librbd1 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 3.1 M 2024-05-04T03:04:48.496 INFO:teuthology.orchestra.run.smithi077.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-04T03:04:48.496 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-04T03:04:48.496 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2024-05-04T03:04:48.496 INFO:teuthology.orchestra.run.smithi077.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-04T03:04:48.496 INFO:teuthology.orchestra.run.smithi077.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-04T03:04:48.496 INFO:teuthology.orchestra.run.smithi077.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-04T03:04:48.496 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-common x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 20 M 2024-05-04T03:04:48.496 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 34 k 2024-05-04T03:04:48.496 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mds x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 2.3 M 2024-05-04T03:04:48.496 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 274 k 2024-05-04T03:04:48.496 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mon x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 4.9 M 2024-05-04T03:04:48.497 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-osd x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 18 M 2024-05-04T03:04:48.497 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3522.gbd4e00a3.el9 ceph-noarch 16 k 2024-05-04T03:04:48.497 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-selinux x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 25 k 2024-05-04T03:04:48.497 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-04T03:04:48.497 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-04T03:04:48.497 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-04T03:04:48.497 INFO:teuthology.orchestra.run.smithi077.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-04T03:04:48.497 INFO:teuthology.orchestra.run.smithi077.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-04T03:04:48.497 INFO:teuthology.orchestra.run.smithi077.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-04T03:04:48.497 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-04T03:04:48.497 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-04T03:04:48.498 INFO:teuthology.orchestra.run.smithi077.stdout: libcephsqlite x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 158 k 2024-05-04T03:04:48.498 INFO:teuthology.orchestra.run.smithi077.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-04T03:04:48.498 INFO:teuthology.orchestra.run.smithi077.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-04T03:04:48.498 INFO:teuthology.orchestra.run.smithi077.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-04T03:04:48.498 INFO:teuthology.orchestra.run.smithi077.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-04T03:04:48.498 INFO:teuthology.orchestra.run.smithi077.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-04T03:04:48.498 INFO:teuthology.orchestra.run.smithi077.stdout: libradosstriper1 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 497 k 2024-05-04T03:04:48.498 INFO:teuthology.orchestra.run.smithi077.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-04T03:04:48.498 INFO:teuthology.orchestra.run.smithi077.stdout: librgw2 x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 5.2 M 2024-05-04T03:04:48.498 INFO:teuthology.orchestra.run.smithi077.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-04T03:04:48.498 INFO:teuthology.orchestra.run.smithi077.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-04T03:04:48.498 INFO:teuthology.orchestra.run.smithi077.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-04T03:04:48.498 INFO:teuthology.orchestra.run.smithi077.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-04T03:04:48.499 INFO:teuthology.orchestra.run.smithi077.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-04T03:04:48.499 INFO:teuthology.orchestra.run.smithi077.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-04T03:04:48.499 INFO:teuthology.orchestra.run.smithi077.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-04T03:04:48.499 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-04T03:04:48.499 INFO:teuthology.orchestra.run.smithi077.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-04T03:04:48.499 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-04T03:04:48.499 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-04T03:04:48.499 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-04T03:04:48.499 INFO:teuthology.orchestra.run.smithi077.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-04T03:04:48.499 INFO:teuthology.orchestra.run.smithi077.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-04T03:04:48.499 INFO:teuthology.orchestra.run.smithi077.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-04T03:04:48.500 INFO:teuthology.orchestra.run.smithi077.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-04T03:04:48.500 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-04T03:04:48.500 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-argparse x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 45 k 2024-05-04T03:04:48.500 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-common x86_64 2:19.0.0-3522.gbd4e00a3.el9 ceph 135 k 2024-05-04T03:04:48.500 INFO:teuthology.orchestra.run.smithi077.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-04T03:04:48.500 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-04T03:04:48.500 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-04T03:04:48.500 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-04T03:04:48.500 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-04T03:04:48.500 INFO:teuthology.orchestra.run.smithi077.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-04T03:04:48.500 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-04T03:04:48.500 INFO:teuthology.orchestra.run.smithi077.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-04T03:04:48.500 INFO:teuthology.orchestra.run.smithi077.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-04T03:04:48.501 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-04T03:04:48.501 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-04T03:04:48.501 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-04T03:04:48.501 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-04T03:04:48.501 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-04T03:04:48.501 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-04T03:04:48.501 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-04T03:04:48.501 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-04T03:04:48.501 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-04T03:04:48.501 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-04T03:04:48.501 INFO:teuthology.orchestra.run.smithi077.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-04T03:04:48.502 INFO:teuthology.orchestra.run.smithi077.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-04T03:04:48.502 INFO:teuthology.orchestra.run.smithi077.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-04T03:04:48.502 INFO:teuthology.orchestra.run.smithi077.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-04T03:04:48.502 INFO:teuthology.orchestra.run.smithi077.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-04T03:04:48.502 INFO:teuthology.orchestra.run.smithi077.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-04T03:04:48.502 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-04T03:04:48.502 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-04T03:04:48.502 INFO:teuthology.orchestra.run.smithi077.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-04T03:04:48.502 INFO:teuthology.orchestra.run.smithi077.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-04T03:04:48.502 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-04T03:04:48.502 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-04T03:04:48.503 INFO:teuthology.orchestra.run.smithi077.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-04T03:04:48.503 INFO:teuthology.orchestra.run.smithi077.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-04T03:04:48.503 INFO:teuthology.orchestra.run.smithi077.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-04T03:04:48.503 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-04T03:04:48.503 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-04T03:04:48.503 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-04T03:04:48.503 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-04T03:04:48.503 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-04T03:04:48.503 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-04T03:04:48.503 INFO:teuthology.orchestra.run.smithi077.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-04T03:04:48.503 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-04T03:04:48.503 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-04T03:04:48.503 INFO:teuthology.orchestra.run.smithi077.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-04T03:04:48.503 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-04T03:04:48.503 INFO:teuthology.orchestra.run.smithi077.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-04T03:04:48.504 INFO:teuthology.orchestra.run.smithi077.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-04T03:04:48.504 INFO:teuthology.orchestra.run.smithi077.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-04T03:04:48.504 INFO:teuthology.orchestra.run.smithi077.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-04T03:04:48.504 INFO:teuthology.orchestra.run.smithi077.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-04T03:04:48.504 INFO:teuthology.orchestra.run.smithi077.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-04T03:04:48.504 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-04T03:04:48.504 INFO:teuthology.orchestra.run.smithi077.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-04T03:04:48.504 INFO:teuthology.orchestra.run.smithi077.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-04T03:04:48.504 INFO:teuthology.orchestra.run.smithi077.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-04T03:04:48.504 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-04T03:04:48.504 INFO:teuthology.orchestra.run.smithi077.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-04T03:04:48.504 INFO:teuthology.orchestra.run.smithi077.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-04T03:04:48.505 INFO:teuthology.orchestra.run.smithi077.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-04T03:04:48.505 INFO:teuthology.orchestra.run.smithi077.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-04T03:04:48.505 INFO:teuthology.orchestra.run.smithi077.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-04T03:04:48.505 INFO:teuthology.orchestra.run.smithi077.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-04T03:04:48.505 INFO:teuthology.orchestra.run.smithi077.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-04T03:04:48.505 INFO:teuthology.orchestra.run.smithi077.stdout:Installing weak dependencies: 2024-05-04T03:04:48.505 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-04T03:04:48.505 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-04T03:04:48.505 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-05-04T03:04:48.505 INFO:teuthology.orchestra.run.smithi077.stdout:======================================================================================= 2024-05-04T03:04:48.505 INFO:teuthology.orchestra.run.smithi077.stdout:Install 148 Packages 2024-05-04T03:04:48.505 INFO:teuthology.orchestra.run.smithi077.stdout:Upgrade 4 Packages 2024-05-04T03:04:48.505 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-04T03:04:48.506 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 214 M 2024-05-04T03:04:48.506 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-05-04T03:04:48.528 INFO:teuthology.orchestra.run.smithi072.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 477 kB/s | 87 kB 00:00 2024-05-04T03:04:48.646 INFO:teuthology.orchestra.run.smithi072.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 708 kB/s | 1.0 MB 00:01 2024-05-04T03:04:48.996 INFO:teuthology.orchestra.run.smithi072.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 697 kB/s | 244 kB 00:00 2024-05-04T03:04:49.316 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2024-05-04T03:04:49.317 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2024-05-04T03:04:49.338 INFO:teuthology.orchestra.run.smithi072.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 725 kB/s | 248 kB 00:00 2024-05-04T03:04:49.439 INFO:teuthology.orchestra.run.smithi072.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 255 kB/s | 26 kB 00:00 2024-05-04T03:04:49.512 INFO:teuthology.orchestra.run.smithi077.stdout:(1/152): ceph-19.0.0-3522.gbd4e00a3.el9.x86_64. 38 kB/s | 6.6 kB 00:00 2024-05-04T03:04:49.548 INFO:teuthology.orchestra.run.smithi072.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 176 kB/s | 19 kB 00:00 2024-05-04T03:04:49.754 INFO:teuthology.orchestra.run.smithi077.stdout:(2/152): ceph-fuse-19.0.0-3522.gbd4e00a3.el9.x8 3.5 MB/s | 863 kB 00:00 2024-05-04T03:04:49.812 INFO:teuthology.orchestra.run.smithi077.stdout:(3/152): ceph-immutable-object-cache-19.0.0-352 2.5 MB/s | 147 kB 00:00 2024-05-04T03:04:49.823 INFO:teuthology.orchestra.run.smithi072.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 625 kB/s | 172 kB 00:00 2024-05-04T03:04:49.932 INFO:teuthology.orchestra.run.smithi072.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 320 kB/s | 35 kB 00:00 2024-05-04T03:04:50.121 INFO:teuthology.orchestra.run.smithi077.stdout:(4/152): ceph-mds-19.0.0-3522.gbd4e00a3.el9.x86 7.3 MB/s | 2.3 MB 00:00 2024-05-04T03:04:50.338 INFO:teuthology.orchestra.run.smithi077.stdout:(5/152): ceph-mgr-19.0.0-3522.gbd4e00a3.el9.x86 6.9 MB/s | 1.5 MB 00:00 2024-05-04T03:04:50.630 INFO:teuthology.orchestra.run.smithi077.stdout:(6/152): ceph-base-19.0.0-3522.gbd4e00a3.el9.x8 4.2 MB/s | 5.4 MB 00:01 2024-05-04T03:04:51.038 INFO:teuthology.orchestra.run.smithi077.stdout:(7/152): ceph-mon-19.0.0-3522.gbd4e00a3.el9.x86 7.0 MB/s | 4.9 MB 00:00 2024-05-04T03:04:51.525 INFO:teuthology.orchestra.run.smithi072.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 1.0 MB/s | 5.8 MB 00:05 2024-05-04T03:04:51.539 INFO:teuthology.orchestra.run.smithi077.stdout:(8/152): ceph-common-19.0.0-3522.gbd4e00a3.el9. 8.9 MB/s | 20 MB 00:02 2024-05-04T03:04:51.614 INFO:teuthology.orchestra.run.smithi077.stdout:(9/152): ceph-selinux-19.0.0-3522.gbd4e00a3.el9 337 kB/s | 25 kB 00:00 2024-05-04T03:04:51.875 INFO:teuthology.orchestra.run.smithi072.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 716 kB/s | 250 kB 00:00 2024-05-04T03:04:52.301 INFO:teuthology.orchestra.run.smithi072.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 522 kB/s | 222 kB 00:00 2024-05-04T03:04:52.485 INFO:teuthology.orchestra.run.smithi072.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 420 kB/s | 77 kB 00:00 2024-05-04T03:04:52.514 INFO:teuthology.orchestra.run.smithi077.stdout:(10/152): ceph-radosgw-19.0.0-3522.gbd4e00a3.el 6.9 MB/s | 10 MB 00:01 2024-05-04T03:04:52.581 INFO:teuthology.orchestra.run.smithi077.stdout:(11/152): cephfs-mirror-19.0.0-3522.gbd4e00a3.e 3.3 MB/s | 224 kB 00:00 2024-05-04T03:04:52.595 INFO:teuthology.orchestra.run.smithi072.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 384 kB/s | 42 kB 00:00 2024-05-04T03:04:52.632 INFO:teuthology.orchestra.run.smithi077.stdout:(12/152): cephfs-shell-19.0.0-3522.gbd4e00a3.el 541 kB/s | 27 kB 00:00 2024-05-04T03:04:52.690 INFO:teuthology.orchestra.run.smithi077.stdout:(13/152): libcephfs-devel-19.0.0-3522.gbd4e00a3 545 kB/s | 32 kB 00:00 2024-05-04T03:04:52.824 INFO:teuthology.orchestra.run.smithi077.stdout:(14/152): libcephfs2-19.0.0-3522.gbd4e00a3.el9. 5.6 MB/s | 756 kB 00:00 2024-05-04T03:04:52.891 INFO:teuthology.orchestra.run.smithi077.stdout:(15/152): libcephsqlite-19.0.0-3522.gbd4e00a3.e 2.3 MB/s | 158 kB 00:00 2024-05-04T03:04:52.958 INFO:teuthology.orchestra.run.smithi077.stdout:(16/152): librados-devel-19.0.0-3522.gbd4e00a3. 1.9 MB/s | 127 kB 00:00 2024-05-04T03:04:53.021 INFO:teuthology.orchestra.run.smithi072.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 633 kB/s | 269 kB 00:00 2024-05-04T03:04:53.075 INFO:teuthology.orchestra.run.smithi077.stdout:(17/152): libradosstriper1-19.0.0-3522.gbd4e00a 4.2 MB/s | 497 kB 00:00 2024-05-04T03:04:53.288 INFO:teuthology.orchestra.run.smithi072.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 595 kB/s | 159 kB 00:00 2024-05-04T03:04:53.714 INFO:teuthology.orchestra.run.smithi072.stdout:(87/152): python3-pyasn1-modules-0.4.8-6.el9.no 655 kB/s | 279 kB 00:00 2024-05-04T03:04:53.800 INFO:teuthology.orchestra.run.smithi077.stdout:(18/152): librgw2-19.0.0-3522.gbd4e00a3.el9.x86 7.1 MB/s | 5.2 MB 00:00 2024-05-04T03:04:53.814 INFO:teuthology.orchestra.run.smithi072.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 509 kB/s | 51 kB 00:00 2024-05-04T03:04:53.842 INFO:teuthology.orchestra.run.smithi077.stdout:(19/152): python3-ceph-argparse-19.0.0-3522.gbd 1.1 MB/s | 45 kB 00:00 2024-05-04T03:04:53.915 INFO:teuthology.orchestra.run.smithi072.stdout:(89/152): python3-requests-oauthlib-1.3.0-12.el 535 kB/s | 54 kB 00:00 2024-05-04T03:04:54.017 INFO:teuthology.orchestra.run.smithi077.stdout:(20/152): ceph-osd-19.0.0-3522.gbd4e00a3.el9.x8 5.2 MB/s | 18 MB 00:03 2024-05-04T03:04:54.043 INFO:teuthology.orchestra.run.smithi077.stdout:(21/152): python3-ceph-common-19.0.0-3522.gbd4e 676 kB/s | 135 kB 00:00 2024-05-04T03:04:54.085 INFO:teuthology.orchestra.run.smithi077.stdout:(22/152): python3-cephfs-19.0.0-3522.gbd4e00a3. 2.4 MB/s | 162 kB 00:00 2024-05-04T03:04:54.110 INFO:teuthology.orchestra.run.smithi077.stdout:(23/152): python3-rados-19.0.0-3522.gbd4e00a3.e 4.7 MB/s | 321 kB 00:00 2024-05-04T03:04:54.170 INFO:teuthology.orchestra.run.smithi077.stdout:(24/152): python3-rbd-19.0.0-3522.gbd4e00a3.el9 3.5 MB/s | 298 kB 00:00 2024-05-04T03:04:54.187 INFO:teuthology.orchestra.run.smithi077.stdout:(25/152): python3-rgw-19.0.0-3522.gbd4e00a3.el9 1.3 MB/s | 100 kB 00:00 2024-05-04T03:04:54.245 INFO:teuthology.orchestra.run.smithi077.stdout:(26/152): rbd-fuse-19.0.0-3522.gbd4e00a3.el9.x8 1.1 MB/s | 85 kB 00:00 2024-05-04T03:04:54.338 INFO:teuthology.orchestra.run.smithi077.stdout:(27/152): rbd-nbd-19.0.0-3522.gbd4e00a3.el9.x86 1.8 MB/s | 170 kB 00:00 2024-05-04T03:04:54.415 INFO:teuthology.orchestra.run.smithi077.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3522.g 457 kB/s | 34 kB 00:00 2024-05-04T03:04:54.495 INFO:teuthology.orchestra.run.smithi077.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3522.gbd4e00a 1.7 MB/s | 142 kB 00:00 2024-05-04T03:04:54.579 INFO:teuthology.orchestra.run.smithi077.stdout:(30/152): rbd-mirror-19.0.0-3522.gbd4e00a3.el9. 7.7 MB/s | 3.0 MB 00:00 2024-05-04T03:04:55.421 INFO:teuthology.orchestra.run.smithi077.stdout:(31/152): ceph-test-19.0.0-3522.gbd4e00a3.el9.x 11 MB/s | 43 MB 00:03 2024-05-04T03:04:55.480 INFO:teuthology.orchestra.run.smithi077.stdout:(32/152): ceph-mgr-dashboard-19.0.0-3522.gbd4e0 3.6 MB/s | 3.5 MB 00:00 2024-05-04T03:04:55.505 INFO:teuthology.orchestra.run.smithi077.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3522.gbd 3.2 MB/s | 274 kB 00:00 2024-05-04T03:04:55.597 INFO:teuthology.orchestra.run.smithi077.stdout:(34/152): ceph-mgr-diskprediction-local-19.0.0- 7.3 MB/s | 7.4 MB 00:01 2024-05-04T03:04:55.622 INFO:teuthology.orchestra.run.smithi077.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3522.gb 141 kB/s | 16 kB 00:00 2024-05-04T03:04:55.648 INFO:teuthology.orchestra.run.smithi077.stdout:(36/152): ceph-mgr-rook-19.0.0-3522.gbd4e00a3.e 294 kB/s | 49 kB 00:00 2024-05-04T03:04:55.650 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2024-05-04T03:04:55.673 INFO:teuthology.orchestra.run.smithi077.stdout:(37/152): ceph-volume-19.0.0-3522.gbd4e00a3.el9 3.5 MB/s | 273 kB 00:00 2024-05-04T03:04:55.706 INFO:teuthology.orchestra.run.smithi077.stdout:(38/152): cephadm-19.0.0-3522.gbd4e00a3.el9.noa 8.9 MB/s | 763 kB 00:00 2024-05-04T03:04:55.731 INFO:teuthology.orchestra.run.smithi077.stdout:(39/152): cephfs-top-19.0.0-3522.gbd4e00a3.el9. 218 kB/s | 18 kB 00:00 2024-05-04T03:04:55.754 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-05-04T03:04:55.790 INFO:teuthology.orchestra.run.smithi077.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 685 kB/s | 80 kB 00:00 2024-05-04T03:04:55.793 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-05-04T03:04:55.846 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-05-04T03:04:55.857 INFO:teuthology.orchestra.run.smithi077.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 195 kB 00:00 2024-05-04T03:04:55.883 INFO:teuthology.orchestra.run.smithi077.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-05-04T03:04:55.888 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-05-04T03:04:55.924 INFO:teuthology.orchestra.run.smithi077.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.6 MB/s | 803 kB 00:00 2024-05-04T03:04:55.934 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-05-04T03:04:55.950 INFO:teuthology.orchestra.run.smithi077.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-05-04T03:04:55.975 INFO:teuthology.orchestra.run.smithi077.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-05-04T03:04:55.982 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-05-04T03:04:56.000 INFO:teuthology.orchestra.run.smithi077.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-05-04T03:04:56.026 INFO:teuthology.orchestra.run.smithi077.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-05-04T03:04:56.059 INFO:teuthology.orchestra.run.smithi077.stdout:(48/152): python3-cryptography-36.0.1-4.el9.x86 6.1 MB/s | 1.2 MB 00:00 2024-05-04T03:04:56.285 INFO:teuthology.orchestra.run.smithi077.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 475 kB/s | 107 kB 00:00 2024-05-04T03:04:56.318 INFO:teuthology.orchestra.run.smithi077.stdout:(50/152): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-05-04T03:04:56.333 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-05-04T03:04:56.369 INFO:teuthology.orchestra.run.smithi077.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 943 kB 00:00 2024-05-04T03:04:56.394 INFO:teuthology.orchestra.run.smithi077.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 197 kB/s | 21 kB 00:00 2024-05-04T03:04:56.419 INFO:teuthology.orchestra.run.smithi077.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-05-04T03:04:56.445 INFO:teuthology.orchestra.run.smithi077.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 432 kB/s | 33 kB 00:00 2024-05-04T03:04:56.455 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 8/156 2024-05-04T03:04:56.486 INFO:teuthology.orchestra.run.smithi077.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 272 kB/s | 18 kB 00:00 2024-05-04T03:04:56.512 INFO:teuthology.orchestra.run.smithi077.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-05-04T03:04:56.573 INFO:teuthology.orchestra.run.smithi072.stdout:(90/152): python3-babel-2.9.1-2.el9.noarch.rpm 759 kB/s | 6.0 MB 00:08 2024-05-04T03:04:56.579 INFO:teuthology.orchestra.run.smithi077.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-05-04T03:04:56.592 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 8/156 2024-05-04T03:04:56.604 INFO:teuthology.orchestra.run.smithi077.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-05-04T03:04:56.615 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 9/156 2024-05-04T03:04:56.629 INFO:teuthology.orchestra.run.smithi077.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 386 kB/s | 45 kB 00:00 2024-05-04T03:04:56.663 INFO:teuthology.orchestra.run.smithi077.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2024-05-04T03:04:56.674 INFO:teuthology.orchestra.run.smithi072.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 416 kB/s | 42 kB 00:00 2024-05-04T03:04:56.697 INFO:teuthology.orchestra.run.smithi077.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 582 kB 00:00 2024-05-04T03:04:56.707 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 9/156 2024-05-04T03:04:56.722 INFO:teuthology.orchestra.run.smithi077.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 276 kB/s | 25 kB 00:00 2024-05-04T03:04:56.736 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rados-2:19.0.0-3522.gbd4e00a3.el9.x86_64 10/156 2024-05-04T03:04:56.747 INFO:teuthology.orchestra.run.smithi077.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 566 kB/s | 47 kB 00:00 2024-05-04T03:04:56.772 INFO:teuthology.orchestra.run.smithi077.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-05-04T03:04:56.774 INFO:teuthology.orchestra.run.smithi072.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 441 kB/s | 44 kB 00:00 2024-05-04T03:04:56.798 INFO:teuthology.orchestra.run.smithi077.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-05-04T03:04:56.823 INFO:teuthology.orchestra.run.smithi077.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 466 kB/s | 35 kB 00:00 2024-05-04T03:04:56.892 INFO:teuthology.orchestra.run.smithi072.stdout:(93/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 750 kB/s | 5.1 MB 00:06 2024-05-04T03:04:56.902 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-05-04T03:04:56.932 INFO:teuthology.orchestra.run.smithi077.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2024-05-04T03:04:56.979 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-05-04T03:04:56.990 INFO:teuthology.orchestra.run.smithi077.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-05-04T03:04:57.000 INFO:teuthology.orchestra.run.smithi072.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 331 kB/s | 36 kB 00:00 2024-05-04T03:04:57.058 INFO:teuthology.orchestra.run.smithi077.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-05-04T03:04:57.082 INFO:teuthology.orchestra.run.smithi077.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 5.9 MB/s | 911 kB 00:00 2024-05-04T03:04:57.101 INFO:teuthology.orchestra.run.smithi072.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-05-04T03:04:57.108 INFO:teuthology.orchestra.run.smithi077.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-05-04T03:04:57.177 INFO:teuthology.orchestra.run.smithi072.stdout:(96/152): qatlib-24.02.0-1.el9.x86_64.rpm 549 kB/s | 221 kB 00:00 2024-05-04T03:04:57.177 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 13/156 2024-05-04T03:04:57.183 INFO:teuthology.orchestra.run.smithi077.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-05-04T03:04:57.208 INFO:teuthology.orchestra.run.smithi077.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 872 kB/s | 87 kB 00:00 2024-05-04T03:04:57.284 INFO:teuthology.orchestra.run.smithi077.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 26 MB/s | 5.8 MB 00:00 2024-05-04T03:04:57.284 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 13/156 2024-05-04T03:04:57.311 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86 14/156 2024-05-04T03:04:57.334 INFO:teuthology.orchestra.run.smithi077.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.9 MB/s | 244 kB 00:00 2024-05-04T03:04:57.359 INFO:teuthology.orchestra.run.smithi077.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-05-04T03:04:57.374 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86 14/156 2024-05-04T03:04:57.396 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-04T03:04:57.416 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-04T03:04:57.429 INFO:teuthology.orchestra.run.smithi072.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 229 kB/s | 58 kB 00:00 2024-05-04T03:04:57.443 INFO:teuthology.orchestra.run.smithi077.stdout:(77/152): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 6.0 MB 00:00 2024-05-04T03:04:57.468 INFO:teuthology.orchestra.run.smithi077.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 191 kB/s | 26 kB 00:00 2024-05-04T03:04:57.494 INFO:teuthology.orchestra.run.smithi077.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 143 kB/s | 19 kB 00:00 2024-05-04T03:04:57.519 INFO:teuthology.orchestra.run.smithi077.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-05-04T03:04:57.544 INFO:teuthology.orchestra.run.smithi077.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 461 kB/s | 35 kB 00:00 2024-05-04T03:04:57.581 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-04T03:04:57.595 INFO:teuthology.orchestra.run.smithi077.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-05-04T03:04:57.613 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-04T03:04:57.620 INFO:teuthology.orchestra.run.smithi077.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-05-04T03:04:57.687 INFO:teuthology.orchestra.run.smithi077.stdout:(84/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 5.1 MB 00:00 2024-05-04T03:04:57.712 INFO:teuthology.orchestra.run.smithi077.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 659 kB/s | 77 kB 00:00 2024-05-04T03:04:57.723 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-05-04T03:04:57.737 INFO:teuthology.orchestra.run.smithi077.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 361 kB/s | 42 kB 00:00 2024-05-04T03:04:57.752 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-05-04T03:04:57.763 INFO:teuthology.orchestra.run.smithi077.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-05-04T03:04:57.788 INFO:teuthology.orchestra.run.smithi077.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-05-04T03:04:57.789 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/156 2024-05-04T03:04:57.813 INFO:teuthology.orchestra.run.smithi077.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-05-04T03:04:57.847 INFO:teuthology.orchestra.run.smithi077.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 610 kB/s | 51 kB 00:00 2024-05-04T03:04:57.863 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/156 2024-05-04T03:04:57.872 INFO:teuthology.orchestra.run.smithi077.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 639 kB/s | 54 kB 00:00 2024-05-04T03:04:57.879 INFO:teuthology.orchestra.run.smithi072.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 392 kB/s | 305 kB 00:00 2024-05-04T03:04:57.905 INFO:teuthology.orchestra.run.smithi072.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 197 kB/s | 94 kB 00:00 2024-05-04T03:04:57.908 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/156 2024-05-04T03:04:57.923 INFO:teuthology.orchestra.run.smithi077.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 552 kB/s | 42 kB 00:00 2024-05-04T03:04:57.956 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/156 2024-05-04T03:04:57.956 INFO:teuthology.orchestra.run.smithi077.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 527 kB/s | 44 kB 00:00 2024-05-04T03:04:57.988 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/156 2024-05-04T03:04:57.998 INFO:teuthology.orchestra.run.smithi077.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-05-04T03:04:58.014 INFO:teuthology.orchestra.run.smithi072.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 413 kB/s | 45 kB 00:00 2024-05-04T03:04:58.015 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/156 2024-05-04T03:04:58.023 INFO:teuthology.orchestra.run.smithi077.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 538 kB/s | 36 kB 00:00 2024-05-04T03:04:58.064 INFO:teuthology.orchestra.run.smithi072.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 345 kB/s | 64 kB 00:00 2024-05-04T03:04:58.065 INFO:teuthology.orchestra.run.smithi077.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 706 kB/s | 47 kB 00:00 2024-05-04T03:04:58.088 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/156 2024-05-04T03:04:58.107 INFO:teuthology.orchestra.run.smithi077.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 305 kB 00:00 2024-05-04T03:04:58.115 INFO:teuthology.orchestra.run.smithi072.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 221 kB/s | 22 kB 00:00 2024-05-04T03:04:58.156 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/156 2024-05-04T03:04:58.185 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/156 2024-05-04T03:04:58.229 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/156 2024-05-04T03:04:58.275 INFO:teuthology.orchestra.run.smithi077.stdout:(98/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-05-04T03:04:58.300 INFO:teuthology.orchestra.run.smithi077.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 247 kB/s | 58 kB 00:00 2024-05-04T03:04:58.308 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/156 2024-05-04T03:04:58.325 INFO:teuthology.orchestra.run.smithi077.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 431 kB/s | 94 kB 00:00 2024-05-04T03:04:58.339 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-ceph-common-2:19.0.0-3522.gbd4e00a3.el9. 30/156 2024-05-04T03:04:58.350 INFO:teuthology.orchestra.run.smithi077.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 845 kB/s | 64 kB 00:00 2024-05-04T03:04:58.375 INFO:teuthology.orchestra.run.smithi077.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 593 kB/s | 45 kB 00:00 2024-05-04T03:04:58.383 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-ceph-argparse-2:19.0.0-3522.gbd4e00a3.el 31/156 2024-05-04T03:04:58.403 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cephfs-2:19.0.0-3522.gbd4e00a3.el9.x86_6 32/156 2024-05-04T03:04:58.416 INFO:teuthology.orchestra.run.smithi072.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.8 MB/s | 548 kB 00:00 2024-05-04T03:04:58.442 INFO:teuthology.orchestra.run.smithi077.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 190 kB/s | 22 kB 00:00 2024-05-04T03:04:58.449 INFO:teuthology.orchestra.run.smithi072.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.2 MB/s | 863 kB 00:00 2024-05-04T03:04:58.478 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/156 2024-05-04T03:04:58.527 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/156 2024-05-04T03:04:58.550 INFO:teuthology.orchestra.run.smithi072.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 2.3 MB/s | 309 kB 00:00 2024-05-04T03:04:58.576 INFO:teuthology.orchestra.run.smithi077.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 3.7 MB/s | 863 kB 00:00 2024-05-04T03:04:58.604 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/156 2024-05-04T03:04:58.617 INFO:teuthology.orchestra.run.smithi072.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 116 kB/s | 19 kB 00:00 2024-05-04T03:04:58.662 INFO:teuthology.orchestra.run.smithi072.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 619 kB/s | 26 kB 00:00 2024-05-04T03:04:58.718 INFO:teuthology.orchestra.run.smithi072.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 830 kB/s | 49 kB 00:00 2024-05-04T03:04:58.752 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/156 2024-05-04T03:04:58.761 INFO:teuthology.orchestra.run.smithi072.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-05-04T03:04:58.805 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/156 2024-05-04T03:04:58.819 INFO:teuthology.orchestra.run.smithi072.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-05-04T03:04:58.951 INFO:teuthology.orchestra.run.smithi077.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 52 kB/s | 19 kB 00:00 2024-05-04T03:04:59.037 INFO:teuthology.orchestra.run.smithi072.stdout:(111/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.8 MB/s | 839 kB 00:00 2024-05-04T03:04:59.135 INFO:teuthology.orchestra.run.smithi077.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 446 kB/s | 309 kB 00:00 2024-05-04T03:04:59.196 INFO:teuthology.orchestra.run.smithi072.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-05-04T03:04:59.239 INFO:teuthology.orchestra.run.smithi072.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-05-04T03:04:59.277 INFO:teuthology.orchestra.run.smithi077.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 608 kB/s | 548 kB 00:00 2024-05-04T03:04:59.281 INFO:teuthology.orchestra.run.smithi072.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 766 kB/s | 32 kB 00:00 2024-05-04T03:04:59.324 INFO:teuthology.orchestra.run.smithi072.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 358 kB/s | 15 kB 00:00 2024-05-04T03:04:59.371 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/156 2024-05-04T03:04:59.399 INFO:teuthology.orchestra.run.smithi072.stdout:(116/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-05-04T03:04:59.461 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/156 2024-05-04T03:04:59.495 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/156 2024-05-04T03:04:59.516 INFO:teuthology.orchestra.run.smithi077.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 69 kB/s | 26 kB 00:00 2024-05-04T03:04:59.517 INFO:teuthology.orchestra.run.smithi072.stdout:(117/152): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-05-04T03:04:59.538 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/156 2024-05-04T03:04:59.584 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librados-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_6 42/156 2024-05-04T03:04:59.610 INFO:teuthology.orchestra.run.smithi072.stdout:(118/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-05-04T03:04:59.627 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/156 2024-05-04T03:04:59.651 INFO:teuthology.orchestra.run.smithi072.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 844 kB/s | 35 kB 00:00 2024-05-04T03:04:59.736 INFO:teuthology.orchestra.run.smithi077.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 106 kB/s | 49 kB 00:00 2024-05-04T03:04:59.752 INFO:teuthology.orchestra.run.smithi072.stdout:(120/152): libarrow-9.0.0-11.el9.x86_64.rpm 3.7 MB/s | 4.4 MB 00:01 2024-05-04T03:04:59.794 INFO:teuthology.orchestra.run.smithi072.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.4 MB/s | 206 kB 00:00 2024-05-04T03:04:59.801 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 44/156 2024-05-04T03:04:59.853 INFO:teuthology.orchestra.run.smithi072.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-05-04T03:04:59.861 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 45/156 2024-05-04T03:04:59.895 INFO:teuthology.orchestra.run.smithi072.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 256 kB/s | 11 kB 00:00 2024-05-04T03:04:59.937 INFO:teuthology.orchestra.run.smithi072.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 422 kB/s | 18 kB 00:00 2024-05-04T03:04:59.958 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 46/156 2024-05-04T03:04:59.980 INFO:teuthology.orchestra.run.smithi072.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 551 kB/s | 23 kB 00:00 2024-05-04T03:05:00.020 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 47/156 2024-05-04T03:05:00.030 INFO:teuthology.orchestra.run.smithi072.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-05-04T03:05:00.045 INFO:teuthology.orchestra.run.smithi077.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 126 kB/s | 67 kB 00:00 2024-05-04T03:05:00.074 INFO:teuthology.orchestra.run.smithi072.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 451 kB/s | 20 kB 00:00 2024-05-04T03:05:00.094 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-05-04T03:05:00.129 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-05-04T03:05:00.166 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-05-04T03:05:00.214 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-05-04T03:05:00.249 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-05-04T03:05:00.291 INFO:teuthology.orchestra.run.smithi072.stdout:(128/152): python3-grpcio-1.46.7-10.el9.x86_64. 3.8 MB/s | 2.0 MB 00:00 2024-05-04T03:05:00.293 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-05-04T03:05:00.333 INFO:teuthology.orchestra.run.smithi072.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-05-04T03:05:00.334 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-05-04T03:05:00.353 INFO:teuthology.orchestra.run.smithi077.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 245 kB/s | 151 kB 00:00 2024-05-04T03:05:00.392 INFO:teuthology.orchestra.run.smithi072.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 3.2 MB/s | 1.0 MB 00:00 2024-05-04T03:05:00.418 INFO:teuthology.orchestra.run.smithi072.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 936 kB/s | 79 kB 00:00 2024-05-04T03:05:00.443 INFO:teuthology.orchestra.run.smithi072.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-05-04T03:05:00.480 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-05-04T03:05:00.485 INFO:teuthology.orchestra.run.smithi072.stdout:(133/152): python3-portend-3.1.0-2.el9.noarch.r 393 kB/s | 16 kB 00:00 2024-05-04T03:05:00.527 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-05-04T03:05:00.536 INFO:teuthology.orchestra.run.smithi072.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-05-04T03:05:00.561 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-05-04T03:05:00.562 INFO:teuthology.orchestra.run.smithi072.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-05-04T03:05:00.587 INFO:teuthology.orchestra.run.smithi072.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 557 kB/s | 28 kB 00:00 2024-05-04T03:05:00.603 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-05-04T03:05:00.613 INFO:teuthology.orchestra.run.smithi072.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 607 kB/s | 31 kB 00:00 2024-05-04T03:05:00.655 INFO:teuthology.orchestra.run.smithi072.stdout:(138/152): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-05-04T03:05:00.697 INFO:teuthology.orchestra.run.smithi072.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-05-04T03:05:00.702 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-05-04T03:05:00.723 INFO:teuthology.orchestra.run.smithi072.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 530 kB/s | 36 kB 00:00 2024-05-04T03:05:00.736 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-05-04T03:05:00.757 INFO:teuthology.orchestra.run.smithi072.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 863 kB/s | 51 kB 00:00 2024-05-04T03:05:00.776 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-05-04T03:05:00.810 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-05-04T03:05:00.817 INFO:teuthology.orchestra.run.smithi072.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-05-04T03:05:00.834 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-05-04T03:05:00.843 INFO:teuthology.orchestra.run.smithi072.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-05-04T03:05:00.885 INFO:teuthology.orchestra.run.smithi072.stdout:(144/152): python3-zc-lockfile-2.0-10.el9.noarc 477 kB/s | 20 kB 00:00 2024-05-04T03:05:00.914 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-05-04T03:05:00.944 INFO:teuthology.orchestra.run.smithi072.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-05-04T03:05:01.003 INFO:teuthology.orchestra.run.smithi072.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-05-04T03:05:01.043 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-05-04T03:05:01.062 INFO:teuthology.orchestra.run.smithi077.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 825 kB/s | 839 kB 00:01 2024-05-04T03:05:01.212 INFO:teuthology.orchestra.run.smithi077.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 639 kB/s | 548 kB 00:00 2024-05-04T03:05:01.268 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-05-04T03:05:01.312 INFO:teuthology.orchestra.run.smithi072.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 4.3 MB/s | 1.6 MB 00:00 2024-05-04T03:05:01.313 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-05-04T03:05:01.347 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-05-04T03:05:01.395 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-05-04T03:05:01.435 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-05-04T03:05:01.464 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-05-04T03:05:01.498 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-05-04T03:05:01.529 INFO:teuthology.orchestra.run.smithi077.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 93 kB/s | 43 kB 00:00 2024-05-04T03:05:01.532 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-05-04T03:05:01.575 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-05-04T03:05:01.579 INFO:teuthology.orchestra.run.smithi072.stdout:(148/152): librados2-19.0.0-3522.gbd4e00a3.el9. 5.8 MB/s | 3.3 MB 00:00 2024-05-04T03:05:01.608 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-05-04T03:05:01.680 INFO:teuthology.orchestra.run.smithi077.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 69 kB/s | 32 kB 00:00 2024-05-04T03:05:01.704 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-05-04T03:05:01.756 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-05-04T03:05:01.794 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-05-04T03:05:01.843 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-05-04T03:05:01.922 INFO:teuthology.orchestra.run.smithi077.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 38 kB/s | 15 kB 00:00 2024-05-04T03:05:01.954 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-05-04T03:05:02.005 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-05-04T03:05:02.255 INFO:teuthology.orchestra.run.smithi077.stdout:(117/152): libarrow-9.0.0-11.el9.x86_64.rpm 1.3 MB/s | 4.4 MB 00:03 2024-05-04T03:05:02.389 INFO:teuthology.orchestra.run.smithi077.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 242 kB/s | 172 kB 00:00 2024-05-04T03:05:02.505 INFO:teuthology.orchestra.run.smithi072.stdout:(149/152): librbd1-19.0.0-3522.gbd4e00a3.el9.x8 2.6 MB/s | 3.1 MB 00:01 2024-05-04T03:05:02.697 INFO:teuthology.orchestra.run.smithi077.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 462 kB/s | 358 kB 00:00 2024-05-04T03:05:02.788 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-05-04T03:05:02.834 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-05-04T03:05:02.856 INFO:teuthology.orchestra.run.smithi077.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 76 kB/s | 35 kB 00:00 2024-05-04T03:05:02.868 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-05-04T03:05:02.975 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-05-04T03:05:03.027 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-05-04T03:05:03.031 INFO:teuthology.orchestra.run.smithi077.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 391 kB/s | 303 kB 00:00 2024-05-04T03:05:03.127 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-05-04T03:05:03.178 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-05-04T03:05:03.348 INFO:teuthology.orchestra.run.smithi077.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 316 kB/s | 206 kB 00:00 2024-05-04T03:05:03.363 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-05-04T03:05:03.400 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-05-04T03:05:03.481 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-05-04T03:05:03.657 INFO:teuthology.orchestra.run.smithi077.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 231 kB/s | 144 kB 00:00 2024-05-04T03:05:03.730 INFO:teuthology.orchestra.run.smithi072.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 643 kB/s | 1.3 MB 00:02 2024-05-04T03:05:03.740 INFO:teuthology.orchestra.run.smithi077.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 27 kB/s | 11 kB 00:00 2024-05-04T03:05:04.058 INFO:teuthology.orchestra.run.smithi077.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 44 kB/s | 18 kB 00:00 2024-05-04T03:05:04.158 INFO:teuthology.orchestra.run.smithi077.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 59 kB/s | 23 kB 00:00 2024-05-04T03:05:04.188 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-05-04T03:05:04.312 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-05-04T03:05:04.467 INFO:teuthology.orchestra.run.smithi077.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 48 kB/s | 19 kB 00:00 2024-05-04T03:05:04.600 INFO:teuthology.orchestra.run.smithi077.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 47 kB/s | 20 kB 00:00 2024-05-04T03:05:04.692 INFO:teuthology.orchestra.run.smithi077.stdout:(129/152): python3-grpcio-1.46.7-10.el9.x86_64. 1.1 MB/s | 2.0 MB 00:01 2024-05-04T03:05:05.109 INFO:teuthology.orchestra.run.smithi077.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 91 kB/s | 46 kB 00:00 2024-05-04T03:05:05.251 INFO:teuthology.orchestra.run.smithi077.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 148 kB/s | 79 kB 00:00 2024-05-04T03:05:05.380 INFO:teuthology.orchestra.run.smithi072.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 852 kB/s | 2.4 MB 00:02 2024-05-04T03:05:05.568 INFO:teuthology.orchestra.run.smithi077.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 126 kB/s | 58 kB 00:00 2024-05-04T03:05:05.637 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-05-04T03:05:05.730 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-05-04T03:05:05.758 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-05-04T03:05:05.791 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-05-04T03:05:05.943 INFO:teuthology.orchestra.run.smithi077.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 393 kB/s | 272 kB 00:00 2024-05-04T03:05:05.960 INFO:teuthology.orchestra.run.smithi077.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 42 kB/s | 16 kB 00:00 2024-05-04T03:05:06.125 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-05-04T03:05:06.161 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 99/156 2024-05-04T03:05:06.211 INFO:teuthology.orchestra.run.smithi077.stdout:(135/152): python3-kubernetes-26.1.0-2.el9.noar 600 kB/s | 1.0 MB 00:01 2024-05-04T03:05:06.253 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 99/156 2024-05-04T03:05:06.283 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 100/156 2024-05-04T03:05:06.320 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-05-04T03:05:06.352 INFO:teuthology.orchestra.run.smithi077.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 72 kB/s | 28 kB 00:00 2024-05-04T03:05:06.486 INFO:teuthology.orchestra.run.smithi077.stdout:(137/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 166 kB/s | 90 kB 00:00 2024-05-04T03:05:06.686 INFO:teuthology.orchestra.run.smithi077.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 65 kB/s | 31 kB 00:00 2024-05-04T03:05:06.835 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-05-04T03:05:06.864 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 103/156 2024-05-04T03:05:06.936 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 103/156 2024-05-04T03:05:06.961 INFO:teuthology.orchestra.run.smithi077.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 124 kB/s | 59 kB 00:00 2024-05-04T03:05:06.966 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 104/156 2024-05-04T03:05:07.062 INFO:teuthology.orchestra.run.smithi077.stdout:(140/152): python3-routes-2.5.1-5.el9.noarch.rp 266 kB/s | 188 kB 00:00 2024-05-04T03:05:07.162 INFO:teuthology.orchestra.run.smithi077.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 75 kB/s | 36 kB 00:00 2024-05-04T03:05:07.471 INFO:teuthology.orchestra.run.smithi077.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 100 kB/s | 51 kB 00:00 2024-05-04T03:05:07.696 INFO:teuthology.orchestra.run.smithi077.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 379 kB/s | 230 kB 00:00 2024-05-04T03:05:07.721 INFO:teuthology.orchestra.run.smithi077.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 161 kB/s | 90 kB 00:00 2024-05-04T03:05:08.080 INFO:teuthology.orchestra.run.smithi077.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 52 kB/s | 20 kB 00:00 2024-05-04T03:05:08.247 INFO:teuthology.orchestra.run.smithi077.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 551 kB/s | 427 kB 00:00 2024-05-04T03:05:08.430 INFO:teuthology.orchestra.run.smithi077.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 270 kB/s | 192 kB 00:00 2024-05-04T03:05:08.873 INFO:teuthology.orchestra.run.smithi077.stdout:(148/152): librados2-19.0.0-3522.gbd4e00a3.el9. 5.3 MB/s | 3.3 MB 00:00 2024-05-04T03:05:09.023 INFO:teuthology.orchestra.run.smithi077.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 9.0 MB/s | 1.3 MB 00:00 2024-05-04T03:05:09.073 INFO:teuthology.orchestra.run.smithi077.stdout:(150/152): librbd1-19.0.0-3522.gbd4e00a3.el9.x8 4.8 MB/s | 3.1 MB 00:00 2024-05-04T03:05:09.132 INFO:teuthology.orchestra.run.smithi077.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-05-04T03:05:09.258 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 105/156 2024-05-04T03:05:09.287 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 105/156 2024-05-04T03:05:09.757 INFO:teuthology.orchestra.run.smithi077.stdout:(152/152): thrift-0.15.0-2.el9.x86_64.rpm 966 kB/s | 1.6 MB 00:01 2024-05-04T03:05:09.761 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-05-04T03:05:09.761 INFO:teuthology.orchestra.run.smithi077.stdout:Total 10 MB/s | 214 MB 00:21 2024-05-04T03:05:09.875 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 105/156 2024-05-04T03:05:09.916 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 106/156 2024-05-04T03:05:10.020 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 106/156 2024-05-04T03:05:10.020 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-04T03:05:10.020 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-04T03:05:10.020 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:05:10.068 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 107/156 2024-05-04T03:05:10.876 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-05-04T03:05:11.021 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-05-04T03:05:11.021 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-05-04T03:05:12.746 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-05-04T03:05:12.747 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-05-04T03:05:16.247 INFO:teuthology.orchestra.run.smithi072.stdout:(152/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 745 kB/s | 16 MB 00:22 2024-05-04T03:05:16.252 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-05-04T03:05:16.253 INFO:teuthology.orchestra.run.smithi072.stdout:Total 4.5 MB/s | 214 MB 00:47 2024-05-04T03:05:17.302 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-05-04T03:05:17.445 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-05-04T03:05:17.445 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-05-04T03:05:18.616 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-05-04T03:05:18.695 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-05-04T03:05:18.766 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-05-04T03:05:19.031 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-05-04T03:05:19.072 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-05-04T03:05:19.097 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-05-04T03:05:19.141 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-05-04T03:05:19.170 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-05-04T03:05:19.170 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-05-04T03:05:19.484 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-05-04T03:05:19.606 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 8/156 2024-05-04T03:05:19.713 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 8/156 2024-05-04T03:05:19.740 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 9/156 2024-05-04T03:05:19.832 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 9/156 2024-05-04T03:05:19.861 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rados-2:19.0.0-3522.gbd4e00a3.el9.x86_64 10/156 2024-05-04T03:05:20.010 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-05-04T03:05:20.058 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-05-04T03:05:20.086 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 13/156 2024-05-04T03:05:20.190 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 13/156 2024-05-04T03:05:20.212 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86 14/156 2024-05-04T03:05:20.275 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86 14/156 2024-05-04T03:05:20.297 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-04T03:05:20.317 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-04T03:05:20.513 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-04T03:05:20.546 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-04T03:05:20.653 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-05-04T03:05:20.686 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-05-04T03:05:20.721 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/156 2024-05-04T03:05:20.786 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/156 2024-05-04T03:05:20.820 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/156 2024-05-04T03:05:20.864 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/156 2024-05-04T03:05:20.897 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/156 2024-05-04T03:05:20.925 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/156 2024-05-04T03:05:20.993 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/156 2024-05-04T03:05:21.046 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/156 2024-05-04T03:05:21.078 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/156 2024-05-04T03:05:21.112 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/156 2024-05-04T03:05:21.188 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/156 2024-05-04T03:05:21.223 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ceph-common-2:19.0.0-3522.gbd4e00a3.el9. 30/156 2024-05-04T03:05:21.267 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ceph-argparse-2:19.0.0-3522.gbd4e00a3.el 31/156 2024-05-04T03:05:21.295 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cephfs-2:19.0.0-3522.gbd4e00a3.el9.x86_6 32/156 2024-05-04T03:05:21.365 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/156 2024-05-04T03:05:21.410 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/156 2024-05-04T03:05:21.484 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/156 2024-05-04T03:05:21.633 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/156 2024-05-04T03:05:21.688 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/156 2024-05-04T03:05:22.236 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/156 2024-05-04T03:05:22.307 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 107/156 2024-05-04T03:05:22.307 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /sys 2024-05-04T03:05:22.307 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /proc 2024-05-04T03:05:22.307 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /mnt 2024-05-04T03:05:22.307 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /var/tmp 2024-05-04T03:05:22.308 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /home 2024-05-04T03:05:22.308 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /root 2024-05-04T03:05:22.308 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /tmp 2024-05-04T03:05:22.308 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:05:22.316 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/156 2024-05-04T03:05:22.345 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/156 2024-05-04T03:05:22.388 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/156 2024-05-04T03:05:22.427 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librados-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_6 42/156 2024-05-04T03:05:22.468 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/156 2024-05-04T03:05:22.489 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 108/156 2024-05-04T03:05:22.505 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 108/156 2024-05-04T03:05:22.505 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T03:05:22.505 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-04T03:05:22.505 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:05:22.620 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 44/156 2024-05-04T03:05:22.671 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 45/156 2024-05-04T03:05:22.762 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 46/156 2024-05-04T03:05:22.818 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 47/156 2024-05-04T03:05:22.899 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-05-04T03:05:22.931 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-05-04T03:05:22.967 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-05-04T03:05:23.022 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-05-04T03:05:23.059 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-05-04T03:05:23.102 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-05-04T03:05:23.144 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-05-04T03:05:23.287 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-05-04T03:05:23.333 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-05-04T03:05:23.363 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-05-04T03:05:23.404 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-05-04T03:05:23.498 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-05-04T03:05:23.538 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-05-04T03:05:23.578 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-05-04T03:05:23.603 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-05-04T03:05:23.628 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-05-04T03:05:23.704 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-05-04T03:05:23.831 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-05-04T03:05:23.861 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-05-04T03:05:23.889 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-05-04T03:05:23.925 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-05-04T03:05:23.962 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-05-04T03:05:24.002 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-05-04T03:05:24.033 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-05-04T03:05:24.067 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-05-04T03:05:24.100 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-05-04T03:05:24.142 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-05-04T03:05:24.144 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 109/156 2024-05-04T03:05:24.176 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-05-04T03:05:24.177 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 109/156 2024-05-04T03:05:24.177 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T03:05:24.177 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-04T03:05:24.177 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-04T03:05:24.177 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-04T03:05:24.177 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:05:24.262 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-05-04T03:05:24.306 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-05-04T03:05:24.337 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-05-04T03:05:24.377 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-05-04T03:05:24.412 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 110/156 2024-05-04T03:05:24.436 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 110/156 2024-05-04T03:05:24.436 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T03:05:24.436 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-04T03:05:24.436 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-04T03:05:24.437 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-04T03:05:24.437 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:05:24.474 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-05-04T03:05:24.522 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-05-04T03:05:24.921 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 111/156 2024-05-04T03:05:24.945 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 111/156 2024-05-04T03:05:24.945 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T03:05:24.945 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-04T03:05:24.945 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-04T03:05:24.945 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-04T03:05:24.946 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:05:25.052 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-05-04T03:05:25.110 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-05-04T03:05:25.140 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-05-04T03:05:25.174 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-05-04T03:05:25.236 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-05-04T03:05:25.256 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-05-04T03:05:25.286 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-05-04T03:05:25.312 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-05-04T03:05:25.321 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-05-04T03:05:25.365 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-05-04T03:05:25.371 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-05-04T03:05:25.409 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-05-04T03:05:25.415 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-05-04T03:05:25.419 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-05-04T03:05:25.445 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-05-04T03:05:25.478 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-05-04T03:05:25.495 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-05-04T03:05:25.496 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-05-04T03:05:25.566 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-05-04T03:05:25.595 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-04T03:05:25.614 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-05-04T03:05:25.739 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-04T03:05:25.781 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-05-04T03:05:25.790 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-05-04T03:05:25.813 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-05-04T03:05:25.821 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-05-04T03:05:25.831 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-05-04T03:05:25.840 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-05-04T03:05:25.897 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-05-04T03:05:26.039 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 8/156 2024-05-04T03:05:26.165 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 8/156 2024-05-04T03:05:26.192 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 9/156 2024-05-04T03:05:26.285 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 9/156 2024-05-04T03:05:26.297 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 123/156 2024-05-04T03:05:26.323 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rados-2:19.0.0-3522.gbd4e00a3.el9.x86_64 10/156 2024-05-04T03:05:26.330 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 123/156 2024-05-04T03:05:26.372 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3522.gbd4e00a3.e 124/156 2024-05-04T03:05:26.440 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3522.gbd4e00a3. 125/156 2024-05-04T03:05:26.470 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noa 126/156 2024-05-04T03:05:26.472 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-05-04T03:05:26.531 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-05-04T03:05:26.534 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-05-04T03:05:26.559 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 13/156 2024-05-04T03:05:26.643 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-05-04T03:05:26.666 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 13/156 2024-05-04T03:05:26.688 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86 14/156 2024-05-04T03:05:26.750 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86 14/156 2024-05-04T03:05:26.782 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-04T03:05:26.807 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-04T03:05:27.007 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-04T03:05:27.041 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-04T03:05:27.149 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-05-04T03:05:27.183 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-05-04T03:05:27.227 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/156 2024-05-04T03:05:27.300 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/156 2024-05-04T03:05:27.345 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/156 2024-05-04T03:05:27.397 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/156 2024-05-04T03:05:27.447 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/156 2024-05-04T03:05:27.486 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/156 2024-05-04T03:05:27.563 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/156 2024-05-04T03:05:27.577 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noa 126/156 2024-05-04T03:05:27.613 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.n 127/156 2024-05-04T03:05:27.617 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/156 2024-05-04T03:05:27.657 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/156 2024-05-04T03:05:27.701 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/156 2024-05-04T03:05:27.777 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/156 2024-05-04T03:05:27.823 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ceph-common-2:19.0.0-3522.gbd4e00a3.el9. 30/156 2024-05-04T03:05:27.857 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-05-04T03:05:27.866 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ceph-argparse-2:19.0.0-3522.gbd4e00a3.el 31/156 2024-05-04T03:05:27.904 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cephfs-2:19.0.0-3522.gbd4e00a3.el9.x86_6 32/156 2024-05-04T03:05:27.941 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-05-04T03:05:27.970 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-05-04T03:05:27.983 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/156 2024-05-04T03:05:28.002 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-05-04T03:05:28.029 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/156 2024-05-04T03:05:28.111 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/156 2024-05-04T03:05:28.253 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/156 2024-05-04T03:05:28.310 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/156 2024-05-04T03:05:28.320 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-05-04T03:05:28.357 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 99/156 2024-05-04T03:05:28.448 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 99/156 2024-05-04T03:05:28.478 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 100/156 2024-05-04T03:05:28.506 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-05-04T03:05:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.n 127/156 2024-05-04T03:05:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4 128/156 2024-05-04T03:05:28.774 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4 128/156 2024-05-04T03:05:28.838 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/156 2024-05-04T03:05:28.925 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/156 2024-05-04T03:05:28.946 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3522.gbd4e00a3.el 129/156 2024-05-04T03:05:28.954 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/156 2024-05-04T03:05:28.982 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 130/156 2024-05-04T03:05:28.992 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-05-04T03:05:28.999 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/156 2024-05-04T03:05:29.007 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 130/156 2024-05-04T03:05:29.007 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T03:05:29.007 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-04T03:05:29.007 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-04T03:05:29.007 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-04T03:05:29.007 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:05:29.018 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 103/156 2024-05-04T03:05:29.038 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librados-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_6 42/156 2024-05-04T03:05:29.058 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 131/156 2024-05-04T03:05:29.069 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 131/156 2024-05-04T03:05:29.080 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/156 2024-05-04T03:05:29.089 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 103/156 2024-05-04T03:05:29.112 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 104/156 2024-05-04T03:05:29.180 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-2:19.0.0-3522.gbd4e00a3.el9.x86_64 132/156 2024-05-04T03:05:29.241 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 133/156 2024-05-04T03:05:29.243 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 44/156 2024-05-04T03:05:29.312 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 45/156 2024-05-04T03:05:29.404 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 46/156 2024-05-04T03:05:29.469 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 47/156 2024-05-04T03:05:29.542 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-05-04T03:05:29.584 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-05-04T03:05:29.620 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-05-04T03:05:29.667 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-05-04T03:05:29.714 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-05-04T03:05:29.758 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-05-04T03:05:29.809 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-05-04T03:05:29.945 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-05-04T03:05:29.993 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-05-04T03:05:30.024 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-05-04T03:05:30.075 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-05-04T03:05:30.169 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-05-04T03:05:30.210 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-05-04T03:05:30.229 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-05-04T03:05:30.242 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-05-04T03:05:30.260 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 135/156 2024-05-04T03:05:30.278 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-05-04T03:05:30.284 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 135/156 2024-05-04T03:05:30.285 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T03:05:30.285 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-04T03:05:30.285 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-04T03:05:30.285 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-04T03:05:30.285 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:05:30.312 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-05-04T03:05:30.328 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3522.gbd4e0 136/156 2024-05-04T03:05:30.351 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3522.gbd4e0 136/156 2024-05-04T03:05:30.351 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T03:05:30.351 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-04T03:05:30.351 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:05:30.394 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-05-04T03:05:30.403 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 137/156 2024-05-04T03:05:30.427 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 137/156 2024-05-04T03:05:30.427 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T03:05:30.427 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-04T03:05:30.427 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-04T03:05:30.427 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-04T03:05:30.427 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:05:30.534 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-05-04T03:05:30.609 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-05-04T03:05:30.646 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-05-04T03:05:30.681 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-05-04T03:05:30.721 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-05-04T03:05:30.755 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 138/156 2024-05-04T03:05:30.761 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-05-04T03:05:30.778 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 138/156 2024-05-04T03:05:30.779 INFO:teuthology.orchestra.run.smithi001.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T03:05:30.779 INFO:teuthology.orchestra.run.smithi001.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-04T03:05:30.779 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-04T03:05:30.779 INFO:teuthology.orchestra.run.smithi001.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-04T03:05:30.779 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:05:30.793 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-05-04T03:05:30.820 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-05-04T03:05:30.853 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-05-04T03:05:30.896 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-05-04T03:05:30.930 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-05-04T03:05:31.017 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-05-04T03:05:31.070 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-05-04T03:05:31.104 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-05-04T03:05:31.152 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-05-04T03:05:31.258 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-05-04T03:05:31.316 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-05-04T03:05:31.419 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 105/156 2024-05-04T03:05:31.451 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 105/156 2024-05-04T03:05:32.009 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 105/156 2024-05-04T03:05:32.035 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-05-04T03:05:32.046 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 106/156 2024-05-04T03:05:32.094 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-05-04T03:05:32.136 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-05-04T03:05:32.138 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 106/156 2024-05-04T03:05:32.138 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-04T03:05:32.138 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-04T03:05:32.138 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-04T03:05:32.173 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 107/156 2024-05-04T03:05:32.235 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-05-04T03:05:32.297 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-05-04T03:05:32.385 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-05-04T03:05:32.426 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-05-04T03:05:32.603 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-05-04T03:05:32.641 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-05-04T03:05:32.728 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-05-04T03:05:33.376 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-05-04T03:05:33.507 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-05-04T03:05:34.724 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-05-04T03:05:34.829 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-05-04T03:05:34.866 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-05-04T03:05:34.899 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-05-04T03:05:35.219 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-05-04T03:05:35.272 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 99/156 2024-05-04T03:05:35.299 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-test-2:19.0.0-3522.gbd4e00a3.el9.x86_64 139/156 2024-05-04T03:05:35.341 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 140/156 2024-05-04T03:05:35.373 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 99/156 2024-05-04T03:05:35.382 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-nbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 141/156 2024-05-04T03:05:35.404 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 100/156 2024-05-04T03:05:35.413 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-05-04T03:05:35.441 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-05-04T03:05:35.441 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-05-04T03:05:35.639 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephfs-shell-2:19.0.0-3522.gbd4e00a3.el9.x86_64 144/156 2024-05-04T03:05:35.687 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-05-04T03:05:35.708 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephfs-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_ 146/156 2024-05-04T03:05:35.932 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-05-04T03:05:35.963 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 103/156 2024-05-04T03:05:36.034 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 103/156 2024-05-04T03:05:36.065 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 104/156 2024-05-04T03:05:36.338 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-04T03:05:36.381 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-04T03:05:36.401 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-04T03:05:37.317 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephfs-top-2:19.0.0-3522.gbd4e00a3.el9.noarch 148/156 2024-05-04T03:05:37.439 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-05-04T03:05:37.486 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-05-04T03:05:37.580 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-05-04T03:05:37.614 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-05-04T03:05:37.615 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-04T03:05:37.682 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-04T03:05:37.715 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-04T03:05:37.715 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-04T03:05:37.802 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-04T03:05:37.875 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-04T03:05:38.315 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 105/156 2024-05-04T03:05:38.360 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 105/156 2024-05-04T03:05:38.916 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 105/156 2024-05-04T03:05:38.959 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 106/156 2024-05-04T03:05:39.051 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 106/156 2024-05-04T03:05:39.051 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-04T03:05:39.051 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-04T03:05:39.051 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-04T03:05:39.088 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 107/156 2024-05-04T03:05:43.769 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 107/156 2024-05-04T03:05:43.770 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /sys 2024-05-04T03:05:43.770 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /proc 2024-05-04T03:05:43.770 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /mnt 2024-05-04T03:05:43.770 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /var/tmp 2024-05-04T03:05:43.770 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /home 2024-05-04T03:05:43.770 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /root 2024-05-04T03:05:43.770 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /tmp 2024-05-04T03:05:43.770 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-04T03:05:43.811 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-04T03:05:43.811 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-2:19.0.0-3522.gbd4e00a3.el9.x86_64 1/156 2024-05-04T03:05:43.811 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2/156 2024-05-04T03:05:43.811 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 3/156 2024-05-04T03:05:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 4/156 2024-05-04T03:05:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3522.gbd4e0 5/156 2024-05-04T03:05:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 6/156 2024-05-04T03:05:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 7/156 2024-05-04T03:05:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 8/156 2024-05-04T03:05:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 9/156 2024-05-04T03:05:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 10/156 2024-05-04T03:05:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 11/156 2024-05-04T03:05:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-test-2:19.0.0-3522.gbd4e00a3.el9.x86_64 12/156 2024-05-04T03:05:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 13/156 2024-05-04T03:05:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephfs-shell-2:19.0.0-3522.gbd4e00a3.el9.x86_64 14/156 2024-05-04T03:05:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephfs-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_ 15/156 2024-05-04T03:05:43.812 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 16/156 2024-05-04T03:05:43.813 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 17/156 2024-05-04T03:05:43.813 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_6 18/156 2024-05-04T03:05:43.813 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86 19/156 2024-05-04T03:05:43.813 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 20/156 2024-05-04T03:05:43.813 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3522.gbd4e00a3.el 21/156 2024-05-04T03:05:43.813 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-ceph-common-2:19.0.0-3522.gbd4e00a3.el9. 22/156 2024-05-04T03:05:43.814 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cephfs-2:19.0.0-3522.gbd4e00a3.el9.x86_6 23/156 2024-05-04T03:05:43.814 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rados-2:19.0.0-3522.gbd4e00a3.el9.x86_64 24/156 2024-05-04T03:05:43.814 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 25/156 2024-05-04T03:05:43.814 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 26/156 2024-05-04T03:05:43.814 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 27/156 2024-05-04T03:05:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 28/156 2024-05-04T03:05:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-nbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 29/156 2024-05-04T03:05:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3522.gbd4e00a3. 30/156 2024-05-04T03:05:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noa 31/156 2024-05-04T03:05:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.n 32/156 2024-05-04T03:05:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4 33/156 2024-05-04T03:05:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3522.gbd4e00a3.el 34/156 2024-05-04T03:05:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 35/156 2024-05-04T03:05:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3522.gbd4e00a3.e 36/156 2024-05-04T03:05:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 37/156 2024-05-04T03:05:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 38/156 2024-05-04T03:05:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephfs-top-2:19.0.0-3522.gbd4e00a3.el9.noarch 39/156 2024-05-04T03:05:43.815 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-05-04T03:05:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-05-04T03:05:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-05-04T03:05:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-05-04T03:05:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-05-04T03:05:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-05-04T03:05:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-05-04T03:05:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-05-04T03:05:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-05-04T03:05:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-05-04T03:05:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-05-04T03:05:43.816 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-05-04T03:05:43.817 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-05-04T03:05:43.817 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-05-04T03:05:43.817 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-05-04T03:05:43.817 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-05-04T03:05:43.817 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-05-04T03:05:43.817 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-05-04T03:05:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-05-04T03:05:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-05-04T03:05:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-05-04T03:05:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-05-04T03:05:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-05-04T03:05:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-05-04T03:05:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-05-04T03:05:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-05-04T03:05:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-05-04T03:05:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-05-04T03:05:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-05-04T03:05:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-05-04T03:05:43.818 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-05-04T03:05:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-05-04T03:05:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-05-04T03:05:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-05-04T03:05:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-05-04T03:05:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-05-04T03:05:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-05-04T03:05:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-05-04T03:05:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-05-04T03:05:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-05-04T03:05:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-05-04T03:05:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-05-04T03:05:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-05-04T03:05:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-05-04T03:05:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-05-04T03:05:43.819 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-05-04T03:05:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-05-04T03:05:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-05-04T03:05:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-05-04T03:05:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-05-04T03:05:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-05-04T03:05:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-05-04T03:05:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-05-04T03:05:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-05-04T03:05:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-05-04T03:05:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-05-04T03:05:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-05-04T03:05:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-05-04T03:05:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-05-04T03:05:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-05-04T03:05:43.820 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-05-04T03:05:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-05-04T03:05:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-05-04T03:05:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-05-04T03:05:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-05-04T03:05:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-05-04T03:05:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-05-04T03:05:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-05-04T03:05:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-05-04T03:05:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-05-04T03:05:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-05-04T03:05:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-05-04T03:05:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-05-04T03:05:43.821 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-05-04T03:05:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-05-04T03:05:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-05-04T03:05:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-05-04T03:05:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-05-04T03:05:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-04T03:05:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-04T03:05:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-05-04T03:05:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-05-04T03:05:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-05-04T03:05:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-05-04T03:05:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-05-04T03:05:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-05-04T03:05:43.822 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-05-04T03:05:43.823 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-05-04T03:05:43.823 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-05-04T03:05:43.823 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-05-04T03:05:43.823 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-05-04T03:05:43.823 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-05-04T03:05:43.823 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-05-04T03:05:43.823 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-05-04T03:05:43.823 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-05-04T03:05:43.823 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-05-04T03:05:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-05-04T03:05:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-05-04T03:05:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-05-04T03:05:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-05-04T03:05:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-05-04T03:05:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-05-04T03:05:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-05-04T03:05:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-05-04T03:05:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-05-04T03:05:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-05-04T03:05:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-05-04T03:05:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-05-04T03:05:43.824 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-05-04T03:05:43.825 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 149/156 2024-05-04T03:05:43.825 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-05-04T03:05:43.825 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 151/156 2024-05-04T03:05:43.825 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-05-04T03:05:43.825 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-05-04T03:05:43.825 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-04T03:05:43.825 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-05-04T03:05:43.930 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 108/156 2024-05-04T03:05:43.945 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 108/156 2024-05-04T03:05:43.945 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T03:05:43.945 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-04T03:05:43.945 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-04T03:05:45.523 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 109/156 2024-05-04T03:05:45.549 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 109/156 2024-05-04T03:05:45.549 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T03:05:45.549 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-04T03:05:45.549 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-04T03:05:45.549 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-04T03:05:45.549 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-04T03:05:45.783 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 110/156 2024-05-04T03:05:45.806 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 110/156 2024-05-04T03:05:45.806 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T03:05:45.806 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-04T03:05:45.806 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-04T03:05:45.806 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-04T03:05:45.806 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-04T03:05:45.842 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-04T03:05:45.843 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:05:45.843 INFO:teuthology.orchestra.run.smithi001.stdout:Upgraded: 2024-05-04T03:05:45.843 INFO:teuthology.orchestra.run.smithi001.stdout: librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.843 INFO:teuthology.orchestra.run.smithi001.stdout: librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.843 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-04T03:05:45.843 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-04T03:05:45.843 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2024-05-04T03:05:45.843 INFO:teuthology.orchestra.run.smithi001.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-04T03:05:45.843 INFO:teuthology.orchestra.run.smithi001.stdout: automake-1.16.2-8.el9.noarch 2024-05-04T03:05:45.843 INFO:teuthology.orchestra.run.smithi001.stdout: bison-3.7.4-5.el9.x86_64 2024-05-04T03:05:45.843 INFO:teuthology.orchestra.run.smithi001.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-04T03:05:45.843 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.844 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.844 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.844 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.844 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-grafana-dashboards-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:05:45.844 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-immutable-object-cache-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.844 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.844 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.844 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:05:45.844 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:05:45.844 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:05:45.844 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-modules-core-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:05:45.844 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:05:45.844 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.844 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.845 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-prometheus-alerts-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:05:45.845 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.845 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.845 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-test-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.845 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:05:45.845 INFO:teuthology.orchestra.run.smithi001.stdout: cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:05:45.845 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.845 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-shell-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.845 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-top-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:05:45.845 INFO:teuthology.orchestra.run.smithi001.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-04T03:05:45.845 INFO:teuthology.orchestra.run.smithi001.stdout: flex-2.6.4-9.el9.x86_64 2024-05-04T03:05:45.845 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-04T03:05:45.845 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-04T03:05:45.845 INFO:teuthology.orchestra.run.smithi001.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-04T03:05:45.846 INFO:teuthology.orchestra.run.smithi001.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-04T03:05:45.846 INFO:teuthology.orchestra.run.smithi001.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-04T03:05:45.846 INFO:teuthology.orchestra.run.smithi001.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-04T03:05:45.846 INFO:teuthology.orchestra.run.smithi001.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-04T03:05:45.846 INFO:teuthology.orchestra.run.smithi001.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-04T03:05:45.846 INFO:teuthology.orchestra.run.smithi001.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-04T03:05:45.846 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.846 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.846 INFO:teuthology.orchestra.run.smithi001.stdout: libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.846 INFO:teuthology.orchestra.run.smithi001.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-04T03:05:45.846 INFO:teuthology.orchestra.run.smithi001.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-04T03:05:45.846 INFO:teuthology.orchestra.run.smithi001.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-04T03:05:45.847 INFO:teuthology.orchestra.run.smithi001.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-04T03:05:45.847 INFO:teuthology.orchestra.run.smithi001.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-04T03:05:45.847 INFO:teuthology.orchestra.run.smithi001.stdout: librados-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.847 INFO:teuthology.orchestra.run.smithi001.stdout: libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.847 INFO:teuthology.orchestra.run.smithi001.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-04T03:05:45.847 INFO:teuthology.orchestra.run.smithi001.stdout: librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.847 INFO:teuthology.orchestra.run.smithi001.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-04T03:05:45.847 INFO:teuthology.orchestra.run.smithi001.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-04T03:05:45.847 INFO:teuthology.orchestra.run.smithi001.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-04T03:05:45.847 INFO:teuthology.orchestra.run.smithi001.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-04T03:05:45.847 INFO:teuthology.orchestra.run.smithi001.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-04T03:05:45.847 INFO:teuthology.orchestra.run.smithi001.stdout: lua-5.4.4-4.el9.x86_64 2024-05-04T03:05:45.847 INFO:teuthology.orchestra.run.smithi001.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-04T03:05:45.848 INFO:teuthology.orchestra.run.smithi001.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-04T03:05:45.848 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-04T03:05:45.848 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-04T03:05:45.848 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-04T03:05:45.848 INFO:teuthology.orchestra.run.smithi001.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-04T03:05:45.848 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-13.14-1.el9.x86_64 2024-05-04T03:05:45.848 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-04T03:05:45.848 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-04T03:05:45.848 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-04T03:05:45.848 INFO:teuthology.orchestra.run.smithi001.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-04T03:05:45.848 INFO:teuthology.orchestra.run.smithi001.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-04T03:05:45.848 INFO:teuthology.orchestra.run.smithi001.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-04T03:05:45.848 INFO:teuthology.orchestra.run.smithi001.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-04T03:05:45.848 INFO:teuthology.orchestra.run.smithi001.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-04T03:05:45.849 INFO:teuthology.orchestra.run.smithi001.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-04T03:05:45.849 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-04T03:05:45.849 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-argparse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.849 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.849 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cephfs-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.849 INFO:teuthology.orchestra.run.smithi001.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-04T03:05:45.849 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-04T03:05:45.849 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-04T03:05:45.849 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-04T03:05:45.849 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-04T03:05:45.849 INFO:teuthology.orchestra.run.smithi001.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-04T03:05:45.849 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-04T03:05:45.849 INFO:teuthology.orchestra.run.smithi001.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-04T03:05:45.849 INFO:teuthology.orchestra.run.smithi001.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-04T03:05:45.850 INFO:teuthology.orchestra.run.smithi001.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-04T03:05:45.850 INFO:teuthology.orchestra.run.smithi001.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-04T03:05:45.850 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-04T03:05:45.850 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-04T03:05:45.850 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-04T03:05:45.850 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-04T03:05:45.850 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-04T03:05:45.850 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-04T03:05:45.850 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-04T03:05:45.850 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-04T03:05:45.850 INFO:teuthology.orchestra.run.smithi001.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-04T03:05:45.850 INFO:teuthology.orchestra.run.smithi001.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-04T03:05:45.850 INFO:teuthology.orchestra.run.smithi001.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-04T03:05:45.850 INFO:teuthology.orchestra.run.smithi001.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-04T03:05:45.851 INFO:teuthology.orchestra.run.smithi001.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-04T03:05:45.851 INFO:teuthology.orchestra.run.smithi001.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-04T03:05:45.851 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-04T03:05:45.851 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-04T03:05:45.851 INFO:teuthology.orchestra.run.smithi001.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-04T03:05:45.851 INFO:teuthology.orchestra.run.smithi001.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-04T03:05:45.851 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-04T03:05:45.851 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ply-3.11-14.el9.noarch 2024-05-04T03:05:45.851 INFO:teuthology.orchestra.run.smithi001.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-04T03:05:45.851 INFO:teuthology.orchestra.run.smithi001.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-04T03:05:45.851 INFO:teuthology.orchestra.run.smithi001.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-04T03:05:45.851 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-04T03:05:45.851 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-04T03:05:45.851 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-04T03:05:45.851 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-04T03:05:45.852 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-04T03:05:45.852 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-04T03:05:45.852 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rados-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.852 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.852 INFO:teuthology.orchestra.run.smithi001.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-04T03:05:45.852 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-04T03:05:45.852 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-04T03:05:45.852 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.852 INFO:teuthology.orchestra.run.smithi001.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-04T03:05:45.852 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-04T03:05:45.852 INFO:teuthology.orchestra.run.smithi001.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-04T03:05:45.852 INFO:teuthology.orchestra.run.smithi001.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-04T03:05:45.852 INFO:teuthology.orchestra.run.smithi001.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-04T03:05:45.852 INFO:teuthology.orchestra.run.smithi001.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-04T03:05:45.852 INFO:teuthology.orchestra.run.smithi001.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-04T03:05:45.853 INFO:teuthology.orchestra.run.smithi001.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-04T03:05:45.853 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-04T03:05:45.853 INFO:teuthology.orchestra.run.smithi001.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-04T03:05:45.853 INFO:teuthology.orchestra.run.smithi001.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-04T03:05:45.853 INFO:teuthology.orchestra.run.smithi001.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-04T03:05:45.853 INFO:teuthology.orchestra.run.smithi001.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-04T03:05:45.853 INFO:teuthology.orchestra.run.smithi001.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-04T03:05:45.853 INFO:teuthology.orchestra.run.smithi001.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-04T03:05:45.853 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.853 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.853 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-nbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:05:45.853 INFO:teuthology.orchestra.run.smithi001.stdout: re2-1:20211101-3.el9.x86_64 2024-05-04T03:05:45.853 INFO:teuthology.orchestra.run.smithi001.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-04T03:05:45.853 INFO:teuthology.orchestra.run.smithi001.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-04T03:05:45.854 INFO:teuthology.orchestra.run.smithi001.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-04T03:05:45.854 INFO:teuthology.orchestra.run.smithi001.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-04T03:05:45.854 INFO:teuthology.orchestra.run.smithi001.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-04T03:05:45.854 INFO:teuthology.orchestra.run.smithi001.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-04T03:05:45.854 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:05:45.854 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2024-05-04T03:05:46.230 DEBUG:teuthology.parallel:result is None 2024-05-04T03:05:46.284 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 111/156 2024-05-04T03:05:46.307 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 111/156 2024-05-04T03:05:46.307 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T03:05:46.307 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-04T03:05:46.307 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-04T03:05:46.308 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-04T03:05:46.308 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-04T03:05:46.415 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-05-04T03:05:46.471 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-05-04T03:05:46.527 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-05-04T03:05:46.721 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-05-04T03:05:46.769 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-05-04T03:05:46.847 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-05-04T03:05:46.934 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-04T03:05:47.073 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-04T03:05:47.121 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-05-04T03:05:47.153 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-05-04T03:05:47.181 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-05-04T03:05:47.540 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 123/156 2024-05-04T03:05:47.570 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 123/156 2024-05-04T03:05:47.610 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3522.gbd4e00a3.e 124/156 2024-05-04T03:05:47.672 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3522.gbd4e00a3. 125/156 2024-05-04T03:05:47.710 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noa 126/156 2024-05-04T03:05:48.749 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noa 126/156 2024-05-04T03:05:48.787 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.n 127/156 2024-05-04T03:05:49.748 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.n 127/156 2024-05-04T03:05:49.779 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4 128/156 2024-05-04T03:05:49.889 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4 128/156 2024-05-04T03:05:50.054 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3522.gbd4e00a3.el 129/156 2024-05-04T03:05:50.081 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 130/156 2024-05-04T03:05:50.103 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 130/156 2024-05-04T03:05:50.103 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T03:05:50.103 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-04T03:05:50.104 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-04T03:05:50.104 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-04T03:05:50.104 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-04T03:05:50.148 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 131/156 2024-05-04T03:05:50.159 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 131/156 2024-05-04T03:05:50.258 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-2:19.0.0-3522.gbd4e00a3.el9.x86_64 132/156 2024-05-04T03:05:50.304 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 133/156 2024-05-04T03:05:51.241 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-05-04T03:05:51.276 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 135/156 2024-05-04T03:05:51.298 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 135/156 2024-05-04T03:05:51.299 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T03:05:51.299 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-04T03:05:51.299 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-04T03:05:51.299 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-04T03:05:51.299 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-04T03:05:51.342 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3522.gbd4e0 136/156 2024-05-04T03:05:51.364 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3522.gbd4e0 136/156 2024-05-04T03:05:51.364 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T03:05:51.364 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-04T03:05:51.364 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-04T03:05:51.410 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 137/156 2024-05-04T03:05:51.432 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 137/156 2024-05-04T03:05:51.432 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T03:05:51.433 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-04T03:05:51.433 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-04T03:05:51.433 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-04T03:05:51.433 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-04T03:05:51.744 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 138/156 2024-05-04T03:05:51.766 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 138/156 2024-05-04T03:05:51.766 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T03:05:51.766 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-04T03:05:51.766 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-04T03:05:51.767 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-04T03:05:51.767 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-04T03:05:54.161 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 107/156 2024-05-04T03:05:54.161 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /sys 2024-05-04T03:05:54.161 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /proc 2024-05-04T03:05:54.161 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /mnt 2024-05-04T03:05:54.161 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /var/tmp 2024-05-04T03:05:54.161 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /home 2024-05-04T03:05:54.161 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /root 2024-05-04T03:05:54.161 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /tmp 2024-05-04T03:05:54.161 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-04T03:05:54.354 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 108/156 2024-05-04T03:05:54.369 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 108/156 2024-05-04T03:05:54.369 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T03:05:54.369 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-04T03:05:54.369 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-04T03:05:55.953 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 109/156 2024-05-04T03:05:55.978 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 109/156 2024-05-04T03:05:55.978 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T03:05:55.979 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-04T03:05:55.979 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-04T03:05:55.979 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-04T03:05:55.979 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-04T03:05:56.039 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-test-2:19.0.0-3522.gbd4e00a3.el9.x86_64 139/156 2024-05-04T03:05:56.081 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 140/156 2024-05-04T03:05:56.121 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-nbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 141/156 2024-05-04T03:05:56.145 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-05-04T03:05:56.172 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-05-04T03:05:56.205 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 110/156 2024-05-04T03:05:56.226 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 110/156 2024-05-04T03:05:56.226 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T03:05:56.226 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-04T03:05:56.227 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-04T03:05:56.227 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-04T03:05:56.227 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-04T03:05:56.353 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephfs-shell-2:19.0.0-3522.gbd4e00a3.el9.x86_64 144/156 2024-05-04T03:05:56.394 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-05-04T03:05:56.423 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephfs-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_ 146/156 2024-05-04T03:05:56.690 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 111/156 2024-05-04T03:05:56.711 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 111/156 2024-05-04T03:05:56.712 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T03:05:56.712 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-04T03:05:56.712 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-04T03:05:56.712 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-04T03:05:56.712 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-04T03:05:56.820 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-05-04T03:05:56.880 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-05-04T03:05:56.944 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-05-04T03:05:57.037 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-04T03:05:57.079 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-04T03:05:57.099 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-04T03:05:57.131 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-05-04T03:05:57.188 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-05-04T03:05:57.291 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-05-04T03:05:57.382 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-04T03:05:57.523 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-04T03:05:57.568 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-05-04T03:05:57.601 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-05-04T03:05:57.637 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-05-04T03:05:57.927 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephfs-top-2:19.0.0-3522.gbd4e00a3.el9.noarch 148/156 2024-05-04T03:05:58.016 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-05-04T03:05:58.057 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-05-04T03:05:58.093 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 123/156 2024-05-04T03:05:58.124 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 123/156 2024-05-04T03:05:58.139 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-05-04T03:05:58.165 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3522.gbd4e00a3.e 124/156 2024-05-04T03:05:58.170 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-05-04T03:05:58.170 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-04T03:05:58.236 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3522.gbd4e00a3. 125/156 2024-05-04T03:05:58.246 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-04T03:05:58.270 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-04T03:05:58.270 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-04T03:05:58.275 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noa 126/156 2024-05-04T03:05:58.358 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-04T03:05:58.427 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-04T03:05:59.313 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noa 126/156 2024-05-04T03:05:59.353 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.n 127/156 2024-05-04T03:06:00.310 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.n 127/156 2024-05-04T03:06:00.338 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4 128/156 2024-05-04T03:06:00.448 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4 128/156 2024-05-04T03:06:00.615 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3522.gbd4e00a3.el 129/156 2024-05-04T03:06:00.643 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 130/156 2024-05-04T03:06:00.666 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 130/156 2024-05-04T03:06:00.666 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T03:06:00.666 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-04T03:06:00.666 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-04T03:06:00.669 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-04T03:06:00.669 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-04T03:06:00.711 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 131/156 2024-05-04T03:06:00.722 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 131/156 2024-05-04T03:06:00.821 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-2:19.0.0-3522.gbd4e00a3.el9.x86_64 132/156 2024-05-04T03:06:00.877 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 133/156 2024-05-04T03:06:01.806 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-05-04T03:06:01.841 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 135/156 2024-05-04T03:06:01.863 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 135/156 2024-05-04T03:06:01.863 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T03:06:01.863 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-04T03:06:01.863 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-04T03:06:01.863 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-04T03:06:01.863 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-04T03:06:01.911 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3522.gbd4e0 136/156 2024-05-04T03:06:01.933 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3522.gbd4e0 136/156 2024-05-04T03:06:01.933 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T03:06:01.933 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-04T03:06:01.933 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-04T03:06:01.988 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 137/156 2024-05-04T03:06:02.009 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 137/156 2024-05-04T03:06:02.009 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T03:06:02.009 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-04T03:06:02.009 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-04T03:06:02.009 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-04T03:06:02.009 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-04T03:06:02.322 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 138/156 2024-05-04T03:06:02.343 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 138/156 2024-05-04T03:06:02.343 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-04T03:06:02.344 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-04T03:06:02.344 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-04T03:06:02.344 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-04T03:06:02.344 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-04T03:06:04.119 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-04T03:06:04.120 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-2:19.0.0-3522.gbd4e00a3.el9.x86_64 1/156 2024-05-04T03:06:04.120 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2/156 2024-05-04T03:06:04.120 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 3/156 2024-05-04T03:06:04.120 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 4/156 2024-05-04T03:06:04.120 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3522.gbd4e0 5/156 2024-05-04T03:06:04.120 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 6/156 2024-05-04T03:06:04.120 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 7/156 2024-05-04T03:06:04.120 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 8/156 2024-05-04T03:06:04.120 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 9/156 2024-05-04T03:06:04.120 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 10/156 2024-05-04T03:06:04.120 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 11/156 2024-05-04T03:06:04.120 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-test-2:19.0.0-3522.gbd4e00a3.el9.x86_64 12/156 2024-05-04T03:06:04.120 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 13/156 2024-05-04T03:06:04.121 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephfs-shell-2:19.0.0-3522.gbd4e00a3.el9.x86_64 14/156 2024-05-04T03:06:04.121 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephfs-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_ 15/156 2024-05-04T03:06:04.121 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 16/156 2024-05-04T03:06:04.121 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 17/156 2024-05-04T03:06:04.121 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_6 18/156 2024-05-04T03:06:04.121 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86 19/156 2024-05-04T03:06:04.121 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 20/156 2024-05-04T03:06:04.121 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3522.gbd4e00a3.el 21/156 2024-05-04T03:06:04.121 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ceph-common-2:19.0.0-3522.gbd4e00a3.el9. 22/156 2024-05-04T03:06:04.121 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cephfs-2:19.0.0-3522.gbd4e00a3.el9.x86_6 23/156 2024-05-04T03:06:04.121 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rados-2:19.0.0-3522.gbd4e00a3.el9.x86_64 24/156 2024-05-04T03:06:04.121 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 25/156 2024-05-04T03:06:04.121 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 26/156 2024-05-04T03:06:04.121 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 27/156 2024-05-04T03:06:04.122 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 28/156 2024-05-04T03:06:04.122 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-nbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 29/156 2024-05-04T03:06:04.122 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3522.gbd4e00a3. 30/156 2024-05-04T03:06:04.134 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noa 31/156 2024-05-04T03:06:04.134 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.n 32/156 2024-05-04T03:06:04.134 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4 33/156 2024-05-04T03:06:04.134 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3522.gbd4e00a3.el 34/156 2024-05-04T03:06:04.134 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 35/156 2024-05-04T03:06:04.134 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3522.gbd4e00a3.e 36/156 2024-05-04T03:06:04.134 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 37/156 2024-05-04T03:06:04.134 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 38/156 2024-05-04T03:06:04.134 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephfs-top-2:19.0.0-3522.gbd4e00a3.el9.noarch 39/156 2024-05-04T03:06:04.134 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-05-04T03:06:04.134 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-05-04T03:06:04.135 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-05-04T03:06:04.135 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-05-04T03:06:04.135 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-05-04T03:06:04.135 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-05-04T03:06:04.135 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-05-04T03:06:04.135 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-05-04T03:06:04.135 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-05-04T03:06:04.135 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-05-04T03:06:04.135 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-05-04T03:06:04.135 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-05-04T03:06:04.135 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-05-04T03:06:04.135 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-05-04T03:06:04.135 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-05-04T03:06:04.135 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-05-04T03:06:04.136 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-05-04T03:06:04.136 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-05-04T03:06:04.136 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-05-04T03:06:04.136 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-05-04T03:06:04.136 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-05-04T03:06:04.136 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-05-04T03:06:04.136 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-05-04T03:06:04.136 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-05-04T03:06:04.136 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-05-04T03:06:04.136 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-05-04T03:06:04.136 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-05-04T03:06:04.136 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-05-04T03:06:04.136 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-05-04T03:06:04.137 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-05-04T03:06:04.137 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-05-04T03:06:04.137 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-05-04T03:06:04.137 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-05-04T03:06:04.137 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-05-04T03:06:04.137 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-05-04T03:06:04.137 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-05-04T03:06:04.137 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-05-04T03:06:04.137 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-05-04T03:06:04.138 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-05-04T03:06:04.138 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-05-04T03:06:04.138 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-05-04T03:06:04.138 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-05-04T03:06:04.138 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-05-04T03:06:04.138 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-05-04T03:06:04.138 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-05-04T03:06:04.138 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-05-04T03:06:04.138 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-05-04T03:06:04.139 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-05-04T03:06:04.139 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-05-04T03:06:04.139 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-05-04T03:06:04.139 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-05-04T03:06:04.139 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-05-04T03:06:04.139 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-05-04T03:06:04.139 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-05-04T03:06:04.139 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-05-04T03:06:04.139 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-05-04T03:06:04.139 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-05-04T03:06:04.139 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-05-04T03:06:04.139 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-05-04T03:06:04.139 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-05-04T03:06:04.140 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-05-04T03:06:04.140 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-05-04T03:06:04.140 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-05-04T03:06:04.140 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-05-04T03:06:04.140 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-05-04T03:06:04.140 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-05-04T03:06:04.140 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-05-04T03:06:04.140 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-05-04T03:06:04.140 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-05-04T03:06:04.140 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-05-04T03:06:04.140 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-05-04T03:06:04.140 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-05-04T03:06:04.140 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-05-04T03:06:04.140 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-05-04T03:06:04.140 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-05-04T03:06:04.141 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-05-04T03:06:04.141 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-05-04T03:06:04.141 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-05-04T03:06:04.141 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-04T03:06:04.141 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-04T03:06:04.141 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-05-04T03:06:04.141 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-05-04T03:06:04.141 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-05-04T03:06:04.141 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-05-04T03:06:04.141 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-05-04T03:06:04.141 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-05-04T03:06:04.141 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-05-04T03:06:04.141 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-05-04T03:06:04.142 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-05-04T03:06:04.142 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-05-04T03:06:04.142 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-05-04T03:06:04.142 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-05-04T03:06:04.142 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-05-04T03:06:04.142 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-05-04T03:06:04.142 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-05-04T03:06:04.142 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-05-04T03:06:04.143 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-05-04T03:06:04.143 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-05-04T03:06:04.143 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-05-04T03:06:04.143 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-05-04T03:06:04.143 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-05-04T03:06:04.143 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-05-04T03:06:04.143 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-05-04T03:06:04.143 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-05-04T03:06:04.143 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-05-04T03:06:04.143 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-05-04T03:06:04.143 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-05-04T03:06:04.143 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-05-04T03:06:04.143 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-05-04T03:06:04.143 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 149/156 2024-05-04T03:06:04.143 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-05-04T03:06:04.144 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 151/156 2024-05-04T03:06:04.144 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-05-04T03:06:04.144 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-05-04T03:06:04.144 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-04T03:06:04.144 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-05-04T03:06:06.288 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-04T03:06:06.288 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-04T03:06:06.288 INFO:teuthology.orchestra.run.smithi077.stdout:Upgraded: 2024-05-04T03:06:06.288 INFO:teuthology.orchestra.run.smithi077.stdout: librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.288 INFO:teuthology.orchestra.run.smithi077.stdout: librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.288 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-04T03:06:06.288 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-04T03:06:06.288 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-05-04T03:06:06.288 INFO:teuthology.orchestra.run.smithi077.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-04T03:06:06.288 INFO:teuthology.orchestra.run.smithi077.stdout: automake-1.16.2-8.el9.noarch 2024-05-04T03:06:06.288 INFO:teuthology.orchestra.run.smithi077.stdout: bison-3.7.4-5.el9.x86_64 2024-05-04T03:06:06.289 INFO:teuthology.orchestra.run.smithi077.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-04T03:06:06.289 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.289 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.289 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.289 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.289 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-grafana-dashboards-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:06:06.289 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-immutable-object-cache-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.289 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.289 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.289 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:06:06.289 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:06:06.289 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:06:06.289 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-modules-core-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:06:06.289 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:06:06.290 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.290 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.290 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-prometheus-alerts-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:06:06.290 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.290 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.290 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-test-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.290 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:06:06.290 INFO:teuthology.orchestra.run.smithi077.stdout: cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:06:06.290 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.290 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-shell-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.290 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-top-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:06:06.290 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-04T03:06:06.290 INFO:teuthology.orchestra.run.smithi077.stdout: flex-2.6.4-9.el9.x86_64 2024-05-04T03:06:06.291 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-04T03:06:06.291 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-04T03:06:06.291 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-04T03:06:06.291 INFO:teuthology.orchestra.run.smithi077.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-04T03:06:06.291 INFO:teuthology.orchestra.run.smithi077.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-04T03:06:06.291 INFO:teuthology.orchestra.run.smithi077.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-04T03:06:06.291 INFO:teuthology.orchestra.run.smithi077.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-04T03:06:06.291 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-04T03:06:06.291 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-04T03:06:06.291 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.291 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.291 INFO:teuthology.orchestra.run.smithi077.stdout: libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.291 INFO:teuthology.orchestra.run.smithi077.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-04T03:06:06.291 INFO:teuthology.orchestra.run.smithi077.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-04T03:06:06.292 INFO:teuthology.orchestra.run.smithi077.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-04T03:06:06.292 INFO:teuthology.orchestra.run.smithi077.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-04T03:06:06.292 INFO:teuthology.orchestra.run.smithi077.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-04T03:06:06.292 INFO:teuthology.orchestra.run.smithi077.stdout: librados-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.292 INFO:teuthology.orchestra.run.smithi077.stdout: libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.292 INFO:teuthology.orchestra.run.smithi077.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-04T03:06:06.292 INFO:teuthology.orchestra.run.smithi077.stdout: librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.292 INFO:teuthology.orchestra.run.smithi077.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-04T03:06:06.292 INFO:teuthology.orchestra.run.smithi077.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-04T03:06:06.292 INFO:teuthology.orchestra.run.smithi077.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-04T03:06:06.292 INFO:teuthology.orchestra.run.smithi077.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-04T03:06:06.292 INFO:teuthology.orchestra.run.smithi077.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-04T03:06:06.292 INFO:teuthology.orchestra.run.smithi077.stdout: lua-5.4.4-4.el9.x86_64 2024-05-04T03:06:06.293 INFO:teuthology.orchestra.run.smithi077.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-04T03:06:06.293 INFO:teuthology.orchestra.run.smithi077.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-04T03:06:06.293 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-04T03:06:06.293 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-04T03:06:06.293 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-04T03:06:06.293 INFO:teuthology.orchestra.run.smithi077.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-04T03:06:06.293 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-13.14-1.el9.x86_64 2024-05-04T03:06:06.293 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-04T03:06:06.293 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-04T03:06:06.293 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-04T03:06:06.293 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-04T03:06:06.293 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-04T03:06:06.293 INFO:teuthology.orchestra.run.smithi077.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-04T03:06:06.293 INFO:teuthology.orchestra.run.smithi077.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-04T03:06:06.294 INFO:teuthology.orchestra.run.smithi077.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-04T03:06:06.294 INFO:teuthology.orchestra.run.smithi077.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-04T03:06:06.294 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-04T03:06:06.294 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-argparse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.294 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.294 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cephfs-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.294 INFO:teuthology.orchestra.run.smithi077.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-04T03:06:06.294 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-04T03:06:06.294 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-04T03:06:06.294 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-04T03:06:06.294 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-04T03:06:06.294 INFO:teuthology.orchestra.run.smithi077.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-04T03:06:06.294 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-04T03:06:06.295 INFO:teuthology.orchestra.run.smithi077.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-04T03:06:06.295 INFO:teuthology.orchestra.run.smithi077.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-04T03:06:06.295 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-04T03:06:06.295 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-04T03:06:06.295 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-04T03:06:06.295 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-04T03:06:06.295 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-04T03:06:06.295 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-04T03:06:06.295 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-04T03:06:06.295 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-04T03:06:06.295 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-04T03:06:06.295 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-04T03:06:06.295 INFO:teuthology.orchestra.run.smithi077.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-04T03:06:06.295 INFO:teuthology.orchestra.run.smithi077.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-04T03:06:06.296 INFO:teuthology.orchestra.run.smithi077.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-04T03:06:06.296 INFO:teuthology.orchestra.run.smithi077.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-04T03:06:06.296 INFO:teuthology.orchestra.run.smithi077.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-04T03:06:06.296 INFO:teuthology.orchestra.run.smithi077.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-04T03:06:06.296 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-04T03:06:06.296 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-04T03:06:06.296 INFO:teuthology.orchestra.run.smithi077.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-04T03:06:06.296 INFO:teuthology.orchestra.run.smithi077.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-04T03:06:06.296 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-04T03:06:06.296 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ply-3.11-14.el9.noarch 2024-05-04T03:06:06.296 INFO:teuthology.orchestra.run.smithi077.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-04T03:06:06.296 INFO:teuthology.orchestra.run.smithi077.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-04T03:06:06.296 INFO:teuthology.orchestra.run.smithi077.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-04T03:06:06.296 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-04T03:06:06.297 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-04T03:06:06.297 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-04T03:06:06.297 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-04T03:06:06.297 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-04T03:06:06.297 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-04T03:06:06.297 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rados-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.297 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.297 INFO:teuthology.orchestra.run.smithi077.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-04T03:06:06.297 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-04T03:06:06.297 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-04T03:06:06.297 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.297 INFO:teuthology.orchestra.run.smithi077.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-04T03:06:06.298 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-04T03:06:06.298 INFO:teuthology.orchestra.run.smithi077.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-04T03:06:06.298 INFO:teuthology.orchestra.run.smithi077.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-04T03:06:06.298 INFO:teuthology.orchestra.run.smithi077.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-04T03:06:06.298 INFO:teuthology.orchestra.run.smithi077.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-04T03:06:06.298 INFO:teuthology.orchestra.run.smithi077.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-04T03:06:06.298 INFO:teuthology.orchestra.run.smithi077.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-04T03:06:06.298 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-04T03:06:06.298 INFO:teuthology.orchestra.run.smithi077.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-04T03:06:06.298 INFO:teuthology.orchestra.run.smithi077.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-04T03:06:06.298 INFO:teuthology.orchestra.run.smithi077.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-04T03:06:06.298 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-04T03:06:06.298 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-04T03:06:06.299 INFO:teuthology.orchestra.run.smithi077.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-04T03:06:06.299 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.299 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.299 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-nbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:06.299 INFO:teuthology.orchestra.run.smithi077.stdout: re2-1:20211101-3.el9.x86_64 2024-05-04T03:06:06.299 INFO:teuthology.orchestra.run.smithi077.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-04T03:06:06.299 INFO:teuthology.orchestra.run.smithi077.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-04T03:06:06.299 INFO:teuthology.orchestra.run.smithi077.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-04T03:06:06.299 INFO:teuthology.orchestra.run.smithi077.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-04T03:06:06.299 INFO:teuthology.orchestra.run.smithi077.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-04T03:06:06.300 INFO:teuthology.orchestra.run.smithi077.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-04T03:06:06.300 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-04T03:06:06.300 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-05-04T03:06:06.639 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-test-2:19.0.0-3522.gbd4e00a3.el9.x86_64 139/156 2024-05-04T03:06:06.675 DEBUG:teuthology.parallel:result is None 2024-05-04T03:06:06.689 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 140/156 2024-05-04T03:06:06.730 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-nbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 141/156 2024-05-04T03:06:06.772 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-05-04T03:06:06.793 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-05-04T03:06:06.967 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephfs-shell-2:19.0.0-3522.gbd4e00a3.el9.x86_64 144/156 2024-05-04T03:06:07.028 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-05-04T03:06:07.058 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephfs-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_ 146/156 2024-05-04T03:06:07.682 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-04T03:06:07.730 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-04T03:06:07.749 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-04T03:06:08.591 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephfs-top-2:19.0.0-3522.gbd4e00a3.el9.noarch 148/156 2024-05-04T03:06:08.700 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-05-04T03:06:08.741 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-05-04T03:06:08.832 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-05-04T03:06:08.876 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-05-04T03:06:08.876 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-04T03:06:08.961 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-04T03:06:08.994 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-04T03:06:08.994 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-04T03:06:09.086 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-04T03:06:09.160 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-04T03:06:19.711 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-04T03:06:19.711 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-2:19.0.0-3522.gbd4e00a3.el9.x86_64 1/156 2024-05-04T03:06:19.711 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2/156 2024-05-04T03:06:19.711 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 3/156 2024-05-04T03:06:19.711 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 4/156 2024-05-04T03:06:19.712 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3522.gbd4e0 5/156 2024-05-04T03:06:19.712 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 6/156 2024-05-04T03:06:19.712 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 7/156 2024-05-04T03:06:19.712 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 8/156 2024-05-04T03:06:19.712 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 9/156 2024-05-04T03:06:19.712 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 10/156 2024-05-04T03:06:19.712 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 11/156 2024-05-04T03:06:19.712 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-test-2:19.0.0-3522.gbd4e00a3.el9.x86_64 12/156 2024-05-04T03:06:19.712 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 13/156 2024-05-04T03:06:19.712 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephfs-shell-2:19.0.0-3522.gbd4e00a3.el9.x86_64 14/156 2024-05-04T03:06:19.712 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephfs-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_ 15/156 2024-05-04T03:06:19.713 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 16/156 2024-05-04T03:06:19.713 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 17/156 2024-05-04T03:06:19.713 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_6 18/156 2024-05-04T03:06:19.713 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86 19/156 2024-05-04T03:06:19.713 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 20/156 2024-05-04T03:06:19.713 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3522.gbd4e00a3.el 21/156 2024-05-04T03:06:19.713 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ceph-common-2:19.0.0-3522.gbd4e00a3.el9. 22/156 2024-05-04T03:06:19.713 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cephfs-2:19.0.0-3522.gbd4e00a3.el9.x86_6 23/156 2024-05-04T03:06:19.713 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rados-2:19.0.0-3522.gbd4e00a3.el9.x86_64 24/156 2024-05-04T03:06:19.713 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 25/156 2024-05-04T03:06:19.713 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 26/156 2024-05-04T03:06:19.713 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 27/156 2024-05-04T03:06:19.713 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 28/156 2024-05-04T03:06:19.714 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-nbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 29/156 2024-05-04T03:06:19.714 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3522.gbd4e00a3. 30/156 2024-05-04T03:06:19.714 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noa 31/156 2024-05-04T03:06:19.728 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.n 32/156 2024-05-04T03:06:19.728 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4 33/156 2024-05-04T03:06:19.728 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3522.gbd4e00a3.el 34/156 2024-05-04T03:06:19.728 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 35/156 2024-05-04T03:06:19.728 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3522.gbd4e00a3.e 36/156 2024-05-04T03:06:19.728 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 37/156 2024-05-04T03:06:19.728 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 38/156 2024-05-04T03:06:19.729 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephfs-top-2:19.0.0-3522.gbd4e00a3.el9.noarch 39/156 2024-05-04T03:06:19.729 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-05-04T03:06:19.729 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-05-04T03:06:19.729 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-05-04T03:06:19.729 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-05-04T03:06:19.729 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-05-04T03:06:19.729 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-05-04T03:06:19.729 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-05-04T03:06:19.729 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-05-04T03:06:19.729 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-05-04T03:06:19.729 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-05-04T03:06:19.729 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-05-04T03:06:19.729 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-05-04T03:06:19.729 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-05-04T03:06:19.730 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-05-04T03:06:19.730 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-05-04T03:06:19.730 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-05-04T03:06:19.730 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-05-04T03:06:19.730 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-05-04T03:06:19.730 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-05-04T03:06:19.730 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-05-04T03:06:19.730 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-05-04T03:06:19.730 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-05-04T03:06:19.730 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-05-04T03:06:19.730 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-05-04T03:06:19.730 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-05-04T03:06:19.730 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-05-04T03:06:19.730 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-05-04T03:06:19.731 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-05-04T03:06:19.731 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-05-04T03:06:19.731 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-05-04T03:06:19.731 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-05-04T03:06:19.731 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-05-04T03:06:19.731 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-05-04T03:06:19.731 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-05-04T03:06:19.731 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-05-04T03:06:19.731 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-05-04T03:06:19.731 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-05-04T03:06:19.731 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-05-04T03:06:19.731 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-05-04T03:06:19.731 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-05-04T03:06:19.731 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-05-04T03:06:19.732 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-05-04T03:06:19.732 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-05-04T03:06:19.732 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-05-04T03:06:19.732 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-05-04T03:06:19.732 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-05-04T03:06:19.732 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-05-04T03:06:19.732 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-05-04T03:06:19.732 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-05-04T03:06:19.732 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-05-04T03:06:19.732 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-05-04T03:06:19.732 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-05-04T03:06:19.732 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-05-04T03:06:19.732 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-05-04T03:06:19.732 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-05-04T03:06:19.733 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-05-04T03:06:19.733 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-05-04T03:06:19.733 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-05-04T03:06:19.733 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-05-04T03:06:19.733 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-05-04T03:06:19.733 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-05-04T03:06:19.733 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-05-04T03:06:19.733 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-05-04T03:06:19.733 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-05-04T03:06:19.733 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-05-04T03:06:19.733 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-05-04T03:06:19.733 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-05-04T03:06:19.733 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-05-04T03:06:19.734 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-05-04T03:06:19.734 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-05-04T03:06:19.734 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-05-04T03:06:19.734 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-05-04T03:06:19.734 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-05-04T03:06:19.734 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-05-04T03:06:19.734 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-05-04T03:06:19.734 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-05-04T03:06:19.734 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-05-04T03:06:19.734 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-05-04T03:06:19.734 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-04T03:06:19.734 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-04T03:06:19.734 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-05-04T03:06:19.734 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-05-04T03:06:19.734 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-05-04T03:06:19.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-05-04T03:06:19.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-05-04T03:06:19.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-05-04T03:06:19.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-05-04T03:06:19.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-05-04T03:06:19.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-05-04T03:06:19.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-05-04T03:06:19.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-05-04T03:06:19.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-05-04T03:06:19.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-05-04T03:06:19.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-05-04T03:06:19.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-05-04T03:06:19.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-05-04T03:06:19.735 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-05-04T03:06:19.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-05-04T03:06:19.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-05-04T03:06:19.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-05-04T03:06:19.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-05-04T03:06:19.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-05-04T03:06:19.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-05-04T03:06:19.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-05-04T03:06:19.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-05-04T03:06:19.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-05-04T03:06:19.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-05-04T03:06:19.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-05-04T03:06:19.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-05-04T03:06:19.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 149/156 2024-05-04T03:06:19.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-05-04T03:06:19.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 151/156 2024-05-04T03:06:19.736 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-05-04T03:06:19.737 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-05-04T03:06:19.737 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-04T03:06:19.737 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-05-04T03:06:22.043 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-04T03:06:22.043 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-04T03:06:22.043 INFO:teuthology.orchestra.run.smithi072.stdout:Upgraded: 2024-05-04T03:06:22.043 INFO:teuthology.orchestra.run.smithi072.stdout: librados2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.043 INFO:teuthology.orchestra.run.smithi072.stdout: librbd1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.043 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-04T03:06:22.043 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-04T03:06:22.043 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-05-04T03:06:22.043 INFO:teuthology.orchestra.run.smithi072.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-04T03:06:22.043 INFO:teuthology.orchestra.run.smithi072.stdout: automake-1.16.2-8.el9.noarch 2024-05-04T03:06:22.044 INFO:teuthology.orchestra.run.smithi072.stdout: bison-3.7.4-5.el9.x86_64 2024-05-04T03:06:22.044 INFO:teuthology.orchestra.run.smithi072.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-04T03:06:22.044 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.044 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-base-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.044 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.044 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.044 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-grafana-dashboards-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:06:22.044 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-immutable-object-cache-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.044 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mds-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.044 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.044 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:06:22.044 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-dashboard-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:06:22.044 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:06:22.045 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-modules-core-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:06:22.045 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-rook-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:06:22.045 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mon-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.045 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-osd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.045 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-prometheus-alerts-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:06:22.045 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-radosgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.045 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-selinux-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.045 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-test-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.045 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-volume-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:06:22.045 INFO:teuthology.orchestra.run.smithi072.stdout: cephadm-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:06:22.045 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.045 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-shell-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.046 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-top-2:19.0.0-3522.gbd4e00a3.el9.noarch 2024-05-04T03:06:22.046 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-04T03:06:22.046 INFO:teuthology.orchestra.run.smithi072.stdout: flex-2.6.4-9.el9.x86_64 2024-05-04T03:06:22.046 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-04T03:06:22.046 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-04T03:06:22.046 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-04T03:06:22.046 INFO:teuthology.orchestra.run.smithi072.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-04T03:06:22.046 INFO:teuthology.orchestra.run.smithi072.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-04T03:06:22.046 INFO:teuthology.orchestra.run.smithi072.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-04T03:06:22.046 INFO:teuthology.orchestra.run.smithi072.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-04T03:06:22.046 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-04T03:06:22.046 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-04T03:06:22.046 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.046 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.047 INFO:teuthology.orchestra.run.smithi072.stdout: libcephsqlite-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.047 INFO:teuthology.orchestra.run.smithi072.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-04T03:06:22.047 INFO:teuthology.orchestra.run.smithi072.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-04T03:06:22.047 INFO:teuthology.orchestra.run.smithi072.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-04T03:06:22.047 INFO:teuthology.orchestra.run.smithi072.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-04T03:06:22.047 INFO:teuthology.orchestra.run.smithi072.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-04T03:06:22.047 INFO:teuthology.orchestra.run.smithi072.stdout: librados-devel-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.047 INFO:teuthology.orchestra.run.smithi072.stdout: libradosstriper1-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.047 INFO:teuthology.orchestra.run.smithi072.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-04T03:06:22.047 INFO:teuthology.orchestra.run.smithi072.stdout: librgw2-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.047 INFO:teuthology.orchestra.run.smithi072.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-04T03:06:22.047 INFO:teuthology.orchestra.run.smithi072.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-04T03:06:22.047 INFO:teuthology.orchestra.run.smithi072.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-04T03:06:22.047 INFO:teuthology.orchestra.run.smithi072.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-04T03:06:22.048 INFO:teuthology.orchestra.run.smithi072.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-04T03:06:22.048 INFO:teuthology.orchestra.run.smithi072.stdout: lua-5.4.4-4.el9.x86_64 2024-05-04T03:06:22.048 INFO:teuthology.orchestra.run.smithi072.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-04T03:06:22.048 INFO:teuthology.orchestra.run.smithi072.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-04T03:06:22.048 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-04T03:06:22.048 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-04T03:06:22.048 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-04T03:06:22.048 INFO:teuthology.orchestra.run.smithi072.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-04T03:06:22.048 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-13.14-1.el9.x86_64 2024-05-04T03:06:22.048 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-04T03:06:22.048 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-04T03:06:22.048 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-04T03:06:22.048 INFO:teuthology.orchestra.run.smithi072.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-04T03:06:22.049 INFO:teuthology.orchestra.run.smithi072.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-04T03:06:22.049 INFO:teuthology.orchestra.run.smithi072.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-04T03:06:22.049 INFO:teuthology.orchestra.run.smithi072.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-04T03:06:22.049 INFO:teuthology.orchestra.run.smithi072.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-04T03:06:22.049 INFO:teuthology.orchestra.run.smithi072.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-04T03:06:22.049 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-04T03:06:22.049 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-argparse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.049 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-common-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.049 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cephfs-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.049 INFO:teuthology.orchestra.run.smithi072.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-04T03:06:22.049 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-04T03:06:22.049 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-04T03:06:22.049 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-04T03:06:22.050 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-04T03:06:22.050 INFO:teuthology.orchestra.run.smithi072.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-04T03:06:22.050 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-04T03:06:22.050 INFO:teuthology.orchestra.run.smithi072.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-04T03:06:22.050 INFO:teuthology.orchestra.run.smithi072.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-04T03:06:22.050 INFO:teuthology.orchestra.run.smithi072.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-04T03:06:22.050 INFO:teuthology.orchestra.run.smithi072.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-04T03:06:22.050 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-04T03:06:22.050 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-04T03:06:22.050 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-04T03:06:22.050 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-04T03:06:22.051 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-04T03:06:22.051 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-04T03:06:22.051 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-04T03:06:22.051 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-04T03:06:22.051 INFO:teuthology.orchestra.run.smithi072.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-04T03:06:22.051 INFO:teuthology.orchestra.run.smithi072.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-04T03:06:22.051 INFO:teuthology.orchestra.run.smithi072.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-04T03:06:22.051 INFO:teuthology.orchestra.run.smithi072.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-04T03:06:22.051 INFO:teuthology.orchestra.run.smithi072.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-04T03:06:22.051 INFO:teuthology.orchestra.run.smithi072.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-04T03:06:22.051 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-04T03:06:22.051 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-04T03:06:22.051 INFO:teuthology.orchestra.run.smithi072.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-04T03:06:22.051 INFO:teuthology.orchestra.run.smithi072.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-04T03:06:22.052 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-04T03:06:22.052 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ply-3.11-14.el9.noarch 2024-05-04T03:06:22.052 INFO:teuthology.orchestra.run.smithi072.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-04T03:06:22.052 INFO:teuthology.orchestra.run.smithi072.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-04T03:06:22.052 INFO:teuthology.orchestra.run.smithi072.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-04T03:06:22.052 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-04T03:06:22.052 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-04T03:06:22.052 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-04T03:06:22.052 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-04T03:06:22.052 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-04T03:06:22.052 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-04T03:06:22.052 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rados-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.052 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.052 INFO:teuthology.orchestra.run.smithi072.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-04T03:06:22.052 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-04T03:06:22.053 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-04T03:06:22.053 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rgw-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.053 INFO:teuthology.orchestra.run.smithi072.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-04T03:06:22.053 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-04T03:06:22.053 INFO:teuthology.orchestra.run.smithi072.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-04T03:06:22.053 INFO:teuthology.orchestra.run.smithi072.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-04T03:06:22.053 INFO:teuthology.orchestra.run.smithi072.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-04T03:06:22.053 INFO:teuthology.orchestra.run.smithi072.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-04T03:06:22.053 INFO:teuthology.orchestra.run.smithi072.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-04T03:06:22.053 INFO:teuthology.orchestra.run.smithi072.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-04T03:06:22.053 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-04T03:06:22.053 INFO:teuthology.orchestra.run.smithi072.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-04T03:06:22.053 INFO:teuthology.orchestra.run.smithi072.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-04T03:06:22.054 INFO:teuthology.orchestra.run.smithi072.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-04T03:06:22.054 INFO:teuthology.orchestra.run.smithi072.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-04T03:06:22.054 INFO:teuthology.orchestra.run.smithi072.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-04T03:06:22.054 INFO:teuthology.orchestra.run.smithi072.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-04T03:06:22.054 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-fuse-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.054 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-mirror-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.054 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-nbd-2:19.0.0-3522.gbd4e00a3.el9.x86_64 2024-05-04T03:06:22.054 INFO:teuthology.orchestra.run.smithi072.stdout: re2-1:20211101-3.el9.x86_64 2024-05-04T03:06:22.054 INFO:teuthology.orchestra.run.smithi072.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-04T03:06:22.054 INFO:teuthology.orchestra.run.smithi072.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-04T03:06:22.054 INFO:teuthology.orchestra.run.smithi072.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-04T03:06:22.054 INFO:teuthology.orchestra.run.smithi072.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-04T03:06:22.054 INFO:teuthology.orchestra.run.smithi072.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-04T03:06:22.055 INFO:teuthology.orchestra.run.smithi072.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-04T03:06:22.055 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-04T03:06:22.055 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-04T03:06:22.528 DEBUG:teuthology.parallel:result is None 2024-05-04T03:06:22.529 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-04T03:06:22.678 DEBUG:teuthology.orchestra.run.smithi001:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-04T03:06:22.709 INFO:teuthology.orchestra.run.smithi001.stdout:19.0.0-3522.gbd4e00a3.el9 2024-05-04T03:06:22.709 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3522.gbd4e00a3.el9 2024-05-04T03:06:22.709 INFO:teuthology.task.install:The correct ceph version 19.0.0-3522.gbd4e00a3 is installed. 2024-05-04T03:06:22.711 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-04T03:06:22.937 DEBUG:teuthology.orchestra.run.smithi072:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-04T03:06:22.959 INFO:teuthology.orchestra.run.smithi072.stdout:19.0.0-3522.gbd4e00a3.el9 2024-05-04T03:06:22.959 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3522.gbd4e00a3.el9 2024-05-04T03:06:22.959 INFO:teuthology.task.install:The correct ceph version 19.0.0-3522.gbd4e00a3 is installed. 2024-05-04T03:06:22.961 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-04T03:06:23.116 DEBUG:teuthology.orchestra.run.smithi077:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-04T03:06:23.143 INFO:teuthology.orchestra.run.smithi077.stdout:19.0.0-3522.gbd4e00a3.el9 2024-05-04T03:06:23.143 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3522.gbd4e00a3.el9 2024-05-04T03:06:23.143 INFO:teuthology.task.install:The correct ceph version 19.0.0-3522.gbd4e00a3 is installed. 2024-05-04T03:06:23.145 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-04T03:06:23.145 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-04T03:06:23.145 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-04T03:06:23.186 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-04T03:06:23.186 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-04T03:06:23.213 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-04T03:06:23.213 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-04T03:06:23.250 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-04T03:06:23.250 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-04T03:06:23.250 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/daemon-helper 2024-05-04T03:06:23.281 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-04T03:06:23.354 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-04T03:06:23.354 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/daemon-helper 2024-05-04T03:06:23.380 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-04T03:06:23.444 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-04T03:06:23.444 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/daemon-helper 2024-05-04T03:06:23.473 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-04T03:06:23.542 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-04T03:06:23.542 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-04T03:06:23.542 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-04T03:06:23.569 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-04T03:06:23.638 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-04T03:06:23.638 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-04T03:06:23.662 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-04T03:06:23.728 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-04T03:06:23.728 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-04T03:06:23.760 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-04T03:06:23.831 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-04T03:06:23.831 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-04T03:06:23.831 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/stdin-killer 2024-05-04T03:06:23.858 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-04T03:06:23.938 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-04T03:06:23.938 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/stdin-killer 2024-05-04T03:06:23.960 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-04T03:06:24.024 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-04T03:06:24.024 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/stdin-killer 2024-05-04T03:06:24.052 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-04T03:06:24.119 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-04T03:06:24.226 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'bd4e00a39d766328b458e4c953f4d722fed96a31', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-05-04T03:06:24.226 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-04T03:06:24.227 INFO:tasks.cephadm:Cluster fsid is 4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:06:24.227 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-04T03:06:24.227 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.1', 'mon.b': '172.21.15.72', 'mon.c': '172.21.15.77'} 2024-05-04T03:06:24.227 INFO:tasks.cephadm:First mon is mon.a on smithi001 2024-05-04T03:06:24.228 INFO:tasks.cephadm:First mgr is x 2024-05-04T03:06:24.228 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-04T03:06:24.228 DEBUG:teuthology.orchestra.run.smithi001:> sudo hostname $(hostname -s) 2024-05-04T03:06:24.253 DEBUG:teuthology.orchestra.run.smithi072:> sudo hostname $(hostname -s) 2024-05-04T03:06:24.276 DEBUG:teuthology.orchestra.run.smithi077:> sudo hostname $(hostname -s) 2024-05-04T03:06:24.304 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-04T03:06:24.305 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-04T03:06:24.646 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'bd4e00a39d766328b458e4c953f4d722fed96a31', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79090/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3522-gbd4e00a3', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3522.gbd4e00a3'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-03 18:03:30.054903', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240503.163550-debug', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-04T03:06:24.892 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-pdonnell-testing-20240503.163550-debug, sha1 bd4e00a39d766328b458e4c953f4d722fed96a31 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-04T03:06:24.894 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/x86_64/flavors/default/cephadm 2024-05-04T03:06:24.894 DEBUG:teuthology.orchestra.run.smithi001:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-04T03:06:25.478 INFO:teuthology.orchestra.run.smithi001.stdout:-rw-r--r--. 1 ubuntu ubuntu 782507 May 4 03:06 /home/ubuntu/cephtest/cephadm 2024-05-04T03:06:25.479 DEBUG:teuthology.orchestra.run.smithi072:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-04T03:06:26.133 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r--. 1 ubuntu ubuntu 782507 May 4 03:06 /home/ubuntu/cephtest/cephadm 2024-05-04T03:06:26.134 DEBUG:teuthology.orchestra.run.smithi077:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240503.163550-debug/bd4e00a39d766328b458e4c953f4d722fed96a31/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-04T03:06:26.688 INFO:teuthology.orchestra.run.smithi077.stdout:-rw-r--r--. 1 ubuntu ubuntu 782507 May 4 03:06 /home/ubuntu/cephtest/cephadm 2024-05-04T03:06:26.688 DEBUG:teuthology.orchestra.run.smithi001:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-04T03:06:26.708 DEBUG:teuthology.orchestra.run.smithi072:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-04T03:06:26.727 DEBUG:teuthology.orchestra.run.smithi077:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-04T03:06:26.762 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 on all hosts... 2024-05-04T03:06:26.762 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 pull 2024-05-04T03:06:26.765 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 pull 2024-05-04T03:06:26.769 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 pull 2024-05-04T03:06:28.285 INFO:teuthology.orchestra.run.smithi001.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31... 2024-05-04T03:06:28.288 INFO:teuthology.orchestra.run.smithi077.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31... 2024-05-04T03:06:32.873 INFO:teuthology.orchestra.run.smithi072.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31... 2024-05-04T03:07:01.759 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-05-04T03:07:01.759 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_version": "ceph version 19.0.0-3522-gbd4e00a3 (bd4e00a39d766328b458e4c953f4d722fed96a31) squid (dev)", 2024-05-04T03:07:01.759 INFO:teuthology.orchestra.run.smithi072.stdout: "image_id": "afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62", 2024-05-04T03:07:01.759 INFO:teuthology.orchestra.run.smithi072.stdout: "repo_digests": [ 2024-05-04T03:07:01.759 INFO:teuthology.orchestra.run.smithi072.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2" 2024-05-04T03:07:01.759 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-05-04T03:07:01.759 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-05-04T03:07:21.188 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-04T03:07:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 19.0.0-3522-gbd4e00a3 (bd4e00a39d766328b458e4c953f4d722fed96a31) squid (dev)", 2024-05-04T03:07:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: "image_id": "afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62", 2024-05-04T03:07:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: "repo_digests": [ 2024-05-04T03:07:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2" 2024-05-04T03:07:21.188 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-04T03:07:21.188 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-04T03:07:23.729 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-04T03:07:23.729 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_version": "ceph version 19.0.0-3522-gbd4e00a3 (bd4e00a39d766328b458e4c953f4d722fed96a31) squid (dev)", 2024-05-04T03:07:23.729 INFO:teuthology.orchestra.run.smithi077.stdout: "image_id": "afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62", 2024-05-04T03:07:23.730 INFO:teuthology.orchestra.run.smithi077.stdout: "repo_digests": [ 2024-05-04T03:07:23.730 INFO:teuthology.orchestra.run.smithi077.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2" 2024-05-04T03:07:23.730 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-05-04T03:07:23.730 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-04T03:07:23.753 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /etc/ceph 2024-05-04T03:07:23.786 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /etc/ceph 2024-05-04T03:07:23.819 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /etc/ceph 2024-05-04T03:07:23.845 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 777 /etc/ceph 2024-05-04T03:07:23.875 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 777 /etc/ceph 2024-05-04T03:07:23.902 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 777 /etc/ceph 2024-05-04T03:07:23.925 INFO:tasks.cephadm:Writing seed config... 2024-05-04T03:07:23.926 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-04T03:07:23.926 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-04T03:07:23.926 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-04T03:07:23.926 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-04T03:07:23.927 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-04T03:07:23.927 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-04T03:07:23.927 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-04T03:07:23.927 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-04T03:07:23.927 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-04T03:07:23.927 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-04T03:07:23.927 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-04T03:07:23.927 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-04T03:07:23.927 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-04T03:07:23.927 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-04T03:07:23.927 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-04T03:07:23.927 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-04T03:07:23.927 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-04T03:07:23.927 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-05-04T03:07:23.927 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-04T03:07:23.927 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-04T03:07:23.928 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-04T03:07:23.928 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-04T03:07:23.928 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-04T03:07:23.928 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-04T03:07:23.928 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-04T03:07:23.928 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-04T03:07:23.928 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-04T03:07:23.928 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-04T03:07:23.928 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-04T03:07:23.928 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-04T03:07:23.928 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-05-04T03:07:23.928 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-05-04T03:07:23.928 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-05-04T03:07:23.928 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-04T03:07:23.928 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-04T03:07:23.929 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-05-04T03:07:23.929 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-05-04T03:07:23.929 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-04T03:07:23.929 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-04T03:07:23.929 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-05-04T03:07:23.929 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-04T03:07:23.929 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-04T03:07:23.929 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-04T03:07:23.929 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-04T03:07:23.929 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-04T03:07:23.929 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-04T03:07:23.929 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-04T03:07:23.929 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-04T03:07:23.930 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-04T03:07:23.930 DEBUG:teuthology.orchestra.run.smithi001:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-04T03:07:23.949 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 4a572aa8-09c3-11ef-bc97-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-04T03:07:23.949 DEBUG:teuthology.orchestra.run.smithi001:mon.a> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mon.a.service 2024-05-04T03:07:23.991 DEBUG:teuthology.orchestra.run.smithi001:mgr.x> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mgr.x.service 2024-05-04T03:07:24.034 INFO:tasks.cephadm:Bootstrapping... 2024-05-04T03:07:24.034 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 -v bootstrap --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.1 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-04T03:07:24.239 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2024-05-04T03:07:24.240 INFO:teuthology.orchestra.run.smithi001.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31', '-v', 'bootstrap', '--fsid', '4a572aa8-09c3-11ef-bc97-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.1', '--skip-admin-label'] 2024-05-04T03:07:24.240 INFO:teuthology.orchestra.run.smithi001.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-04T03:07:24.240 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying podman|docker is present... 2024-05-04T03:07:24.265 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-04T03:07:24.266 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying lvm2 is present... 2024-05-04T03:07:24.266 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying time synchronization is in place... 2024-05-04T03:07:24.273 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-04T03:07:24.273 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-04T03:07:24.279 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-04T03:07:24.279 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-05-04T03:07:24.286 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout enabled 2024-05-04T03:07:24.292 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout active 2024-05-04T03:07:24.292 INFO:teuthology.orchestra.run.smithi001.stdout:Unit chronyd.service is enabled and running 2024-05-04T03:07:24.292 INFO:teuthology.orchestra.run.smithi001.stdout:Repeating the final host check... 2024-05-04T03:07:24.316 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-04T03:07:24.316 INFO:teuthology.orchestra.run.smithi001.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-04T03:07:24.316 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl is present 2024-05-04T03:07:24.316 INFO:teuthology.orchestra.run.smithi001.stdout:lvcreate is present 2024-05-04T03:07:24.323 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-04T03:07:24.323 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-04T03:07:24.329 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-04T03:07:24.329 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-05-04T03:07:24.336 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout enabled 2024-05-04T03:07:24.342 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout active 2024-05-04T03:07:24.342 INFO:teuthology.orchestra.run.smithi001.stdout:Unit chronyd.service is enabled and running 2024-05-04T03:07:24.342 INFO:teuthology.orchestra.run.smithi001.stdout:Host looks OK 2024-05-04T03:07:24.342 INFO:teuthology.orchestra.run.smithi001.stdout:Cluster fsid: 4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:07:24.343 INFO:teuthology.orchestra.run.smithi001.stdout:Acquiring lock 140380780150736 on /run/cephadm/4a572aa8-09c3-11ef-bc97-c7b262605968.lock 2024-05-04T03:07:24.343 INFO:teuthology.orchestra.run.smithi001.stdout:Lock 140380780150736 acquired on /run/cephadm/4a572aa8-09c3-11ef-bc97-c7b262605968.lock 2024-05-04T03:07:24.343 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying IP 172.21.15.1 port 3300 ... 2024-05-04T03:07:24.343 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying IP 172.21.15.1 port 6789 ... 2024-05-04T03:07:24.343 INFO:teuthology.orchestra.run.smithi001.stdout:Base mon IP(s) is [172.21.15.1:3300, 172.21.15.1:6789], mon addrv is [v2:172.21.15.1:3300,v1:172.21.15.1:6789] 2024-05-04T03:07:24.346 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.1 metric 100 2024-05-04T03:07:24.347 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.1 metric 100 2024-05-04T03:07:24.349 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-04T03:07:24.350 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-04T03:07:24.350 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-04T03:07:24.352 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-04T03:07:24.352 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-04T03:07:24.352 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-04T03:07:24.352 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-04T03:07:24.352 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15e8/64 scope link noprefixroute 2024-05-04T03:07:24.352 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-04T03:07:24.353 INFO:teuthology.orchestra.run.smithi001.stdout:Mon IP `172.21.15.1` is in CIDR network `172.21.0.0/20` 2024-05-04T03:07:24.353 INFO:teuthology.orchestra.run.smithi001.stdout:Mon IP `172.21.15.1` is in CIDR network `172.21.0.0/20` 2024-05-04T03:07:24.353 INFO:teuthology.orchestra.run.smithi001.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-04T03:07:24.353 INFO:teuthology.orchestra.run.smithi001.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-04T03:07:24.354 INFO:teuthology.orchestra.run.smithi001.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31... 2024-05-04T03:07:24.824 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stdout afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 2024-05-04T03:07:24.824 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31... 2024-05-04T03:07:24.824 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Getting image source signatures 2024-05-04T03:07:24.824 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Copying blob sha256:096f198f13bcfe0dfab6408b113f6513b5ae0fab440afa5e44aec6470f68ac59 2024-05-04T03:07:24.824 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Copying blob sha256:e2f2fd2adca92139da4b4b907fa90a6775d3eb4eddaa977c42ef37c66eea70bd 2024-05-04T03:07:24.824 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Copying config sha256:afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 2024-05-04T03:07:24.825 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-04T03:07:25.947 INFO:teuthology.orchestra.run.smithi001.stdout:ceph: stdout ceph version 19.0.0-3522-gbd4e00a3 (bd4e00a39d766328b458e4c953f4d722fed96a31) squid (dev) 2024-05-04T03:07:25.948 INFO:teuthology.orchestra.run.smithi001.stdout:Ceph version: ceph version 19.0.0-3522-gbd4e00a3 (bd4e00a39d766328b458e4c953f4d722fed96a31) squid (dev) 2024-05-04T03:07:25.948 INFO:teuthology.orchestra.run.smithi001.stdout:Extracting ceph user uid/gid from container image... 2024-05-04T03:07:26.929 INFO:teuthology.orchestra.run.smithi001.stdout:stat: stdout 167 167 2024-05-04T03:07:26.929 INFO:teuthology.orchestra.run.smithi001.stdout:Creating initial keys... 2024-05-04T03:07:27.809 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph-authtool: stdout AQBvpjVm2JeCDxAADy/CZTnMUg2yqHs9xQcbOg== 2024-05-04T03:07:28.672 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph-authtool: stdout AQBwpjVm0AL8CBAAW7zSC6DLSCD7UKAgQ2SP+Q== 2024-05-04T03:07:29.578 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph-authtool: stdout AQBxpjVmwILWAhAAp7/ghDjTa40aNFuwz+E1UQ== 2024-05-04T03:07:29.578 INFO:teuthology.orchestra.run.smithi001.stdout:Creating initial monmap... 2024-05-04T03:07:30.467 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-04T03:07:30.467 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-04T03:07:30.468 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:07:30.468 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-04T03:07:30.468 INFO:teuthology.orchestra.run.smithi001.stdout:monmaptool for a [v2:172.21.15.1:3300,v1:172.21.15.1:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-04T03:07:30.468 INFO:teuthology.orchestra.run.smithi001.stdout:setting min_mon_release = quincy 2024-05-04T03:07:30.468 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: set fsid to 4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:07:30.468 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-04T03:07:30.468 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:07:30.468 INFO:teuthology.orchestra.run.smithi001.stdout:Creating mon... 2024-05-04T03:07:31.298 INFO:teuthology.orchestra.run.smithi001.stdout:create mon.a on 2024-05-04T03:07:31.611 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-04T03:07:31.753 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-04T03:07:31.926 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-4a572aa8-09c3-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-4a572aa8-09c3-11ef-bc97-c7b262605968.target. 2024-05-04T03:07:31.926 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-4a572aa8-09c3-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-4a572aa8-09c3-11ef-bc97-c7b262605968.target. 2024-05-04T03:07:32.165 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mon.a 2024-05-04T03:07:32.166 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Failed to reset failed state of unit ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mon.a.service: Unit ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mon.a.service not loaded. 2024-05-04T03:07:32.329 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-4a572aa8-09c3-11ef-bc97-c7b262605968.target.wants/ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mon.a.service → /etc/systemd/system/ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@.service. 2024-05-04T03:07:32.594 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:32 smithi001 podman[29342]: 2024-05-04 03:07:32.497302724 +0000 UTC m=+0.018072917 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-04T03:07:32.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:32 smithi001 podman[29342]: 2024-05-04 03:07:32.791074033 +0000 UTC m=+0.311844227 container init 49bba26e3b4b5f1d771115309c9ec0bd035378d83763b5e02673a41aa65eed13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-04T03:07:32.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:32 smithi001 podman[29342]: 2024-05-04 03:07:32.793939763 +0000 UTC m=+0.314709959 container start 49bba26e3b4b5f1d771115309c9ec0bd035378d83763b5e02673a41aa65eed13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mon-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-04T03:07:32.889 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-04T03:07:32.889 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout disabled 2024-05-04T03:07:32.896 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-04T03:07:32.896 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-05-04T03:07:32.896 INFO:teuthology.orchestra.run.smithi001.stdout:firewalld.service is not enabled 2024-05-04T03:07:32.896 INFO:teuthology.orchestra.run.smithi001.stdout:Not possible to enable service . firewalld.service is not available 2024-05-04T03:07:32.897 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mon to start... 2024-05-04T03:07:32.897 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mon... 2024-05-04T03:07:33.117 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:32 smithi001 ceph-mon[29361]: mkfs 4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:07:33.117 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:32 smithi001 ceph-mon[29361]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-04T03:07:33.118 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:32 smithi001 bash[29342]: 49bba26e3b4b5f1d771115309c9ec0bd035378d83763b5e02673a41aa65eed13 2024-05-04T03:07:33.118 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:32 smithi001 systemd[1]: Started Ceph mon.a for 4a572aa8-09c3-11ef-bc97-c7b262605968. 2024-05-04T03:07:33.931 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout cluster: 2024-05-04T03:07:33.931 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout id: 4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:07:33.931 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-04T03:07:33.931 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-05-04T03:07:33.931 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout services: 2024-05-04T03:07:33.931 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.568267s) 2024-05-04T03:07:33.931 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-04T03:07:33.931 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-04T03:07:33.931 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-05-04T03:07:33.931 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout data: 2024-05-04T03:07:33.931 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-04T03:07:33.931 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-04T03:07:33.932 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-04T03:07:33.932 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout pgs: 2024-05-04T03:07:33.932 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-05-04T03:07:33.932 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.330+0000 7fe1ff0c9700 1 Processor -- start 2024-05-04T03:07:33.932 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.330+0000 7fe1ff0c9700 1 -- start start 2024-05-04T03:07:33.932 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.330+0000 7fe1ff0c9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1f8109c40 0x7fe1f810a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:33.933 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.330+0000 7fe1ff0c9700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f810a640 con 0x7fe1f8109c40 2024-05-04T03:07:33.933 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.331+0000 7fe1fce65700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1f8109c40 0x7fe1f810a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:33.933 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.331+0000 7fe1fce65700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1f8109c40 0x7fe1f810a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33836/0 (socket says 172.21.15.1:33836) 2024-05-04T03:07:33.933 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.331+0000 7fe1fce65700 1 -- 172.21.15.1:0/2636579554 learned_addr learned my addr 172.21.15.1:0/2636579554 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:07:33.933 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.331+0000 7fe1fce65700 1 -- 172.21.15.1:0/2636579554 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f810ae10 con 0x7fe1f8109c40 2024-05-04T03:07:33.933 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.331+0000 7fe1fce65700 1 --2- 172.21.15.1:0/2636579554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1f8109c40 0x7fe1f810a050 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec0098a0 tx=0x7fe1ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=536328ddd894c64e server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:33.933 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.332+0000 7fe1f77fe700 1 -- 172.21.15.1:0/2636579554 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1ec004160 con 0x7fe1f8109c40 2024-05-04T03:07:33.933 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.332+0000 7fe1f77fe700 1 -- 172.21.15.1:0/2636579554 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1ec004300 con 0x7fe1f8109c40 2024-05-04T03:07:33.933 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.332+0000 7fe1f77fe700 1 -- 172.21.15.1:0/2636579554 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1ec0036a0 con 0x7fe1f8109c40 2024-05-04T03:07:33.933 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.332+0000 7fe1ff0c9700 1 -- 172.21.15.1:0/2636579554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1f8109c40 msgr2=0x7fe1f810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:33.933 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.332+0000 7fe1ff0c9700 1 --2- 172.21.15.1:0/2636579554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1f8109c40 0x7fe1f810a050 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec0098a0 tx=0x7fe1ec009bb0 comp rx=0 tx=0).stop 2024-05-04T03:07:33.933 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.332+0000 7fe1ff0c9700 1 -- 172.21.15.1:0/2636579554 shutdown_connections 2024-05-04T03:07:33.933 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.332+0000 7fe1ff0c9700 1 --2- 172.21.15.1:0/2636579554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1f8109c40 0x7fe1f810a050 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:33.934 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.332+0000 7fe1ff0c9700 1 -- 172.21.15.1:0/2636579554 >> 172.21.15.1:0/2636579554 conn(0x7fe1f81052c0 msgr2=0x7fe1f8107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:33.934 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.332+0000 7fe1ff0c9700 1 -- 172.21.15.1:0/2636579554 shutdown_connections 2024-05-04T03:07:33.934 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.333+0000 7fe1ff0c9700 1 -- 172.21.15.1:0/2636579554 wait complete. 2024-05-04T03:07:33.934 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.333+0000 7fe1ff0c9700 1 Processor -- start 2024-05-04T03:07:33.934 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.333+0000 7fe1ff0c9700 1 -- start start 2024-05-04T03:07:33.934 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.333+0000 7fe1ff0c9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1f8109c40 0x7fe1f819d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:33.934 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.333+0000 7fe1ff0c9700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f810b340 con 0x7fe1f8109c40 2024-05-04T03:07:33.934 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.333+0000 7fe1fce65700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1f8109c40 0x7fe1f819d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:33.934 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.334+0000 7fe1fce65700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1f8109c40 0x7fe1f819d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33848/0 (socket says 172.21.15.1:33848) 2024-05-04T03:07:33.934 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.334+0000 7fe1fce65700 1 -- 172.21.15.1:0/2322178156 learned_addr learned my addr 172.21.15.1:0/2322178156 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:07:33.934 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.334+0000 7fe1fce65700 1 -- 172.21.15.1:0/2322178156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f819dd60 con 0x7fe1f8109c40 2024-05-04T03:07:33.934 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.334+0000 7fe1fce65700 1 --2- 172.21.15.1:0/2322178156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1f8109c40 0x7fe1f819d820 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec000c00 tx=0x7fe1ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:33.934 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.334+0000 7fe1f5ffb700 1 -- 172.21.15.1:0/2322178156 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1ec003db0 con 0x7fe1f8109c40 2024-05-04T03:07:33.935 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.334+0000 7fe1f5ffb700 1 -- 172.21.15.1:0/2322178156 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1ec011630 con 0x7fe1f8109c40 2024-05-04T03:07:33.935 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.334+0000 7fe1f5ffb700 1 -- 172.21.15.1:0/2322178156 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1ec011950 con 0x7fe1f8109c40 2024-05-04T03:07:33.935 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.334+0000 7fe1ff0c9700 1 -- 172.21.15.1:0/2322178156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f819dff0 con 0x7fe1f8109c40 2024-05-04T03:07:33.935 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.334+0000 7fe1ff0c9700 1 -- 172.21.15.1:0/2322178156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f81a0c00 con 0x7fe1f8109c40 2024-05-04T03:07:33.935 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.335+0000 7fe1f5ffb700 1 -- 172.21.15.1:0/2322178156 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe1ec01a460 con 0x7fe1f8109c40 2024-05-04T03:07:33.935 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.335+0000 7fe1f5ffb700 1 -- 172.21.15.1:0/2322178156 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe1ec01b780 con 0x7fe1f8109c40 2024-05-04T03:07:33.935 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.335+0000 7fe1ff0c9700 1 -- 172.21.15.1:0/2322178156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1f8064e20 con 0x7fe1f8109c40 2024-05-04T03:07:33.935 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.338+0000 7fe1f5ffb700 1 -- 172.21.15.1:0/2322178156 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fe1ec031020 con 0x7fe1f8109c40 2024-05-04T03:07:33.935 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.411+0000 7fe1ff0c9700 1 -- 172.21.15.1:0/2322178156 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fe1f802e870 con 0x7fe1f8109c40 2024-05-04T03:07:33.935 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.412+0000 7fe1f5ffb700 1 -- 172.21.15.1:0/2322178156 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fe1ec01b9d0 con 0x7fe1f8109c40 2024-05-04T03:07:33.935 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.413+0000 7fe1ff0c9700 1 -- 172.21.15.1:0/2322178156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1f8109c40 msgr2=0x7fe1f819d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:33.935 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.413+0000 7fe1ff0c9700 1 --2- 172.21.15.1:0/2322178156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1f8109c40 0x7fe1f819d820 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec000c00 tx=0x7fe1ec004750 comp rx=0 tx=0).stop 2024-05-04T03:07:33.935 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.413+0000 7fe1ff0c9700 1 -- 172.21.15.1:0/2322178156 shutdown_connections 2024-05-04T03:07:33.935 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.413+0000 7fe1ff0c9700 1 --2- 172.21.15.1:0/2322178156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1f8109c40 0x7fe1f819d820 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:33.936 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.413+0000 7fe1ff0c9700 1 -- 172.21.15.1:0/2322178156 >> 172.21.15.1:0/2322178156 conn(0x7fe1f81052c0 msgr2=0x7fe1f8193ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:33.936 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.413+0000 7fe1ff0c9700 1 -- 172.21.15.1:0/2322178156 shutdown_connections 2024-05-04T03:07:33.936 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:33.413+0000 7fe1ff0c9700 1 -- 172.21.15.1:0/2322178156 wait complete. 2024-05-04T03:07:33.936 INFO:teuthology.orchestra.run.smithi001.stdout:mon is available 2024-05-04T03:07:33.936 INFO:teuthology.orchestra.run.smithi001.stdout:Assimilating anything we can from ceph.conf... 2024-05-04T03:07:34.048 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:33 smithi001 ceph-mon[29361]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-04T03:07:34.048 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:33 smithi001 ceph-mon[29361]: monmap epoch 1 2024-05-04T03:07:34.048 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:33 smithi001 ceph-mon[29361]: fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:07:34.048 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:33 smithi001 ceph-mon[29361]: last_changed 2024-05-04T03:07:29.984656+0000 2024-05-04T03:07:34.048 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:33 smithi001 ceph-mon[29361]: created 2024-05-04T03:07:29.984656+0000 2024-05-04T03:07:34.048 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:33 smithi001 ceph-mon[29361]: min_mon_release 19 (squid) 2024-05-04T03:07:34.048 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:33 smithi001 ceph-mon[29361]: election_strategy: 1 2024-05-04T03:07:34.048 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:33 smithi001 ceph-mon[29361]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-05-04T03:07:34.048 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:33 smithi001 ceph-mon[29361]: fsmap 2024-05-04T03:07:34.048 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:33 smithi001 ceph-mon[29361]: osdmap e1: 0 total, 0 up, 0 in 2024-05-04T03:07:34.048 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:33 smithi001 ceph-mon[29361]: mgrmap e1: no daemons active 2024-05-04T03:07:34.049 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:33 smithi001 ceph-mon[29361]: from='client.? 172.21.15.1:0/2322178156' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-04T03:07:35.003 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:34 smithi001 ceph-mon[29361]: from='client.? 172.21.15.1:0/1268747993' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-04T03:07:35.003 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:34 smithi001 ceph-mon[29361]: from='client.? 172.21.15.1:0/1268747993' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-04T03:07:35.008 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-05-04T03:07:35.008 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [global] 2024-05-04T03:07:35.008 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout fsid = 4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:07:35.008 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-04T03:07:35.008 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.1:3300,v1:172.21.15.1:6789] 2024-05-04T03:07:35.008 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-04T03:07:35.008 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-04T03:07:35.008 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-04T03:07:35.008 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-04T03:07:35.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-05-04T03:07:35.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-04T03:07:35.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-04T03:07:35.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-05-04T03:07:35.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [osd] 2024-05-04T03:07:35.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-05-04T03:07:35.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-04T03:07:35.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-04T03:07:35.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-04T03:07:35.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-04T03:07:35.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-04T03:07:35.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.374+0000 7fb6d8433700 1 Processor -- start 2024-05-04T03:07:35.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.374+0000 7fb6d8433700 1 -- start start 2024-05-04T03:07:35.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.375+0000 7fb6d8433700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6d0109cd0 0x7fb6d010a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:35.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.375+0000 7fb6d8433700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d010a6d0 con 0x7fb6d0109cd0 2024-05-04T03:07:35.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.375+0000 7fb6d61cf700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6d0109cd0 0x7fb6d010a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:35.014 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.375+0000 7fb6d61cf700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6d0109cd0 0x7fb6d010a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33850/0 (socket says 172.21.15.1:33850) 2024-05-04T03:07:35.014 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.375+0000 7fb6d61cf700 1 -- 172.21.15.1:0/527982506 learned_addr learned my addr 172.21.15.1:0/527982506 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:07:35.014 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.375+0000 7fb6d61cf700 1 -- 172.21.15.1:0/527982506 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d010aea0 con 0x7fb6d0109cd0 2024-05-04T03:07:35.014 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.375+0000 7fb6d61cf700 1 --2- 172.21.15.1:0/527982506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6d0109cd0 0x7fb6d010a0e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00098a0 tx=0x7fb6c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=310c77280e949f2c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:35.014 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.376+0000 7fb6d51cd700 1 -- 172.21.15.1:0/527982506 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6c0004160 con 0x7fb6d0109cd0 2024-05-04T03:07:35.014 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.376+0000 7fb6d51cd700 1 -- 172.21.15.1:0/527982506 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6c0004300 con 0x7fb6d0109cd0 2024-05-04T03:07:35.014 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.376+0000 7fb6d51cd700 1 -- 172.21.15.1:0/527982506 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6c00036a0 con 0x7fb6d0109cd0 2024-05-04T03:07:35.014 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.376+0000 7fb6d8433700 1 -- 172.21.15.1:0/527982506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6d0109cd0 msgr2=0x7fb6d010a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:35.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.376+0000 7fb6d8433700 1 --2- 172.21.15.1:0/527982506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6d0109cd0 0x7fb6d010a0e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00098a0 tx=0x7fb6c0009bb0 comp rx=0 tx=0).stop 2024-05-04T03:07:35.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.376+0000 7fb6d8433700 1 -- 172.21.15.1:0/527982506 shutdown_connections 2024-05-04T03:07:35.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.376+0000 7fb6d8433700 1 --2- 172.21.15.1:0/527982506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6d0109cd0 0x7fb6d010a0e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:35.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.376+0000 7fb6d8433700 1 -- 172.21.15.1:0/527982506 >> 172.21.15.1:0/527982506 conn(0x7fb6d0105330 msgr2=0x7fb6d01077c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:35.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.376+0000 7fb6d8433700 1 -- 172.21.15.1:0/527982506 shutdown_connections 2024-05-04T03:07:35.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.377+0000 7fb6d8433700 1 -- 172.21.15.1:0/527982506 wait complete. 2024-05-04T03:07:35.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.377+0000 7fb6d8433700 1 Processor -- start 2024-05-04T03:07:35.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.377+0000 7fb6d8433700 1 -- start start 2024-05-04T03:07:35.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.377+0000 7fb6d8433700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6d0109cd0 0x7fb6d019d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:35.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.377+0000 7fb6d8433700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d010b3d0 con 0x7fb6d0109cd0 2024-05-04T03:07:35.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.378+0000 7fb6d61cf700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6d0109cd0 0x7fb6d019d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:35.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.378+0000 7fb6d61cf700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6d0109cd0 0x7fb6d019d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33860/0 (socket says 172.21.15.1:33860) 2024-05-04T03:07:35.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.378+0000 7fb6d61cf700 1 -- 172.21.15.1:0/1268747993 learned_addr learned my addr 172.21.15.1:0/1268747993 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:07:35.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.378+0000 7fb6d61cf700 1 -- 172.21.15.1:0/1268747993 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d019dcb0 con 0x7fb6d0109cd0 2024-05-04T03:07:35.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.378+0000 7fb6d61cf700 1 --2- 172.21.15.1:0/1268747993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6d0109cd0 0x7fb6d019d770 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0000c00 tx=0x7fb6c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:35.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.378+0000 7fb6c77fe700 1 -- 172.21.15.1:0/1268747993 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6c0003db0 con 0x7fb6d0109cd0 2024-05-04T03:07:35.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.378+0000 7fb6c77fe700 1 -- 172.21.15.1:0/1268747993 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6c0011630 con 0x7fb6d0109cd0 2024-05-04T03:07:35.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.378+0000 7fb6c77fe700 1 -- 172.21.15.1:0/1268747993 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6c0011950 con 0x7fb6d0109cd0 2024-05-04T03:07:35.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.378+0000 7fb6d8433700 1 -- 172.21.15.1:0/1268747993 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d019df40 con 0x7fb6d0109cd0 2024-05-04T03:07:35.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.378+0000 7fb6d8433700 1 -- 172.21.15.1:0/1268747993 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d01a0b50 con 0x7fb6d0109cd0 2024-05-04T03:07:35.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.378+0000 7fb6c77fe700 1 -- 172.21.15.1:0/1268747993 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb6c0011b80 con 0x7fb6d0109cd0 2024-05-04T03:07:35.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.378+0000 7fb6c77fe700 1 -- 172.21.15.1:0/1268747993 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb6c001c450 con 0x7fb6d0109cd0 2024-05-04T03:07:35.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.379+0000 7fb6d8433700 1 -- 172.21.15.1:0/1268747993 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6d0064e20 con 0x7fb6d0109cd0 2024-05-04T03:07:35.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.382+0000 7fb6c77fe700 1 -- 172.21.15.1:0/1268747993 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fb6c0032020 con 0x7fb6d0109cd0 2024-05-04T03:07:35.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.453+0000 7fb6d8433700 1 -- 172.21.15.1:0/1268747993 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fb6d002e8f0 con 0x7fb6d0109cd0 2024-05-04T03:07:35.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.456+0000 7fb6c77fe700 1 -- 172.21.15.1:0/1268747993 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+490 (secure 0 0 0) 0x7fb6c0004260 con 0x7fb6d0109cd0 2024-05-04T03:07:35.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.457+0000 7fb6c77fe700 1 -- 172.21.15.1:0/1268747993 <== mon.0 v2:172.21.15.1:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb6c001c6a0 con 0x7fb6d0109cd0 2024-05-04T03:07:35.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.458+0000 7fb6d8433700 1 -- 172.21.15.1:0/1268747993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6d0109cd0 msgr2=0x7fb6d019d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:35.017 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.458+0000 7fb6d8433700 1 --2- 172.21.15.1:0/1268747993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6d0109cd0 0x7fb6d019d770 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0000c00 tx=0x7fb6c0004750 comp rx=0 tx=0).stop 2024-05-04T03:07:35.017 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.458+0000 7fb6d8433700 1 -- 172.21.15.1:0/1268747993 shutdown_connections 2024-05-04T03:07:35.017 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.458+0000 7fb6d8433700 1 --2- 172.21.15.1:0/1268747993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb6d0109cd0 0x7fb6d019d770 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:35.017 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.458+0000 7fb6d8433700 1 -- 172.21.15.1:0/1268747993 >> 172.21.15.1:0/1268747993 conn(0x7fb6d0105330 msgr2=0x7fb6d0193e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:35.017 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.458+0000 7fb6d8433700 1 -- 172.21.15.1:0/1268747993 shutdown_connections 2024-05-04T03:07:35.017 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:34.458+0000 7fb6d8433700 1 -- 172.21.15.1:0/1268747993 wait complete. 2024-05-04T03:07:35.017 INFO:teuthology.orchestra.run.smithi001.stdout:Generating new minimal ceph.conf... 2024-05-04T03:07:36.049 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.444+0000 7fd5effaf700 1 Processor -- start 2024-05-04T03:07:36.050 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.444+0000 7fd5effaf700 1 -- start start 2024-05-04T03:07:36.050 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.444+0000 7fd5effaf700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5e810a0f0 0x7fd5e810a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:36.050 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.444+0000 7fd5effaf700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e810ab10 con 0x7fd5e810a0f0 2024-05-04T03:07:36.050 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.445+0000 7fd5edd4b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5e810a0f0 0x7fd5e810a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:36.050 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.445+0000 7fd5edd4b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5e810a0f0 0x7fd5e810a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33866/0 (socket says 172.21.15.1:33866) 2024-05-04T03:07:36.050 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.445+0000 7fd5edd4b700 1 -- 172.21.15.1:0/3827570204 learned_addr learned my addr 172.21.15.1:0/3827570204 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:07:36.050 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.445+0000 7fd5edd4b700 1 -- 172.21.15.1:0/3827570204 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e8108d20 con 0x7fd5e810a0f0 2024-05-04T03:07:36.050 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.445+0000 7fd5edd4b700 1 --2- 172.21.15.1:0/3827570204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5e810a0f0 0x7fd5e810a500 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40098a0 tx=0x7fd5e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ef21da61766503 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:36.050 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.445+0000 7fd5ecd49700 1 -- 172.21.15.1:0/3827570204 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5e4004160 con 0x7fd5e810a0f0 2024-05-04T03:07:36.050 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.445+0000 7fd5ecd49700 1 -- 172.21.15.1:0/3827570204 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd5e4004300 con 0x7fd5e810a0f0 2024-05-04T03:07:36.050 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.445+0000 7fd5ecd49700 1 -- 172.21.15.1:0/3827570204 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5e4011af0 con 0x7fd5e810a0f0 2024-05-04T03:07:36.050 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.446+0000 7fd5effaf700 1 -- 172.21.15.1:0/3827570204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5e810a0f0 msgr2=0x7fd5e810a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:36.051 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.446+0000 7fd5effaf700 1 --2- 172.21.15.1:0/3827570204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5e810a0f0 0x7fd5e810a500 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40098a0 tx=0x7fd5e4009bb0 comp rx=0 tx=0).stop 2024-05-04T03:07:36.051 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.446+0000 7fd5effaf700 1 -- 172.21.15.1:0/3827570204 shutdown_connections 2024-05-04T03:07:36.051 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.446+0000 7fd5effaf700 1 --2- 172.21.15.1:0/3827570204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5e810a0f0 0x7fd5e810a500 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:36.051 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.446+0000 7fd5effaf700 1 -- 172.21.15.1:0/3827570204 >> 172.21.15.1:0/3827570204 conn(0x7fd5e8103900 msgr2=0x7fd5e8105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:36.051 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.446+0000 7fd5effaf700 1 -- 172.21.15.1:0/3827570204 shutdown_connections 2024-05-04T03:07:36.051 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.446+0000 7fd5effaf700 1 -- 172.21.15.1:0/3827570204 wait complete. 2024-05-04T03:07:36.051 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.447+0000 7fd5effaf700 1 Processor -- start 2024-05-04T03:07:36.051 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.447+0000 7fd5effaf700 1 -- start start 2024-05-04T03:07:36.051 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.447+0000 7fd5effaf700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5e810a0f0 0x7fd5e8105af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:36.051 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.447+0000 7fd5effaf700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e810d0e0 con 0x7fd5e810a0f0 2024-05-04T03:07:36.051 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.447+0000 7fd5edd4b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5e810a0f0 0x7fd5e8105af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:36.052 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.447+0000 7fd5edd4b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5e810a0f0 0x7fd5e8105af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33868/0 (socket says 172.21.15.1:33868) 2024-05-04T03:07:36.052 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.447+0000 7fd5edd4b700 1 -- 172.21.15.1:0/2518325041 learned_addr learned my addr 172.21.15.1:0/2518325041 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:07:36.052 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.448+0000 7fd5edd4b700 1 -- 172.21.15.1:0/2518325041 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e8106030 con 0x7fd5e810a0f0 2024-05-04T03:07:36.052 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.448+0000 7fd5edd4b700 1 --2- 172.21.15.1:0/2518325041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5e810a0f0 0x7fd5e8105af0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40117b0 tx=0x7fd5e4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:36.052 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.448+0000 7fd5deffd700 1 -- 172.21.15.1:0/2518325041 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5e40039c0 con 0x7fd5e810a0f0 2024-05-04T03:07:36.052 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.448+0000 7fd5deffd700 1 -- 172.21.15.1:0/2518325041 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd5e4003b60 con 0x7fd5e810a0f0 2024-05-04T03:07:36.052 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.448+0000 7fd5effaf700 1 -- 172.21.15.1:0/2518325041 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e8104280 con 0x7fd5e810a0f0 2024-05-04T03:07:36.052 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.448+0000 7fd5deffd700 1 -- 172.21.15.1:0/2518325041 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5e40196a0 con 0x7fd5e810a0f0 2024-05-04T03:07:36.052 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.448+0000 7fd5effaf700 1 -- 172.21.15.1:0/2518325041 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e8104760 con 0x7fd5e810a0f0 2024-05-04T03:07:36.052 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.448+0000 7fd5deffd700 1 -- 172.21.15.1:0/2518325041 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd5e4019ca0 con 0x7fd5e810a0f0 2024-05-04T03:07:36.052 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.448+0000 7fd5deffd700 1 -- 172.21.15.1:0/2518325041 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd5e4024730 con 0x7fd5e810a0f0 2024-05-04T03:07:36.052 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.449+0000 7fd5effaf700 1 -- 172.21.15.1:0/2518325041 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e8064e20 con 0x7fd5e810a0f0 2024-05-04T03:07:36.052 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.451+0000 7fd5deffd700 1 -- 172.21.15.1:0/2518325041 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fd5e40042a0 con 0x7fd5e810a0f0 2024-05-04T03:07:36.052 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.522+0000 7fd5effaf700 1 -- 172.21.15.1:0/2518325041 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fd5e8197160 con 0x7fd5e810a0f0 2024-05-04T03:07:36.053 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.522+0000 7fd5deffd700 1 -- 172.21.15.1:0/2518325041 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+173 (secure 0 0 0) 0x7fd5e401abb0 con 0x7fd5e810a0f0 2024-05-04T03:07:36.053 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.523+0000 7fd5effaf700 1 -- 172.21.15.1:0/2518325041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5e810a0f0 msgr2=0x7fd5e8105af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:36.053 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.523+0000 7fd5effaf700 1 --2- 172.21.15.1:0/2518325041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5e810a0f0 0x7fd5e8105af0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40117b0 tx=0x7fd5e4011b70 comp rx=0 tx=0).stop 2024-05-04T03:07:36.053 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.523+0000 7fd5effaf700 1 -- 172.21.15.1:0/2518325041 shutdown_connections 2024-05-04T03:07:36.053 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.523+0000 7fd5effaf700 1 --2- 172.21.15.1:0/2518325041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5e810a0f0 0x7fd5e8105af0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:36.053 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.523+0000 7fd5effaf700 1 -- 172.21.15.1:0/2518325041 >> 172.21.15.1:0/2518325041 conn(0x7fd5e8103900 msgr2=0x7fd5e8194650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:36.053 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.523+0000 7fd5effaf700 1 -- 172.21.15.1:0/2518325041 shutdown_connections 2024-05-04T03:07:36.053 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:35.523+0000 7fd5effaf700 1 -- 172.21.15.1:0/2518325041 wait complete. 2024-05-04T03:07:36.053 INFO:teuthology.orchestra.run.smithi001.stdout:Restarting the monitor... 2024-05-04T03:07:36.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:35 smithi001 ceph-mon[29361]: from='client.? 172.21.15.1:0/2518325041' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:07:36.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:36 smithi001 systemd[1]: Stopping Ceph mon.a for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:07:36.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:36 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mon-a[29357]: 2024-05-04T03:07:36.312+0000 7fbe99c12700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:07:36.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:36 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mon-a[29357]: 2024-05-04T03:07:36.312+0000 7fbe99c12700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-04T03:07:36.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:36 smithi001 podman[29608]: 2024-05-04 03:07:36.364005691 +0000 UTC m=+0.149490240 container died 49bba26e3b4b5f1d771115309c9ec0bd035378d83763b5e02673a41aa65eed13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mon-a, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-04T03:07:36.970 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:36 smithi001 podman[29608]: 2024-05-04 03:07:36.737030984 +0000 UTC m=+0.522515538 container cleanup 49bba26e3b4b5f1d771115309c9ec0bd035378d83763b5e02673a41aa65eed13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430) 2024-05-04T03:07:36.971 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:36 smithi001 bash[29608]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mon-a 2024-05-04T03:07:37.305 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:36 smithi001 podman[29622]: 2024-05-04 03:07:36.971165259 +0000 UTC m=+0.598460852 container remove 49bba26e3b4b5f1d771115309c9ec0bd035378d83763b5e02673a41aa65eed13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430) 2024-05-04T03:07:37.305 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 systemd[1]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-04T03:07:37.305 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 systemd[1]: Stopped Ceph mon.a for 4a572aa8-09c3-11ef-bc97-c7b262605968. 2024-05-04T03:07:37.305 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 systemd[1]: Starting Ceph mon.a for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:07:37.558 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 podman[29712]: 2024-05-04 03:07:37.305231628 +0000 UTC m=+0.102178838 container create f52e77ebe41e330bd75fe7f4e8036b1d7589c717670b79ca11530e843113e826 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-04T03:07:37.558 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 podman[29712]: 2024-05-04 03:07:37.221330043 +0000 UTC m=+0.018277267 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-04T03:07:37.558 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 podman[29712]: 2024-05-04 03:07:37.51628768 +0000 UTC m=+0.313234901 container init f52e77ebe41e330bd75fe7f4e8036b1d7589c717670b79ca11530e843113e826 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mon-a, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-04T03:07:37.559 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 podman[29712]: 2024-05-04 03:07:37.519462066 +0000 UTC m=+0.316409276 container start f52e77ebe41e330bd75fe7f4e8036b1d7589c717670b79ca11530e843113e826 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mon-a, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-04T03:07:37.559 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: set uid:gid to 167:167 (ceph:ceph) 2024-05-04T03:07:37.559 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: ceph version 19.0.0-3522-gbd4e00a3 (bd4e00a39d766328b458e4c953f4d722fed96a31) squid (dev), process ceph-mon, pid 2 2024-05-04T03:07:37.559 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: pidfile_write: ignore empty --pid-file 2024-05-04T03:07:37.559 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: load: jerasure load: lrc 2024-05-04T03:07:37.612 INFO:teuthology.orchestra.run.smithi001.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-04T03:07:37.841 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: RocksDB version: 7.9.2 2024-05-04T03:07:37.841 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Git sha 0 2024-05-04T03:07:37.841 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Compile date 2024-05-03 18:51:23 2024-05-04T03:07:37.841 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: DB SUMMARY 2024-05-04T03:07:37.841 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: DB Session ID: 2XFO2AJJ1YN30V1PUD9Y 2024-05-04T03:07:37.842 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: CURRENT file: CURRENT 2024-05-04T03:07:37.842 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: IDENTITY file: IDENTITY 2024-05-04T03:07:37.842 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-04T03:07:37.842 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-04T03:07:37.842 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104976 ; 2024-05-04T03:07:37.842 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.error_if_exists: 0 2024-05-04T03:07:37.842 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.create_if_missing: 0 2024-05-04T03:07:37.842 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.paranoid_checks: 1 2024-05-04T03:07:37.842 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-04T03:07:37.842 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-04T03:07:37.842 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-04T03:07:37.842 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.env: 0x557d94663500 2024-05-04T03:07:37.842 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.fs: PosixFileSystem 2024-05-04T03:07:37.843 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.info_log: 0x557d96c86020 2024-05-04T03:07:37.843 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.max_file_opening_threads: 16 2024-05-04T03:07:37.843 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.statistics: (nil) 2024-05-04T03:07:37.843 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.use_fsync: 0 2024-05-04T03:07:37.843 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.max_log_file_size: 0 2024-05-04T03:07:37.843 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-04T03:07:37.843 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-04T03:07:37.843 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.keep_log_file_num: 1000 2024-05-04T03:07:37.843 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.recycle_log_file_num: 0 2024-05-04T03:07:37.843 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.allow_fallocate: 1 2024-05-04T03:07:37.843 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.allow_mmap_reads: 0 2024-05-04T03:07:37.843 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.allow_mmap_writes: 0 2024-05-04T03:07:37.843 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.use_direct_reads: 0 2024-05-04T03:07:37.844 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-04T03:07:37.844 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.create_missing_column_families: 0 2024-05-04T03:07:37.844 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.db_log_dir: 2024-05-04T03:07:37.844 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.wal_dir: 2024-05-04T03:07:37.844 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-04T03:07:37.844 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-04T03:07:37.844 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-04T03:07:37.844 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-04T03:07:37.845 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-04T03:07:37.845 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-04T03:07:37.846 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.advise_random_on_open: 1 2024-05-04T03:07:37.846 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.db_write_buffer_size: 0 2024-05-04T03:07:37.846 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.write_buffer_manager: 0x557d96b508c0 2024-05-04T03:07:37.846 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-04T03:07:37.846 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-04T03:07:37.846 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-04T03:07:37.846 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.rate_limiter: (nil) 2024-05-04T03:07:37.846 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-04T03:07:37.846 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.wal_recovery_mode: 2 2024-05-04T03:07:37.846 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.enable_thread_tracking: 0 2024-05-04T03:07:37.846 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.enable_pipelined_write: 0 2024-05-04T03:07:37.846 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.unordered_write: 0 2024-05-04T03:07:37.846 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-04T03:07:37.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-04T03:07:37.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-04T03:07:37.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-04T03:07:37.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.row_cache: None 2024-05-04T03:07:37.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.wal_filter: None 2024-05-04T03:07:37.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-04T03:07:37.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.allow_ingest_behind: 0 2024-05-04T03:07:37.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.two_write_queues: 0 2024-05-04T03:07:37.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.manual_wal_flush: 0 2024-05-04T03:07:37.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.wal_compression: 0 2024-05-04T03:07:37.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.atomic_flush: 0 2024-05-04T03:07:37.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-04T03:07:37.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-04T03:07:37.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-04T03:07:37.848 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.log_readahead_size: 0 2024-05-04T03:07:37.848 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-04T03:07:37.848 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.best_efforts_recovery: 0 2024-05-04T03:07:37.848 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-04T03:07:37.848 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-04T03:07:37.848 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.allow_data_in_errors: 0 2024-05-04T03:07:37.848 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.db_host_id: __hostname__ 2024-05-04T03:07:37.848 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-04T03:07:37.848 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.max_background_jobs: 2 2024-05-04T03:07:37.848 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.max_background_compactions: -1 2024-05-04T03:07:37.848 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.max_subcompactions: 1 2024-05-04T03:07:37.848 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-04T03:07:37.848 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-04T03:07:37.848 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-04T03:07:37.848 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.max_total_wal_size: 0 2024-05-04T03:07:37.848 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-04T03:07:37.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-04T03:07:37.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-04T03:07:37.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-04T03:07:37.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.max_open_files: -1 2024-05-04T03:07:37.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.bytes_per_sync: 0 2024-05-04T03:07:37.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-04T03:07:37.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-04T03:07:37.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.compaction_readahead_size: 0 2024-05-04T03:07:37.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.max_background_flushes: -1 2024-05-04T03:07:37.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Compression algorithms supported: 2024-05-04T03:07:37.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-04T03:07:37.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: kZSTD supported: 0 2024-05-04T03:07:37.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: kXpressCompression supported: 0 2024-05-04T03:07:37.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: kLZ4HCCompression supported: 1 2024-05-04T03:07:37.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: kZlibCompression supported: 1 2024-05-04T03:07:37.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: kSnappyCompression supported: 1 2024-05-04T03:07:37.849 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: kLZ4Compression supported: 1 2024-05-04T03:07:37.850 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: kBZip2Compression supported: 0 2024-05-04T03:07:37.850 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-04T03:07:37.850 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-04T03:07:37.850 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-04T03:07:37.850 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-04T03:07:37.850 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-04T03:07:37.850 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.merge_operator: 2024-05-04T03:07:37.850 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.compaction_filter: None 2024-05-04T03:07:37.850 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.compaction_filter_factory: None 2024-05-04T03:07:37.850 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.sst_partitioner_factory: None 2024-05-04T03:07:37.850 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-04T03:07:37.850 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-04T03:07:37.850 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557d96c861e0) 2024-05-04T03:07:37.850 INFO:journalctl@ceph.mon.a.smithi001.stdout: cache_index_and_filter_blocks: 1 2024-05-04T03:07:37.850 INFO:journalctl@ceph.mon.a.smithi001.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-04T03:07:37.851 INFO:journalctl@ceph.mon.a.smithi001.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-04T03:07:37.851 INFO:journalctl@ceph.mon.a.smithi001.stdout: pin_top_level_index_and_filter: 1 2024-05-04T03:07:37.851 INFO:journalctl@ceph.mon.a.smithi001.stdout: index_type: 0 2024-05-04T03:07:37.851 INFO:journalctl@ceph.mon.a.smithi001.stdout: data_block_index_type: 0 2024-05-04T03:07:37.851 INFO:journalctl@ceph.mon.a.smithi001.stdout: index_shortening: 1 2024-05-04T03:07:37.851 INFO:journalctl@ceph.mon.a.smithi001.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-04T03:07:37.851 INFO:journalctl@ceph.mon.a.smithi001.stdout: checksum: 4 2024-05-04T03:07:37.851 INFO:journalctl@ceph.mon.a.smithi001.stdout: no_block_cache: 0 2024-05-04T03:07:37.851 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_cache: 0x557d96b831f0 2024-05-04T03:07:37.851 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_cache_name: BinnedLRUCache 2024-05-04T03:07:37.851 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_cache_options: 2024-05-04T03:07:37.851 INFO:journalctl@ceph.mon.a.smithi001.stdout: capacity : 536870912 2024-05-04T03:07:37.851 INFO:journalctl@ceph.mon.a.smithi001.stdout: num_shard_bits : 4 2024-05-04T03:07:37.851 INFO:journalctl@ceph.mon.a.smithi001.stdout: strict_capacity_limit : 0 2024-05-04T03:07:37.851 INFO:journalctl@ceph.mon.a.smithi001.stdout: high_pri_pool_ratio: 0.000 2024-05-04T03:07:37.851 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_cache_compressed: (nil) 2024-05-04T03:07:37.852 INFO:journalctl@ceph.mon.a.smithi001.stdout: persistent_cache: (nil) 2024-05-04T03:07:37.852 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_size: 4096 2024-05-04T03:07:37.852 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_size_deviation: 10 2024-05-04T03:07:37.852 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_restart_interval: 16 2024-05-04T03:07:37.852 INFO:journalctl@ceph.mon.a.smithi001.stdout: index_block_restart_interval: 1 2024-05-04T03:07:37.852 INFO:journalctl@ceph.mon.a.smithi001.stdout: metadata_block_size: 4096 2024-05-04T03:07:37.852 INFO:journalctl@ceph.mon.a.smithi001.stdout: partition_filters: 0 2024-05-04T03:07:37.852 INFO:journalctl@ceph.mon.a.smithi001.stdout: use_delta_encoding: 1 2024-05-04T03:07:37.852 INFO:journalctl@ceph.mon.a.smithi001.stdout: filter_policy: bloomfilter 2024-05-04T03:07:37.852 INFO:journalctl@ceph.mon.a.smithi001.stdout: whole_key_filtering: 1 2024-05-04T03:07:37.852 INFO:journalctl@ceph.mon.a.smithi001.stdout: verify_compression: 0 2024-05-04T03:07:37.852 INFO:journalctl@ceph.mon.a.smithi001.stdout: read_amp_bytes_per_bit: 0 2024-05-04T03:07:37.852 INFO:journalctl@ceph.mon.a.smithi001.stdout: format_version: 5 2024-05-04T03:07:37.852 INFO:journalctl@ceph.mon.a.smithi001.stdout: enable_index_compression: 1 2024-05-04T03:07:37.852 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_align: 0 2024-05-04T03:07:37.852 INFO:journalctl@ceph.mon.a.smithi001.stdout: max_auto_readahead_size: 262144 2024-05-04T03:07:37.853 INFO:journalctl@ceph.mon.a.smithi001.stdout: prepopulate_block_cache: 0 2024-05-04T03:07:37.853 INFO:journalctl@ceph.mon.a.smithi001.stdout: initial_auto_readahead_size: 8192 2024-05-04T03:07:37.853 INFO:journalctl@ceph.mon.a.smithi001.stdout: num_file_reads_for_auto_readahead: 2 2024-05-04T03:07:37.853 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.write_buffer_size: 33554432 2024-05-04T03:07:37.853 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.max_write_buffer_number: 2 2024-05-04T03:07:37.853 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.compression: NoCompression 2024-05-04T03:07:37.853 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.bottommost_compression: Disabled 2024-05-04T03:07:37.853 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.prefix_extractor: nullptr 2024-05-04T03:07:37.853 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-04T03:07:37.853 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.num_levels: 7 2024-05-04T03:07:37.853 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-04T03:07:37.853 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-04T03:07:37.853 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-04T03:07:37.853 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-04T03:07:37.853 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-04T03:07:37.853 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-04T03:07:37.853 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-04T03:07:37.854 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-04T03:07:37.854 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-04T03:07:37.854 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-04T03:07:37.854 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-04T03:07:37.854 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-04T03:07:37.854 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-04T03:07:37.854 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.compression_opts.level: 32767 2024-05-04T03:07:37.854 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.compression_opts.strategy: 0 2024-05-04T03:07:37.854 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-04T03:07:37.854 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-04T03:07:37.854 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-04T03:07:37.854 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-04T03:07:37.854 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.compression_opts.enabled: false 2024-05-04T03:07:37.855 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-04T03:07:37.855 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-04T03:07:37.855 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-04T03:07:37.855 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-04T03:07:37.855 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.target_file_size_base: 67108864 2024-05-04T03:07:37.855 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-04T03:07:37.855 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-04T03:07:37.855 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-04T03:07:37.855 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-04T03:07:37.855 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-04T03:07:37.855 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-04T03:07:37.855 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-04T03:07:37.855 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-04T03:07:37.855 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-04T03:07:37.855 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-04T03:07:37.856 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-04T03:07:37.856 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-04T03:07:37.856 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-04T03:07:37.856 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-04T03:07:37.856 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.arena_block_size: 1048576 2024-05-04T03:07:37.856 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-04T03:07:37.856 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-04T03:07:37.856 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.disable_auto_compactions: 0 2024-05-04T03:07:37.856 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-04T03:07:37.856 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-04T03:07:37.856 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-04T03:07:37.856 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-04T03:07:37.856 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-04T03:07:37.856 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-04T03:07:37.856 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-04T03:07:37.856 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-04T03:07:37.856 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-04T03:07:37.857 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-04T03:07:37.857 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-04T03:07:37.857 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.inplace_update_support: 0 2024-05-04T03:07:37.857 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-04T03:07:37.857 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-04T03:07:37.857 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-04T03:07:37.857 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-04T03:07:37.857 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.bloom_locality: 0 2024-05-04T03:07:37.857 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.max_successive_merges: 0 2024-05-04T03:07:37.857 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-04T03:07:37.857 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.paranoid_file_checks: 0 2024-05-04T03:07:37.857 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.force_consistency_checks: 1 2024-05-04T03:07:37.857 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.report_bg_io_stats: 0 2024-05-04T03:07:37.857 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.ttl: 2592000 2024-05-04T03:07:37.857 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-04T03:07:37.857 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-04T03:07:37.858 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-04T03:07:37.858 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.enable_blob_files: false 2024-05-04T03:07:37.858 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.min_blob_size: 0 2024-05-04T03:07:37.858 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.blob_file_size: 268435456 2024-05-04T03:07:37.858 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-04T03:07:37.858 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-04T03:07:37.858 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-04T03:07:37.858 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-04T03:07:37.858 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-04T03:07:37.858 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.blob_file_starting_level: 0 2024-05-04T03:07:37.858 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-04T03:07:37.858 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-04T03:07:37.858 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-04T03:07:37.858 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e1e6581d-5b0a-42b1-a397-4deef68e8644 2024-05-04T03:07:37.858 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714792057561179, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-04T03:07:37.859 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-04T03:07:37.859 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714792057562315, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100153, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 98037, "index_size": 308, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14161, "raw_average_key_size": 49, "raw_value_size": 90320, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714792057, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e1e6581d-5b0a-42b1-a397-4deef68e8644", "db_session_id": "2XFO2AJJ1YN30V1PUD9Y", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-04T03:07:37.859 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714792057562398, "job": 1, "event": "recovery_finished"} 2024-05-04T03:07:37.859 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-04T03:07:37.859 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-04T03:07:37.859 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557d96b67c00 2024-05-04T03:07:37.859 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: DB pointer 0x557d9b95a000 2024-05-04T03:07:37.859 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-04T03:07:37.859 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-04T03:07:37.859 INFO:journalctl@ceph.mon.a.smithi001.stdout: ** DB Stats ** 2024-05-04T03:07:37.859 INFO:journalctl@ceph.mon.a.smithi001.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-04T03:07:37.859 INFO:journalctl@ceph.mon.a.smithi001.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-04T03:07:37.859 INFO:journalctl@ceph.mon.a.smithi001.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-04T03:07:37.859 INFO:journalctl@ceph.mon.a.smithi001.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-04T03:07:37.859 INFO:journalctl@ceph.mon.a.smithi001.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-04T03:07:37.860 INFO:journalctl@ceph.mon.a.smithi001.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-04T03:07:37.860 INFO:journalctl@ceph.mon.a.smithi001.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-04T03:07:37.860 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-05-04T03:07:37.860 INFO:journalctl@ceph.mon.a.smithi001.stdout: ** Compaction Stats [default] ** 2024-05-04T03:07:37.860 INFO:journalctl@ceph.mon.a.smithi001.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-04T03:07:37.860 INFO:journalctl@ceph.mon.a.smithi001.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-04T03:07:37.860 INFO:journalctl@ceph.mon.a.smithi001.stdout: L0 2/0 99.67 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-04T03:07:37.860 INFO:journalctl@ceph.mon.a.smithi001.stdout: Sum 2/0 99.67 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-04T03:07:37.860 INFO:journalctl@ceph.mon.a.smithi001.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-04T03:07:37.860 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-05-04T03:07:37.860 INFO:journalctl@ceph.mon.a.smithi001.stdout: ** Compaction Stats [default] ** 2024-05-04T03:07:37.860 INFO:journalctl@ceph.mon.a.smithi001.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-04T03:07:37.860 INFO:journalctl@ceph.mon.a.smithi001.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-04T03:07:37.860 INFO:journalctl@ceph.mon.a.smithi001.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 150.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-04T03:07:37.860 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-05-04T03:07:37.860 INFO:journalctl@ceph.mon.a.smithi001.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-04T03:07:37.861 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-05-04T03:07:37.861 INFO:journalctl@ceph.mon.a.smithi001.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-04T03:07:37.861 INFO:journalctl@ceph.mon.a.smithi001.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-04T03:07:37.861 INFO:journalctl@ceph.mon.a.smithi001.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-04T03:07:37.861 INFO:journalctl@ceph.mon.a.smithi001.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-04T03:07:37.861 INFO:journalctl@ceph.mon.a.smithi001.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-04T03:07:37.861 INFO:journalctl@ceph.mon.a.smithi001.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-04T03:07:37.861 INFO:journalctl@ceph.mon.a.smithi001.stdout: Cumulative compaction: 0.00 GB write, 28.73 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-04T03:07:37.861 INFO:journalctl@ceph.mon.a.smithi001.stdout: Interval compaction: 0.00 GB write, 28.73 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-04T03:07:37.861 INFO:journalctl@ceph.mon.a.smithi001.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-04T03:07:37.861 INFO:journalctl@ceph.mon.a.smithi001.stdout: Block cache BinnedLRUCache@0x557d96b831f0#2 capacity: 512.00 MB usage: 1.53 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-04T03:07:37.861 INFO:journalctl@ceph.mon.a.smithi001.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.03 KB,0.000196695%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-04T03:07:37.861 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-05-04T03:07:37.861 INFO:journalctl@ceph.mon.a.smithi001.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-04T03:07:37.861 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: starting mon.a rank 0 at public addrs [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] at bind addrs [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:07:37.861 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: mon.a@-1(???) e1 preinit fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:07:37.861 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: mon.a@-1(???).mds e1 new map 2024-05-04T03:07:37.862 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: mon.a@-1(???).mds e1 print_map 2024-05-04T03:07:37.862 INFO:journalctl@ceph.mon.a.smithi001.stdout: e1 2024-05-04T03:07:37.862 INFO:journalctl@ceph.mon.a.smithi001.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-04T03:07:37.862 INFO:journalctl@ceph.mon.a.smithi001.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-04T03:07:37.862 INFO:journalctl@ceph.mon.a.smithi001.stdout: legacy client fscid: -1 2024-05-04T03:07:37.862 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-05-04T03:07:37.862 INFO:journalctl@ceph.mon.a.smithi001.stdout: No filesystems configured 2024-05-04T03:07:37.862 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-04T03:07:37.862 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-04T03:07:37.862 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-04T03:07:37.862 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-04T03:07:37.862 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-04T03:07:37.862 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 bash[29712]: f52e77ebe41e330bd75fe7f4e8036b1d7589c717670b79ca11530e843113e826 2024-05-04T03:07:37.862 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 systemd[1]: Started Ceph mon.a for 4a572aa8-09c3-11ef-bc97-c7b262605968. 2024-05-04T03:07:37.863 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-04T03:07:37.863 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: monmap epoch 1 2024-05-04T03:07:37.863 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:07:37.863 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: last_changed 2024-05-04T03:07:29.984656+0000 2024-05-04T03:07:37.863 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: created 2024-05-04T03:07:29.984656+0000 2024-05-04T03:07:37.863 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: min_mon_release 19 (squid) 2024-05-04T03:07:37.863 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: election_strategy: 1 2024-05-04T03:07:37.863 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-05-04T03:07:37.863 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: fsmap 2024-05-04T03:07:37.863 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: osdmap e1: 0 total, 0 up, 0 in 2024-05-04T03:07:37.863 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:37 smithi001 ceph-mon[29731]: mgrmap e1: no daemons active 2024-05-04T03:07:38.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.052+0000 7fbd66739700 1 Processor -- start 2024-05-04T03:07:38.644 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.052+0000 7fbd66739700 1 -- start start 2024-05-04T03:07:38.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.052+0000 7fbd66739700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd60109c40 0x7fbd6010a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:38.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.052+0000 7fbd66739700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6010a640 con 0x7fbd60109c40 2024-05-04T03:07:38.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.052+0000 7fbd5ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd60109c40 0x7fbd6010a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:38.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.053+0000 7fbd5ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd60109c40 0x7fbd6010a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33876/0 (socket says 172.21.15.1:33876) 2024-05-04T03:07:38.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.053+0000 7fbd5ffff700 1 -- 172.21.15.1:0/4240962566 learned_addr learned my addr 172.21.15.1:0/4240962566 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:07:38.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.053+0000 7fbd5ffff700 1 -- 172.21.15.1:0/4240962566 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd6010ae10 con 0x7fbd60109c40 2024-05-04T03:07:38.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.053+0000 7fbd5ffff700 1 --2- 172.21.15.1:0/4240962566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd60109c40 0x7fbd6010a050 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbd500098a0 tx=0x7fbd50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83dc7481ed5a44bb server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:38.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.053+0000 7fbd5effd700 1 -- 172.21.15.1:0/4240962566 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd50004160 con 0x7fbd60109c40 2024-05-04T03:07:38.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.053+0000 7fbd5effd700 1 -- 172.21.15.1:0/4240962566 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbd50004300 con 0x7fbd60109c40 2024-05-04T03:07:38.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.053+0000 7fbd5effd700 1 -- 172.21.15.1:0/4240962566 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd50011af0 con 0x7fbd60109c40 2024-05-04T03:07:38.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.054+0000 7fbd66739700 1 -- 172.21.15.1:0/4240962566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd60109c40 msgr2=0x7fbd6010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:38.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.054+0000 7fbd66739700 1 --2- 172.21.15.1:0/4240962566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd60109c40 0x7fbd6010a050 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbd500098a0 tx=0x7fbd50009bb0 comp rx=0 tx=0).stop 2024-05-04T03:07:38.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.054+0000 7fbd66739700 1 -- 172.21.15.1:0/4240962566 shutdown_connections 2024-05-04T03:07:38.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.054+0000 7fbd66739700 1 --2- 172.21.15.1:0/4240962566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd60109c40 0x7fbd6010a050 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:38.645 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.054+0000 7fbd66739700 1 -- 172.21.15.1:0/4240962566 >> 172.21.15.1:0/4240962566 conn(0x7fbd601052c0 msgr2=0x7fbd60107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:38.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.054+0000 7fbd66739700 1 -- 172.21.15.1:0/4240962566 shutdown_connections 2024-05-04T03:07:38.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.054+0000 7fbd66739700 1 -- 172.21.15.1:0/4240962566 wait complete. 2024-05-04T03:07:38.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.055+0000 7fbd66739700 1 Processor -- start 2024-05-04T03:07:38.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.055+0000 7fbd66739700 1 -- start start 2024-05-04T03:07:38.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.055+0000 7fbd66739700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd60109c40 0x7fbd6019e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:38.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.055+0000 7fbd66739700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6010e1c0 con 0x7fbd60109c40 2024-05-04T03:07:38.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.055+0000 7fbd5ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd60109c40 0x7fbd6019e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:38.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.055+0000 7fbd5ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd60109c40 0x7fbd6019e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33882/0 (socket says 172.21.15.1:33882) 2024-05-04T03:07:38.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.055+0000 7fbd5ffff700 1 -- 172.21.15.1:0/21087686 learned_addr learned my addr 172.21.15.1:0/21087686 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:07:38.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.055+0000 7fbd5ffff700 1 -- 172.21.15.1:0/21087686 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd6019e690 con 0x7fbd60109c40 2024-05-04T03:07:38.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.056+0000 7fbd5ffff700 1 --2- 172.21.15.1:0/21087686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd60109c40 0x7fbd6019e150 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbd500090e0 tx=0x7fbd50011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:38.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.056+0000 7fbd5d7fa700 1 -- 172.21.15.1:0/21087686 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd500039e0 con 0x7fbd60109c40 2024-05-04T03:07:38.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.056+0000 7fbd5d7fa700 1 -- 172.21.15.1:0/21087686 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbd50003b80 con 0x7fbd60109c40 2024-05-04T03:07:38.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.056+0000 7fbd5d7fa700 1 -- 172.21.15.1:0/21087686 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd50024690 con 0x7fbd60109c40 2024-05-04T03:07:38.646 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.056+0000 7fbd66739700 1 -- 172.21.15.1:0/21087686 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd6019e920 con 0x7fbd60109c40 2024-05-04T03:07:38.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.056+0000 7fbd66739700 1 -- 172.21.15.1:0/21087686 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd601a1530 con 0x7fbd60109c40 2024-05-04T03:07:38.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.056+0000 7fbd5d7fa700 1 -- 172.21.15.1:0/21087686 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbd50022020 con 0x7fbd60109c40 2024-05-04T03:07:38.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.056+0000 7fbd5d7fa700 1 -- 172.21.15.1:0/21087686 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbd5002dbe0 con 0x7fbd60109c40 2024-05-04T03:07:38.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.057+0000 7fbd66739700 1 -- 172.21.15.1:0/21087686 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd60064e20 con 0x7fbd60109c40 2024-05-04T03:07:38.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.060+0000 7fbd5d7fa700 1 -- 172.21.15.1:0/21087686 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fbd5002dde0 con 0x7fbd60109c40 2024-05-04T03:07:38.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.130+0000 7fbd66739700 1 -- 172.21.15.1:0/21087686 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fbd601a1840 con 0x7fbd60109c40 2024-05-04T03:07:38.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.132+0000 7fbd5d7fa700 1 -- 172.21.15.1:0/21087686 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fbd5001a9c0 con 0x7fbd60109c40 2024-05-04T03:07:38.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.133+0000 7fbd66739700 1 -- 172.21.15.1:0/21087686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd60109c40 msgr2=0x7fbd6019e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:38.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.133+0000 7fbd66739700 1 --2- 172.21.15.1:0/21087686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd60109c40 0x7fbd6019e150 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbd500090e0 tx=0x7fbd50011940 comp rx=0 tx=0).stop 2024-05-04T03:07:38.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.133+0000 7fbd66739700 1 -- 172.21.15.1:0/21087686 shutdown_connections 2024-05-04T03:07:38.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.133+0000 7fbd66739700 1 --2- 172.21.15.1:0/21087686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd60109c40 0x7fbd6019e150 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:38.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.133+0000 7fbd66739700 1 -- 172.21.15.1:0/21087686 >> 172.21.15.1:0/21087686 conn(0x7fbd601052c0 msgr2=0x7fbd60105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:38.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.133+0000 7fbd66739700 1 -- 172.21.15.1:0/21087686 shutdown_connections 2024-05-04T03:07:38.647 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:38.133+0000 7fbd66739700 1 -- 172.21.15.1:0/21087686 wait complete. 2024-05-04T03:07:38.648 INFO:teuthology.orchestra.run.smithi001.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-04T03:07:38.663 INFO:teuthology.orchestra.run.smithi001.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-04T03:07:38.663 INFO:teuthology.orchestra.run.smithi001.stdout:Creating mgr... 2024-05-04T03:07:38.663 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-04T03:07:38.664 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-04T03:07:38.915 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mgr.x 2024-05-04T03:07:38.915 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Failed to reset failed state of unit ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mgr.x.service: Unit ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mgr.x.service not loaded. 2024-05-04T03:07:39.067 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-4a572aa8-09c3-11ef-bc97-c7b262605968.target.wants/ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@.service. 2024-05-04T03:07:39.299 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:39 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/21087686' entity='client.admin' 2024-05-04T03:07:39.601 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:39 smithi001 podman[29950]: 2024-05-04 03:07:39.520802378 +0000 UTC m=+0.329233090 container start 5918334689e48eb373233fd1754c9eb2f79e4cb4d013f157ca7aa3272a080a18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-04T03:07:39.614 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-04T03:07:39.614 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout disabled 2024-05-04T03:07:39.622 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-04T03:07:39.622 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-05-04T03:07:39.622 INFO:teuthology.orchestra.run.smithi001.stdout:firewalld.service is not enabled 2024-05-04T03:07:39.622 INFO:teuthology.orchestra.run.smithi001.stdout:Not possible to enable service . firewalld.service is not available 2024-05-04T03:07:39.630 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-04T03:07:39.631 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout disabled 2024-05-04T03:07:39.638 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-04T03:07:39.638 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-05-04T03:07:39.638 INFO:teuthology.orchestra.run.smithi001.stdout:firewalld.service is not enabled 2024-05-04T03:07:39.639 INFO:teuthology.orchestra.run.smithi001.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-04T03:07:39.639 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mgr to start... 2024-05-04T03:07:39.639 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mgr... 2024-05-04T03:07:39.855 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:39 smithi001 bash[29950]: 5918334689e48eb373233fd1754c9eb2f79e4cb4d013f157ca7aa3272a080a18 2024-05-04T03:07:39.855 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:39 smithi001 systemd[1]: Started Ceph mgr.x for 4a572aa8-09c3-11ef-bc97-c7b262605968. 2024-05-04T03:07:39.855 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:39 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:39.690+0000 7f6e6998e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-04T03:07:40.128 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:39 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:39.886+0000 7f6e6998e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-04T03:07:40.128 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:39 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:39.992+0000 7f6e6998e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-04T03:07:40.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:40 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/4258961441' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-04T03:07:40.628 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:40 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:40.329+0000 7f6e6998e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-04T03:07:40.628 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:40 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:40.417+0000 7f6e6998e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-04T03:07:40.794 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-05-04T03:07:40.803 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-05-04T03:07:40.804 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsid": "4a572aa8-09c3-11ef-bc97-c7b262605968", 2024-05-04T03:07:40.804 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "health": { 2024-05-04T03:07:40.804 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-04T03:07:40.804 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-04T03:07:40.804 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-04T03:07:40.804 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-05-04T03:07:40.804 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-04T03:07:40.804 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-04T03:07:40.804 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 0 2024-05-04T03:07:40.804 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-05-04T03:07:40.804 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-04T03:07:40.804 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "a" 2024-05-04T03:07:40.804 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-05-04T03:07:40.804 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-04T03:07:40.804 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-04T03:07:40.805 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-04T03:07:40.805 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-04T03:07:40.805 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-04T03:07:40.805 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-05-04T03:07:40.805 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-04T03:07:40.805 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-04T03:07:40.805 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-04T03:07:40.805 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-04T03:07:40.805 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-04T03:07:40.805 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-04T03:07:40.805 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-04T03:07:40.805 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-04T03:07:40.805 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-05-04T03:07:40.805 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-04T03:07:40.806 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-04T03:07:40.806 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-04T03:07:40.806 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-04T03:07:40.806 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-04T03:07:40.806 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-04T03:07:40.806 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-04T03:07:40.806 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-04T03:07:40.806 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-04T03:07:40.806 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-05-04T03:07:40.806 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-04T03:07:40.806 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-04T03:07:40.806 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-04T03:07:40.806 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-04T03:07:40.806 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-05-04T03:07:40.807 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-04T03:07:40.807 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-04T03:07:40.807 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-04T03:07:40.807 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-04T03:07:40.807 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-04T03:07:40.807 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-04T03:07:40.807 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "restful" 2024-05-04T03:07:40.807 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-05-04T03:07:40.807 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-04T03:07:40.807 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-05-04T03:07:40.807 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-04T03:07:40.807 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-04T03:07:40.807 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modified": "2024-05-04T03:07:32.846160+0000", 2024-05-04T03:07:40.807 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-04T03:07:40.807 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-05-04T03:07:40.807 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-04T03:07:40.807 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-05-04T03:07:40.808 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.084+0000 7f9e05c70700 1 Processor -- start 2024-05-04T03:07:40.808 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.084+0000 7f9e05c70700 1 -- start start 2024-05-04T03:07:40.808 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.084+0000 7f9e05c70700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e0010a170 0x7f9e0010a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:40.808 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.084+0000 7f9e05c70700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0010ab50 con 0x7f9e0010a170 2024-05-04T03:07:40.808 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.085+0000 7f9dff7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e0010a170 0x7f9e0010a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:40.808 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.085+0000 7f9dff7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e0010a170 0x7f9e0010a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33910/0 (socket says 172.21.15.1:33910) 2024-05-04T03:07:40.808 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.085+0000 7f9dff7fe700 1 -- 172.21.15.1:0/2124384849 learned_addr learned my addr 172.21.15.1:0/2124384849 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:07:40.808 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.085+0000 7f9dff7fe700 1 -- 172.21.15.1:0/2124384849 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e00108df0 con 0x7f9e0010a170 2024-05-04T03:07:40.808 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.085+0000 7f9dff7fe700 1 --2- 172.21.15.1:0/2124384849 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e0010a170 0x7f9e0010a580 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9df00098a0 tx=0x7f9df0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c854b972c3669093 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:40.808 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.085+0000 7f9dfe7fc700 1 -- 172.21.15.1:0/2124384849 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9df0004160 con 0x7f9e0010a170 2024-05-04T03:07:40.808 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.085+0000 7f9dfe7fc700 1 -- 172.21.15.1:0/2124384849 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9df0004300 con 0x7f9e0010a170 2024-05-04T03:07:40.808 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.085+0000 7f9dfe7fc700 1 -- 172.21.15.1:0/2124384849 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9df0011af0 con 0x7f9e0010a170 2024-05-04T03:07:40.808 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.086+0000 7f9e05c70700 1 -- 172.21.15.1:0/2124384849 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e0010a170 msgr2=0x7f9e0010a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:40.808 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.086+0000 7f9e05c70700 1 --2- 172.21.15.1:0/2124384849 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e0010a170 0x7f9e0010a580 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9df00098a0 tx=0x7f9df0009bb0 comp rx=0 tx=0).stop 2024-05-04T03:07:40.809 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.086+0000 7f9e05c70700 1 -- 172.21.15.1:0/2124384849 shutdown_connections 2024-05-04T03:07:40.809 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.086+0000 7f9e05c70700 1 --2- 172.21.15.1:0/2124384849 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e0010a170 0x7f9e0010a580 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:40.809 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.086+0000 7f9e05c70700 1 -- 172.21.15.1:0/2124384849 >> 172.21.15.1:0/2124384849 conn(0x7f9e00103930 msgr2=0x7f9e00105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:40.809 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.086+0000 7f9e05c70700 1 -- 172.21.15.1:0/2124384849 shutdown_connections 2024-05-04T03:07:40.809 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.086+0000 7f9e05c70700 1 -- 172.21.15.1:0/2124384849 wait complete. 2024-05-04T03:07:40.809 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.087+0000 7f9e05c70700 1 Processor -- start 2024-05-04T03:07:40.809 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.087+0000 7f9e05c70700 1 -- start start 2024-05-04T03:07:40.809 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.088+0000 7f9e05c70700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e0010a170 0x7f9e00105b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:40.809 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.088+0000 7f9e05c70700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0010d1b0 con 0x7f9e0010a170 2024-05-04T03:07:40.809 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.088+0000 7f9dff7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e0010a170 0x7f9e00105b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:40.809 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.088+0000 7f9dff7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e0010a170 0x7f9e00105b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33922/0 (socket says 172.21.15.1:33922) 2024-05-04T03:07:40.809 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.088+0000 7f9dff7fe700 1 -- 172.21.15.1:0/4258961441 learned_addr learned my addr 172.21.15.1:0/4258961441 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:07:40.809 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.088+0000 7f9dff7fe700 1 -- 172.21.15.1:0/4258961441 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e00106080 con 0x7f9e0010a170 2024-05-04T03:07:40.809 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.088+0000 7f9dff7fe700 1 --2- 172.21.15.1:0/4258961441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e0010a170 0x7f9e00105b40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9df0011890 tx=0x7f9df0011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:40.809 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.088+0000 7f9dfcff9700 1 -- 172.21.15.1:0/4258961441 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9df00039e0 con 0x7f9e0010a170 2024-05-04T03:07:40.809 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.088+0000 7f9dfcff9700 1 -- 172.21.15.1:0/4258961441 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9df0003b80 con 0x7f9e0010a170 2024-05-04T03:07:40.810 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.088+0000 7f9dfcff9700 1 -- 172.21.15.1:0/4258961441 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9df0024750 con 0x7f9e0010a170 2024-05-04T03:07:40.810 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.088+0000 7f9e05c70700 1 -- 172.21.15.1:0/4258961441 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e001042d0 con 0x7f9e0010a170 2024-05-04T03:07:40.810 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.089+0000 7f9e05c70700 1 -- 172.21.15.1:0/4258961441 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e001047b0 con 0x7f9e0010a170 2024-05-04T03:07:40.810 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.089+0000 7f9dfcff9700 1 -- 172.21.15.1:0/4258961441 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9df0022020 con 0x7f9e0010a170 2024-05-04T03:07:40.810 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.089+0000 7f9dfcff9700 1 -- 172.21.15.1:0/4258961441 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9df002dd90 con 0x7f9e0010a170 2024-05-04T03:07:40.810 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.089+0000 7f9e05c70700 1 -- 172.21.15.1:0/4258961441 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e00064e20 con 0x7f9e0010a170 2024-05-04T03:07:40.810 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.092+0000 7f9dfcff9700 1 -- 172.21.15.1:0/4258961441 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f9df001ab50 con 0x7f9e0010a170 2024-05-04T03:07:40.810 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.168+0000 7f9e05c70700 1 -- 172.21.15.1:0/4258961441 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9e00197250 con 0x7f9e0010a170 2024-05-04T03:07:40.810 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.168+0000 7f9dfcff9700 1 -- 172.21.15.1:0/4258961441 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f9df002d3f0 con 0x7f9e0010a170 2024-05-04T03:07:40.810 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.169+0000 7f9e05c70700 1 -- 172.21.15.1:0/4258961441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e0010a170 msgr2=0x7f9e00105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:40.810 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.169+0000 7f9e05c70700 1 --2- 172.21.15.1:0/4258961441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e0010a170 0x7f9e00105b40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9df0011890 tx=0x7f9df0011970 comp rx=0 tx=0).stop 2024-05-04T03:07:40.810 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.169+0000 7f9e05c70700 1 -- 172.21.15.1:0/4258961441 shutdown_connections 2024-05-04T03:07:40.810 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.169+0000 7f9e05c70700 1 --2- 172.21.15.1:0/4258961441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e0010a170 0x7f9e00105b40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:40.810 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.169+0000 7f9e05c70700 1 -- 172.21.15.1:0/4258961441 >> 172.21.15.1:0/4258961441 conn(0x7f9e00103930 msgr2=0x7f9e001946b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:40.810 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.169+0000 7f9e05c70700 1 -- 172.21.15.1:0/4258961441 shutdown_connections 2024-05-04T03:07:40.810 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:40.169+0000 7f9e05c70700 1 -- 172.21.15.1:0/4258961441 wait complete. 2024-05-04T03:07:40.811 INFO:teuthology.orchestra.run.smithi001.stdout:mgr not available, waiting (1/15)... 2024-05-04T03:07:42.036 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:41 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:41.684+0000 7f6e6998e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-04T03:07:42.036 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:41 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:41.859+0000 7f6e6998e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-04T03:07:42.036 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:42 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:42.035+0000 7f6e6998e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-04T03:07:42.628 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:42 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:42.333+0000 7f6e6998e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-04T03:07:42.628 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:42 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:42.428+0000 7f6e6998e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-04T03:07:42.925 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:42 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:42.638+0000 7f6e6998e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-04T03:07:42.925 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:42 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:42.839+0000 7f6e6998e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-04T03:07:43.478 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:43 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/866035904' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-04T03:07:43.878 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:43 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:43.477+0000 7f6e6998e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-04T03:07:43.878 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:43 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:43.708+0000 7f6e6998e200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-04T03:07:43.925 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-05-04T03:07:43.925 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-05-04T03:07:43.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsid": "4a572aa8-09c3-11ef-bc97-c7b262605968", 2024-05-04T03:07:43.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "health": { 2024-05-04T03:07:43.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-04T03:07:43.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-04T03:07:43.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-04T03:07:43.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-05-04T03:07:43.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-04T03:07:43.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-04T03:07:43.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 0 2024-05-04T03:07:43.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-05-04T03:07:43.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-04T03:07:43.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "a" 2024-05-04T03:07:43.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-05-04T03:07:43.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-04T03:07:43.927 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-04T03:07:43.927 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-04T03:07:43.927 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-04T03:07:43.927 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-04T03:07:43.927 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-05-04T03:07:43.927 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-04T03:07:43.927 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-04T03:07:43.935 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-04T03:07:43.935 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-04T03:07:43.935 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-04T03:07:43.935 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-04T03:07:43.935 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-04T03:07:43.935 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-04T03:07:43.935 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-05-04T03:07:43.935 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-04T03:07:43.935 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-04T03:07:43.935 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-04T03:07:43.936 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-04T03:07:43.936 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-04T03:07:43.936 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-04T03:07:43.936 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-04T03:07:43.936 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-04T03:07:43.936 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-04T03:07:43.936 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-05-04T03:07:43.936 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-04T03:07:43.936 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-04T03:07:43.936 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-04T03:07:43.936 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-04T03:07:43.936 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-05-04T03:07:43.936 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-04T03:07:43.936 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-04T03:07:43.936 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-04T03:07:43.936 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-04T03:07:43.937 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-04T03:07:43.937 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-04T03:07:43.937 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "restful" 2024-05-04T03:07:43.937 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-05-04T03:07:43.937 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-04T03:07:43.937 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-05-04T03:07:43.937 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-04T03:07:43.937 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-04T03:07:43.937 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modified": "2024-05-04T03:07:32.846160+0000", 2024-05-04T03:07:43.937 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-04T03:07:43.937 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-05-04T03:07:43.937 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-04T03:07:43.937 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-05-04T03:07:43.937 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.255+0000 7f0a54076700 1 Processor -- start 2024-05-04T03:07:43.937 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.255+0000 7f0a54076700 1 -- start start 2024-05-04T03:07:43.937 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.255+0000 7f0a54076700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a4c108b80 0x7f0a4c108f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:43.938 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.255+0000 7f0a54076700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c109580 con 0x7f0a4c108b80 2024-05-04T03:07:43.938 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.256+0000 7f0a51e12700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a4c108b80 0x7f0a4c108f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:43.938 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.256+0000 7f0a51e12700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a4c108b80 0x7f0a4c108f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51964/0 (socket says 172.21.15.1:51964) 2024-05-04T03:07:43.938 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.256+0000 7f0a51e12700 1 -- 172.21.15.1:0/2374058083 learned_addr learned my addr 172.21.15.1:0/2374058083 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:07:43.938 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.256+0000 7f0a51e12700 1 -- 172.21.15.1:0/2374058083 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4c109d50 con 0x7f0a4c108b80 2024-05-04T03:07:43.938 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.256+0000 7f0a51e12700 1 --2- 172.21.15.1:0/2374058083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a4c108b80 0x7f0a4c108f90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c00bb40 tx=0x7f0a3c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f215d9e7a8d63ddf server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:43.938 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.256+0000 7f0a50e10700 1 -- 172.21.15.1:0/2374058083 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a3c00dbe0 con 0x7f0a4c108b80 2024-05-04T03:07:43.938 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.256+0000 7f0a50e10700 1 -- 172.21.15.1:0/2374058083 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0a3c0044c0 con 0x7f0a4c108b80 2024-05-04T03:07:43.938 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.256+0000 7f0a50e10700 1 -- 172.21.15.1:0/2374058083 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a3c010470 con 0x7f0a4c108b80 2024-05-04T03:07:43.938 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.257+0000 7f0a54076700 1 -- 172.21.15.1:0/2374058083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a4c108b80 msgr2=0x7f0a4c108f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:43.938 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.257+0000 7f0a54076700 1 --2- 172.21.15.1:0/2374058083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a4c108b80 0x7f0a4c108f90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c00bb40 tx=0x7f0a3c00be50 comp rx=0 tx=0).stop 2024-05-04T03:07:43.938 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.257+0000 7f0a54076700 1 -- 172.21.15.1:0/2374058083 shutdown_connections 2024-05-04T03:07:43.938 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.257+0000 7f0a54076700 1 --2- 172.21.15.1:0/2374058083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a4c108b80 0x7f0a4c108f90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:43.938 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.257+0000 7f0a54076700 1 -- 172.21.15.1:0/2374058083 >> 172.21.15.1:0/2374058083 conn(0x7f0a4c104240 msgr2=0x7f0a4c106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.257+0000 7f0a54076700 1 -- 172.21.15.1:0/2374058083 shutdown_connections 2024-05-04T03:07:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.257+0000 7f0a54076700 1 -- 172.21.15.1:0/2374058083 wait complete. 2024-05-04T03:07:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.258+0000 7f0a54076700 1 Processor -- start 2024-05-04T03:07:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.258+0000 7f0a54076700 1 -- start start 2024-05-04T03:07:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.258+0000 7f0a54076700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a4c108b80 0x7f0a4c19be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.258+0000 7f0a54076700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c076440 con 0x7f0a4c108b80 2024-05-04T03:07:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.258+0000 7f0a51e12700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a4c108b80 0x7f0a4c19be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.258+0000 7f0a51e12700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a4c108b80 0x7f0a4c19be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51976/0 (socket says 172.21.15.1:51976) 2024-05-04T03:07:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.258+0000 7f0a51e12700 1 -- 172.21.15.1:0/866035904 learned_addr learned my addr 172.21.15.1:0/866035904 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:07:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.259+0000 7f0a51e12700 1 -- 172.21.15.1:0/866035904 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4c19c360 con 0x7f0a4c108b80 2024-05-04T03:07:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.259+0000 7f0a51e12700 1 --2- 172.21.15.1:0/866035904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a4c108b80 0x7f0a4c19be20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c003b90 tx=0x7f0a3c003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.259+0000 7f0a42ffd700 1 -- 172.21.15.1:0/866035904 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a3c027070 con 0x7f0a4c108b80 2024-05-04T03:07:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.259+0000 7f0a42ffd700 1 -- 172.21.15.1:0/866035904 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0a3c010c00 con 0x7f0a4c108b80 2024-05-04T03:07:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.259+0000 7f0a54076700 1 -- 172.21.15.1:0/866035904 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a4c19c5f0 con 0x7f0a4c108b80 2024-05-04T03:07:43.939 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.259+0000 7f0a42ffd700 1 -- 172.21.15.1:0/866035904 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a3c022be0 con 0x7f0a4c108b80 2024-05-04T03:07:43.940 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.259+0000 7f0a54076700 1 -- 172.21.15.1:0/866035904 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a4c19f200 con 0x7f0a4c108b80 2024-05-04T03:07:43.940 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.259+0000 7f0a42ffd700 1 -- 172.21.15.1:0/866035904 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0a3c02a070 con 0x7f0a4c108b80 2024-05-04T03:07:43.940 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.259+0000 7f0a42ffd700 1 -- 172.21.15.1:0/866035904 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0a3c01f070 con 0x7f0a4c108b80 2024-05-04T03:07:43.940 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.260+0000 7f0a54076700 1 -- 172.21.15.1:0/866035904 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a4c064e20 con 0x7f0a4c108b80 2024-05-04T03:07:43.940 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.262+0000 7f0a42ffd700 1 -- 172.21.15.1:0/866035904 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f0a3c031030 con 0x7f0a4c108b80 2024-05-04T03:07:43.940 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.338+0000 7f0a54076700 1 -- 172.21.15.1:0/866035904 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0a4c195030 con 0x7f0a4c108b80 2024-05-04T03:07:43.940 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.339+0000 7f0a42ffd700 1 -- 172.21.15.1:0/866035904 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f0a4c195030 con 0x7f0a4c108b80 2024-05-04T03:07:43.940 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.340+0000 7f0a54076700 1 -- 172.21.15.1:0/866035904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a4c108b80 msgr2=0x7f0a4c19be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:43.940 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.340+0000 7f0a54076700 1 --2- 172.21.15.1:0/866035904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a4c108b80 0x7f0a4c19be20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c003b90 tx=0x7f0a3c003dc0 comp rx=0 tx=0).stop 2024-05-04T03:07:43.940 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.340+0000 7f0a54076700 1 -- 172.21.15.1:0/866035904 shutdown_connections 2024-05-04T03:07:43.940 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.340+0000 7f0a54076700 1 --2- 172.21.15.1:0/866035904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a4c108b80 0x7f0a4c19be20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:43.940 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.340+0000 7f0a54076700 1 -- 172.21.15.1:0/866035904 >> 172.21.15.1:0/866035904 conn(0x7f0a4c104240 msgr2=0x7f0a4c1920f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:43.940 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.340+0000 7f0a54076700 1 -- 172.21.15.1:0/866035904 shutdown_connections 2024-05-04T03:07:43.940 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:43.340+0000 7f0a54076700 1 -- 172.21.15.1:0/866035904 wait complete. 2024-05-04T03:07:43.940 INFO:teuthology.orchestra.run.smithi001.stdout:mgr not available, waiting (2/15)... 2024-05-04T03:07:44.247 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:44 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:44.147+0000 7f6e6998e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-04T03:07:44.247 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:44 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:44.247+0000 7f6e6998e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-04T03:07:45.004 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:44 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:44.633+0000 7f6e6998e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-04T03:07:45.005 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:45 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:45.004+0000 7f6e6998e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-04T03:07:45.351 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:45 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:45.099+0000 7f6e6998e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-04T03:07:45.628 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:45 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:45.351+0000 7f6e6998e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-04T03:07:45.628 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:45 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:45.489+0000 7f6e6998e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-04T03:07:46.128 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:45 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:45.804+0000 7f6e6998e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-04T03:07:46.128 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:45 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:45.924+0000 7f6e6998e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-04T03:07:46.128 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:46 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:46.020+0000 7f6e6998e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-04T03:07:46.928 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:46 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:46.648+0000 7f6e6998e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-04T03:07:46.928 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:46 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:46.873+0000 7f6e6998e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-04T03:07:47.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:46 smithi001 ceph-mon[29731]: Activating manager daemon x 2024-05-04T03:07:47.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:46 smithi001 ceph-mon[29731]: mgrmap e2: x(active, starting, since 0.00385526s) 2024-05-04T03:07:47.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:46 smithi001 ceph-mon[29731]: from='mgr.14100 172.21.15.1:0/671105802' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-04T03:07:47.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:46 smithi001 ceph-mon[29731]: from='mgr.14100 172.21.15.1:0/671105802' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-04T03:07:47.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:46 smithi001 ceph-mon[29731]: from='mgr.14100 172.21.15.1:0/671105802' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-04T03:07:47.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:46 smithi001 ceph-mon[29731]: from='mgr.14100 172.21.15.1:0/671105802' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:07:47.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:46 smithi001 ceph-mon[29731]: from='mgr.14100 172.21.15.1:0/671105802' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-04T03:07:47.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:46 smithi001 ceph-mon[29731]: Manager daemon x is now available 2024-05-04T03:07:47.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:46 smithi001 ceph-mon[29731]: from='mgr.14100 172.21.15.1:0/671105802' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-04T03:07:47.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:46 smithi001 ceph-mon[29731]: from='mgr.14100 172.21.15.1:0/671105802' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-04T03:07:47.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:46 smithi001 ceph-mon[29731]: from='mgr.14100 172.21.15.1:0/671105802' entity='mgr.x' 2024-05-04T03:07:47.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:46 smithi001 ceph-mon[29731]: from='mgr.14100 172.21.15.1:0/671105802' entity='mgr.x' 2024-05-04T03:07:47.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:46 smithi001 ceph-mon[29731]: from='mgr.14100 172.21.15.1:0/671105802' entity='mgr.x' 2024-05-04T03:07:49.173 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:48 smithi001 ceph-mon[29731]: mgrmap e3: x(active, since 1.00862s) 2024-05-04T03:07:50.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:49 smithi001 ceph-mon[29731]: mgrmap e4: x(active, since 2s) 2024-05-04T03:07:50.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:49 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/1375875464' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-04T03:07:55.177 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-05-04T03:07:55.177 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-05-04T03:07:55.177 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsid": "4a572aa8-09c3-11ef-bc97-c7b262605968", 2024-05-04T03:07:55.178 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "health": { 2024-05-04T03:07:55.178 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-04T03:07:55.178 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-04T03:07:55.178 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-04T03:07:55.178 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-05-04T03:07:55.178 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-04T03:07:55.178 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-04T03:07:55.178 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 0 2024-05-04T03:07:55.178 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-05-04T03:07:55.178 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-04T03:07:55.178 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "a" 2024-05-04T03:07:55.179 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-05-04T03:07:55.179 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-05-04T03:07:55.179 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-04T03:07:55.179 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-04T03:07:55.179 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-04T03:07:55.179 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-04T03:07:55.179 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-05-04T03:07:55.179 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-04T03:07:55.179 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-04T03:07:55.179 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-04T03:07:55.179 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-04T03:07:55.191 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-04T03:07:55.191 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-04T03:07:55.191 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-04T03:07:55.192 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-04T03:07:55.192 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-05-04T03:07:55.192 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-04T03:07:55.192 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-04T03:07:55.192 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-04T03:07:55.192 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-04T03:07:55.192 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-04T03:07:55.192 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-04T03:07:55.192 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-04T03:07:55.192 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-04T03:07:55.192 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-04T03:07:55.193 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-05-04T03:07:55.193 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-04T03:07:55.193 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-04T03:07:55.193 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-04T03:07:55.193 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-04T03:07:55.193 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-05-04T03:07:55.193 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-04T03:07:55.193 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-04T03:07:55.193 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-04T03:07:55.193 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-04T03:07:55.193 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-04T03:07:55.194 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-04T03:07:55.194 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "restful" 2024-05-04T03:07:55.194 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-05-04T03:07:55.194 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-04T03:07:55.194 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-05-04T03:07:55.194 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-04T03:07:55.194 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-04T03:07:55.194 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modified": "2024-05-04T03:07:32.846160+0000", 2024-05-04T03:07:55.194 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-04T03:07:55.194 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-05-04T03:07:55.194 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-04T03:07:55.195 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-05-04T03:07:55.195 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.171+0000 7f740bf76700 1 Processor -- start 2024-05-04T03:07:55.195 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.171+0000 7f740bf76700 1 -- start start 2024-05-04T03:07:55.195 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.172+0000 7f740bf76700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7404108560 0x7f7404108970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:55.195 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.172+0000 7f740bf76700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f740407cdf0 con 0x7f7404108560 2024-05-04T03:07:55.195 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.172+0000 7f7409d12700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7404108560 0x7f7404108970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:55.195 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.172+0000 7f7409d12700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7404108560 0x7f7404108970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52032/0 (socket says 172.21.15.1:52032) 2024-05-04T03:07:55.195 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.172+0000 7f7409d12700 1 -- 172.21.15.1:0/2896741861 learned_addr learned my addr 172.21.15.1:0/2896741861 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:07:55.195 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.172+0000 7f7409d12700 1 -- 172.21.15.1:0/2896741861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7404108eb0 con 0x7f7404108560 2024-05-04T03:07:55.195 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.172+0000 7f7409d12700 1 --2- 172.21.15.1:0/2896741861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7404108560 0x7f7404108970 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f74000098a0 tx=0x7f7400009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b092f3033cd78a2d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:55.195 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.172+0000 7f7408d10700 1 -- 172.21.15.1:0/2896741861 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7400004160 con 0x7f7404108560 2024-05-04T03:07:55.195 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.173+0000 7f7408d10700 1 -- 172.21.15.1:0/2896741861 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7400004300 con 0x7f7404108560 2024-05-04T03:07:55.196 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.173+0000 7f7408d10700 1 -- 172.21.15.1:0/2896741861 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7400011af0 con 0x7f7404108560 2024-05-04T03:07:55.196 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.173+0000 7f740bf76700 1 -- 172.21.15.1:0/2896741861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7404108560 msgr2=0x7f7404108970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:55.196 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.173+0000 7f740bf76700 1 --2- 172.21.15.1:0/2896741861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7404108560 0x7f7404108970 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f74000098a0 tx=0x7f7400009bb0 comp rx=0 tx=0).stop 2024-05-04T03:07:55.196 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.173+0000 7f740bf76700 1 -- 172.21.15.1:0/2896741861 shutdown_connections 2024-05-04T03:07:55.196 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.173+0000 7f740bf76700 1 --2- 172.21.15.1:0/2896741861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7404108560 0x7f7404108970 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:55.196 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.173+0000 7f740bf76700 1 -- 172.21.15.1:0/2896741861 >> 172.21.15.1:0/2896741861 conn(0x7f7404104190 msgr2=0x7f7404106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:55.196 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.173+0000 7f740bf76700 1 -- 172.21.15.1:0/2896741861 shutdown_connections 2024-05-04T03:07:55.196 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.173+0000 7f740bf76700 1 -- 172.21.15.1:0/2896741861 wait complete. 2024-05-04T03:07:55.196 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.174+0000 7f740bf76700 1 Processor -- start 2024-05-04T03:07:55.196 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.174+0000 7f740bf76700 1 -- start start 2024-05-04T03:07:55.196 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.174+0000 7f740bf76700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7404108560 0x7f74041a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.174+0000 7f740bf76700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7404109f30 con 0x7f7404108560 2024-05-04T03:07:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.174+0000 7f7409d12700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7404108560 0x7f74041a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.174+0000 7f7409d12700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7404108560 0x7f74041a0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52046/0 (socket says 172.21.15.1:52046) 2024-05-04T03:07:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.174+0000 7f7409d12700 1 -- 172.21.15.1:0/1375875464 learned_addr learned my addr 172.21.15.1:0/1375875464 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:07:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.175+0000 7f7409d12700 1 -- 172.21.15.1:0/1375875464 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74041a0850 con 0x7f7404108560 2024-05-04T03:07:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.175+0000 7f7409d12700 1 --2- 172.21.15.1:0/1375875464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7404108560 0x7f74041a0310 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f74000117b0 tx=0x7f7400011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.175+0000 7f73faffd700 1 -- 172.21.15.1:0/1375875464 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7400003af0 con 0x7f7404108560 2024-05-04T03:07:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.175+0000 7f73faffd700 1 -- 172.21.15.1:0/1375875464 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7400020070 con 0x7f7404108560 2024-05-04T03:07:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.175+0000 7f740bf76700 1 -- 172.21.15.1:0/1375875464 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74041a0ae0 con 0x7f7404108560 2024-05-04T03:07:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.175+0000 7f73faffd700 1 -- 172.21.15.1:0/1375875464 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7400019630 con 0x7f7404108560 2024-05-04T03:07:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.175+0000 7f740bf76700 1 -- 172.21.15.1:0/1375875464 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74041a1520 con 0x7f7404108560 2024-05-04T03:07:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.176+0000 7f73faffd700 1 -- 172.21.15.1:0/1375875464 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f74000198f0 con 0x7f7404108560 2024-05-04T03:07:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.176+0000 7f740bf76700 1 -- 172.21.15.1:0/1375875464 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7404064e20 con 0x7f7404108560 2024-05-04T03:07:55.197 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.176+0000 7f73faffd700 1 --2- 172.21.15.1:0/1375875464 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7f73f003e420 0x7f73f0040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:55.198 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.176+0000 7f73faffd700 1 -- 172.21.15.1:0/1375875464 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7400053a80 con 0x7f7404108560 2024-05-04T03:07:55.198 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.176+0000 7f7409511700 1 --2- 172.21.15.1:0/1375875464 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7f73f003e420 0x7f73f0040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:55.198 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.177+0000 7f7409511700 1 --2- 172.21.15.1:0/1375875464 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7f73f003e420 0x7f73f0040930 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f73f4006fd0 tx=0x7f73f4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:55.198 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.182+0000 7f73faffd700 1 -- 172.21.15.1:0/1375875464 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7400055e20 con 0x7f7404108560 2024-05-04T03:07:55.198 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.494+0000 7f740bf76700 1 -- 172.21.15.1:0/1375875464 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f74041a22a0 con 0x7f7404108560 2024-05-04T03:07:55.198 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.494+0000 7f73faffd700 1 -- 172.21.15.1:0/1375875464 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f7400016260 con 0x7f7404108560 2024-05-04T03:07:55.198 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.498+0000 7f740bf76700 1 -- 172.21.15.1:0/1375875464 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7f73f003e420 msgr2=0x7f73f0040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:55.198 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.498+0000 7f740bf76700 1 --2- 172.21.15.1:0/1375875464 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7f73f003e420 0x7f73f0040930 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f73f4006fd0 tx=0x7f73f4006e40 comp rx=0 tx=0).stop 2024-05-04T03:07:55.198 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.498+0000 7f740bf76700 1 -- 172.21.15.1:0/1375875464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7404108560 msgr2=0x7f74041a0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:55.198 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.498+0000 7f740bf76700 1 --2- 172.21.15.1:0/1375875464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7404108560 0x7f74041a0310 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f74000117b0 tx=0x7f7400011b70 comp rx=0 tx=0).stop 2024-05-04T03:07:55.198 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.498+0000 7f740bf76700 1 -- 172.21.15.1:0/1375875464 shutdown_connections 2024-05-04T03:07:55.198 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.498+0000 7f740bf76700 1 --2- 172.21.15.1:0/1375875464 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7f73f003e420 0x7f73f0040930 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:55.198 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.498+0000 7f740bf76700 1 --2- 172.21.15.1:0/1375875464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7404108560 0x7f74041a0310 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:55.198 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.498+0000 7f740bf76700 1 -- 172.21.15.1:0/1375875464 >> 172.21.15.1:0/1375875464 conn(0x7f7404104190 msgr2=0x7f7404194210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:55.199 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.498+0000 7f740bf76700 1 -- 172.21.15.1:0/1375875464 shutdown_connections 2024-05-04T03:07:55.199 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:49.498+0000 7f740bf76700 1 -- 172.21.15.1:0/1375875464 wait complete. 2024-05-04T03:07:55.199 INFO:teuthology.orchestra.run.smithi001.stdout:mgr is available 2024-05-04T03:07:57.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:57 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3209416646' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-04T03:07:57.917 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-05-04T03:07:57.919 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [global] 2024-05-04T03:07:57.919 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout fsid = 4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:07:57.920 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-04T03:07:57.920 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-04T03:07:57.920 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-04T03:07:57.920 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-04T03:07:57.920 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-04T03:07:57.920 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-05-04T03:07:57.920 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-04T03:07:57.920 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-04T03:07:57.920 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-05-04T03:07:57.920 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [osd] 2024-05-04T03:07:57.920 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-05-04T03:07:57.920 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-04T03:07:57.920 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-04T03:07:57.921 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-04T03:07:57.921 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-04T03:07:57.921 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-04T03:07:57.921 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.152+0000 7fb710189700 1 Processor -- start 2024-05-04T03:07:57.921 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.152+0000 7fb710189700 1 -- start start 2024-05-04T03:07:57.921 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.153+0000 7fb710189700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb708109c60 0x7fb70810a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:57.921 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.153+0000 7fb710189700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb70810a660 con 0x7fb708109c60 2024-05-04T03:07:57.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.153+0000 7fb70df25700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb708109c60 0x7fb70810a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:57.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.153+0000 7fb70df25700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb708109c60 0x7fb70810a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46652/0 (socket says 172.21.15.1:46652) 2024-05-04T03:07:57.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.153+0000 7fb70df25700 1 -- 172.21.15.1:0/4122899890 learned_addr learned my addr 172.21.15.1:0/4122899890 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:07:57.926 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.153+0000 7fb70df25700 1 -- 172.21.15.1:0/4122899890 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70810ae30 con 0x7fb708109c60 2024-05-04T03:07:57.927 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.153+0000 7fb70df25700 1 --2- 172.21.15.1:0/4122899890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb708109c60 0x7fb70810a070 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb700009e80 tx=0x7fb70000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edaddaf82b51f99f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:57.927 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.154+0000 7fb70cf23700 1 -- 172.21.15.1:0/4122899890 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb700017070 con 0x7fb708109c60 2024-05-04T03:07:57.927 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.154+0000 7fb70cf23700 1 -- 172.21.15.1:0/4122899890 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb70000b8c0 con 0x7fb708109c60 2024-05-04T03:07:57.927 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.154+0000 7fb70cf23700 1 -- 172.21.15.1:0/4122899890 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb700004490 con 0x7fb708109c60 2024-05-04T03:07:57.927 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.154+0000 7fb710189700 1 -- 172.21.15.1:0/4122899890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb708109c60 msgr2=0x7fb70810a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:57.927 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.154+0000 7fb710189700 1 --2- 172.21.15.1:0/4122899890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb708109c60 0x7fb70810a070 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb700009e80 tx=0x7fb70000b330 comp rx=0 tx=0).stop 2024-05-04T03:07:57.927 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.155+0000 7fb710189700 1 -- 172.21.15.1:0/4122899890 shutdown_connections 2024-05-04T03:07:57.927 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.155+0000 7fb710189700 1 --2- 172.21.15.1:0/4122899890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb708109c60 0x7fb70810a070 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:57.927 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.155+0000 7fb710189700 1 -- 172.21.15.1:0/4122899890 >> 172.21.15.1:0/4122899890 conn(0x7fb7081052e0 msgr2=0x7fb708107750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:57.927 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.155+0000 7fb710189700 1 -- 172.21.15.1:0/4122899890 shutdown_connections 2024-05-04T03:07:57.927 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.155+0000 7fb710189700 1 -- 172.21.15.1:0/4122899890 wait complete. 2024-05-04T03:07:57.927 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.155+0000 7fb710189700 1 Processor -- start 2024-05-04T03:07:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.155+0000 7fb710189700 1 -- start start 2024-05-04T03:07:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.156+0000 7fb710189700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb708109c60 0x7fb70807e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.156+0000 7fb710189700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb70810e1e0 con 0x7fb708109c60 2024-05-04T03:07:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.156+0000 7fb70df25700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb708109c60 0x7fb70807e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.156+0000 7fb70df25700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb708109c60 0x7fb70807e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46668/0 (socket says 172.21.15.1:46668) 2024-05-04T03:07:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.156+0000 7fb70df25700 1 -- 172.21.15.1:0/3209416646 learned_addr learned my addr 172.21.15.1:0/3209416646 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:07:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.156+0000 7fb70df25700 1 -- 172.21.15.1:0/3209416646 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70807e8c0 con 0x7fb708109c60 2024-05-04T03:07:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.156+0000 7fb70df25700 1 --2- 172.21.15.1:0/3209416646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb708109c60 0x7fb70807e380 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb700009fd0 tx=0x7fb70000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.156+0000 7fb706ffd700 1 -- 172.21.15.1:0/3209416646 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb700020020 con 0x7fb708109c60 2024-05-04T03:07:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.156+0000 7fb706ffd700 1 -- 172.21.15.1:0/3209416646 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb700012430 con 0x7fb708109c60 2024-05-04T03:07:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.156+0000 7fb710189700 1 -- 172.21.15.1:0/3209416646 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb70807cb10 con 0x7fb708109c60 2024-05-04T03:07:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.156+0000 7fb706ffd700 1 -- 172.21.15.1:0/3209416646 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb700017040 con 0x7fb708109c60 2024-05-04T03:07:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.157+0000 7fb710189700 1 -- 172.21.15.1:0/3209416646 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb70807d050 con 0x7fb708109c60 2024-05-04T03:07:57.928 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.158+0000 7fb706ffd700 1 -- 172.21.15.1:0/3209416646 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fb700023020 con 0x7fb708109c60 2024-05-04T03:07:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.158+0000 7fb710189700 1 -- 172.21.15.1:0/3209416646 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb708064e20 con 0x7fb708109c60 2024-05-04T03:07:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.158+0000 7fb706ffd700 1 --2- 172.21.15.1:0/3209416646 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7fb6f003e420 0x7fb6f0040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.158+0000 7fb706ffd700 1 -- 172.21.15.1:0/3209416646 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb70000f050 con 0x7fb708109c60 2024-05-04T03:07:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.158+0000 7fb70d724700 1 --2- 172.21.15.1:0/3209416646 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7fb6f003e420 0x7fb6f0040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.159+0000 7fb70d724700 1 --2- 172.21.15.1:0/3209416646 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7fb6f003e420 0x7fb6f0040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc006fd0 tx=0x7fb6fc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.163+0000 7fb706ffd700 1 -- 172.21.15.1:0/3209416646 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb70002a020 con 0x7fb708109c60 2024-05-04T03:07:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.375+0000 7fb710189700 1 -- 172.21.15.1:0/3209416646 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fb70807d740 con 0x7fb708109c60 2024-05-04T03:07:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.377+0000 7fb706ffd700 1 -- 172.21.15.1:0/3209416646 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7fb70002dd60 con 0x7fb708109c60 2024-05-04T03:07:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.379+0000 7fb710189700 1 -- 172.21.15.1:0/3209416646 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7fb6f003e420 msgr2=0x7fb6f0040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.379+0000 7fb710189700 1 --2- 172.21.15.1:0/3209416646 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7fb6f003e420 0x7fb6f0040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc006fd0 tx=0x7fb6fc006e40 comp rx=0 tx=0).stop 2024-05-04T03:07:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.379+0000 7fb710189700 1 -- 172.21.15.1:0/3209416646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb708109c60 msgr2=0x7fb70807e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.379+0000 7fb710189700 1 --2- 172.21.15.1:0/3209416646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb708109c60 0x7fb70807e380 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb700009fd0 tx=0x7fb70000bd50 comp rx=0 tx=0).stop 2024-05-04T03:07:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.379+0000 7fb710189700 1 -- 172.21.15.1:0/3209416646 shutdown_connections 2024-05-04T03:07:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.379+0000 7fb710189700 1 --2- 172.21.15.1:0/3209416646 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7fb6f003e420 0x7fb6f0040930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:57.929 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.379+0000 7fb710189700 1 --2- 172.21.15.1:0/3209416646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb708109c60 0x7fb70807e380 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:57.930 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.379+0000 7fb710189700 1 -- 172.21.15.1:0/3209416646 >> 172.21.15.1:0/3209416646 conn(0x7fb7081052e0 msgr2=0x7fb708105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:57.930 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.380+0000 7fb710189700 1 -- 172.21.15.1:0/3209416646 shutdown_connections 2024-05-04T03:07:57.930 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:57.380+0000 7fb710189700 1 -- 172.21.15.1:0/3209416646 wait complete. 2024-05-04T03:07:57.930 INFO:teuthology.orchestra.run.smithi001.stdout:Enabling cephadm module... 2024-05-04T03:07:58.890 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:58 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/368201252' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-04T03:07:58.890 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:58 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: ignoring --setuser ceph since I am not root 2024-05-04T03:07:58.890 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:58 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: ignoring --setgroup ceph since I am not root 2024-05-04T03:07:58.890 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:58 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:58.889+0000 7fd48392b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-04T03:07:59.191 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:59 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:59.084+0000 7fd48392b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-04T03:07:59.191 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:59 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:59.191+0000 7fd48392b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-04T03:07:59.202 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.345+0000 7fc7673ef700 1 Processor -- start 2024-05-04T03:07:59.202 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.345+0000 7fc7673ef700 1 -- start start 2024-05-04T03:07:59.203 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.346+0000 7fc7673ef700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc76010a7a0 0x7fc76010cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:59.203 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.346+0000 7fc7673ef700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc760077130 con 0x7fc76010a7a0 2024-05-04T03:07:59.203 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.346+0000 7fc76518b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc76010a7a0 0x7fc76010cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:59.203 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.346+0000 7fc76518b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc76010a7a0 0x7fc76010cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46670/0 (socket says 172.21.15.1:46670) 2024-05-04T03:07:59.203 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.346+0000 7fc76518b700 1 -- 172.21.15.1:0/476845867 learned_addr learned my addr 172.21.15.1:0/476845867 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:07:59.203 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.346+0000 7fc76518b700 1 -- 172.21.15.1:0/476845867 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76010d100 con 0x7fc76010a7a0 2024-05-04T03:07:59.203 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.346+0000 7fc76518b700 1 --2- 172.21.15.1:0/476845867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc76010a7a0 0x7fc76010cbc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc75c0098a0 tx=0x7fc75c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97d6e4d2db70ae5 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:59.203 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.346+0000 7fc757fff700 1 -- 172.21.15.1:0/476845867 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc75c004160 con 0x7fc76010a7a0 2024-05-04T03:07:59.203 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.346+0000 7fc757fff700 1 -- 172.21.15.1:0/476845867 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc75c004300 con 0x7fc76010a7a0 2024-05-04T03:07:59.203 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.346+0000 7fc757fff700 1 -- 172.21.15.1:0/476845867 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc75c011af0 con 0x7fc76010a7a0 2024-05-04T03:07:59.203 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.347+0000 7fc7673ef700 1 -- 172.21.15.1:0/476845867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc76010a7a0 msgr2=0x7fc76010cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:59.203 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.347+0000 7fc7673ef700 1 --2- 172.21.15.1:0/476845867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc76010a7a0 0x7fc76010cbc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc75c0098a0 tx=0x7fc75c009bb0 comp rx=0 tx=0).stop 2024-05-04T03:07:59.204 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.347+0000 7fc7673ef700 1 -- 172.21.15.1:0/476845867 shutdown_connections 2024-05-04T03:07:59.204 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.347+0000 7fc7673ef700 1 --2- 172.21.15.1:0/476845867 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc76010a7a0 0x7fc76010cbc0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:59.204 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.347+0000 7fc7673ef700 1 -- 172.21.15.1:0/476845867 >> 172.21.15.1:0/476845867 conn(0x7fc760104240 msgr2=0x7fc760106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:59.204 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.347+0000 7fc7673ef700 1 -- 172.21.15.1:0/476845867 shutdown_connections 2024-05-04T03:07:59.204 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.347+0000 7fc7673ef700 1 -- 172.21.15.1:0/476845867 wait complete. 2024-05-04T03:07:59.204 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.348+0000 7fc7673ef700 1 Processor -- start 2024-05-04T03:07:59.204 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.348+0000 7fc7673ef700 1 -- start start 2024-05-04T03:07:59.204 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.348+0000 7fc7673ef700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc76010a7a0 0x7fc76019e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:59.204 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.348+0000 7fc7673ef700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc76010e060 con 0x7fc76010a7a0 2024-05-04T03:07:59.204 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.348+0000 7fc76518b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc76010a7a0 0x7fc76019e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:59.204 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.348+0000 7fc76518b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc76010a7a0 0x7fc76019e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46686/0 (socket says 172.21.15.1:46686) 2024-05-04T03:07:59.204 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.348+0000 7fc76518b700 1 -- 172.21.15.1:0/368201252 learned_addr learned my addr 172.21.15.1:0/368201252 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:07:59.204 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.349+0000 7fc76518b700 1 -- 172.21.15.1:0/368201252 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76019e5a0 con 0x7fc76010a7a0 2024-05-04T03:07:59.205 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.349+0000 7fc76518b700 1 --2- 172.21.15.1:0/368201252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc76010a7a0 0x7fc76019e060 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009fb0 tx=0x7fc75c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:59.205 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.349+0000 7fc7567fc700 1 -- 172.21.15.1:0/368201252 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc75c0036a0 con 0x7fc76010a7a0 2024-05-04T03:07:59.205 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.349+0000 7fc7567fc700 1 -- 172.21.15.1:0/368201252 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc75c01a430 con 0x7fc76010a7a0 2024-05-04T03:07:59.205 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.349+0000 7fc7673ef700 1 -- 172.21.15.1:0/368201252 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc76019e830 con 0x7fc76010a7a0 2024-05-04T03:07:59.205 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.349+0000 7fc7567fc700 1 -- 172.21.15.1:0/368201252 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc75c019550 con 0x7fc76010a7a0 2024-05-04T03:07:59.205 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.349+0000 7fc7673ef700 1 -- 172.21.15.1:0/368201252 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7601a1440 con 0x7fc76010a7a0 2024-05-04T03:07:59.205 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.350+0000 7fc7567fc700 1 -- 172.21.15.1:0/368201252 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fc75c019800 con 0x7fc76010a7a0 2024-05-04T03:07:59.205 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.350+0000 7fc7567fc700 1 --2- 172.21.15.1:0/368201252 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7fc74c03e420 0x7fc74c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:07:59.205 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.350+0000 7fc7673ef700 1 -- 172.21.15.1:0/368201252 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc760064e20 con 0x7fc76010a7a0 2024-05-04T03:07:59.205 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.350+0000 7fc7567fc700 1 -- 172.21.15.1:0/368201252 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc75c01a660 con 0x7fc76010a7a0 2024-05-04T03:07:59.205 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.350+0000 7fc76498a700 1 --2- 172.21.15.1:0/368201252 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7fc74c03e420 0x7fc74c040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:07:59.205 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.351+0000 7fc76498a700 1 --2- 172.21.15.1:0/368201252 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7fc74c03e420 0x7fc74c040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc750006fd0 tx=0x7fc750006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:07:59.206 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.356+0000 7fc7567fc700 1 -- 172.21.15.1:0/368201252 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc75c017e90 con 0x7fc76010a7a0 2024-05-04T03:07:59.206 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.629+0000 7fc7673ef700 1 -- 172.21.15.1:0/368201252 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fc7601a1610 con 0x7fc76010a7a0 2024-05-04T03:07:59.206 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.681+0000 7fc7567fc700 1 -- 172.21.15.1:0/368201252 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fc7601a1610 con 0x7fc76010a7a0 2024-05-04T03:07:59.206 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.683+0000 7fc7567fc700 1 -- 172.21.15.1:0/368201252 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fc75c00fd80 con 0x7fc76010a7a0 2024-05-04T03:07:59.206 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.684+0000 7fc7673ef700 1 -- 172.21.15.1:0/368201252 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7fc74c03e420 msgr2=0x7fc74c040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:59.206 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.684+0000 7fc7673ef700 1 --2- 172.21.15.1:0/368201252 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7fc74c03e420 0x7fc74c040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc750006fd0 tx=0x7fc750006e40 comp rx=0 tx=0).stop 2024-05-04T03:07:59.206 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.684+0000 7fc7673ef700 1 -- 172.21.15.1:0/368201252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc76010a7a0 msgr2=0x7fc76019e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:07:59.206 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.684+0000 7fc7673ef700 1 --2- 172.21.15.1:0/368201252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc76010a7a0 0x7fc76019e060 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009fb0 tx=0x7fc75c011940 comp rx=0 tx=0).stop 2024-05-04T03:07:59.206 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.684+0000 7fc7673ef700 1 -- 172.21.15.1:0/368201252 shutdown_connections 2024-05-04T03:07:59.206 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.684+0000 7fc7673ef700 1 --2- 172.21.15.1:0/368201252 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7fc74c03e420 0x7fc74c040930 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:59.206 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.684+0000 7fc7673ef700 1 --2- 172.21.15.1:0/368201252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc76010a7a0 0x7fc76019e060 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:07:59.206 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.684+0000 7fc7673ef700 1 -- 172.21.15.1:0/368201252 >> 172.21.15.1:0/368201252 conn(0x7fc760104240 msgr2=0x7fc760105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:07:59.207 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.684+0000 7fc7673ef700 1 -- 172.21.15.1:0/368201252 shutdown_connections 2024-05-04T03:07:59.207 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:58.684+0000 7fc7673ef700 1 -- 172.21.15.1:0/368201252 wait complete. 2024-05-04T03:07:59.858 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:59 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/368201252' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-04T03:07:59.859 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:07:59 smithi001 ceph-mon[29731]: mgrmap e5: x(active, since 11s) 2024-05-04T03:07:59.859 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:59 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:59.523+0000 7fd48392b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-04T03:07:59.859 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:07:59 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:07:59.615+0000 7fd48392b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-04T03:08:00.484 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-05-04T03:08:00.484 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-04T03:08:00.484 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-04T03:08:00.484 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-04T03:08:00.484 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-04T03:08:00.484 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-05-04T03:08:00.484 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.651+0000 7f39e39f5700 1 Processor -- start 2024-05-04T03:08:00.484 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.651+0000 7f39e39f5700 1 -- start start 2024-05-04T03:08:00.484 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.651+0000 7f39e39f5700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39dc10a6f0 0x7f39dc10cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:00.485 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.651+0000 7f39e39f5700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc07ce70 con 0x7f39dc10a6f0 2024-05-04T03:08:00.485 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.652+0000 7f39e1791700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39dc10a6f0 0x7f39dc10cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:00.485 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.652+0000 7f39e1791700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39dc10a6f0 0x7f39dc10cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46722/0 (socket says 172.21.15.1:46722) 2024-05-04T03:08:00.485 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.652+0000 7f39e1791700 1 -- 172.21.15.1:0/1966464312 learned_addr learned my addr 172.21.15.1:0/1966464312 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:00.485 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.652+0000 7f39e1791700 1 -- 172.21.15.1:0/1966464312 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39dc10d050 con 0x7f39dc10a6f0 2024-05-04T03:08:00.485 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.652+0000 7f39e1791700 1 --2- 172.21.15.1:0/1966464312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39dc10a6f0 0x7f39dc10cb10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f39d00098a0 tx=0x7f39d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4cb2a097962060 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:00.485 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.652+0000 7f39cffff700 1 -- 172.21.15.1:0/1966464312 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39d0004160 con 0x7f39dc10a6f0 2024-05-04T03:08:00.485 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.652+0000 7f39cffff700 1 -- 172.21.15.1:0/1966464312 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f39d0004300 con 0x7f39dc10a6f0 2024-05-04T03:08:00.485 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.652+0000 7f39cffff700 1 -- 172.21.15.1:0/1966464312 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39d0011af0 con 0x7f39dc10a6f0 2024-05-04T03:08:00.485 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.653+0000 7f39e39f5700 1 -- 172.21.15.1:0/1966464312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39dc10a6f0 msgr2=0x7f39dc10cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:00.485 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.653+0000 7f39e39f5700 1 --2- 172.21.15.1:0/1966464312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39dc10a6f0 0x7f39dc10cb10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f39d00098a0 tx=0x7f39d0009bb0 comp rx=0 tx=0).stop 2024-05-04T03:08:00.485 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.653+0000 7f39e39f5700 1 -- 172.21.15.1:0/1966464312 shutdown_connections 2024-05-04T03:08:00.485 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.653+0000 7f39e39f5700 1 --2- 172.21.15.1:0/1966464312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39dc10a6f0 0x7f39dc10cb10 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:00.486 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.653+0000 7f39e39f5700 1 -- 172.21.15.1:0/1966464312 >> 172.21.15.1:0/1966464312 conn(0x7f39dc104190 msgr2=0x7f39dc106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:00.486 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.653+0000 7f39e39f5700 1 -- 172.21.15.1:0/1966464312 shutdown_connections 2024-05-04T03:08:00.486 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.653+0000 7f39e39f5700 1 -- 172.21.15.1:0/1966464312 wait complete. 2024-05-04T03:08:00.486 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.654+0000 7f39e39f5700 1 Processor -- start 2024-05-04T03:08:00.486 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.654+0000 7f39e39f5700 1 -- start start 2024-05-04T03:08:00.486 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.654+0000 7f39e39f5700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39dc10a6f0 0x7f39dc19dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:00.486 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.654+0000 7f39e39f5700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc10e010 con 0x7f39dc10a6f0 2024-05-04T03:08:00.486 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.654+0000 7f39e1791700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39dc10a6f0 0x7f39dc19dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:00.486 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.654+0000 7f39e1791700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39dc10a6f0 0x7f39dc19dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46736/0 (socket says 172.21.15.1:46736) 2024-05-04T03:08:00.486 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.654+0000 7f39e1791700 1 -- 172.21.15.1:0/3168225229 learned_addr learned my addr 172.21.15.1:0/3168225229 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:00.486 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.655+0000 7f39e1791700 1 -- 172.21.15.1:0/3168225229 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39dc19e4f0 con 0x7f39dc10a6f0 2024-05-04T03:08:00.486 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.655+0000 7f39e1791700 1 --2- 172.21.15.1:0/3168225229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39dc10a6f0 0x7f39dc19dfb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f39d0009fb0 tx=0x7f39d0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:00.486 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.655+0000 7f39ce7fc700 1 -- 172.21.15.1:0/3168225229 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39d00036a0 con 0x7f39dc10a6f0 2024-05-04T03:08:00.487 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.655+0000 7f39e39f5700 1 -- 172.21.15.1:0/3168225229 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39dc19e780 con 0x7f39dc10a6f0 2024-05-04T03:08:00.487 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.655+0000 7f39e39f5700 1 -- 172.21.15.1:0/3168225229 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39dc1a1390 con 0x7f39dc10a6f0 2024-05-04T03:08:00.487 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.655+0000 7f39ce7fc700 1 -- 172.21.15.1:0/3168225229 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f39d001a430 con 0x7f39dc10a6f0 2024-05-04T03:08:00.487 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.655+0000 7f39ce7fc700 1 -- 172.21.15.1:0/3168225229 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39d0019550 con 0x7f39dc10a6f0 2024-05-04T03:08:00.487 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.656+0000 7f39ce7fc700 1 -- 172.21.15.1:0/3168225229 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f39d0019800 con 0x7f39dc10a6f0 2024-05-04T03:08:00.487 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.656+0000 7f39e39f5700 1 -- 172.21.15.1:0/3168225229 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39c0005360 con 0x7f39dc10a6f0 2024-05-04T03:08:00.487 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.657+0000 7f39ce7fc700 1 --2- 172.21.15.1:0/3168225229 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7f39c803e420 0x7f39c8040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:00.487 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.657+0000 7f39ce7fc700 1 -- 172.21.15.1:0/3168225229 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f39d00539c0 con 0x7f39dc10a6f0 2024-05-04T03:08:00.487 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.657+0000 7f39e0f90700 1 -- 172.21.15.1:0/3168225229 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7f39c803e420 msgr2=0x7f39c8040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/4200529863 2024-05-04T03:08:00.487 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.657+0000 7f39e0f90700 1 --2- 172.21.15.1:0/3168225229 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7f39c803e420 0x7f39c8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-04T03:08:00.487 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.662+0000 7f39ce7fc700 1 -- 172.21.15.1:0/3168225229 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f39d0017e90 con 0x7f39dc10a6f0 2024-05-04T03:08:00.487 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.858+0000 7f39e0f90700 1 -- 172.21.15.1:0/3168225229 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7f39c803e420 msgr2=0x7f39c8040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/4200529863 2024-05-04T03:08:00.487 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.858+0000 7f39e0f90700 1 --2- 172.21.15.1:0/3168225229 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7f39c803e420 0x7f39c8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-04T03:08:00.487 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.961+0000 7f39e39f5700 1 -- 172.21.15.1:0/3168225229 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f39c0005e60 con 0x7f39dc10a6f0 2024-05-04T03:08:00.487 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.961+0000 7f39ce7fc700 1 -- 172.21.15.1:0/3168225229 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f39d0023390 con 0x7f39dc10a6f0 2024-05-04T03:08:00.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.963+0000 7f39e39f5700 1 -- 172.21.15.1:0/3168225229 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7f39c803e420 msgr2=0x7f39c8040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-04T03:08:00.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.963+0000 7f39e39f5700 1 --2- 172.21.15.1:0/3168225229 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7f39c803e420 0x7f39c8040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:00.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.963+0000 7f39e39f5700 1 -- 172.21.15.1:0/3168225229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39dc10a6f0 msgr2=0x7f39dc19dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:00.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.963+0000 7f39e39f5700 1 --2- 172.21.15.1:0/3168225229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39dc10a6f0 0x7f39dc19dfb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f39d0009fb0 tx=0x7f39d0011940 comp rx=0 tx=0).stop 2024-05-04T03:08:00.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.963+0000 7f39e39f5700 1 -- 172.21.15.1:0/3168225229 shutdown_connections 2024-05-04T03:08:00.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.963+0000 7f39e39f5700 1 --2- 172.21.15.1:0/3168225229 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7f39c803e420 0x7f39c8040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:00.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.963+0000 7f39e39f5700 1 --2- 172.21.15.1:0/3168225229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39dc10a6f0 0x7f39dc19dfb0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:00.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.963+0000 7f39e39f5700 1 -- 172.21.15.1:0/3168225229 >> 172.21.15.1:0/3168225229 conn(0x7f39dc104190 msgr2=0x7f39dc104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:00.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.964+0000 7f39e39f5700 1 -- 172.21.15.1:0/3168225229 shutdown_connections 2024-05-04T03:08:00.488 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:07:59.964+0000 7f39e39f5700 1 -- 172.21.15.1:0/3168225229 wait complete. 2024-05-04T03:08:00.488 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for the mgr to restart... 2024-05-04T03:08:00.488 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mgr epoch 5... 2024-05-04T03:08:00.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:00 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3168225229' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-04T03:08:01.187 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:00 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:00.902+0000 7fd48392b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-04T03:08:01.187 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:01 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:01.083+0000 7fd48392b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-04T03:08:01.570 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:01 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:01.265+0000 7fd48392b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-04T03:08:01.571 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:01 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:01.570+0000 7fd48392b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-04T03:08:01.829 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:01 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:01.667+0000 7fd48392b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-04T03:08:02.083 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:01 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:01.880+0000 7fd48392b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-04T03:08:02.355 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:02 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:02.082+0000 7fd48392b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-04T03:08:02.944 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:02 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:02.712+0000 7fd48392b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-04T03:08:02.945 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:02 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:02.944+0000 7fd48392b200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-04T03:08:03.875 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:03 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:03.387+0000 7fd48392b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-04T03:08:03.875 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:03 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:03.489+0000 7fd48392b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-04T03:08:03.875 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:03 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:03.874+0000 7fd48392b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-04T03:08:04.602 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:04 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:04.249+0000 7fd48392b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-04T03:08:04.602 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:04 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:04.346+0000 7fd48392b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-04T03:08:04.878 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:04 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:04.601+0000 7fd48392b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-04T03:08:04.878 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:04 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:04.741+0000 7fd48392b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-04T03:08:05.378 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:05 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:05.056+0000 7fd48392b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-04T03:08:05.378 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:05 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:05.177+0000 7fd48392b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-04T03:08:05.378 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:05 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:05.274+0000 7fd48392b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-04T03:08:06.187 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:05 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:05.906+0000 7fd48392b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-04T03:08:06.187 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:06 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:06.133+0000 7fd48392b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-04T03:08:06.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:06 smithi001 ceph-mon[29731]: Active manager daemon x restarted 2024-05-04T03:08:06.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:06 smithi001 ceph-mon[29731]: Activating manager daemon x 2024-05-04T03:08:06.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:06 smithi001 ceph-mon[29731]: osdmap e2: 0 total, 0 up, 0 in 2024-05-04T03:08:06.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:06 smithi001 ceph-mon[29731]: mgrmap e6: x(active, starting, since 0.00407798s) 2024-05-04T03:08:06.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:06 smithi001 ceph-mon[29731]: from='mgr.14118 172.21.15.1:0/3287545089' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:08:06.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:06 smithi001 ceph-mon[29731]: from='mgr.14118 172.21.15.1:0/3287545089' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-04T03:08:06.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:06 smithi001 ceph-mon[29731]: from='mgr.14118 172.21.15.1:0/3287545089' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-04T03:08:06.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:06 smithi001 ceph-mon[29731]: from='mgr.14118 172.21.15.1:0/3287545089' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-04T03:08:06.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:06 smithi001 ceph-mon[29731]: from='mgr.14118 172.21.15.1:0/3287545089' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-04T03:08:06.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:06 smithi001 ceph-mon[29731]: Manager daemon x is now available 2024-05-04T03:08:06.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:06 smithi001 ceph-mon[29731]: from='mgr.14118 172.21.15.1:0/3287545089' entity='mgr.x' 2024-05-04T03:08:06.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:06 smithi001 ceph-mon[29731]: from='mgr.14118 172.21.15.1:0/3287545089' entity='mgr.x' 2024-05-04T03:08:06.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:06 smithi001 ceph-mon[29731]: from='mgr.14118 172.21.15.1:0/3287545089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:08:06.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:06 smithi001 ceph-mon[29731]: from='mgr.14118 172.21.15.1:0/3287545089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:08:07.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:07 smithi001 ceph-mon[29731]: Found migration_current of "None". Setting to last migration. 2024-05-04T03:08:07.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:07 smithi001 ceph-mon[29731]: from='mgr.14118 172.21.15.1:0/3287545089' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-04T03:08:07.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:07 smithi001 ceph-mon[29731]: from='mgr.14118 172.21.15.1:0/3287545089' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-04T03:08:07.448 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:07 smithi001 ceph-mon[29731]: from='mgr.14118 172.21.15.1:0/3287545089' entity='mgr.x' 2024-05-04T03:08:07.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:07 smithi001 ceph-mon[29731]: from='mgr.14118 172.21.15.1:0/3287545089' entity='mgr.x' 2024-05-04T03:08:07.449 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:07 smithi001 ceph-mon[29731]: mgrmap e7: x(active, since 1.0071s) 2024-05-04T03:08:07.734 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-05-04T03:08:07.734 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-04T03:08:07.735 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-04T03:08:07.735 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-05-04T03:08:07.735 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.220+0000 7f0078bb4700 1 Processor -- start 2024-05-04T03:08:07.735 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.220+0000 7f0078bb4700 1 -- start start 2024-05-04T03:08:07.735 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.220+0000 7f0078bb4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0074076fc0 0x7f007410c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:07.735 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.220+0000 7f0078bb4700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f007410ce60 con 0x7f0074076fc0 2024-05-04T03:08:07.735 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.221+0000 7f007259c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0074076fc0 0x7f007410c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:07.735 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.221+0000 7f007259c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0074076fc0 0x7f007410c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46742/0 (socket says 172.21.15.1:46742) 2024-05-04T03:08:07.735 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.221+0000 7f007259c700 1 -- 172.21.15.1:0/2058553640 learned_addr learned my addr 172.21.15.1:0/2058553640 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:07.735 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.221+0000 7f007259c700 1 -- 172.21.15.1:0/2058553640 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f007410cfe0 con 0x7f0074076fc0 2024-05-04T03:08:07.735 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.221+0000 7f007259c700 1 --2- 172.21.15.1:0/2058553640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0074076fc0 0x7f007410c850 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f00680098a0 tx=0x7f0068009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69734f807d2efc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:07.735 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.221+0000 7f007159a700 1 -- 172.21.15.1:0/2058553640 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0068004160 con 0x7f0074076fc0 2024-05-04T03:08:07.735 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.221+0000 7f007159a700 1 -- 172.21.15.1:0/2058553640 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0068004300 con 0x7f0074076fc0 2024-05-04T03:08:07.736 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.221+0000 7f007159a700 1 -- 172.21.15.1:0/2058553640 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0068011af0 con 0x7f0074076fc0 2024-05-04T03:08:07.736 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.222+0000 7f0078bb4700 1 -- 172.21.15.1:0/2058553640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0074076fc0 msgr2=0x7f007410c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:07.736 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.222+0000 7f0078bb4700 1 --2- 172.21.15.1:0/2058553640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0074076fc0 0x7f007410c850 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f00680098a0 tx=0x7f0068009bb0 comp rx=0 tx=0).stop 2024-05-04T03:08:07.736 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.222+0000 7f0078bb4700 1 -- 172.21.15.1:0/2058553640 shutdown_connections 2024-05-04T03:08:07.736 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.222+0000 7f0078bb4700 1 --2- 172.21.15.1:0/2058553640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0074076fc0 0x7f007410c850 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:07.736 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.222+0000 7f0078bb4700 1 -- 172.21.15.1:0/2058553640 >> 172.21.15.1:0/2058553640 conn(0x7f0074104240 msgr2=0x7f0074106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:07.736 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.222+0000 7f0078bb4700 1 -- 172.21.15.1:0/2058553640 shutdown_connections 2024-05-04T03:08:07.736 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.222+0000 7f0078bb4700 1 -- 172.21.15.1:0/2058553640 wait complete. 2024-05-04T03:08:07.736 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.223+0000 7f0078bb4700 1 Processor -- start 2024-05-04T03:08:07.736 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.223+0000 7f0078bb4700 1 -- start start 2024-05-04T03:08:07.736 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.224+0000 7f0078bb4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0074076fc0 0x7f007419e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:07.736 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.224+0000 7f0078bb4700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f007410e060 con 0x7f0074076fc0 2024-05-04T03:08:07.736 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.224+0000 7f007259c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0074076fc0 0x7f007419e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:07.737 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.224+0000 7f007259c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0074076fc0 0x7f007419e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46746/0 (socket says 172.21.15.1:46746) 2024-05-04T03:08:07.737 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.224+0000 7f007259c700 1 -- 172.21.15.1:0/3976936174 learned_addr learned my addr 172.21.15.1:0/3976936174 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:07.737 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.224+0000 7f007259c700 1 -- 172.21.15.1:0/3976936174 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f007419e580 con 0x7f0074076fc0 2024-05-04T03:08:07.737 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.224+0000 7f007259c700 1 --2- 172.21.15.1:0/3976936174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0074076fc0 0x7f007419e040 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f00680117b0 tx=0x7f0068011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:07.737 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.225+0000 7f00637fe700 1 -- 172.21.15.1:0/3976936174 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0068003af0 con 0x7f0074076fc0 2024-05-04T03:08:07.737 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.225+0000 7f00637fe700 1 -- 172.21.15.1:0/3976936174 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0068020070 con 0x7f0074076fc0 2024-05-04T03:08:07.737 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.225+0000 7f00637fe700 1 -- 172.21.15.1:0/3976936174 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0068019630 con 0x7f0074076fc0 2024-05-04T03:08:07.737 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.226+0000 7f0078bb4700 1 -- 172.21.15.1:0/3976936174 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f007419e810 con 0x7f0074076fc0 2024-05-04T03:08:07.737 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.226+0000 7f0078bb4700 1 -- 172.21.15.1:0/3976936174 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00741a1420 con 0x7f0074076fc0 2024-05-04T03:08:07.737 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.227+0000 7f00637fe700 1 -- 172.21.15.1:0/3976936174 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f006801a9c0 con 0x7f0074076fc0 2024-05-04T03:08:07.737 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.227+0000 7f00637fe700 1 --2- 172.21.15.1:0/3976936174 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7f005c03e4c0 0x7f005c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:07.737 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.227+0000 7f00637fe700 1 -- 172.21.15.1:0/3976936174 --> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f005c0410e0 con 0x7f005c03e4c0 2024-05-04T03:08:07.737 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.227+0000 7f0071d9b700 1 -- 172.21.15.1:0/3976936174 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7f005c03e4c0 msgr2=0x7f005c0409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/4200529863 2024-05-04T03:08:07.738 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.227+0000 7f00637fe700 1 -- 172.21.15.1:0/3976936174 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0068053a80 con 0x7f0074076fc0 2024-05-04T03:08:07.738 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.227+0000 7f0071d9b700 1 --2- 172.21.15.1:0/3976936174 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7f005c03e4c0 0x7f005c0409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-04T03:08:07.738 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.428+0000 7f0071d9b700 1 -- 172.21.15.1:0/3976936174 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7f005c03e4c0 msgr2=0x7f005c0409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/4200529863 2024-05-04T03:08:07.738 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.428+0000 7f0071d9b700 1 --2- 172.21.15.1:0/3976936174 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7f005c03e4c0 0x7f005c0409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-04T03:08:07.738 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.829+0000 7f0071d9b700 1 -- 172.21.15.1:0/3976936174 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7f005c03e4c0 msgr2=0x7f005c0409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/4200529863 2024-05-04T03:08:07.738 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:01.829+0000 7f0071d9b700 1 --2- 172.21.15.1:0/3976936174 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7f005c03e4c0 0x7f005c0409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-04T03:08:07.738 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:02.630+0000 7f0071d9b700 1 -- 172.21.15.1:0/3976936174 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7f005c03e4c0 msgr2=0x7f005c0409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/4200529863 2024-05-04T03:08:07.738 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:02.630+0000 7f0071d9b700 1 --2- 172.21.15.1:0/3976936174 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7f005c03e4c0 0x7f005c0409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-04T03:08:07.738 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:04.231+0000 7f0071d9b700 1 -- 172.21.15.1:0/3976936174 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7f005c03e4c0 msgr2=0x7f005c0409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/4200529863 2024-05-04T03:08:07.738 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:04.231+0000 7f0071d9b700 1 --2- 172.21.15.1:0/3976936174 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7f005c03e4c0 0x7f005c0409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-04T03:08:07.738 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:06.137+0000 7f00637fe700 1 -- 172.21.15.1:0/3976936174 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7f006801a430 con 0x7f0074076fc0 2024-05-04T03:08:07.738 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:06.137+0000 7f00637fe700 1 -- 172.21.15.1:0/3976936174 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7f005c03e4c0 msgr2=0x7f005c0409d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-04T03:08:07.738 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:06.137+0000 7f00637fe700 1 --2- 172.21.15.1:0/3976936174 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7f005c03e4c0 0x7f005c0409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:07.738 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:07.139+0000 7f00637fe700 1 -- 172.21.15.1:0/3976936174 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f0068019a90 con 0x7f0074076fc0 2024-05-04T03:08:07.739 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:07.139+0000 7f00637fe700 1 --2- 172.21.15.1:0/3976936174 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f005c041e00 0x7f005c044220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:07.739 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:07.139+0000 7f00637fe700 1 -- 172.21.15.1:0/3976936174 --> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f006800ae50 con 0x7f005c041e00 2024-05-04T03:08:07.739 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:07.140+0000 7f0071d9b700 1 --2- 172.21.15.1:0/3976936174 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f005c041e00 0x7f005c044220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:07.739 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:07.140+0000 7f0071d9b700 1 --2- 172.21.15.1:0/3976936174 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f005c041e00 0x7f005c044220 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0064003a10 tx=0x7f00640092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:07.739 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:07.141+0000 7f00637fe700 1 -- 172.21.15.1:0/3976936174 <== mgr.14118 v2:172.21.15.1:6800/128793132 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f006800ae50 con 0x7f005c041e00 2024-05-04T03:08:07.739 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:07.150+0000 7f0078bb4700 1 -- 172.21.15.1:0/3976936174 --> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f0074064e20 con 0x7f005c041e00 2024-05-04T03:08:07.739 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:07.150+0000 7f00637fe700 1 -- 172.21.15.1:0/3976936174 <== mgr.14118 v2:172.21.15.1:6800/128793132 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f0074064e20 con 0x7f005c041e00 2024-05-04T03:08:07.739 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:07.150+0000 7f0078bb4700 1 -- 172.21.15.1:0/3976936174 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f005c041e00 msgr2=0x7f005c044220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:07.739 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:07.150+0000 7f0078bb4700 1 --2- 172.21.15.1:0/3976936174 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f005c041e00 0x7f005c044220 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0064003a10 tx=0x7f00640092b0 comp rx=0 tx=0).stop 2024-05-04T03:08:07.739 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:07.150+0000 7f0078bb4700 1 -- 172.21.15.1:0/3976936174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0074076fc0 msgr2=0x7f007419e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:07.739 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:07.150+0000 7f0078bb4700 1 --2- 172.21.15.1:0/3976936174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0074076fc0 0x7f007419e040 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f00680117b0 tx=0x7f0068011b70 comp rx=0 tx=0).stop 2024-05-04T03:08:07.739 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:07.150+0000 7f0078bb4700 1 -- 172.21.15.1:0/3976936174 shutdown_connections 2024-05-04T03:08:07.739 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:07.150+0000 7f0078bb4700 1 --2- 172.21.15.1:0/3976936174 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f005c041e00 0x7f005c044220 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:07.739 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:07.150+0000 7f0078bb4700 1 --2- 172.21.15.1:0/3976936174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0074076fc0 0x7f007419e040 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:07.740 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:07.150+0000 7f0078bb4700 1 --2- 172.21.15.1:0/3976936174 >> [v2:172.21.15.1:6800/4200529863,v1:172.21.15.1:6801/4200529863] conn(0x7f005c03e4c0 0x7f005c0409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:07.740 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:07.150+0000 7f0078bb4700 1 -- 172.21.15.1:0/3976936174 >> 172.21.15.1:0/3976936174 conn(0x7f0074104240 msgr2=0x7f0074105b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:07.740 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:07.150+0000 7f0078bb4700 1 -- 172.21.15.1:0/3976936174 shutdown_connections 2024-05-04T03:08:07.740 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:07.150+0000 7f0078bb4700 1 -- 172.21.15.1:0/3976936174 wait complete. 2024-05-04T03:08:07.740 INFO:teuthology.orchestra.run.smithi001.stdout:mgr epoch 5 is available 2024-05-04T03:08:07.740 INFO:teuthology.orchestra.run.smithi001.stdout:Setting orchestrator backend to cephadm... 2024-05-04T03:08:08.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:08 smithi001 ceph-mon[29731]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-04T03:08:08.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:08 smithi001 ceph-mon[29731]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-04T03:08:08.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:08 smithi001 ceph-mon[29731]: [04/May/2024:03:08:07] ENGINE Bus STARTING 2024-05-04T03:08:08.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:08 smithi001 ceph-mon[29731]: [04/May/2024:03:08:07] ENGINE Serving on http://172.21.15.1:8765 2024-05-04T03:08:08.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:08 smithi001 ceph-mon[29731]: from='mgr.14118 172.21.15.1:0/3287545089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:08:09.010 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.194+0000 7f508615c700 1 Processor -- start 2024-05-04T03:08:09.011 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.194+0000 7f508615c700 1 -- start start 2024-05-04T03:08:09.011 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.194+0000 7f508615c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f508010bf10 0x7f508010c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:09.011 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.194+0000 7f508615c700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f508010c8f0 con 0x7f508010bf10 2024-05-04T03:08:09.011 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.194+0000 7f507f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f508010bf10 0x7f508010c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:09.011 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.194+0000 7f507f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f508010bf10 0x7f508010c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43346/0 (socket says 172.21.15.1:43346) 2024-05-04T03:08:09.011 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.194+0000 7f507f7fe700 1 -- 172.21.15.1:0/4020300882 learned_addr learned my addr 172.21.15.1:0/4020300882 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:09.011 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.194+0000 7f507f7fe700 1 -- 172.21.15.1:0/4020300882 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508010cff0 con 0x7f508010bf10 2024-05-04T03:08:09.011 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.195+0000 7f507f7fe700 1 --2- 172.21.15.1:0/4020300882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f508010bf10 0x7f508010c320 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f50700098a0 tx=0x7f5070009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73eaabf817d75cd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:09.011 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.195+0000 7f507e7fc700 1 -- 172.21.15.1:0/4020300882 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5070004160 con 0x7f508010bf10 2024-05-04T03:08:09.011 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.195+0000 7f507e7fc700 1 -- 172.21.15.1:0/4020300882 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5070004300 con 0x7f508010bf10 2024-05-04T03:08:09.011 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.195+0000 7f507e7fc700 1 -- 172.21.15.1:0/4020300882 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5070011af0 con 0x7f508010bf10 2024-05-04T03:08:09.011 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.196+0000 7f508615c700 1 -- 172.21.15.1:0/4020300882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f508010bf10 msgr2=0x7f508010c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:09.012 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.196+0000 7f508615c700 1 --2- 172.21.15.1:0/4020300882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f508010bf10 0x7f508010c320 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f50700098a0 tx=0x7f5070009bb0 comp rx=0 tx=0).stop 2024-05-04T03:08:09.012 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.196+0000 7f508615c700 1 -- 172.21.15.1:0/4020300882 shutdown_connections 2024-05-04T03:08:09.012 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.196+0000 7f508615c700 1 --2- 172.21.15.1:0/4020300882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f508010bf10 0x7f508010c320 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:09.012 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.196+0000 7f508615c700 1 -- 172.21.15.1:0/4020300882 >> 172.21.15.1:0/4020300882 conn(0x7f5080107680 msgr2=0x7f5080109aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:09.012 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.196+0000 7f508615c700 1 -- 172.21.15.1:0/4020300882 shutdown_connections 2024-05-04T03:08:09.012 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.196+0000 7f508615c700 1 -- 172.21.15.1:0/4020300882 wait complete. 2024-05-04T03:08:09.012 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.196+0000 7f508615c700 1 Processor -- start 2024-05-04T03:08:09.012 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.197+0000 7f508615c700 1 -- start start 2024-05-04T03:08:09.012 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.197+0000 7f508615c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f508010bf10 0x7f50801a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:09.012 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.197+0000 7f508615c700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5080110430 con 0x7f508010bf10 2024-05-04T03:08:09.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.197+0000 7f507f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f508010bf10 0x7f50801a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:09.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.197+0000 7f507f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f508010bf10 0x7f50801a0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43348/0 (socket says 172.21.15.1:43348) 2024-05-04T03:08:09.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.197+0000 7f507f7fe700 1 -- 172.21.15.1:0/1617861819 learned_addr learned my addr 172.21.15.1:0/1617861819 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:09.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.197+0000 7f507f7fe700 1 -- 172.21.15.1:0/1617861819 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50801a0750 con 0x7f508010bf10 2024-05-04T03:08:09.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.197+0000 7f507f7fe700 1 --2- 172.21.15.1:0/1617861819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f508010bf10 0x7f50801a0210 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5070011890 tx=0x7f5070011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:09.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.198+0000 7f507cff9700 1 -- 172.21.15.1:0/1617861819 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50700039e0 con 0x7f508010bf10 2024-05-04T03:08:09.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.198+0000 7f507cff9700 1 -- 172.21.15.1:0/1617861819 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5070003b80 con 0x7f508010bf10 2024-05-04T03:08:09.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.198+0000 7f508615c700 1 -- 172.21.15.1:0/1617861819 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50801a09e0 con 0x7f508010bf10 2024-05-04T03:08:09.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.198+0000 7f507cff9700 1 -- 172.21.15.1:0/1617861819 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5070024750 con 0x7f508010bf10 2024-05-04T03:08:09.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.198+0000 7f508615c700 1 -- 172.21.15.1:0/1617861819 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50801a35f0 con 0x7f508010bf10 2024-05-04T03:08:09.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.199+0000 7f507cff9700 1 -- 172.21.15.1:0/1617861819 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f5070022020 con 0x7f508010bf10 2024-05-04T03:08:09.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.199+0000 7f507cff9700 1 --2- 172.21.15.1:0/1617861819 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f506803e4c0 0x7f50680409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:09.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.199+0000 7f507cff9700 1 -- 172.21.15.1:0/1617861819 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5070054760 con 0x7f508010bf10 2024-05-04T03:08:09.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.199+0000 7f508615c700 1 -- 172.21.15.1:0/1617861819 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5080064e20 con 0x7f508010bf10 2024-05-04T03:08:09.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.199+0000 7f507effd700 1 --2- 172.21.15.1:0/1617861819 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f506803e4c0 0x7f50680409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:09.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.199+0000 7f507effd700 1 --2- 172.21.15.1:0/1617861819 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f506803e4c0 0x7f50680409d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5074006fd0 tx=0x7f5074006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:09.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.205+0000 7f507cff9700 1 -- 172.21.15.1:0/1617861819 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f507002c350 con 0x7f508010bf10 2024-05-04T03:08:09.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.438+0000 7f508615c700 1 -- 172.21.15.1:0/1617861819 --> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f50801086b0 con 0x7f506803e4c0 2024-05-04T03:08:09.017 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.443+0000 7f507cff9700 1 -- 172.21.15.1:0/1617861819 <== mgr.14118 v2:172.21.15.1:6800/128793132 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f50801086b0 con 0x7f506803e4c0 2024-05-04T03:08:09.017 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.445+0000 7f508615c700 1 -- 172.21.15.1:0/1617861819 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f506803e4c0 msgr2=0x7f50680409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:09.017 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.445+0000 7f508615c700 1 --2- 172.21.15.1:0/1617861819 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f506803e4c0 0x7f50680409d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5074006fd0 tx=0x7f5074006e40 comp rx=0 tx=0).stop 2024-05-04T03:08:09.017 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.445+0000 7f508615c700 1 -- 172.21.15.1:0/1617861819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f508010bf10 msgr2=0x7f50801a0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:09.017 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.445+0000 7f508615c700 1 --2- 172.21.15.1:0/1617861819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f508010bf10 0x7f50801a0210 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5070011890 tx=0x7f5070011970 comp rx=0 tx=0).stop 2024-05-04T03:08:09.017 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.445+0000 7f508615c700 1 -- 172.21.15.1:0/1617861819 shutdown_connections 2024-05-04T03:08:09.017 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.445+0000 7f508615c700 1 --2- 172.21.15.1:0/1617861819 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f506803e4c0 0x7f50680409d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:09.017 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.445+0000 7f508615c700 1 --2- 172.21.15.1:0/1617861819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f508010bf10 0x7f50801a0210 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:09.017 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.445+0000 7f508615c700 1 -- 172.21.15.1:0/1617861819 >> 172.21.15.1:0/1617861819 conn(0x7f5080107680 msgr2=0x7f5080107f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:09.017 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.446+0000 7f508615c700 1 -- 172.21.15.1:0/1617861819 shutdown_connections 2024-05-04T03:08:09.017 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:08.446+0000 7f508615c700 1 -- 172.21.15.1:0/1617861819 wait complete. 2024-05-04T03:08:09.325 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:09 smithi001 ceph-mon[29731]: [04/May/2024:03:08:07] ENGINE Serving on https://172.21.15.1:7150 2024-05-04T03:08:09.325 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:09 smithi001 ceph-mon[29731]: [04/May/2024:03:08:07] ENGINE Bus STARTED 2024-05-04T03:08:09.325 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:09 smithi001 ceph-mon[29731]: mgrmap e8: x(active, since 2s) 2024-05-04T03:08:09.325 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:09 smithi001 ceph-mon[29731]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:08:09.325 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:09 smithi001 ceph-mon[29731]: from='mgr.14118 172.21.15.1:0/3287545089' entity='mgr.x' 2024-05-04T03:08:09.325 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:09 smithi001 ceph-mon[29731]: from='mgr.14118 172.21.15.1:0/3287545089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:08:10.237 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:10 smithi001 ceph-mon[29731]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:08:10.245 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-04T03:08:10.246 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.446+0000 7f2a5a066700 1 Processor -- start 2024-05-04T03:08:10.246 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.446+0000 7f2a5a066700 1 -- start start 2024-05-04T03:08:10.246 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.446+0000 7f2a5a066700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a5410a6e0 0x7f2a5410cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:10.246 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.446+0000 7f2a5a066700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5407ce70 con 0x7f2a5410a6e0 2024-05-04T03:08:10.246 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.446+0000 7f2a537fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a5410a6e0 0x7f2a5410cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:10.246 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.446+0000 7f2a537fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a5410a6e0 0x7f2a5410cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43350/0 (socket says 172.21.15.1:43350) 2024-05-04T03:08:10.246 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.446+0000 7f2a537fe700 1 -- 172.21.15.1:0/2863359921 learned_addr learned my addr 172.21.15.1:0/2863359921 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:10.246 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.447+0000 7f2a537fe700 1 -- 172.21.15.1:0/2863359921 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5410d040 con 0x7f2a5410a6e0 2024-05-04T03:08:10.246 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.447+0000 7f2a537fe700 1 --2- 172.21.15.1:0/2863359921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a5410a6e0 0x7f2a5410cb00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2a440098a0 tx=0x7f2a44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf52517a6da7cde server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:10.246 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.447+0000 7f2a527fc700 1 -- 172.21.15.1:0/2863359921 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a44004160 con 0x7f2a5410a6e0 2024-05-04T03:08:10.246 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.447+0000 7f2a527fc700 1 -- 172.21.15.1:0/2863359921 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2a44004300 con 0x7f2a5410a6e0 2024-05-04T03:08:10.247 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.447+0000 7f2a527fc700 1 -- 172.21.15.1:0/2863359921 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a44011af0 con 0x7f2a5410a6e0 2024-05-04T03:08:10.247 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.448+0000 7f2a5a066700 1 -- 172.21.15.1:0/2863359921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a5410a6e0 msgr2=0x7f2a5410cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:10.247 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.448+0000 7f2a5a066700 1 --2- 172.21.15.1:0/2863359921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a5410a6e0 0x7f2a5410cb00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2a440098a0 tx=0x7f2a44009bb0 comp rx=0 tx=0).stop 2024-05-04T03:08:10.247 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.448+0000 7f2a5a066700 1 -- 172.21.15.1:0/2863359921 shutdown_connections 2024-05-04T03:08:10.247 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.448+0000 7f2a5a066700 1 --2- 172.21.15.1:0/2863359921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a5410a6e0 0x7f2a5410cb00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:10.247 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.448+0000 7f2a5a066700 1 -- 172.21.15.1:0/2863359921 >> 172.21.15.1:0/2863359921 conn(0x7f2a541041e0 msgr2=0x7f2a54106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:10.247 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.448+0000 7f2a5a066700 1 -- 172.21.15.1:0/2863359921 shutdown_connections 2024-05-04T03:08:10.247 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.448+0000 7f2a5a066700 1 -- 172.21.15.1:0/2863359921 wait complete. 2024-05-04T03:08:10.247 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.448+0000 7f2a5a066700 1 Processor -- start 2024-05-04T03:08:10.247 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.449+0000 7f2a5a066700 1 -- start start 2024-05-04T03:08:10.247 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.449+0000 7f2a5a066700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a5410a6e0 0x7f2a5419e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:10.247 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.449+0000 7f2a5a066700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5410e000 con 0x7f2a5410a6e0 2024-05-04T03:08:10.247 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.449+0000 7f2a537fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a5410a6e0 0x7f2a5419e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:10.247 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.449+0000 7f2a537fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a5410a6e0 0x7f2a5419e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43360/0 (socket says 172.21.15.1:43360) 2024-05-04T03:08:10.248 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.449+0000 7f2a537fe700 1 -- 172.21.15.1:0/2483219619 learned_addr learned my addr 172.21.15.1:0/2483219619 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:10.248 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.449+0000 7f2a537fe700 1 -- 172.21.15.1:0/2483219619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5419e5e0 con 0x7f2a5410a6e0 2024-05-04T03:08:10.248 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.449+0000 7f2a537fe700 1 --2- 172.21.15.1:0/2483219619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a5410a6e0 0x7f2a5419e0a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2a440117b0 tx=0x7f2a44011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:10.248 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.450+0000 7f2a50ff9700 1 -- 172.21.15.1:0/2483219619 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a440039c0 con 0x7f2a5410a6e0 2024-05-04T03:08:10.248 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.450+0000 7f2a50ff9700 1 -- 172.21.15.1:0/2483219619 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2a44003b60 con 0x7f2a5410a6e0 2024-05-04T03:08:10.248 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.450+0000 7f2a5a066700 1 -- 172.21.15.1:0/2483219619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a5419e870 con 0x7f2a5410a6e0 2024-05-04T03:08:10.248 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.450+0000 7f2a50ff9700 1 -- 172.21.15.1:0/2483219619 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a440196a0 con 0x7f2a5410a6e0 2024-05-04T03:08:10.248 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.450+0000 7f2a5a066700 1 -- 172.21.15.1:0/2483219619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a541a1480 con 0x7f2a5410a6e0 2024-05-04T03:08:10.248 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.451+0000 7f2a5a066700 1 -- 172.21.15.1:0/2483219619 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a54064e20 con 0x7f2a5410a6e0 2024-05-04T03:08:10.248 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.451+0000 7f2a50ff9700 1 -- 172.21.15.1:0/2483219619 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f2a44019910 con 0x7f2a5410a6e0 2024-05-04T03:08:10.248 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.451+0000 7f2a50ff9700 1 --2- 172.21.15.1:0/2483219619 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f2a3c03e4c0 0x7f2a3c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:10.248 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.451+0000 7f2a50ff9700 1 -- 172.21.15.1:0/2483219619 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2a44053920 con 0x7f2a5410a6e0 2024-05-04T03:08:10.248 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.451+0000 7f2a52ffd700 1 --2- 172.21.15.1:0/2483219619 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f2a3c03e4c0 0x7f2a3c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:10.248 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.451+0000 7f2a52ffd700 1 --2- 172.21.15.1:0/2483219619 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f2a3c03e4c0 0x7f2a3c0409d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2a48006fd0 tx=0x7f2a48006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:10.249 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.456+0000 7f2a50ff9700 1 -- 172.21.15.1:0/2483219619 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2a440042a0 con 0x7f2a5410a6e0 2024-05-04T03:08:10.249 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.674+0000 7f2a5a066700 1 -- 172.21.15.1:0/2483219619 --> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f2a541063f0 con 0x7f2a3c03e4c0 2024-05-04T03:08:10.249 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.675+0000 7f2a50ff9700 1 -- 172.21.15.1:0/2483219619 <== mgr.14118 v2:172.21.15.1:6800/128793132 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f2a541063f0 con 0x7f2a3c03e4c0 2024-05-04T03:08:10.249 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.677+0000 7f2a5a066700 1 -- 172.21.15.1:0/2483219619 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f2a3c03e4c0 msgr2=0x7f2a3c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:10.249 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.677+0000 7f2a5a066700 1 --2- 172.21.15.1:0/2483219619 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f2a3c03e4c0 0x7f2a3c0409d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2a48006fd0 tx=0x7f2a48006e40 comp rx=0 tx=0).stop 2024-05-04T03:08:10.249 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.677+0000 7f2a5a066700 1 -- 172.21.15.1:0/2483219619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a5410a6e0 msgr2=0x7f2a5419e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:10.249 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.677+0000 7f2a5a066700 1 --2- 172.21.15.1:0/2483219619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a5410a6e0 0x7f2a5419e0a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2a440117b0 tx=0x7f2a44011b70 comp rx=0 tx=0).stop 2024-05-04T03:08:10.249 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.677+0000 7f2a5a066700 1 -- 172.21.15.1:0/2483219619 shutdown_connections 2024-05-04T03:08:10.249 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.677+0000 7f2a5a066700 1 --2- 172.21.15.1:0/2483219619 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f2a3c03e4c0 0x7f2a3c0409d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:10.249 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.677+0000 7f2a5a066700 1 --2- 172.21.15.1:0/2483219619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a5410a6e0 0x7f2a5419e0a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:10.249 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.677+0000 7f2a5a066700 1 -- 172.21.15.1:0/2483219619 >> 172.21.15.1:0/2483219619 conn(0x7f2a541041e0 msgr2=0x7f2a54105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:10.249 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.677+0000 7f2a5a066700 1 -- 172.21.15.1:0/2483219619 shutdown_connections 2024-05-04T03:08:10.249 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:09.677+0000 7f2a5a066700 1 -- 172.21.15.1:0/2483219619 wait complete. 2024-05-04T03:08:10.249 INFO:teuthology.orchestra.run.smithi001.stdout:Generating ssh key... 2024-05-04T03:08:11.334 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:11 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: Generating public/private rsa key pair. 2024-05-04T03:08:11.335 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:11 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: Your identification has been saved in /tmp/tmp4sm1lynu/key. 2024-05-04T03:08:11.335 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:11 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: Your public key has been saved in /tmp/tmp4sm1lynu/key.pub. 2024-05-04T03:08:11.335 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:11 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: The key fingerprint is: 2024-05-04T03:08:11.335 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:11 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: SHA256:uZGVHRMbz17HH2KaPGOvpWieP3IvEbcGFD32N2TC2mM ceph-4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:08:11.335 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:11 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: The key's randomart image is: 2024-05-04T03:08:11.335 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:11 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: +---[RSA 3072]----+ 2024-05-04T03:08:11.335 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:11 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: | .O. | 2024-05-04T03:08:11.335 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:11 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: | + & + | 2024-05-04T03:08:11.335 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:11 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: | + *oO.+| 2024-05-04T03:08:11.335 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:11 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: | +.++E.+=| 2024-05-04T03:08:11.335 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:11 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: | S B= + +| 2024-05-04T03:08:11.335 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:11 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: | o..+o | 2024-05-04T03:08:11.336 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:11 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: | . oo | 2024-05-04T03:08:11.336 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:11 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: | oo++ | 2024-05-04T03:08:11.336 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:11 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: | o++++. | 2024-05-04T03:08:11.336 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:11 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: +----[SHA256]-----+ 2024-05-04T03:08:11.626 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.671+0000 7fd7162e4700 1 Processor -- start 2024-05-04T03:08:11.626 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.671+0000 7fd7162e4700 1 -- start start 2024-05-04T03:08:11.626 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.671+0000 7fd7162e4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd71010a6e0 0x7fd71010cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:11.626 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.671+0000 7fd7162e4700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd71007ce70 con 0x7fd71010a6e0 2024-05-04T03:08:11.626 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.672+0000 7fd70ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd71010a6e0 0x7fd71010cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:11.626 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.672+0000 7fd70ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd71010a6e0 0x7fd71010cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43376/0 (socket says 172.21.15.1:43376) 2024-05-04T03:08:11.627 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.672+0000 7fd70ffff700 1 -- 172.21.15.1:0/714326946 learned_addr learned my addr 172.21.15.1:0/714326946 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:11.627 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.672+0000 7fd70ffff700 1 -- 172.21.15.1:0/714326946 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd71010d040 con 0x7fd71010a6e0 2024-05-04T03:08:11.627 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.672+0000 7fd70ffff700 1 --2- 172.21.15.1:0/714326946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd71010a6e0 0x7fd71010cb00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd7000098a0 tx=0x7fd700009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee1c17bf344c32c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:11.627 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.672+0000 7fd70effd700 1 -- 172.21.15.1:0/714326946 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd700004160 con 0x7fd71010a6e0 2024-05-04T03:08:11.627 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.672+0000 7fd70effd700 1 -- 172.21.15.1:0/714326946 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd700004300 con 0x7fd71010a6e0 2024-05-04T03:08:11.627 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.672+0000 7fd70effd700 1 -- 172.21.15.1:0/714326946 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd700011af0 con 0x7fd71010a6e0 2024-05-04T03:08:11.627 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.673+0000 7fd7162e4700 1 -- 172.21.15.1:0/714326946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd71010a6e0 msgr2=0x7fd71010cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:11.627 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.673+0000 7fd7162e4700 1 --2- 172.21.15.1:0/714326946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd71010a6e0 0x7fd71010cb00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd7000098a0 tx=0x7fd700009bb0 comp rx=0 tx=0).stop 2024-05-04T03:08:11.627 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.673+0000 7fd7162e4700 1 -- 172.21.15.1:0/714326946 shutdown_connections 2024-05-04T03:08:11.627 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.673+0000 7fd7162e4700 1 --2- 172.21.15.1:0/714326946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd71010a6e0 0x7fd71010cb00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:11.627 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.673+0000 7fd7162e4700 1 -- 172.21.15.1:0/714326946 >> 172.21.15.1:0/714326946 conn(0x7fd7101041e0 msgr2=0x7fd710106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:11.627 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.673+0000 7fd7162e4700 1 -- 172.21.15.1:0/714326946 shutdown_connections 2024-05-04T03:08:11.627 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.673+0000 7fd7162e4700 1 -- 172.21.15.1:0/714326946 wait complete. 2024-05-04T03:08:11.627 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.674+0000 7fd7162e4700 1 Processor -- start 2024-05-04T03:08:11.628 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.674+0000 7fd7162e4700 1 -- start start 2024-05-04T03:08:11.628 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.674+0000 7fd7162e4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd71010a6e0 0x7fd71019e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:11.628 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.674+0000 7fd7162e4700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd71010e000 con 0x7fd71010a6e0 2024-05-04T03:08:11.628 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.674+0000 7fd70ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd71010a6e0 0x7fd71019e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:11.628 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.674+0000 7fd70ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd71010a6e0 0x7fd71019e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43382/0 (socket says 172.21.15.1:43382) 2024-05-04T03:08:11.628 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.674+0000 7fd70ffff700 1 -- 172.21.15.1:0/3021289361 learned_addr learned my addr 172.21.15.1:0/3021289361 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:11.628 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.675+0000 7fd70ffff700 1 -- 172.21.15.1:0/3021289361 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd71019e5c0 con 0x7fd71010a6e0 2024-05-04T03:08:11.628 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.675+0000 7fd70ffff700 1 --2- 172.21.15.1:0/3021289361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd71010a6e0 0x7fd71019e080 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd700009fb0 tx=0x7fd700011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:11.628 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.675+0000 7fd70d7fa700 1 -- 172.21.15.1:0/3021289361 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd700011bc0 con 0x7fd71010a6e0 2024-05-04T03:08:11.628 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.675+0000 7fd70d7fa700 1 -- 172.21.15.1:0/3021289361 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd700024440 con 0x7fd71010a6e0 2024-05-04T03:08:11.628 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.675+0000 7fd7162e4700 1 -- 172.21.15.1:0/3021289361 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd71019e850 con 0x7fd71010a6e0 2024-05-04T03:08:11.628 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.675+0000 7fd70d7fa700 1 -- 172.21.15.1:0/3021289361 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd70001a5c0 con 0x7fd71010a6e0 2024-05-04T03:08:11.628 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.675+0000 7fd7162e4700 1 -- 172.21.15.1:0/3021289361 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7101a1460 con 0x7fd71010a6e0 2024-05-04T03:08:11.628 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.676+0000 7fd70d7fa700 1 -- 172.21.15.1:0/3021289361 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fd70001a830 con 0x7fd71010a6e0 2024-05-04T03:08:11.628 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.676+0000 7fd70d7fa700 1 --2- 172.21.15.1:0/3021289361 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7fd6f803e470 0x7fd6f8040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:11.629 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.676+0000 7fd7162e4700 1 -- 172.21.15.1:0/3021289361 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd710064e20 con 0x7fd71010a6e0 2024-05-04T03:08:11.629 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.676+0000 7fd70d7fa700 1 -- 172.21.15.1:0/3021289361 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd7000538f0 con 0x7fd71010a6e0 2024-05-04T03:08:11.629 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.677+0000 7fd70f7fe700 1 --2- 172.21.15.1:0/3021289361 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7fd6f803e470 0x7fd6f8040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:11.629 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.677+0000 7fd70f7fe700 1 --2- 172.21.15.1:0/3021289361 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7fd6f803e470 0x7fd6f8040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd704006fd0 tx=0x7fd704006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:11.629 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.682+0000 7fd70d7fa700 1 -- 172.21.15.1:0/3021289361 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd70000b3e0 con 0x7fd71010a6e0 2024-05-04T03:08:11.629 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:10.905+0000 7fd7162e4700 1 -- 172.21.15.1:0/3021289361 --> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fd710106380 con 0x7fd6f803e470 2024-05-04T03:08:11.629 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:11.042+0000 7fd70d7fa700 1 -- 172.21.15.1:0/3021289361 <== mgr.14118 v2:172.21.15.1:6800/128793132 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd710106380 con 0x7fd6f803e470 2024-05-04T03:08:11.629 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:11.044+0000 7fd7162e4700 1 -- 172.21.15.1:0/3021289361 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7fd6f803e470 msgr2=0x7fd6f8040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:11.629 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:11.044+0000 7fd7162e4700 1 --2- 172.21.15.1:0/3021289361 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7fd6f803e470 0x7fd6f8040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd704006fd0 tx=0x7fd704006e40 comp rx=0 tx=0).stop 2024-05-04T03:08:11.629 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:11.044+0000 7fd7162e4700 1 -- 172.21.15.1:0/3021289361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd71010a6e0 msgr2=0x7fd71019e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:11.629 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:11.044+0000 7fd7162e4700 1 --2- 172.21.15.1:0/3021289361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd71010a6e0 0x7fd71019e080 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd700009fb0 tx=0x7fd700011890 comp rx=0 tx=0).stop 2024-05-04T03:08:11.629 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:11.044+0000 7fd7162e4700 1 -- 172.21.15.1:0/3021289361 shutdown_connections 2024-05-04T03:08:11.629 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:11.044+0000 7fd7162e4700 1 --2- 172.21.15.1:0/3021289361 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7fd6f803e470 0x7fd6f8040980 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:11.630 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:11.044+0000 7fd7162e4700 1 --2- 172.21.15.1:0/3021289361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd71010a6e0 0x7fd71019e080 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:11.630 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:11.044+0000 7fd7162e4700 1 -- 172.21.15.1:0/3021289361 >> 172.21.15.1:0/3021289361 conn(0x7fd7101041e0 msgr2=0x7fd710105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:11.630 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:11.044+0000 7fd7162e4700 1 -- 172.21.15.1:0/3021289361 shutdown_connections 2024-05-04T03:08:11.630 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:11.044+0000 7fd7162e4700 1 -- 172.21.15.1:0/3021289361 wait complete. 2024-05-04T03:08:12.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:12 smithi001 ceph-mon[29731]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:08:12.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:12 smithi001 ceph-mon[29731]: Generating ssh key... 2024-05-04T03:08:12.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:12 smithi001 ceph-mon[29731]: from='mgr.14118 172.21.15.1:0/3287545089' entity='mgr.x' 2024-05-04T03:08:12.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:12 smithi001 ceph-mon[29731]: from='mgr.14118 172.21.15.1:0/3287545089' entity='mgr.x' 2024-05-04T03:08:12.874 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGVlXZK1X8uZZfL/hJsK8EueQx2js91wvEokJdSqH5Eimh4brSlWcuYmmf0mxR9mPT1+s5ymBO34rSSYfKFEfu1PiDM9sA3ov2wcME3Ax3lRwZJsK/qvpJ/4mcaOZMX7+Xg55ytxhUCe9pgHAivSdG4ufjDHnH3BKt4OR/89ZCRSBRGpkxiwM+1oMQzlwSKgju6wDWUBFrYBSvmcEIs/Biw1In+/4WhWnNYd63/AjXsGUI/6NPOaaqRFTeHVo7FtLvp50wZXxNeu5Td+NgCSvPdPE+YOAKM6coc9fM4NGSE+K+6rCN7p+XOqS+MF4Xq3g1BNI6gT6/GwXMTN6MX5hPwAIrTCuJJOJFDj32yw8EAGkP/k629ysBee7N8Bp6ORkUByyBbL181/pU/KQ3wRqHl2y8Z7k2DatyDP8Xdp0R25J9mQ65i7Mbxt1+s8GF8G2SNDba/awGwWHVjRRQpRrvoyAVREQWs7JUnkb8INyQjmZd8WXWrkOyTFL9wYgjUtE= ceph-4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:08:12.875 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.094+0000 7f328bc8c700 1 Processor -- start 2024-05-04T03:08:12.875 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.095+0000 7f328bc8c700 1 -- start start 2024-05-04T03:08:12.875 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.095+0000 7f328bc8c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3284109c40 0x7f328410a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:12.875 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.095+0000 7f328bc8c700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f328410a640 con 0x7f3284109c40 2024-05-04T03:08:12.875 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.095+0000 7f3289a28700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3284109c40 0x7f328410a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:12.875 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.095+0000 7f3289a28700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3284109c40 0x7f328410a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56408/0 (socket says 172.21.15.1:56408) 2024-05-04T03:08:12.875 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.095+0000 7f3289a28700 1 -- 172.21.15.1:0/2663678854 learned_addr learned my addr 172.21.15.1:0/2663678854 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:12.875 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.096+0000 7f3289a28700 1 -- 172.21.15.1:0/2663678854 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f328410ae10 con 0x7f3284109c40 2024-05-04T03:08:12.875 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.096+0000 7f3289a28700 1 --2- 172.21.15.1:0/2663678854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3284109c40 0x7f328410a050 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f32740098a0 tx=0x7f3274009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce28da37ff7d2025 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:12.875 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.096+0000 7f3288a26700 1 -- 172.21.15.1:0/2663678854 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3274004160 con 0x7f3284109c40 2024-05-04T03:08:12.875 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.096+0000 7f3288a26700 1 -- 172.21.15.1:0/2663678854 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3274004300 con 0x7f3284109c40 2024-05-04T03:08:12.876 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.096+0000 7f3288a26700 1 -- 172.21.15.1:0/2663678854 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3274011af0 con 0x7f3284109c40 2024-05-04T03:08:12.876 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.097+0000 7f328bc8c700 1 -- 172.21.15.1:0/2663678854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3284109c40 msgr2=0x7f328410a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:12.876 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.097+0000 7f328bc8c700 1 --2- 172.21.15.1:0/2663678854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3284109c40 0x7f328410a050 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f32740098a0 tx=0x7f3274009bb0 comp rx=0 tx=0).stop 2024-05-04T03:08:12.876 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.097+0000 7f328bc8c700 1 -- 172.21.15.1:0/2663678854 shutdown_connections 2024-05-04T03:08:12.876 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.097+0000 7f328bc8c700 1 --2- 172.21.15.1:0/2663678854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3284109c40 0x7f328410a050 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:12.876 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.097+0000 7f328bc8c700 1 -- 172.21.15.1:0/2663678854 >> 172.21.15.1:0/2663678854 conn(0x7f32841052c0 msgr2=0x7f3284107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:12.876 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.097+0000 7f328bc8c700 1 -- 172.21.15.1:0/2663678854 shutdown_connections 2024-05-04T03:08:12.876 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.097+0000 7f328bc8c700 1 -- 172.21.15.1:0/2663678854 wait complete. 2024-05-04T03:08:12.876 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.097+0000 7f328bc8c700 1 Processor -- start 2024-05-04T03:08:12.876 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.098+0000 7f328bc8c700 1 -- start start 2024-05-04T03:08:12.876 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.098+0000 7f328bc8c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3284109c40 0x7f328407ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:12.876 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.098+0000 7f328bc8c700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f328410e1c0 con 0x7f3284109c40 2024-05-04T03:08:12.876 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.098+0000 7f3289a28700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3284109c40 0x7f328407ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:12.876 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.098+0000 7f3289a28700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3284109c40 0x7f328407ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56424/0 (socket says 172.21.15.1:56424) 2024-05-04T03:08:12.876 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.098+0000 7f3289a28700 1 -- 172.21.15.1:0/2863150540 learned_addr learned my addr 172.21.15.1:0/2863150540 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:12.877 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.098+0000 7f3289a28700 1 -- 172.21.15.1:0/2863150540 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f328407efa0 con 0x7f3284109c40 2024-05-04T03:08:12.877 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.098+0000 7f3289a28700 1 --2- 172.21.15.1:0/2863150540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3284109c40 0x7f328407ea60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f32740090e0 tx=0x7f3274011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:12.877 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.099+0000 7f327affd700 1 -- 172.21.15.1:0/2863150540 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32740039e0 con 0x7f3284109c40 2024-05-04T03:08:12.877 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.099+0000 7f327affd700 1 -- 172.21.15.1:0/2863150540 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3274003b80 con 0x7f3284109c40 2024-05-04T03:08:12.877 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.099+0000 7f328bc8c700 1 -- 172.21.15.1:0/2863150540 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f328407d1f0 con 0x7f3284109c40 2024-05-04T03:08:12.877 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.099+0000 7f327affd700 1 -- 172.21.15.1:0/2863150540 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3274024690 con 0x7f3284109c40 2024-05-04T03:08:12.877 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.099+0000 7f328bc8c700 1 -- 172.21.15.1:0/2863150540 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f328407d6d0 con 0x7f3284109c40 2024-05-04T03:08:12.877 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.100+0000 7f328bc8c700 1 -- 172.21.15.1:0/2863150540 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3284064e20 con 0x7f3284109c40 2024-05-04T03:08:12.877 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.100+0000 7f327affd700 1 -- 172.21.15.1:0/2863150540 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f3274022020 con 0x7f3284109c40 2024-05-04T03:08:12.877 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.100+0000 7f327affd700 1 --2- 172.21.15.1:0/2863150540 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f327003e470 0x7f3270040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:12.877 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.100+0000 7f327affd700 1 -- 172.21.15.1:0/2863150540 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3274053a00 con 0x7f3284109c40 2024-05-04T03:08:12.877 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.100+0000 7f3289227700 1 --2- 172.21.15.1:0/2863150540 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f327003e470 0x7f3270040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:12.877 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.101+0000 7f3289227700 1 --2- 172.21.15.1:0/2863150540 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f327003e470 0x7f3270040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3280006fd0 tx=0x7f3280006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:12.877 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.105+0000 7f327affd700 1 -- 172.21.15.1:0/2863150540 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f32740172b0 con 0x7f3284109c40 2024-05-04T03:08:12.878 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.325+0000 7f328bc8c700 1 -- 172.21.15.1:0/2863150540 --> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f328407df50 con 0x7f327003e470 2024-05-04T03:08:12.878 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.326+0000 7f327affd700 1 -- 172.21.15.1:0/2863150540 <== mgr.14118 v2:172.21.15.1:6800/128793132 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f328407df50 con 0x7f327003e470 2024-05-04T03:08:12.878 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.328+0000 7f328bc8c700 1 -- 172.21.15.1:0/2863150540 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f327003e470 msgr2=0x7f3270040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:12.878 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.328+0000 7f328bc8c700 1 --2- 172.21.15.1:0/2863150540 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f327003e470 0x7f3270040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3280006fd0 tx=0x7f3280006e40 comp rx=0 tx=0).stop 2024-05-04T03:08:12.878 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.328+0000 7f328bc8c700 1 -- 172.21.15.1:0/2863150540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3284109c40 msgr2=0x7f328407ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:12.878 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.328+0000 7f328bc8c700 1 --2- 172.21.15.1:0/2863150540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3284109c40 0x7f328407ea60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f32740090e0 tx=0x7f3274011940 comp rx=0 tx=0).stop 2024-05-04T03:08:12.878 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.328+0000 7f328bc8c700 1 -- 172.21.15.1:0/2863150540 shutdown_connections 2024-05-04T03:08:12.878 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.328+0000 7f328bc8c700 1 --2- 172.21.15.1:0/2863150540 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f327003e470 0x7f3270040980 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:12.878 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.328+0000 7f328bc8c700 1 --2- 172.21.15.1:0/2863150540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3284109c40 0x7f328407ea60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:12.878 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.328+0000 7f328bc8c700 1 -- 172.21.15.1:0/2863150540 >> 172.21.15.1:0/2863150540 conn(0x7f32841052c0 msgr2=0x7f3284105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:12.878 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.328+0000 7f328bc8c700 1 -- 172.21.15.1:0/2863150540 shutdown_connections 2024-05-04T03:08:12.878 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:12.328+0000 7f328bc8c700 1 -- 172.21.15.1:0/2863150540 wait complete. 2024-05-04T03:08:12.878 INFO:teuthology.orchestra.run.smithi001.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-04T03:08:12.879 INFO:teuthology.orchestra.run.smithi001.stdout:Adding key to root@localhost authorized_keys... 2024-05-04T03:08:12.879 INFO:teuthology.orchestra.run.smithi001.stdout:Adding host smithi001... 2024-05-04T03:08:13.302 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:13 smithi001 ceph-mon[29731]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:08:14.180 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:14 smithi001 ceph-mon[29731]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi001", "addr": "172.21.15.1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:08:15.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:15 smithi001 ceph-mon[29731]: Deploying cephadm binary to smithi001 2024-05-04T03:08:18.342 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout Added host 'smithi001' with addr '172.21.15.1' 2024-05-04T03:08:18.342 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.347+0000 7f5f316f6700 1 Processor -- start 2024-05-04T03:08:18.342 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.347+0000 7f5f316f6700 1 -- start start 2024-05-04T03:08:18.342 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.348+0000 7f5f316f6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f2c10a6e0 0x7f5f2c10cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:18.342 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.348+0000 7f5f316f6700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c07ce70 con 0x7f5f2c10a6e0 2024-05-04T03:08:18.342 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.348+0000 7f5f2affd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f2c10a6e0 0x7f5f2c10cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:18.342 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.348+0000 7f5f2affd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f2c10a6e0 0x7f5f2c10cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56428/0 (socket says 172.21.15.1:56428) 2024-05-04T03:08:18.342 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.348+0000 7f5f2affd700 1 -- 172.21.15.1:0/1145402987 learned_addr learned my addr 172.21.15.1:0/1145402987 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:18.342 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.348+0000 7f5f2affd700 1 -- 172.21.15.1:0/1145402987 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2c10d040 con 0x7f5f2c10a6e0 2024-05-04T03:08:18.342 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.348+0000 7f5f2affd700 1 --2- 172.21.15.1:0/1145402987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f2c10a6e0 0x7f5f2c10cb00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0098a0 tx=0x7f5f1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46df7da4da1ef76 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:18.343 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.349+0000 7f5f29ffb700 1 -- 172.21.15.1:0/1145402987 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f1c004160 con 0x7f5f2c10a6e0 2024-05-04T03:08:18.343 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.349+0000 7f5f29ffb700 1 -- 172.21.15.1:0/1145402987 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5f1c004300 con 0x7f5f2c10a6e0 2024-05-04T03:08:18.343 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.349+0000 7f5f29ffb700 1 -- 172.21.15.1:0/1145402987 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f1c011af0 con 0x7f5f2c10a6e0 2024-05-04T03:08:18.343 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.349+0000 7f5f316f6700 1 -- 172.21.15.1:0/1145402987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f2c10a6e0 msgr2=0x7f5f2c10cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:18.343 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.349+0000 7f5f316f6700 1 --2- 172.21.15.1:0/1145402987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f2c10a6e0 0x7f5f2c10cb00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0098a0 tx=0x7f5f1c009bb0 comp rx=0 tx=0).stop 2024-05-04T03:08:18.343 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.349+0000 7f5f316f6700 1 -- 172.21.15.1:0/1145402987 shutdown_connections 2024-05-04T03:08:18.343 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.349+0000 7f5f316f6700 1 --2- 172.21.15.1:0/1145402987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f2c10a6e0 0x7f5f2c10cb00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:18.343 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.349+0000 7f5f316f6700 1 -- 172.21.15.1:0/1145402987 >> 172.21.15.1:0/1145402987 conn(0x7f5f2c1041e0 msgr2=0x7f5f2c106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:18.343 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.349+0000 7f5f316f6700 1 -- 172.21.15.1:0/1145402987 shutdown_connections 2024-05-04T03:08:18.343 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.349+0000 7f5f316f6700 1 -- 172.21.15.1:0/1145402987 wait complete. 2024-05-04T03:08:18.343 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.350+0000 7f5f316f6700 1 Processor -- start 2024-05-04T03:08:18.343 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.350+0000 7f5f316f6700 1 -- start start 2024-05-04T03:08:18.343 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.350+0000 7f5f316f6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f2c10a6e0 0x7f5f2c109e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:18.343 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.350+0000 7f5f316f6700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c10e000 con 0x7f5f2c10a6e0 2024-05-04T03:08:18.344 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.351+0000 7f5f2affd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f2c10a6e0 0x7f5f2c109e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:18.344 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.351+0000 7f5f2affd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f2c10a6e0 0x7f5f2c109e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56442/0 (socket says 172.21.15.1:56442) 2024-05-04T03:08:18.344 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.351+0000 7f5f2affd700 1 -- 172.21.15.1:0/3224782830 learned_addr learned my addr 172.21.15.1:0/3224782830 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:18.344 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.351+0000 7f5f2affd700 1 -- 172.21.15.1:0/3224782830 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2c10a3a0 con 0x7f5f2c10a6e0 2024-05-04T03:08:18.344 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.351+0000 7f5f2affd700 1 --2- 172.21.15.1:0/3224782830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f2c10a6e0 0x7f5f2c109e60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0117b0 tx=0x7f5f1c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:18.344 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.351+0000 7f5f13fff700 1 -- 172.21.15.1:0/3224782830 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f1c0039c0 con 0x7f5f2c10a6e0 2024-05-04T03:08:18.344 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.351+0000 7f5f316f6700 1 -- 172.21.15.1:0/3224782830 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f2c1085f0 con 0x7f5f2c10a6e0 2024-05-04T03:08:18.344 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.351+0000 7f5f13fff700 1 -- 172.21.15.1:0/3224782830 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5f1c003b60 con 0x7f5f2c10a6e0 2024-05-04T03:08:18.345 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.351+0000 7f5f13fff700 1 -- 172.21.15.1:0/3224782830 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f1c0196a0 con 0x7f5f2c10a6e0 2024-05-04T03:08:18.345 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.351+0000 7f5f316f6700 1 -- 172.21.15.1:0/3224782830 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f2c108ad0 con 0x7f5f2c10a6e0 2024-05-04T03:08:18.345 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.352+0000 7f5f316f6700 1 -- 172.21.15.1:0/3224782830 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f2c064e20 con 0x7f5f2c10a6e0 2024-05-04T03:08:18.345 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.352+0000 7f5f13fff700 1 -- 172.21.15.1:0/3224782830 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f5f1c019960 con 0x7f5f2c10a6e0 2024-05-04T03:08:18.345 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.353+0000 7f5f13fff700 1 --2- 172.21.15.1:0/3224782830 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f5f1403e470 0x7f5f14040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:18.345 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.353+0000 7f5f13fff700 1 -- 172.21.15.1:0/3224782830 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5f1c053b00 con 0x7f5f2c10a6e0 2024-05-04T03:08:18.345 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.353+0000 7f5f2a7fc700 1 --2- 172.21.15.1:0/3224782830 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f5f1403e470 0x7f5f14040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:18.345 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.353+0000 7f5f2a7fc700 1 --2- 172.21.15.1:0/3224782830 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f5f1403e470 0x7f5f14040980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5f20006fd0 tx=0x7f5f20006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:18.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.358+0000 7f5f13fff700 1 -- 172.21.15.1:0/3224782830 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5f1c017e90 con 0x7f5f2c10a6e0 2024-05-04T03:08:18.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:13.577+0000 7f5f316f6700 1 -- 172.21.15.1:0/3224782830 --> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi001", "addr": "172.21.15.1", "target": ["mon-mgr", ""]}) -- 0x7f5f2c109280 con 0x7f5f1403e470 2024-05-04T03:08:18.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:17.774+0000 7f5f13fff700 1 -- 172.21.15.1:0/3224782830 <== mgr.14118 v2:172.21.15.1:6800/128793132 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7f5f2c109280 con 0x7f5f1403e470 2024-05-04T03:08:18.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:17.777+0000 7f5f316f6700 1 -- 172.21.15.1:0/3224782830 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f5f1403e470 msgr2=0x7f5f14040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:18.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:17.777+0000 7f5f316f6700 1 --2- 172.21.15.1:0/3224782830 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f5f1403e470 0x7f5f14040980 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5f20006fd0 tx=0x7f5f20006e40 comp rx=0 tx=0).stop 2024-05-04T03:08:18.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:17.777+0000 7f5f316f6700 1 -- 172.21.15.1:0/3224782830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f2c10a6e0 msgr2=0x7f5f2c109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:18.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:17.777+0000 7f5f316f6700 1 --2- 172.21.15.1:0/3224782830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f2c10a6e0 0x7f5f2c109e60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0117b0 tx=0x7f5f1c011b70 comp rx=0 tx=0).stop 2024-05-04T03:08:18.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:17.777+0000 7f5f316f6700 1 -- 172.21.15.1:0/3224782830 shutdown_connections 2024-05-04T03:08:18.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:17.777+0000 7f5f316f6700 1 --2- 172.21.15.1:0/3224782830 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f5f1403e470 0x7f5f14040980 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:18.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:17.777+0000 7f5f316f6700 1 --2- 172.21.15.1:0/3224782830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f2c10a6e0 0x7f5f2c109e60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:18.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:17.777+0000 7f5f316f6700 1 -- 172.21.15.1:0/3224782830 >> 172.21.15.1:0/3224782830 conn(0x7f5f2c1041e0 msgr2=0x7f5f2c105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:18.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:17.777+0000 7f5f316f6700 1 -- 172.21.15.1:0/3224782830 shutdown_connections 2024-05-04T03:08:18.346 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:17.777+0000 7f5f316f6700 1 -- 172.21.15.1:0/3224782830 wait complete. 2024-05-04T03:08:18.346 INFO:teuthology.orchestra.run.smithi001.stdout:Deploying unmanaged mon service... 2024-05-04T03:08:18.997 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:18 smithi001 ceph-mon[29731]: from='mgr.14118 172.21.15.1:0/3287545089' entity='mgr.x' 2024-05-04T03:08:18.998 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:18 smithi001 ceph-mon[29731]: Added host smithi001 2024-05-04T03:08:18.998 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:18 smithi001 ceph-mon[29731]: from='mgr.14118 172.21.15.1:0/3287545089' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:08:19.591 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-04T03:08:19.591 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.757+0000 7fed50168700 1 Processor -- start 2024-05-04T03:08:19.591 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.757+0000 7fed50168700 1 -- start start 2024-05-04T03:08:19.591 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.757+0000 7fed50168700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed48108b00 0x7fed48108f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:19.591 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.757+0000 7fed50168700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fed48109500 con 0x7fed48108b00 2024-05-04T03:08:19.591 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.757+0000 7fed4df04700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed48108b00 0x7fed48108f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:19.591 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.757+0000 7fed4df04700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed48108b00 0x7fed48108f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56444/0 (socket says 172.21.15.1:56444) 2024-05-04T03:08:19.591 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.757+0000 7fed4df04700 1 -- 172.21.15.1:0/1572261639 learned_addr learned my addr 172.21.15.1:0/1572261639 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:19.591 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.758+0000 7fed4df04700 1 -- 172.21.15.1:0/1572261639 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed48109cd0 con 0x7fed48108b00 2024-05-04T03:08:19.591 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.758+0000 7fed4df04700 1 --2- 172.21.15.1:0/1572261639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed48108b00 0x7fed48108f10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fed440098a0 tx=0x7fed44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce87ee63427c328 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:19.591 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.758+0000 7fed4cf02700 1 -- 172.21.15.1:0/1572261639 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed44004160 con 0x7fed48108b00 2024-05-04T03:08:19.592 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.758+0000 7fed4cf02700 1 -- 172.21.15.1:0/1572261639 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fed44004300 con 0x7fed48108b00 2024-05-04T03:08:19.592 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.758+0000 7fed4cf02700 1 -- 172.21.15.1:0/1572261639 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed44011af0 con 0x7fed48108b00 2024-05-04T03:08:19.592 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.759+0000 7fed50168700 1 -- 172.21.15.1:0/1572261639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed48108b00 msgr2=0x7fed48108f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:19.592 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.759+0000 7fed50168700 1 --2- 172.21.15.1:0/1572261639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed48108b00 0x7fed48108f10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fed440098a0 tx=0x7fed44009bb0 comp rx=0 tx=0).stop 2024-05-04T03:08:19.592 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.759+0000 7fed50168700 1 -- 172.21.15.1:0/1572261639 shutdown_connections 2024-05-04T03:08:19.592 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.759+0000 7fed50168700 1 --2- 172.21.15.1:0/1572261639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed48108b00 0x7fed48108f10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:19.592 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.759+0000 7fed50168700 1 -- 172.21.15.1:0/1572261639 >> 172.21.15.1:0/1572261639 conn(0x7fed481041e0 msgr2=0x7fed48106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:19.592 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.759+0000 7fed50168700 1 -- 172.21.15.1:0/1572261639 shutdown_connections 2024-05-04T03:08:19.592 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.759+0000 7fed50168700 1 -- 172.21.15.1:0/1572261639 wait complete. 2024-05-04T03:08:19.592 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.760+0000 7fed50168700 1 Processor -- start 2024-05-04T03:08:19.592 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.760+0000 7fed50168700 1 -- start start 2024-05-04T03:08:19.592 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.760+0000 7fed50168700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed48108b00 0x7fed4807c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:19.592 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.760+0000 7fed50168700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fed4810d080 con 0x7fed48108b00 2024-05-04T03:08:19.592 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.760+0000 7fed4df04700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed48108b00 0x7fed4807c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:19.592 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.760+0000 7fed4df04700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed48108b00 0x7fed4807c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56454/0 (socket says 172.21.15.1:56454) 2024-05-04T03:08:19.593 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.760+0000 7fed4df04700 1 -- 172.21.15.1:0/945790406 learned_addr learned my addr 172.21.15.1:0/945790406 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:19.593 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.760+0000 7fed4df04700 1 -- 172.21.15.1:0/945790406 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed4807cee0 con 0x7fed48108b00 2024-05-04T03:08:19.593 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.760+0000 7fed4df04700 1 --2- 172.21.15.1:0/945790406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed48108b00 0x7fed4807c9a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fed440090e0 tx=0x7fed44011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:19.593 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.761+0000 7fed3effd700 1 -- 172.21.15.1:0/945790406 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed440039e0 con 0x7fed48108b00 2024-05-04T03:08:19.593 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.761+0000 7fed3effd700 1 -- 172.21.15.1:0/945790406 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fed44003b80 con 0x7fed48108b00 2024-05-04T03:08:19.593 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.761+0000 7fed50168700 1 -- 172.21.15.1:0/945790406 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed481a0e40 con 0x7fed48108b00 2024-05-04T03:08:19.593 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.761+0000 7fed3effd700 1 -- 172.21.15.1:0/945790406 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed44024690 con 0x7fed48108b00 2024-05-04T03:08:19.593 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.761+0000 7fed50168700 1 -- 172.21.15.1:0/945790406 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed481a1320 con 0x7fed48108b00 2024-05-04T03:08:19.593 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.762+0000 7fed50168700 1 -- 172.21.15.1:0/945790406 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed48064e20 con 0x7fed48108b00 2024-05-04T03:08:19.593 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.762+0000 7fed3effd700 1 -- 172.21.15.1:0/945790406 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fed44022020 con 0x7fed48108b00 2024-05-04T03:08:19.593 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.762+0000 7fed3effd700 1 --2- 172.21.15.1:0/945790406 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7fed3403e4c0 0x7fed340409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:19.593 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.762+0000 7fed3effd700 1 -- 172.21.15.1:0/945790406 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fed44053840 con 0x7fed48108b00 2024-05-04T03:08:19.594 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.762+0000 7fed4d703700 1 --2- 172.21.15.1:0/945790406 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7fed3403e4c0 0x7fed340409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:19.594 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.763+0000 7fed4d703700 1 --2- 172.21.15.1:0/945790406 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7fed3403e4c0 0x7fed340409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fed38006fd0 tx=0x7fed38006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:19.594 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.767+0000 7fed3effd700 1 -- 172.21.15.1:0/945790406 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fed440249f0 con 0x7fed48108b00 2024-05-04T03:08:19.594 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:18.997+0000 7fed50168700 1 -- 172.21.15.1:0/945790406 --> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fed48106370 con 0x7fed3403e4c0 2024-05-04T03:08:19.594 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.002+0000 7fed3effd700 1 -- 172.21.15.1:0/945790406 <== mgr.14118 v2:172.21.15.1:6800/128793132 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fed48106370 con 0x7fed3403e4c0 2024-05-04T03:08:19.594 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.004+0000 7fed50168700 1 -- 172.21.15.1:0/945790406 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7fed3403e4c0 msgr2=0x7fed340409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:19.594 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.004+0000 7fed50168700 1 --2- 172.21.15.1:0/945790406 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7fed3403e4c0 0x7fed340409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fed38006fd0 tx=0x7fed38006e40 comp rx=0 tx=0).stop 2024-05-04T03:08:19.594 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.004+0000 7fed50168700 1 -- 172.21.15.1:0/945790406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed48108b00 msgr2=0x7fed4807c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:19.594 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.004+0000 7fed50168700 1 --2- 172.21.15.1:0/945790406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed48108b00 0x7fed4807c9a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fed440090e0 tx=0x7fed44011940 comp rx=0 tx=0).stop 2024-05-04T03:08:19.594 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.005+0000 7fed50168700 1 -- 172.21.15.1:0/945790406 shutdown_connections 2024-05-04T03:08:19.594 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.005+0000 7fed50168700 1 --2- 172.21.15.1:0/945790406 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7fed3403e4c0 0x7fed340409d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:19.594 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.005+0000 7fed50168700 1 --2- 172.21.15.1:0/945790406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed48108b00 0x7fed4807c9a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:19.595 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.005+0000 7fed50168700 1 -- 172.21.15.1:0/945790406 >> 172.21.15.1:0/945790406 conn(0x7fed481041e0 msgr2=0x7fed48105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:19.595 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.005+0000 7fed50168700 1 -- 172.21.15.1:0/945790406 shutdown_connections 2024-05-04T03:08:19.595 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.005+0000 7fed50168700 1 -- 172.21.15.1:0/945790406 wait complete. 2024-05-04T03:08:19.595 INFO:teuthology.orchestra.run.smithi001.stdout:Deploying unmanaged mgr service... 2024-05-04T03:08:20.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:20 smithi001 ceph-mon[29731]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:08:20.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:20 smithi001 ceph-mon[29731]: Saving service mon spec with placement count:5 2024-05-04T03:08:20.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:20 smithi001 ceph-mon[29731]: from='mgr.14118 172.21.15.1:0/3287545089' entity='mgr.x' 2024-05-04T03:08:21.404 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-04T03:08:21.405 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.976+0000 7fbc5efb6700 1 Processor -- start 2024-05-04T03:08:21.405 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.976+0000 7fbc5efb6700 1 -- start start 2024-05-04T03:08:21.405 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.976+0000 7fbc5efb6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc5810a7c0 0x7fbc5810cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:21.405 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.976+0000 7fbc5efb6700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbc58077110 con 0x7fbc5810a7c0 2024-05-04T03:08:21.405 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.976+0000 7fbc5cd52700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc5810a7c0 0x7fbc5810cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:21.405 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.976+0000 7fbc5cd52700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc5810a7c0 0x7fbc5810cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56466/0 (socket says 172.21.15.1:56466) 2024-05-04T03:08:21.405 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.976+0000 7fbc5cd52700 1 -- 172.21.15.1:0/190212168 learned_addr learned my addr 172.21.15.1:0/190212168 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:21.405 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.977+0000 7fbc5cd52700 1 -- 172.21.15.1:0/190212168 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5810d120 con 0x7fbc5810a7c0 2024-05-04T03:08:21.405 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.977+0000 7fbc5cd52700 1 --2- 172.21.15.1:0/190212168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc5810a7c0 0x7fbc5810cbe0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0098a0 tx=0x7fbc4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f9cabc38f47d23 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:21.405 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.977+0000 7fbc577fe700 1 -- 172.21.15.1:0/190212168 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc4c004160 con 0x7fbc5810a7c0 2024-05-04T03:08:21.405 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.977+0000 7fbc577fe700 1 -- 172.21.15.1:0/190212168 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbc4c004300 con 0x7fbc5810a7c0 2024-05-04T03:08:21.405 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.977+0000 7fbc577fe700 1 -- 172.21.15.1:0/190212168 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc4c011af0 con 0x7fbc5810a7c0 2024-05-04T03:08:21.405 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.978+0000 7fbc5efb6700 1 -- 172.21.15.1:0/190212168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc5810a7c0 msgr2=0x7fbc5810cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:21.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.978+0000 7fbc5efb6700 1 --2- 172.21.15.1:0/190212168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc5810a7c0 0x7fbc5810cbe0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0098a0 tx=0x7fbc4c009bb0 comp rx=0 tx=0).stop 2024-05-04T03:08:21.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.978+0000 7fbc5efb6700 1 -- 172.21.15.1:0/190212168 shutdown_connections 2024-05-04T03:08:21.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.978+0000 7fbc5efb6700 1 --2- 172.21.15.1:0/190212168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc5810a7c0 0x7fbc5810cbe0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:21.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.978+0000 7fbc5efb6700 1 -- 172.21.15.1:0/190212168 >> 172.21.15.1:0/190212168 conn(0x7fbc58104240 msgr2=0x7fbc58106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:21.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.978+0000 7fbc5efb6700 1 -- 172.21.15.1:0/190212168 shutdown_connections 2024-05-04T03:08:21.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.978+0000 7fbc5efb6700 1 -- 172.21.15.1:0/190212168 wait complete. 2024-05-04T03:08:21.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.979+0000 7fbc5efb6700 1 Processor -- start 2024-05-04T03:08:21.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.979+0000 7fbc5efb6700 1 -- start start 2024-05-04T03:08:21.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.979+0000 7fbc5efb6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc5810a7c0 0x7fbc5819e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:21.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.979+0000 7fbc5efb6700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5810e080 con 0x7fbc5810a7c0 2024-05-04T03:08:21.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.979+0000 7fbc5cd52700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc5810a7c0 0x7fbc5819e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:21.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.979+0000 7fbc5cd52700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc5810a7c0 0x7fbc5819e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56474/0 (socket says 172.21.15.1:56474) 2024-05-04T03:08:21.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.979+0000 7fbc5cd52700 1 -- 172.21.15.1:0/2475997025 learned_addr learned my addr 172.21.15.1:0/2475997025 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:21.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.979+0000 7fbc5cd52700 1 -- 172.21.15.1:0/2475997025 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5819e580 con 0x7fbc5810a7c0 2024-05-04T03:08:21.407 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.979+0000 7fbc5cd52700 1 --2- 172.21.15.1:0/2475997025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc5810a7c0 0x7fbc5819e040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009fb0 tx=0x7fbc4c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:21.407 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.980+0000 7fbc55ffb700 1 -- 172.21.15.1:0/2475997025 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc4c0036a0 con 0x7fbc5810a7c0 2024-05-04T03:08:21.407 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.980+0000 7fbc55ffb700 1 -- 172.21.15.1:0/2475997025 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbc4c01a430 con 0x7fbc5810a7c0 2024-05-04T03:08:21.407 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.980+0000 7fbc5efb6700 1 -- 172.21.15.1:0/2475997025 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc5819e810 con 0x7fbc5810a7c0 2024-05-04T03:08:21.407 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.980+0000 7fbc55ffb700 1 -- 172.21.15.1:0/2475997025 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc4c019550 con 0x7fbc5810a7c0 2024-05-04T03:08:21.407 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.980+0000 7fbc5efb6700 1 -- 172.21.15.1:0/2475997025 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc581a1420 con 0x7fbc5810a7c0 2024-05-04T03:08:21.407 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.981+0000 7fbc55ffb700 1 -- 172.21.15.1:0/2475997025 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fbc4c019800 con 0x7fbc5810a7c0 2024-05-04T03:08:21.407 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.981+0000 7fbc5efb6700 1 -- 172.21.15.1:0/2475997025 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc58064e20 con 0x7fbc5810a7c0 2024-05-04T03:08:21.407 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.981+0000 7fbc55ffb700 1 --2- 172.21.15.1:0/2475997025 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7fbc4803e470 0x7fbc48040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:21.407 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.981+0000 7fbc55ffb700 1 -- 172.21.15.1:0/2475997025 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbc4c0538a0 con 0x7fbc5810a7c0 2024-05-04T03:08:21.407 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.981+0000 7fbc57fff700 1 --2- 172.21.15.1:0/2475997025 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7fbc4803e470 0x7fbc48040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:21.407 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.982+0000 7fbc57fff700 1 --2- 172.21.15.1:0/2475997025 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7fbc4803e470 0x7fbc48040980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbc44006fd0 tx=0x7fbc44006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:21.407 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:19.986+0000 7fbc55ffb700 1 -- 172.21.15.1:0/2475997025 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbc4c017e90 con 0x7fbc5810a7c0 2024-05-04T03:08:21.408 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:20.222+0000 7fbc5efb6700 1 -- 172.21.15.1:0/2475997025 --> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fbc581062a0 con 0x7fbc4803e470 2024-05-04T03:08:21.408 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:20.227+0000 7fbc55ffb700 1 -- 172.21.15.1:0/2475997025 <== mgr.14118 v2:172.21.15.1:6800/128793132 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fbc581062a0 con 0x7fbc4803e470 2024-05-04T03:08:21.408 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:20.229+0000 7fbc5efb6700 1 -- 172.21.15.1:0/2475997025 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7fbc4803e470 msgr2=0x7fbc48040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:21.408 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:20.229+0000 7fbc5efb6700 1 --2- 172.21.15.1:0/2475997025 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7fbc4803e470 0x7fbc48040980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbc44006fd0 tx=0x7fbc44006e40 comp rx=0 tx=0).stop 2024-05-04T03:08:21.408 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:20.229+0000 7fbc5efb6700 1 -- 172.21.15.1:0/2475997025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc5810a7c0 msgr2=0x7fbc5819e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:21.408 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:20.229+0000 7fbc5efb6700 1 --2- 172.21.15.1:0/2475997025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc5810a7c0 0x7fbc5819e040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009fb0 tx=0x7fbc4c011940 comp rx=0 tx=0).stop 2024-05-04T03:08:21.408 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:20.230+0000 7fbc5efb6700 1 -- 172.21.15.1:0/2475997025 shutdown_connections 2024-05-04T03:08:21.408 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:20.230+0000 7fbc5efb6700 1 --2- 172.21.15.1:0/2475997025 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7fbc4803e470 0x7fbc48040980 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:21.408 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:20.230+0000 7fbc5efb6700 1 --2- 172.21.15.1:0/2475997025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc5810a7c0 0x7fbc5819e040 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:21.408 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:20.230+0000 7fbc5efb6700 1 -- 172.21.15.1:0/2475997025 >> 172.21.15.1:0/2475997025 conn(0x7fbc58104240 msgr2=0x7fbc58105b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:21.408 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:20.230+0000 7fbc5efb6700 1 -- 172.21.15.1:0/2475997025 shutdown_connections 2024-05-04T03:08:21.408 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:20.230+0000 7fbc5efb6700 1 -- 172.21.15.1:0/2475997025 wait complete. 2024-05-04T03:08:21.488 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:21 smithi001 ceph-mon[29731]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:08:21.488 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:21 smithi001 ceph-mon[29731]: Saving service mgr spec with placement count:2 2024-05-04T03:08:21.488 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:21 smithi001 ceph-mon[29731]: from='mgr.14118 172.21.15.1:0/3287545089' entity='mgr.x' 2024-05-04T03:08:21.488 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:21 smithi001 ceph-mon[29731]: from='mgr.14118 172.21.15.1:0/3287545089' entity='mgr.x' 2024-05-04T03:08:22.663 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.861+0000 7f994efef700 1 Processor -- start 2024-05-04T03:08:22.664 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.861+0000 7f994efef700 1 -- start start 2024-05-04T03:08:22.664 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.861+0000 7f994efef700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9948109cd0 0x7f994810a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:22.664 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.861+0000 7f994efef700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f994810a6d0 con 0x7f9948109cd0 2024-05-04T03:08:22.664 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.861+0000 7f994cd8b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9948109cd0 0x7f994810a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:22.664 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.861+0000 7f994cd8b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9948109cd0 0x7f994810a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59392/0 (socket says 172.21.15.1:59392) 2024-05-04T03:08:22.664 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.861+0000 7f994cd8b700 1 -- 172.21.15.1:0/3469628753 learned_addr learned my addr 172.21.15.1:0/3469628753 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:22.665 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.862+0000 7f994cd8b700 1 -- 172.21.15.1:0/3469628753 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994810aea0 con 0x7f9948109cd0 2024-05-04T03:08:22.665 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.862+0000 7f994cd8b700 1 --2- 172.21.15.1:0/3469628753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9948109cd0 0x7f994810a0e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f993c0098a0 tx=0x7f993c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dee0820d4f89313 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:22.665 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.862+0000 7f99477fe700 1 -- 172.21.15.1:0/3469628753 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f993c004160 con 0x7f9948109cd0 2024-05-04T03:08:22.665 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.862+0000 7f99477fe700 1 -- 172.21.15.1:0/3469628753 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f993c004300 con 0x7f9948109cd0 2024-05-04T03:08:22.665 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.862+0000 7f99477fe700 1 -- 172.21.15.1:0/3469628753 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f993c011a20 con 0x7f9948109cd0 2024-05-04T03:08:22.665 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.863+0000 7f994efef700 1 -- 172.21.15.1:0/3469628753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9948109cd0 msgr2=0x7f994810a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:22.666 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.863+0000 7f994efef700 1 --2- 172.21.15.1:0/3469628753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9948109cd0 0x7f994810a0e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f993c0098a0 tx=0x7f993c009bb0 comp rx=0 tx=0).stop 2024-05-04T03:08:22.666 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.864+0000 7f994efef700 1 -- 172.21.15.1:0/3469628753 shutdown_connections 2024-05-04T03:08:22.666 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.864+0000 7f994efef700 1 --2- 172.21.15.1:0/3469628753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9948109cd0 0x7f994810a0e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:22.666 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.864+0000 7f994efef700 1 -- 172.21.15.1:0/3469628753 >> 172.21.15.1:0/3469628753 conn(0x7f9948105330 msgr2=0x7f99481077c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:22.666 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.864+0000 7f994efef700 1 -- 172.21.15.1:0/3469628753 shutdown_connections 2024-05-04T03:08:22.667 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.864+0000 7f994efef700 1 -- 172.21.15.1:0/3469628753 wait complete. 2024-05-04T03:08:22.667 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.865+0000 7f994efef700 1 Processor -- start 2024-05-04T03:08:22.667 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.865+0000 7f994efef700 1 -- start start 2024-05-04T03:08:22.667 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.865+0000 7f994efef700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9948109cd0 0x7f994819e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:22.667 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.865+0000 7f994efef700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f994810e250 con 0x7f9948109cd0 2024-05-04T03:08:22.667 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.865+0000 7f994cd8b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9948109cd0 0x7f994819e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:22.667 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.865+0000 7f994cd8b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9948109cd0 0x7f994819e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59402/0 (socket says 172.21.15.1:59402) 2024-05-04T03:08:22.668 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.865+0000 7f994cd8b700 1 -- 172.21.15.1:0/1541960283 learned_addr learned my addr 172.21.15.1:0/1541960283 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:22.668 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.865+0000 7f994cd8b700 1 -- 172.21.15.1:0/1541960283 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994819e570 con 0x7f9948109cd0 2024-05-04T03:08:22.668 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.866+0000 7f994cd8b700 1 --2- 172.21.15.1:0/1541960283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9948109cd0 0x7f994819e030 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f993c009fd0 tx=0x7f993c003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:22.668 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.866+0000 7f9945ffb700 1 -- 172.21.15.1:0/1541960283 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f993c011a20 con 0x7f9948109cd0 2024-05-04T03:08:22.668 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.866+0000 7f994efef700 1 -- 172.21.15.1:0/1541960283 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994819e800 con 0x7f9948109cd0 2024-05-04T03:08:22.668 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.866+0000 7f9945ffb700 1 -- 172.21.15.1:0/1541960283 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f993c003b50 con 0x7f9948109cd0 2024-05-04T03:08:22.669 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.866+0000 7f9945ffb700 1 -- 172.21.15.1:0/1541960283 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f993c019620 con 0x7f9948109cd0 2024-05-04T03:08:22.669 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.866+0000 7f994efef700 1 -- 172.21.15.1:0/1541960283 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99481a1410 con 0x7f9948109cd0 2024-05-04T03:08:22.669 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.867+0000 7f9945ffb700 1 -- 172.21.15.1:0/1541960283 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f993c0198d0 con 0x7f9948109cd0 2024-05-04T03:08:22.669 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.867+0000 7f994efef700 1 -- 172.21.15.1:0/1541960283 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9948064e20 con 0x7f9948109cd0 2024-05-04T03:08:22.669 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.868+0000 7f9945ffb700 1 --2- 172.21.15.1:0/1541960283 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f993803e470 0x7f9938040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:22.669 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.868+0000 7f9945ffb700 1 -- 172.21.15.1:0/1541960283 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f993c054bd0 con 0x7f9948109cd0 2024-05-04T03:08:22.670 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.868+0000 7f9947fff700 1 --2- 172.21.15.1:0/1541960283 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f993803e470 0x7f9938040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:22.670 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.868+0000 7f9947fff700 1 --2- 172.21.15.1:0/1541960283 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f993803e470 0x7f9938040980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9934006fd0 tx=0x7f9934006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:22.670 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:21.873+0000 7f9945ffb700 1 -- 172.21.15.1:0/1541960283 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f993c0044b0 con 0x7f9948109cd0 2024-05-04T03:08:22.670 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:22.090+0000 7f994efef700 1 -- 172.21.15.1:0/1541960283 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f99481a1a50 con 0x7f9948109cd0 2024-05-04T03:08:22.670 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:22.093+0000 7f9945ffb700 1 -- 172.21.15.1:0/1541960283 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f993c023390 con 0x7f9948109cd0 2024-05-04T03:08:22.671 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:22.096+0000 7f994efef700 1 -- 172.21.15.1:0/1541960283 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f993803e470 msgr2=0x7f9938040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:22.671 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:22.096+0000 7f994efef700 1 --2- 172.21.15.1:0/1541960283 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f993803e470 0x7f9938040980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9934006fd0 tx=0x7f9934006e40 comp rx=0 tx=0).stop 2024-05-04T03:08:22.671 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:22.096+0000 7f994efef700 1 -- 172.21.15.1:0/1541960283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9948109cd0 msgr2=0x7f994819e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:22.671 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:22.096+0000 7f994efef700 1 --2- 172.21.15.1:0/1541960283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9948109cd0 0x7f994819e030 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f993c009fd0 tx=0x7f993c003980 comp rx=0 tx=0).stop 2024-05-04T03:08:22.671 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:22.096+0000 7f994efef700 1 -- 172.21.15.1:0/1541960283 shutdown_connections 2024-05-04T03:08:22.672 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:22.096+0000 7f994efef700 1 --2- 172.21.15.1:0/1541960283 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f993803e470 0x7f9938040980 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:22.672 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:22.096+0000 7f994efef700 1 --2- 172.21.15.1:0/1541960283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9948109cd0 0x7f994819e030 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:22.672 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:22.096+0000 7f994efef700 1 -- 172.21.15.1:0/1541960283 >> 172.21.15.1:0/1541960283 conn(0x7f9948105330 msgr2=0x7f9948105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:22.672 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:22.096+0000 7f994efef700 1 -- 172.21.15.1:0/1541960283 shutdown_connections 2024-05-04T03:08:22.672 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:22.096+0000 7f994efef700 1 -- 172.21.15.1:0/1541960283 wait complete. 2024-05-04T03:08:23.005 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:22 smithi001 ceph-mon[29731]: from='mgr.14118 172.21.15.1:0/3287545089' entity='mgr.x' 2024-05-04T03:08:23.006 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:22 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/1541960283' entity='client.admin' 2024-05-04T03:08:24.428 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.124+0000 7f4531664700 1 Processor -- start 2024-05-04T03:08:24.428 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.124+0000 7f4531664700 1 -- start start 2024-05-04T03:08:24.428 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.124+0000 7f4531664700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f452c108b00 0x7f452c108f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:24.428 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.124+0000 7f4531664700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f452c109500 con 0x7f452c108b00 2024-05-04T03:08:24.428 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.124+0000 7f452affd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f452c108b00 0x7f452c108f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:24.429 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.124+0000 7f452affd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f452c108b00 0x7f452c108f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59404/0 (socket says 172.21.15.1:59404) 2024-05-04T03:08:24.429 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.124+0000 7f452affd700 1 -- 172.21.15.1:0/1618743555 learned_addr learned my addr 172.21.15.1:0/1618743555 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:24.429 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.125+0000 7f452affd700 1 -- 172.21.15.1:0/1618743555 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452c109cd0 con 0x7f452c108b00 2024-05-04T03:08:24.429 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.125+0000 7f452affd700 1 --2- 172.21.15.1:0/1618743555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f452c108b00 0x7f452c108f10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f451c0098a0 tx=0x7f451c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af1c0e45f99099e server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:24.429 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.125+0000 7f4529ffb700 1 -- 172.21.15.1:0/1618743555 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f451c004160 con 0x7f452c108b00 2024-05-04T03:08:24.429 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.125+0000 7f4529ffb700 1 -- 172.21.15.1:0/1618743555 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f451c004300 con 0x7f452c108b00 2024-05-04T03:08:24.430 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.125+0000 7f4529ffb700 1 -- 172.21.15.1:0/1618743555 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f451c011a20 con 0x7f452c108b00 2024-05-04T03:08:24.430 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.126+0000 7f4531664700 1 -- 172.21.15.1:0/1618743555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f452c108b00 msgr2=0x7f452c108f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:24.430 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.126+0000 7f4531664700 1 --2- 172.21.15.1:0/1618743555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f452c108b00 0x7f452c108f10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f451c0098a0 tx=0x7f451c009bb0 comp rx=0 tx=0).stop 2024-05-04T03:08:24.430 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.126+0000 7f4531664700 1 -- 172.21.15.1:0/1618743555 shutdown_connections 2024-05-04T03:08:24.430 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.126+0000 7f4531664700 1 --2- 172.21.15.1:0/1618743555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f452c108b00 0x7f452c108f10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:24.430 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.126+0000 7f4531664700 1 -- 172.21.15.1:0/1618743555 >> 172.21.15.1:0/1618743555 conn(0x7f452c1041e0 msgr2=0x7f452c106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:24.431 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.126+0000 7f4531664700 1 -- 172.21.15.1:0/1618743555 shutdown_connections 2024-05-04T03:08:24.431 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.126+0000 7f4531664700 1 -- 172.21.15.1:0/1618743555 wait complete. 2024-05-04T03:08:24.431 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.126+0000 7f4531664700 1 Processor -- start 2024-05-04T03:08:24.431 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.127+0000 7f4531664700 1 -- start start 2024-05-04T03:08:24.431 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.127+0000 7f4531664700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f452c108b00 0x7f452c07c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:24.431 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.127+0000 7f4531664700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f452c10d080 con 0x7f452c108b00 2024-05-04T03:08:24.432 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.127+0000 7f452affd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f452c108b00 0x7f452c07c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:24.432 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.127+0000 7f452affd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f452c108b00 0x7f452c07c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59420/0 (socket says 172.21.15.1:59420) 2024-05-04T03:08:24.432 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.127+0000 7f452affd700 1 -- 172.21.15.1:0/295646730 learned_addr learned my addr 172.21.15.1:0/295646730 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:24.432 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.127+0000 7f452affd700 1 -- 172.21.15.1:0/295646730 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452c07ce80 con 0x7f452c108b00 2024-05-04T03:08:24.432 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.127+0000 7f452affd700 1 --2- 172.21.15.1:0/295646730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f452c108b00 0x7f452c07c940 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f451c009130 tx=0x7f451c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:24.432 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.127+0000 7f4513fff700 1 -- 172.21.15.1:0/295646730 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f451c003aa0 con 0x7f452c108b00 2024-05-04T03:08:24.433 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.128+0000 7f4531664700 1 -- 172.21.15.1:0/295646730 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452c1a0e50 con 0x7f452c108b00 2024-05-04T03:08:24.433 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.128+0000 7f4513fff700 1 -- 172.21.15.1:0/295646730 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f451c01a8e0 con 0x7f452c108b00 2024-05-04T03:08:24.433 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.128+0000 7f4513fff700 1 -- 172.21.15.1:0/295646730 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f451c0248f0 con 0x7f452c108b00 2024-05-04T03:08:24.433 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.128+0000 7f4531664700 1 -- 172.21.15.1:0/295646730 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f452c1a12d0 con 0x7f452c108b00 2024-05-04T03:08:24.433 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.129+0000 7f4513fff700 1 -- 172.21.15.1:0/295646730 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f451c022020 con 0x7f452c108b00 2024-05-04T03:08:24.433 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.129+0000 7f4513fff700 1 --2- 172.21.15.1:0/295646730 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f451403e470 0x7f4514040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:24.433 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.129+0000 7f4513fff700 1 -- 172.21.15.1:0/295646730 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f451c0547c0 con 0x7f452c108b00 2024-05-04T03:08:24.434 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.129+0000 7f4531664700 1 -- 172.21.15.1:0/295646730 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f452c064e20 con 0x7f452c108b00 2024-05-04T03:08:24.434 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.129+0000 7f452a7fc700 1 --2- 172.21.15.1:0/295646730 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f451403e470 0x7f4514040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:24.434 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.129+0000 7f452a7fc700 1 --2- 172.21.15.1:0/295646730 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f451403e470 0x7f4514040980 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4520006fd0 tx=0x7f4520006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:24.434 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.134+0000 7f4513fff700 1 -- 172.21.15.1:0/295646730 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f451c029df0 con 0x7f452c108b00 2024-05-04T03:08:24.434 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.371+0000 7f4531664700 1 -- 172.21.15.1:0/295646730 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f452c1a1690 con 0x7f452c108b00 2024-05-04T03:08:24.434 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.374+0000 7f4513fff700 1 -- 172.21.15.1:0/295646730 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f451c01aa90 con 0x7f452c108b00 2024-05-04T03:08:24.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.376+0000 7f4531664700 1 -- 172.21.15.1:0/295646730 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f451403e470 msgr2=0x7f4514040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:24.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.376+0000 7f4531664700 1 --2- 172.21.15.1:0/295646730 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f451403e470 0x7f4514040980 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4520006fd0 tx=0x7f4520006e40 comp rx=0 tx=0).stop 2024-05-04T03:08:24.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.376+0000 7f4531664700 1 -- 172.21.15.1:0/295646730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f452c108b00 msgr2=0x7f452c07c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:24.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.376+0000 7f4531664700 1 --2- 172.21.15.1:0/295646730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f452c108b00 0x7f452c07c940 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f451c009130 tx=0x7f451c011aa0 comp rx=0 tx=0).stop 2024-05-04T03:08:24.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.376+0000 7f4531664700 1 -- 172.21.15.1:0/295646730 shutdown_connections 2024-05-04T03:08:24.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.376+0000 7f4531664700 1 --2- 172.21.15.1:0/295646730 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f451403e470 0x7f4514040980 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:24.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.376+0000 7f4531664700 1 --2- 172.21.15.1:0/295646730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f452c108b00 0x7f452c07c940 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:24.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.376+0000 7f4531664700 1 -- 172.21.15.1:0/295646730 >> 172.21.15.1:0/295646730 conn(0x7f452c1041e0 msgr2=0x7f452c105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:24.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.376+0000 7f4531664700 1 -- 172.21.15.1:0/295646730 shutdown_connections 2024-05-04T03:08:24.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:23.376+0000 7f4531664700 1 -- 172.21.15.1:0/295646730 wait complete. 2024-05-04T03:08:24.436 INFO:teuthology.orchestra.run.smithi001.stdout:Enabling the dashboard module... 2024-05-04T03:08:24.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:24 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/295646730' entity='client.admin' 2024-05-04T03:08:25.794 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:25 smithi001 ceph-mon[29731]: from='mgr.14118 172.21.15.1:0/3287545089' entity='mgr.x' 2024-05-04T03:08:25.794 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:25 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2394961966' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-04T03:08:25.795 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:25 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: ignoring --setuser ceph since I am not root 2024-05-04T03:08:25.795 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:25 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: ignoring --setgroup ceph since I am not root 2024-05-04T03:08:25.795 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:25 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:25.726+0000 7f406913a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-04T03:08:26.053 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:25 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:25.943+0000 7f406913a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-04T03:08:26.054 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:26 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:26.050+0000 7f406913a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-04T03:08:26.181 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.914+0000 7f2f3441f700 1 Processor -- start 2024-05-04T03:08:26.181 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.914+0000 7f2f3441f700 1 -- start start 2024-05-04T03:08:26.182 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.915+0000 7f2f3441f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f2c10a750 0x7f2f2c10cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:26.182 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.915+0000 7f2f3441f700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c07ce10 con 0x7f2f2c10a750 2024-05-04T03:08:26.182 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.915+0000 7f2f321bb700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f2c10a750 0x7f2f2c10cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:26.182 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.915+0000 7f2f321bb700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f2c10a750 0x7f2f2c10cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59434/0 (socket says 172.21.15.1:59434) 2024-05-04T03:08:26.182 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.915+0000 7f2f321bb700 1 -- 172.21.15.1:0/655586958 learned_addr learned my addr 172.21.15.1:0/655586958 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:26.183 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.916+0000 7f2f321bb700 1 -- 172.21.15.1:0/655586958 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f2c10d0b0 con 0x7f2f2c10a750 2024-05-04T03:08:26.183 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.916+0000 7f2f321bb700 1 --2- 172.21.15.1:0/655586958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f2c10a750 0x7f2f2c10cb70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2f280098a0 tx=0x7f2f28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5428ac4d23ee922e server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:26.187 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.916+0000 7f2f311b9700 1 -- 172.21.15.1:0/655586958 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f28004160 con 0x7f2f2c10a750 2024-05-04T03:08:26.187 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.916+0000 7f2f311b9700 1 -- 172.21.15.1:0/655586958 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f28004300 con 0x7f2f2c10a750 2024-05-04T03:08:26.187 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.916+0000 7f2f311b9700 1 -- 172.21.15.1:0/655586958 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f28011a20 con 0x7f2f2c10a750 2024-05-04T03:08:26.187 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.917+0000 7f2f3441f700 1 -- 172.21.15.1:0/655586958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f2c10a750 msgr2=0x7f2f2c10cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:26.187 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.917+0000 7f2f3441f700 1 --2- 172.21.15.1:0/655586958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f2c10a750 0x7f2f2c10cb70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2f280098a0 tx=0x7f2f28009bb0 comp rx=0 tx=0).stop 2024-05-04T03:08:26.187 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.917+0000 7f2f3441f700 1 -- 172.21.15.1:0/655586958 shutdown_connections 2024-05-04T03:08:26.188 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.917+0000 7f2f3441f700 1 --2- 172.21.15.1:0/655586958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f2c10a750 0x7f2f2c10cb70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:26.188 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.917+0000 7f2f3441f700 1 -- 172.21.15.1:0/655586958 >> 172.21.15.1:0/655586958 conn(0x7f2f2c104190 msgr2=0x7f2f2c106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:26.188 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.917+0000 7f2f3441f700 1 -- 172.21.15.1:0/655586958 shutdown_connections 2024-05-04T03:08:26.188 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.917+0000 7f2f3441f700 1 -- 172.21.15.1:0/655586958 wait complete. 2024-05-04T03:08:26.188 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.918+0000 7f2f3441f700 1 Processor -- start 2024-05-04T03:08:26.188 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.918+0000 7f2f3441f700 1 -- start start 2024-05-04T03:08:26.188 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.918+0000 7f2f3441f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f2c10a750 0x7f2f2c19dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:26.188 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.918+0000 7f2f3441f700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c10e050 con 0x7f2f2c10a750 2024-05-04T03:08:26.188 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.918+0000 7f2f321bb700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f2c10a750 0x7f2f2c19dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:26.188 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.918+0000 7f2f321bb700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f2c10a750 0x7f2f2c19dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59442/0 (socket says 172.21.15.1:59442) 2024-05-04T03:08:26.188 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.918+0000 7f2f321bb700 1 -- 172.21.15.1:0/2394961966 learned_addr learned my addr 172.21.15.1:0/2394961966 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:26.188 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.919+0000 7f2f321bb700 1 -- 172.21.15.1:0/2394961966 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f2c19e4f0 con 0x7f2f2c10a750 2024-05-04T03:08:26.188 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.919+0000 7f2f321bb700 1 --2- 172.21.15.1:0/2394961966 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f2c10a750 0x7f2f2c19dfb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2f280119f0 tx=0x7f2f28011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:26.188 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.919+0000 7f2f237fe700 1 -- 172.21.15.1:0/2394961966 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f280039e0 con 0x7f2f2c10a750 2024-05-04T03:08:26.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.919+0000 7f2f3441f700 1 -- 172.21.15.1:0/2394961966 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f2c19e780 con 0x7f2f2c10a750 2024-05-04T03:08:26.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.919+0000 7f2f237fe700 1 -- 172.21.15.1:0/2394961966 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f2801b8f0 con 0x7f2f2c10a750 2024-05-04T03:08:26.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.919+0000 7f2f237fe700 1 -- 172.21.15.1:0/2394961966 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f280237a0 con 0x7f2f2c10a750 2024-05-04T03:08:26.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.919+0000 7f2f3441f700 1 -- 172.21.15.1:0/2394961966 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f2c1a1390 con 0x7f2f2c10a750 2024-05-04T03:08:26.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.920+0000 7f2f3441f700 1 -- 172.21.15.1:0/2394961966 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f2c064e20 con 0x7f2f2c10a750 2024-05-04T03:08:26.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.920+0000 7f2f237fe700 1 -- 172.21.15.1:0/2394961966 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f2f28023aa0 con 0x7f2f2c10a750 2024-05-04T03:08:26.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.920+0000 7f2f237fe700 1 --2- 172.21.15.1:0/2394961966 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f2f1803e420 0x7f2f18040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:26.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.920+0000 7f2f237fe700 1 -- 172.21.15.1:0/2394961966 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2f28053f50 con 0x7f2f2c10a750 2024-05-04T03:08:26.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.920+0000 7f2f319ba700 1 --2- 172.21.15.1:0/2394961966 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f2f1803e420 0x7f2f18040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:26.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.921+0000 7f2f319ba700 1 --2- 172.21.15.1:0/2394961966 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f2f1803e420 0x7f2f18040930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c006fd0 tx=0x7f2f1c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:26.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:24.925+0000 7f2f237fe700 1 -- 172.21.15.1:0/2394961966 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2f2801f030 con 0x7f2f2c10a750 2024-05-04T03:08:26.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:25.210+0000 7f2f3441f700 1 -- 172.21.15.1:0/2394961966 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f2f2c1a1560 con 0x7f2f2c10a750 2024-05-04T03:08:26.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:25.495+0000 7f2f237fe700 1 -- 172.21.15.1:0/2394961966 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f2f2800b3e0 con 0x7f2f2c10a750 2024-05-04T03:08:26.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:25.498+0000 7f2f237fe700 1 -- 172.21.15.1:0/2394961966 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f2f2801acb0 con 0x7f2f2c10a750 2024-05-04T03:08:26.189 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:25.499+0000 7f2f3441f700 1 -- 172.21.15.1:0/2394961966 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f2f1803e420 msgr2=0x7f2f18040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:26.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:25.499+0000 7f2f3441f700 1 --2- 172.21.15.1:0/2394961966 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f2f1803e420 0x7f2f18040930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c006fd0 tx=0x7f2f1c006e40 comp rx=0 tx=0).stop 2024-05-04T03:08:26.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:25.499+0000 7f2f3441f700 1 -- 172.21.15.1:0/2394961966 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f2c10a750 msgr2=0x7f2f2c19dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:26.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:25.499+0000 7f2f3441f700 1 --2- 172.21.15.1:0/2394961966 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f2c10a750 0x7f2f2c19dfb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2f280119f0 tx=0x7f2f28011ad0 comp rx=0 tx=0).stop 2024-05-04T03:08:26.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:25.499+0000 7f2f3441f700 1 -- 172.21.15.1:0/2394961966 shutdown_connections 2024-05-04T03:08:26.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:25.499+0000 7f2f3441f700 1 --2- 172.21.15.1:0/2394961966 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f2f1803e420 0x7f2f18040930 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:26.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:25.499+0000 7f2f3441f700 1 --2- 172.21.15.1:0/2394961966 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f2c10a750 0x7f2f2c19dfb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:26.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:25.499+0000 7f2f3441f700 1 -- 172.21.15.1:0/2394961966 >> 172.21.15.1:0/2394961966 conn(0x7f2f2c104190 msgr2=0x7f2f2c1049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:26.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:25.499+0000 7f2f3441f700 1 -- 172.21.15.1:0/2394961966 shutdown_connections 2024-05-04T03:08:26.190 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:25.499+0000 7f2f3441f700 1 -- 172.21.15.1:0/2394961966 wait complete. 2024-05-04T03:08:26.597 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:26 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2394961966' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-04T03:08:26.598 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:26 smithi001 ceph-mon[29731]: mgrmap e9: x(active, since 19s) 2024-05-04T03:08:26.598 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:26 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:26.388+0000 7f406913a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-04T03:08:26.598 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:26 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:26.480+0000 7f406913a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-04T03:08:27.470 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-05-04T03:08:27.472 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-04T03:08:27.472 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-04T03:08:27.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-04T03:08:27.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-04T03:08:27.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-05-04T03:08:27.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.638+0000 7fa1c1a3f700 1 Processor -- start 2024-05-04T03:08:27.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.638+0000 7fa1c1a3f700 1 -- start start 2024-05-04T03:08:27.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.638+0000 7fa1c1a3f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1bc10a7a0 0x7fa1bc10cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:27.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.638+0000 7fa1c1a3f700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc077130 con 0x7fa1bc10a7a0 2024-05-04T03:08:27.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.638+0000 7fa1baffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1bc10a7a0 0x7fa1bc10cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:27.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.638+0000 7fa1baffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1bc10a7a0 0x7fa1bc10cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59462/0 (socket says 172.21.15.1:59462) 2024-05-04T03:08:27.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.638+0000 7fa1baffd700 1 -- 172.21.15.1:0/3126139476 learned_addr learned my addr 172.21.15.1:0/3126139476 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:27.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.639+0000 7fa1baffd700 1 -- 172.21.15.1:0/3126139476 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1bc10d100 con 0x7fa1bc10a7a0 2024-05-04T03:08:27.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.639+0000 7fa1baffd700 1 --2- 172.21.15.1:0/3126139476 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1bc10a7a0 0x7fa1bc10cbc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac0098a0 tx=0x7fa1ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32830c6aa93ccd65 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:27.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.639+0000 7fa1ba7fc700 1 -- 172.21.15.1:0/3126139476 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1ac004160 con 0x7fa1bc10a7a0 2024-05-04T03:08:27.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.639+0000 7fa1ba7fc700 1 -- 172.21.15.1:0/3126139476 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1ac004300 con 0x7fa1bc10a7a0 2024-05-04T03:08:27.473 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.639+0000 7fa1ba7fc700 1 -- 172.21.15.1:0/3126139476 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1ac011a20 con 0x7fa1bc10a7a0 2024-05-04T03:08:27.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.640+0000 7fa1c1a3f700 1 -- 172.21.15.1:0/3126139476 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1bc10a7a0 msgr2=0x7fa1bc10cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:27.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.640+0000 7fa1c1a3f700 1 --2- 172.21.15.1:0/3126139476 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1bc10a7a0 0x7fa1bc10cbc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac0098a0 tx=0x7fa1ac009bb0 comp rx=0 tx=0).stop 2024-05-04T03:08:27.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.640+0000 7fa1c1a3f700 1 -- 172.21.15.1:0/3126139476 shutdown_connections 2024-05-04T03:08:27.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.640+0000 7fa1c1a3f700 1 --2- 172.21.15.1:0/3126139476 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1bc10a7a0 0x7fa1bc10cbc0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:27.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.640+0000 7fa1c1a3f700 1 -- 172.21.15.1:0/3126139476 >> 172.21.15.1:0/3126139476 conn(0x7fa1bc104240 msgr2=0x7fa1bc106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:27.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.640+0000 7fa1c1a3f700 1 -- 172.21.15.1:0/3126139476 shutdown_connections 2024-05-04T03:08:27.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.640+0000 7fa1c1a3f700 1 -- 172.21.15.1:0/3126139476 wait complete. 2024-05-04T03:08:27.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.641+0000 7fa1c1a3f700 1 Processor -- start 2024-05-04T03:08:27.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.641+0000 7fa1c1a3f700 1 -- start start 2024-05-04T03:08:27.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.641+0000 7fa1c1a3f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1bc10a7a0 0x7fa1bc19e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:27.474 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.641+0000 7fa1c1a3f700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc10e0a0 con 0x7fa1bc10a7a0 2024-05-04T03:08:27.477 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.641+0000 7fa1baffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1bc10a7a0 0x7fa1bc19e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:27.477 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.641+0000 7fa1baffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1bc10a7a0 0x7fa1bc19e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59478/0 (socket says 172.21.15.1:59478) 2024-05-04T03:08:27.478 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.641+0000 7fa1baffd700 1 -- 172.21.15.1:0/3651484759 learned_addr learned my addr 172.21.15.1:0/3651484759 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:27.478 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.641+0000 7fa1baffd700 1 -- 172.21.15.1:0/3651484759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1bc19e560 con 0x7fa1bc10a7a0 2024-05-04T03:08:27.478 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.642+0000 7fa1baffd700 1 --2- 172.21.15.1:0/3651484759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1bc10a7a0 0x7fa1bc19e020 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac000c00 tx=0x7fa1ac011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:27.478 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.642+0000 7fa1c0a3d700 1 -- 172.21.15.1:0/3651484759 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1ac003aa0 con 0x7fa1bc10a7a0 2024-05-04T03:08:27.478 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.642+0000 7fa1c1a3f700 1 -- 172.21.15.1:0/3651484759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1bc19e7f0 con 0x7fa1bc10a7a0 2024-05-04T03:08:27.478 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.642+0000 7fa1c0a3d700 1 -- 172.21.15.1:0/3651484759 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1ac01b8f0 con 0x7fa1bc10a7a0 2024-05-04T03:08:27.478 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.642+0000 7fa1c0a3d700 1 -- 172.21.15.1:0/3651484759 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1ac023890 con 0x7fa1bc10a7a0 2024-05-04T03:08:27.478 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.642+0000 7fa1c1a3f700 1 -- 172.21.15.1:0/3651484759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1bc1a1400 con 0x7fa1bc10a7a0 2024-05-04T03:08:27.478 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.643+0000 7fa1c1a3f700 1 -- 172.21.15.1:0/3651484759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1bc064e20 con 0x7fa1bc10a7a0 2024-05-04T03:08:27.478 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.643+0000 7fa1c0a3d700 1 -- 172.21.15.1:0/3651484759 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fa1ac02c3f0 con 0x7fa1bc10a7a0 2024-05-04T03:08:27.478 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.643+0000 7fa1c0a3d700 1 --2- 172.21.15.1:0/3651484759 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7fa1a803e520 0x7fa1a8040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:27.478 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.643+0000 7fa1c0a3d700 1 -- 172.21.15.1:0/3651484759 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa1ac053df0 con 0x7fa1bc10a7a0 2024-05-04T03:08:27.479 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.643+0000 7fa1b3fff700 1 -- 172.21.15.1:0/3651484759 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7fa1a803e520 msgr2=0x7fa1a8040a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/128793132 2024-05-04T03:08:27.479 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.643+0000 7fa1b3fff700 1 --2- 172.21.15.1:0/3651484759 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7fa1a803e520 0x7fa1a8040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-04T03:08:27.479 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.649+0000 7fa1c0a3d700 1 -- 172.21.15.1:0/3651484759 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa1ac01f030 con 0x7fa1bc10a7a0 2024-05-04T03:08:27.479 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.844+0000 7fa1b3fff700 1 -- 172.21.15.1:0/3651484759 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7fa1a803e520 msgr2=0x7fa1a8040a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/128793132 2024-05-04T03:08:27.479 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.844+0000 7fa1b3fff700 1 --2- 172.21.15.1:0/3651484759 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7fa1a803e520 0x7fa1a8040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-04T03:08:27.479 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.968+0000 7fa1c1a3f700 1 -- 172.21.15.1:0/3651484759 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fa1bc1a15d0 con 0x7fa1bc10a7a0 2024-05-04T03:08:27.479 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.968+0000 7fa1c0a3d700 1 -- 172.21.15.1:0/3651484759 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fa1bc1a15d0 con 0x7fa1bc10a7a0 2024-05-04T03:08:27.479 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.971+0000 7fa1c1a3f700 1 -- 172.21.15.1:0/3651484759 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7fa1a803e520 msgr2=0x7fa1a8040a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-04T03:08:27.479 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.971+0000 7fa1c1a3f700 1 --2- 172.21.15.1:0/3651484759 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7fa1a803e520 0x7fa1a8040a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:27.480 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.971+0000 7fa1c1a3f700 1 -- 172.21.15.1:0/3651484759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1bc10a7a0 msgr2=0x7fa1bc19e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:27.480 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.971+0000 7fa1c1a3f700 1 --2- 172.21.15.1:0/3651484759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1bc10a7a0 0x7fa1bc19e020 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac000c00 tx=0x7fa1ac011aa0 comp rx=0 tx=0).stop 2024-05-04T03:08:27.480 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.971+0000 7fa1c1a3f700 1 -- 172.21.15.1:0/3651484759 shutdown_connections 2024-05-04T03:08:27.480 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.971+0000 7fa1c1a3f700 1 --2- 172.21.15.1:0/3651484759 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7fa1a803e520 0x7fa1a8040a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:27.480 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.971+0000 7fa1c1a3f700 1 --2- 172.21.15.1:0/3651484759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1bc10a7a0 0x7fa1bc19e020 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:27.480 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.971+0000 7fa1c1a3f700 1 -- 172.21.15.1:0/3651484759 >> 172.21.15.1:0/3651484759 conn(0x7fa1bc104240 msgr2=0x7fa1bc105ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:27.480 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.971+0000 7fa1c1a3f700 1 -- 172.21.15.1:0/3651484759 shutdown_connections 2024-05-04T03:08:27.480 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:26.971+0000 7fa1c1a3f700 1 -- 172.21.15.1:0/3651484759 wait complete. 2024-05-04T03:08:27.480 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for the mgr to restart... 2024-05-04T03:08:27.480 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mgr epoch 9... 2024-05-04T03:08:27.593 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:27 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3651484759' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-04T03:08:27.878 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:27 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:27.748+0000 7f406913a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-04T03:08:28.378 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:27 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:27.938+0000 7f406913a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-04T03:08:28.378 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:28 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:28.117+0000 7f406913a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-04T03:08:28.738 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:28 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:28.426+0000 7f406913a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-04T03:08:28.738 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:28 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:28.524+0000 7f406913a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-04T03:08:28.738 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:28 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:28.737+0000 7f406913a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-04T03:08:29.128 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:28 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:28.942+0000 7f406913a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-04T03:08:29.878 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:29 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:29.576+0000 7f406913a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-04T03:08:29.878 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:29 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:29.813+0000 7f406913a200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-04T03:08:30.628 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:30 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:30.262+0000 7f406913a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-04T03:08:30.628 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:30 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:30.364+0000 7f406913a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-04T03:08:31.128 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:30 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:30.758+0000 7f406913a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-04T03:08:31.493 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:31 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:31.135+0000 7f406913a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-04T03:08:31.493 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:31 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:31.234+0000 7f406913a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-04T03:08:31.772 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:31 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:31.492+0000 7f406913a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-04T03:08:31.772 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:31 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:31.633+0000 7f406913a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-04T03:08:32.075 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:31 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:31.951+0000 7f406913a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-04T03:08:32.075 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:32 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:32.074+0000 7f406913a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-04T03:08:32.374 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:32 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:32.173+0000 7f406913a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-04T03:08:33.097 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:32 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:32.811+0000 7f406913a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-04T03:08:33.097 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:08:33 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:08:33.043+0000 7f406913a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-04T03:08:33.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:33 smithi001 ceph-mon[29731]: Active manager daemon x restarted 2024-05-04T03:08:33.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:33 smithi001 ceph-mon[29731]: Activating manager daemon x 2024-05-04T03:08:33.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:33 smithi001 ceph-mon[29731]: osdmap e3: 0 total, 0 up, 0 in 2024-05-04T03:08:33.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:33 smithi001 ceph-mon[29731]: mgrmap e10: x(active, starting, since 0.00376316s) 2024-05-04T03:08:33.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:33 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:08:33.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:33 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-04T03:08:33.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:33 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-04T03:08:33.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:33 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-04T03:08:33.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:33 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-04T03:08:33.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:33 smithi001 ceph-mon[29731]: Manager daemon x is now available 2024-05-04T03:08:33.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:33 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:08:34.256 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:34 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-04T03:08:34.256 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:34 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-04T03:08:34.256 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:34 smithi001 ceph-mon[29731]: mgrmap e11: x(active, since 1.00714s) 2024-05-04T03:08:34.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-05-04T03:08:34.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-04T03:08:34.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-04T03:08:34.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-05-04T03:08:34.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.908+0000 7f3efad90700 1 Processor -- start 2024-05-04T03:08:34.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.908+0000 7f3efad90700 1 -- start start 2024-05-04T03:08:34.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.909+0000 7f3efad90700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ef410a760 0x7f3ef410cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:34.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.909+0000 7f3efad90700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef4077170 con 0x7f3ef410a760 2024-05-04T03:08:34.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.909+0000 7f3ef8b2c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ef410a760 0x7f3ef410cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:34.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.909+0000 7f3ef8b2c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ef410a760 0x7f3ef410cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59488/0 (socket says 172.21.15.1:59488) 2024-05-04T03:08:34.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.909+0000 7f3ef8b2c700 1 -- 172.21.15.1:0/2600313124 learned_addr learned my addr 172.21.15.1:0/2600313124 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:34.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.909+0000 7f3ef8b2c700 1 -- 172.21.15.1:0/2600313124 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef410d0c0 con 0x7f3ef410a760 2024-05-04T03:08:34.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.909+0000 7f3ef8b2c700 1 --2- 172.21.15.1:0/2600313124 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ef410a760 0x7f3ef410cb80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3ee80098a0 tx=0x7f3ee8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ceeea6d5f068df4 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:34.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.910+0000 7f3ef37fe700 1 -- 172.21.15.1:0/2600313124 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ee8004160 con 0x7f3ef410a760 2024-05-04T03:08:34.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.910+0000 7f3ef37fe700 1 -- 172.21.15.1:0/2600313124 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ee8004300 con 0x7f3ef410a760 2024-05-04T03:08:34.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.910+0000 7f3ef37fe700 1 -- 172.21.15.1:0/2600313124 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ee8011a20 con 0x7f3ef410a760 2024-05-04T03:08:34.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.911+0000 7f3efad90700 1 -- 172.21.15.1:0/2600313124 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ef410a760 msgr2=0x7f3ef410cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:34.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.911+0000 7f3efad90700 1 --2- 172.21.15.1:0/2600313124 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ef410a760 0x7f3ef410cb80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3ee80098a0 tx=0x7f3ee8009bb0 comp rx=0 tx=0).stop 2024-05-04T03:08:34.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.911+0000 7f3efad90700 1 -- 172.21.15.1:0/2600313124 shutdown_connections 2024-05-04T03:08:34.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.911+0000 7f3efad90700 1 --2- 172.21.15.1:0/2600313124 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ef410a760 0x7f3ef410cb80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:34.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.911+0000 7f3efad90700 1 -- 172.21.15.1:0/2600313124 >> 172.21.15.1:0/2600313124 conn(0x7f3ef4104240 msgr2=0x7f3ef4106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:34.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.911+0000 7f3efad90700 1 -- 172.21.15.1:0/2600313124 shutdown_connections 2024-05-04T03:08:34.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.911+0000 7f3efad90700 1 -- 172.21.15.1:0/2600313124 wait complete. 2024-05-04T03:08:34.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.911+0000 7f3efad90700 1 Processor -- start 2024-05-04T03:08:34.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.911+0000 7f3efad90700 1 -- start start 2024-05-04T03:08:34.964 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.912+0000 7f3efad90700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ef410a760 0x7f3ef4109ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:34.964 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.912+0000 7f3efad90700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef410e0c0 con 0x7f3ef410a760 2024-05-04T03:08:34.964 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.912+0000 7f3ef8b2c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ef410a760 0x7f3ef4109ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:34.964 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.912+0000 7f3ef8b2c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ef410a760 0x7f3ef4109ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59500/0 (socket says 172.21.15.1:59500) 2024-05-04T03:08:34.964 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.912+0000 7f3ef8b2c700 1 -- 172.21.15.1:0/3294054288 learned_addr learned my addr 172.21.15.1:0/3294054288 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:34.964 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.912+0000 7f3ef8b2c700 1 -- 172.21.15.1:0/3294054288 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef410a420 con 0x7f3ef410a760 2024-05-04T03:08:34.964 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.912+0000 7f3ef8b2c700 1 --2- 172.21.15.1:0/3294054288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ef410a760 0x7f3ef4109ee0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3ee80119f0 tx=0x7f3ee8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:34.964 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.912+0000 7f3ef1ffb700 1 -- 172.21.15.1:0/3294054288 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ee80039e0 con 0x7f3ef410a760 2024-05-04T03:08:34.964 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.912+0000 7f3ef1ffb700 1 -- 172.21.15.1:0/3294054288 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ee801b8f0 con 0x7f3ef410a760 2024-05-04T03:08:34.964 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.912+0000 7f3ef1ffb700 1 -- 172.21.15.1:0/3294054288 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ee80237a0 con 0x7f3ef410a760 2024-05-04T03:08:34.964 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.912+0000 7f3efad90700 1 -- 172.21.15.1:0/3294054288 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef4108670 con 0x7f3ef410a760 2024-05-04T03:08:34.964 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.913+0000 7f3efad90700 1 -- 172.21.15.1:0/3294054288 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef4108b50 con 0x7f3ef410a760 2024-05-04T03:08:34.964 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.914+0000 7f3ef1ffb700 1 -- 172.21.15.1:0/3294054288 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f3ee8023a50 con 0x7f3ef410a760 2024-05-04T03:08:34.964 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.914+0000 7f3ef1ffb700 1 --2- 172.21.15.1:0/3294054288 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f3ee403e4d0 0x7f3ee40409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:34.965 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.914+0000 7f3ef1ffb700 1 -- 172.21.15.1:0/3294054288 --> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3ee40410f0 con 0x7f3ee403e4d0 2024-05-04T03:08:34.965 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.914+0000 7f3ef1ffb700 1 -- 172.21.15.1:0/3294054288 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3ee8054dd0 con 0x7f3ef410a760 2024-05-04T03:08:34.965 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.914+0000 7f3ef3fff700 1 -- 172.21.15.1:0/3294054288 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f3ee403e4d0 msgr2=0x7f3ee40409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/128793132 2024-05-04T03:08:34.965 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:27.914+0000 7f3ef3fff700 1 --2- 172.21.15.1:0/3294054288 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f3ee403e4d0 0x7f3ee40409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-04T03:08:34.965 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:28.115+0000 7f3ef3fff700 1 -- 172.21.15.1:0/3294054288 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f3ee403e4d0 msgr2=0x7f3ee40409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/128793132 2024-05-04T03:08:34.965 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:28.115+0000 7f3ef3fff700 1 --2- 172.21.15.1:0/3294054288 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f3ee403e4d0 0x7f3ee40409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-04T03:08:34.965 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:28.516+0000 7f3ef3fff700 1 -- 172.21.15.1:0/3294054288 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f3ee403e4d0 msgr2=0x7f3ee40409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/128793132 2024-05-04T03:08:34.965 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:28.516+0000 7f3ef3fff700 1 --2- 172.21.15.1:0/3294054288 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f3ee403e4d0 0x7f3ee40409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-04T03:08:34.965 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:29.317+0000 7f3ef3fff700 1 -- 172.21.15.1:0/3294054288 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f3ee403e4d0 msgr2=0x7f3ee40409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/128793132 2024-05-04T03:08:34.965 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:29.317+0000 7f3ef3fff700 1 --2- 172.21.15.1:0/3294054288 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f3ee403e4d0 0x7f3ee40409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-04T03:08:34.965 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:30.918+0000 7f3ef3fff700 1 -- 172.21.15.1:0/3294054288 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f3ee403e4d0 msgr2=0x7f3ee40409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/128793132 2024-05-04T03:08:34.965 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:30.918+0000 7f3ef3fff700 1 --2- 172.21.15.1:0/3294054288 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f3ee403e4d0 0x7f3ee40409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-04T03:08:34.965 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:33.047+0000 7f3ef1ffb700 1 -- 172.21.15.1:0/3294054288 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7f3ee802c3f0 con 0x7f3ef410a760 2024-05-04T03:08:34.966 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:33.047+0000 7f3ef1ffb700 1 -- 172.21.15.1:0/3294054288 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f3ee403e4d0 msgr2=0x7f3ee40409e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-04T03:08:34.966 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:33.047+0000 7f3ef1ffb700 1 --2- 172.21.15.1:0/3294054288 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f3ee403e4d0 0x7f3ee40409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:34.966 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:34.049+0000 7f3ef1ffb700 1 -- 172.21.15.1:0/3294054288 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f3ee802b350 con 0x7f3ef410a760 2024-05-04T03:08:34.966 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:34.049+0000 7f3ef1ffb700 1 --2- 172.21.15.1:0/3294054288 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f3ee8054960 0x7f3ee4043fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:34.966 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:34.050+0000 7f3ef1ffb700 1 -- 172.21.15.1:0/3294054288 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3ee40410f0 con 0x7f3ee8054960 2024-05-04T03:08:34.966 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:34.050+0000 7f3ef3fff700 1 --2- 172.21.15.1:0/3294054288 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f3ee8054960 0x7f3ee4043fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:34.966 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:34.051+0000 7f3ef3fff700 1 --2- 172.21.15.1:0/3294054288 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f3ee8054960 0x7f3ee4043fd0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0003a10 tx=0x7f3ee0009270 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:34.966 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:34.052+0000 7f3ef1ffb700 1 -- 172.21.15.1:0/3294054288 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f3ee40410f0 con 0x7f3ee8054960 2024-05-04T03:08:34.966 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:34.060+0000 7f3efad90700 1 -- 172.21.15.1:0/3294054288 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f3ef4064e20 con 0x7f3ee8054960 2024-05-04T03:08:34.966 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:34.061+0000 7f3ef1ffb700 1 -- 172.21.15.1:0/3294054288 <== mgr.14150 v2:172.21.15.1:6800/402647351 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f3ef4064e20 con 0x7f3ee8054960 2024-05-04T03:08:34.966 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:34.061+0000 7f3efad90700 1 -- 172.21.15.1:0/3294054288 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f3ee8054960 msgr2=0x7f3ee4043fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:34.966 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:34.061+0000 7f3efad90700 1 --2- 172.21.15.1:0/3294054288 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f3ee8054960 0x7f3ee4043fd0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0003a10 tx=0x7f3ee0009270 comp rx=0 tx=0).stop 2024-05-04T03:08:34.966 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:34.061+0000 7f3efad90700 1 -- 172.21.15.1:0/3294054288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ef410a760 msgr2=0x7f3ef4109ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:34.966 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:34.061+0000 7f3efad90700 1 --2- 172.21.15.1:0/3294054288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ef410a760 0x7f3ef4109ee0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3ee80119f0 tx=0x7f3ee8011ad0 comp rx=0 tx=0).stop 2024-05-04T03:08:34.967 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:34.061+0000 7f3efad90700 1 -- 172.21.15.1:0/3294054288 shutdown_connections 2024-05-04T03:08:34.967 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:34.061+0000 7f3efad90700 1 --2- 172.21.15.1:0/3294054288 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f3ee8054960 0x7f3ee4043fd0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:34.967 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:34.061+0000 7f3efad90700 1 --2- 172.21.15.1:0/3294054288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ef410a760 0x7f3ef4109ee0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:34.967 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:34.061+0000 7f3efad90700 1 --2- 172.21.15.1:0/3294054288 >> [v2:172.21.15.1:6800/128793132,v1:172.21.15.1:6801/128793132] conn(0x7f3ee403e4d0 0x7f3ee40409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:34.967 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:34.061+0000 7f3efad90700 1 -- 172.21.15.1:0/3294054288 >> 172.21.15.1:0/3294054288 conn(0x7f3ef4104240 msgr2=0x7f3ef4105ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:34.967 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:34.061+0000 7f3efad90700 1 -- 172.21.15.1:0/3294054288 shutdown_connections 2024-05-04T03:08:34.967 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:34.061+0000 7f3efad90700 1 -- 172.21.15.1:0/3294054288 wait complete. 2024-05-04T03:08:34.967 INFO:teuthology.orchestra.run.smithi001.stdout:mgr epoch 9 is available 2024-05-04T03:08:34.967 INFO:teuthology.orchestra.run.smithi001.stdout:Generating a dashboard self-signed certificate... 2024-05-04T03:08:35.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:35 smithi001 ceph-mon[29731]: [04/May/2024:03:08:33] ENGINE Bus STARTING 2024-05-04T03:08:35.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:35 smithi001 ceph-mon[29731]: [04/May/2024:03:08:33] ENGINE Serving on http://172.21.15.1:8765 2024-05-04T03:08:35.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:35 smithi001 ceph-mon[29731]: [04/May/2024:03:08:34] ENGINE Serving on https://172.21.15.1:7150 2024-05-04T03:08:35.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:35 smithi001 ceph-mon[29731]: [04/May/2024:03:08:34] ENGINE Bus STARTED 2024-05-04T03:08:35.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:35 smithi001 ceph-mon[29731]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-04T03:08:35.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:35 smithi001 ceph-mon[29731]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-04T03:08:35.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:35 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:08:36.399 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-04T03:08:36.399 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.419+0000 7f0a9d84e700 1 Processor -- start 2024-05-04T03:08:36.399 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.419+0000 7f0a9d84e700 1 -- start start 2024-05-04T03:08:36.399 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.420+0000 7f0a9d84e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a9810a6e0 0x7f0a9810cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:36.399 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.420+0000 7f0a9d84e700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9807ce70 con 0x7f0a9810a6e0 2024-05-04T03:08:36.399 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.420+0000 7f0a96ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a9810a6e0 0x7f0a9810cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:36.399 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.420+0000 7f0a96ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a9810a6e0 0x7f0a9810cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40610/0 (socket says 172.21.15.1:40610) 2024-05-04T03:08:36.399 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.420+0000 7f0a96ffd700 1 -- 172.21.15.1:0/1344102170 learned_addr learned my addr 172.21.15.1:0/1344102170 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:36.400 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.420+0000 7f0a96ffd700 1 -- 172.21.15.1:0/1344102170 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9810d040 con 0x7f0a9810a6e0 2024-05-04T03:08:36.400 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.420+0000 7f0a96ffd700 1 --2- 172.21.15.1:0/1344102170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a9810a6e0 0x7f0a9810cb00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0a880098a0 tx=0x7f0a88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a945cdb7aa1296 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:36.400 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.421+0000 7f0a95ffb700 1 -- 172.21.15.1:0/1344102170 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a88004160 con 0x7f0a9810a6e0 2024-05-04T03:08:36.400 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.421+0000 7f0a95ffb700 1 -- 172.21.15.1:0/1344102170 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a88004300 con 0x7f0a9810a6e0 2024-05-04T03:08:36.400 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.421+0000 7f0a95ffb700 1 -- 172.21.15.1:0/1344102170 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a88011a20 con 0x7f0a9810a6e0 2024-05-04T03:08:36.400 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.421+0000 7f0a9d84e700 1 -- 172.21.15.1:0/1344102170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a9810a6e0 msgr2=0x7f0a9810cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:36.400 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.421+0000 7f0a9d84e700 1 --2- 172.21.15.1:0/1344102170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a9810a6e0 0x7f0a9810cb00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0a880098a0 tx=0x7f0a88009bb0 comp rx=0 tx=0).stop 2024-05-04T03:08:36.400 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.421+0000 7f0a9d84e700 1 -- 172.21.15.1:0/1344102170 shutdown_connections 2024-05-04T03:08:36.400 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.421+0000 7f0a9d84e700 1 --2- 172.21.15.1:0/1344102170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a9810a6e0 0x7f0a9810cb00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:36.400 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.421+0000 7f0a9d84e700 1 -- 172.21.15.1:0/1344102170 >> 172.21.15.1:0/1344102170 conn(0x7f0a981041e0 msgr2=0x7f0a98106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:36.400 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.422+0000 7f0a9d84e700 1 -- 172.21.15.1:0/1344102170 shutdown_connections 2024-05-04T03:08:36.400 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.422+0000 7f0a9d84e700 1 -- 172.21.15.1:0/1344102170 wait complete. 2024-05-04T03:08:36.400 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.422+0000 7f0a9d84e700 1 Processor -- start 2024-05-04T03:08:36.401 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.422+0000 7f0a9d84e700 1 -- start start 2024-05-04T03:08:36.401 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.423+0000 7f0a9d84e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a9810a6e0 0x7f0a9819e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:36.401 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.423+0000 7f0a9d84e700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9810e040 con 0x7f0a9810a6e0 2024-05-04T03:08:36.401 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.423+0000 7f0a96ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a9810a6e0 0x7f0a9819e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:36.401 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.423+0000 7f0a96ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a9810a6e0 0x7f0a9819e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40618/0 (socket says 172.21.15.1:40618) 2024-05-04T03:08:36.401 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.423+0000 7f0a96ffd700 1 -- 172.21.15.1:0/4110787894 learned_addr learned my addr 172.21.15.1:0/4110787894 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:36.401 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.423+0000 7f0a96ffd700 1 -- 172.21.15.1:0/4110787894 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9819e560 con 0x7f0a9810a6e0 2024-05-04T03:08:36.401 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.423+0000 7f0a96ffd700 1 --2- 172.21.15.1:0/4110787894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a9810a6e0 0x7f0a9819e020 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0a88009520 tx=0x7f0a88011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:36.401 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.423+0000 7f0a9c84c700 1 -- 172.21.15.1:0/4110787894 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a880039e0 con 0x7f0a9810a6e0 2024-05-04T03:08:36.401 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.423+0000 7f0a9c84c700 1 -- 172.21.15.1:0/4110787894 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a8801b8f0 con 0x7f0a9810a6e0 2024-05-04T03:08:36.401 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.423+0000 7f0a9d84e700 1 -- 172.21.15.1:0/4110787894 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a9819e7f0 con 0x7f0a9810a6e0 2024-05-04T03:08:36.401 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.423+0000 7f0a9d84e700 1 -- 172.21.15.1:0/4110787894 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a981a1400 con 0x7f0a9810a6e0 2024-05-04T03:08:36.401 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.424+0000 7f0a9c84c700 1 -- 172.21.15.1:0/4110787894 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a880197d0 con 0x7f0a9810a6e0 2024-05-04T03:08:36.401 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.424+0000 7f0a9d84e700 1 -- 172.21.15.1:0/4110787894 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a84005360 con 0x7f0a9810a6e0 2024-05-04T03:08:36.402 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.425+0000 7f0a9c84c700 1 -- 172.21.15.1:0/4110787894 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f0a88019b10 con 0x7f0a9810a6e0 2024-05-04T03:08:36.402 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.425+0000 7f0a9c84c700 1 --2- 172.21.15.1:0/4110787894 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0a8003e3a0 0x7f0a800408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:36.402 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.425+0000 7f0a9c84c700 1 -- 172.21.15.1:0/4110787894 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0a88054000 con 0x7f0a9810a6e0 2024-05-04T03:08:36.402 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.425+0000 7f0a967fc700 1 --2- 172.21.15.1:0/4110787894 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0a8003e3a0 0x7f0a800408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:36.402 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.426+0000 7f0a967fc700 1 --2- 172.21.15.1:0/4110787894 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0a8003e3a0 0x7f0a800408b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c006fd0 tx=0x7f0a8c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:36.402 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.431+0000 7f0a9c84c700 1 -- 172.21.15.1:0/4110787894 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0a8801f030 con 0x7f0a9810a6e0 2024-05-04T03:08:36.402 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.606+0000 7f0a9c84c700 1 -- 172.21.15.1:0/4110787894 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f0a8801a3d0 con 0x7f0a9810a6e0 2024-05-04T03:08:36.402 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.655+0000 7f0a9d84e700 1 -- 172.21.15.1:0/4110787894 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f0a84000bf0 con 0x7f0a8003e3a0 2024-05-04T03:08:36.402 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.836+0000 7f0a9c84c700 1 -- 172.21.15.1:0/4110787894 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f0a84000bf0 con 0x7f0a8003e3a0 2024-05-04T03:08:36.402 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.838+0000 7f0a9d84e700 1 -- 172.21.15.1:0/4110787894 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0a8003e3a0 msgr2=0x7f0a800408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:36.402 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.838+0000 7f0a9d84e700 1 --2- 172.21.15.1:0/4110787894 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0a8003e3a0 0x7f0a800408b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c006fd0 tx=0x7f0a8c006e40 comp rx=0 tx=0).stop 2024-05-04T03:08:36.402 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.838+0000 7f0a9d84e700 1 -- 172.21.15.1:0/4110787894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a9810a6e0 msgr2=0x7f0a9819e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:36.403 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.838+0000 7f0a9d84e700 1 --2- 172.21.15.1:0/4110787894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a9810a6e0 0x7f0a9819e020 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0a88009520 tx=0x7f0a88011aa0 comp rx=0 tx=0).stop 2024-05-04T03:08:36.403 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.838+0000 7f0a9d84e700 1 -- 172.21.15.1:0/4110787894 shutdown_connections 2024-05-04T03:08:36.403 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.838+0000 7f0a9d84e700 1 --2- 172.21.15.1:0/4110787894 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0a8003e3a0 0x7f0a800408b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:36.403 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.838+0000 7f0a9d84e700 1 --2- 172.21.15.1:0/4110787894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a9810a6e0 0x7f0a9819e020 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:36.403 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.838+0000 7f0a9d84e700 1 -- 172.21.15.1:0/4110787894 >> 172.21.15.1:0/4110787894 conn(0x7f0a981041e0 msgr2=0x7f0a98105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:36.403 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.838+0000 7f0a9d84e700 1 -- 172.21.15.1:0/4110787894 shutdown_connections 2024-05-04T03:08:36.403 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:35.838+0000 7f0a9d84e700 1 -- 172.21.15.1:0/4110787894 wait complete. 2024-05-04T03:08:36.403 INFO:teuthology.orchestra.run.smithi001.stdout:Creating initial admin user... 2024-05-04T03:08:36.777 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:36 smithi001 ceph-mon[29731]: mgrmap e12: x(active, since 2s) 2024-05-04T03:08:36.777 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:36 smithi001 ceph-mon[29731]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:08:36.777 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:08:36.777 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:08:36.777 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:08:38.737 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:38 smithi001 ceph-mon[29731]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:08:38.737 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:38 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:08:38.744 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$jIMhdPEuTEQGQnH9SLCM7.UqXH0wEj2n53.zipz85qD3IY44ybELi", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714792117, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-04T03:08:38.744 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.898+0000 7fa9a7b08700 1 Processor -- start 2024-05-04T03:08:38.745 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.898+0000 7fa9a7b08700 1 -- start start 2024-05-04T03:08:38.745 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.898+0000 7fa9a7b08700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9a010a750 0x7fa9a010cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:38.745 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.898+0000 7fa9a7b08700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a007ce10 con 0x7fa9a010a750 2024-05-04T03:08:38.745 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.898+0000 7fa9a58a4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9a010a750 0x7fa9a010cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:38.745 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.898+0000 7fa9a58a4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9a010a750 0x7fa9a010cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40628/0 (socket says 172.21.15.1:40628) 2024-05-04T03:08:38.745 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.898+0000 7fa9a58a4700 1 -- 172.21.15.1:0/3188555053 learned_addr learned my addr 172.21.15.1:0/3188555053 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:38.745 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.899+0000 7fa9a58a4700 1 -- 172.21.15.1:0/3188555053 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a010d0b0 con 0x7fa9a010a750 2024-05-04T03:08:38.745 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.899+0000 7fa9a58a4700 1 --2- 172.21.15.1:0/3188555053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9a010a750 0x7fa9a010cb70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa99c0098a0 tx=0x7fa99c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316e5932c18b97bf server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:38.745 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.899+0000 7fa9a48a2700 1 -- 172.21.15.1:0/3188555053 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa99c004160 con 0x7fa9a010a750 2024-05-04T03:08:38.745 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.899+0000 7fa9a48a2700 1 -- 172.21.15.1:0/3188555053 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa99c004300 con 0x7fa9a010a750 2024-05-04T03:08:38.745 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.899+0000 7fa9a48a2700 1 -- 172.21.15.1:0/3188555053 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa99c011a20 con 0x7fa9a010a750 2024-05-04T03:08:38.745 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.900+0000 7fa9a7b08700 1 -- 172.21.15.1:0/3188555053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9a010a750 msgr2=0x7fa9a010cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:38.745 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.900+0000 7fa9a7b08700 1 --2- 172.21.15.1:0/3188555053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9a010a750 0x7fa9a010cb70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa99c0098a0 tx=0x7fa99c009bb0 comp rx=0 tx=0).stop 2024-05-04T03:08:38.746 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.900+0000 7fa9a7b08700 1 -- 172.21.15.1:0/3188555053 shutdown_connections 2024-05-04T03:08:38.746 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.900+0000 7fa9a7b08700 1 --2- 172.21.15.1:0/3188555053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9a010a750 0x7fa9a010cb70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:38.746 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.900+0000 7fa9a7b08700 1 -- 172.21.15.1:0/3188555053 >> 172.21.15.1:0/3188555053 conn(0x7fa9a0104190 msgr2=0x7fa9a0106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:38.746 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.900+0000 7fa9a7b08700 1 -- 172.21.15.1:0/3188555053 shutdown_connections 2024-05-04T03:08:38.746 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.900+0000 7fa9a7b08700 1 -- 172.21.15.1:0/3188555053 wait complete. 2024-05-04T03:08:38.746 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.901+0000 7fa9a7b08700 1 Processor -- start 2024-05-04T03:08:38.746 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.901+0000 7fa9a7b08700 1 -- start start 2024-05-04T03:08:38.746 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.901+0000 7fa9a7b08700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9a010a750 0x7fa9a0109ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:38.746 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.901+0000 7fa9a7b08700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a010e050 con 0x7fa9a010a750 2024-05-04T03:08:38.746 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.901+0000 7fa9a58a4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9a010a750 0x7fa9a0109ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:38.746 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.901+0000 7fa9a58a4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9a010a750 0x7fa9a0109ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40634/0 (socket says 172.21.15.1:40634) 2024-05-04T03:08:38.746 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.901+0000 7fa9a58a4700 1 -- 172.21.15.1:0/2845675907 learned_addr learned my addr 172.21.15.1:0/2845675907 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:38.746 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.902+0000 7fa9a58a4700 1 -- 172.21.15.1:0/2845675907 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a010a410 con 0x7fa9a010a750 2024-05-04T03:08:38.747 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.902+0000 7fa9a58a4700 1 --2- 172.21.15.1:0/2845675907 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9a010a750 0x7fa9a0109ed0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa99c0119f0 tx=0x7fa99c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:38.747 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.902+0000 7fa992ffd700 1 -- 172.21.15.1:0/2845675907 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa99c0039e0 con 0x7fa9a010a750 2024-05-04T03:08:38.747 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.902+0000 7fa992ffd700 1 -- 172.21.15.1:0/2845675907 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa99c01b8f0 con 0x7fa9a010a750 2024-05-04T03:08:38.747 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.902+0000 7fa992ffd700 1 -- 172.21.15.1:0/2845675907 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa99c0237a0 con 0x7fa9a010a750 2024-05-04T03:08:38.747 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.902+0000 7fa9a7b08700 1 -- 172.21.15.1:0/2845675907 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a0108660 con 0x7fa9a010a750 2024-05-04T03:08:38.747 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.902+0000 7fa9a7b08700 1 -- 172.21.15.1:0/2845675907 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a0108b40 con 0x7fa9a010a750 2024-05-04T03:08:38.747 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.903+0000 7fa992ffd700 1 -- 172.21.15.1:0/2845675907 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fa99c02c3f0 con 0x7fa9a010a750 2024-05-04T03:08:38.747 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.903+0000 7fa9a7b08700 1 -- 172.21.15.1:0/2845675907 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a0064e20 con 0x7fa9a010a750 2024-05-04T03:08:38.747 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.904+0000 7fa992ffd700 1 --2- 172.21.15.1:0/2845675907 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa98c03e4d0 0x7fa98c0409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:38.747 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.904+0000 7fa992ffd700 1 -- 172.21.15.1:0/2845675907 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa99c054ee0 con 0x7fa9a010a750 2024-05-04T03:08:38.747 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.904+0000 7fa9a50a3700 1 --2- 172.21.15.1:0/2845675907 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa98c03e4d0 0x7fa98c0409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:38.747 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.904+0000 7fa9a50a3700 1 --2- 172.21.15.1:0/2845675907 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa98c03e4d0 0x7fa98c0409e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa994006fd0 tx=0x7fa994006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:38.747 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:36.910+0000 7fa992ffd700 1 -- 172.21.15.1:0/2845675907 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa99c01f030 con 0x7fa9a010a750 2024-05-04T03:08:38.747 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:37.159+0000 7fa9a7b08700 1 -- 172.21.15.1:0/2845675907 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fa9a01091f0 con 0x7fa98c03e4d0 2024-05-04T03:08:38.748 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:37.405+0000 7fa992ffd700 1 -- 172.21.15.1:0/2845675907 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fa9a01091f0 con 0x7fa98c03e4d0 2024-05-04T03:08:38.748 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:37.408+0000 7fa9a7b08700 1 -- 172.21.15.1:0/2845675907 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa98c03e4d0 msgr2=0x7fa98c0409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:38.748 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:37.408+0000 7fa9a7b08700 1 --2- 172.21.15.1:0/2845675907 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa98c03e4d0 0x7fa98c0409e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa994006fd0 tx=0x7fa994006e40 comp rx=0 tx=0).stop 2024-05-04T03:08:38.748 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:37.408+0000 7fa9a7b08700 1 -- 172.21.15.1:0/2845675907 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9a010a750 msgr2=0x7fa9a0109ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:38.748 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:37.408+0000 7fa9a7b08700 1 --2- 172.21.15.1:0/2845675907 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9a010a750 0x7fa9a0109ed0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa99c0119f0 tx=0x7fa99c011ad0 comp rx=0 tx=0).stop 2024-05-04T03:08:38.748 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:37.408+0000 7fa9a7b08700 1 -- 172.21.15.1:0/2845675907 shutdown_connections 2024-05-04T03:08:38.748 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:37.408+0000 7fa9a7b08700 1 --2- 172.21.15.1:0/2845675907 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa98c03e4d0 0x7fa98c0409e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:38.748 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:37.408+0000 7fa9a7b08700 1 --2- 172.21.15.1:0/2845675907 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9a010a750 0x7fa9a0109ed0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:38.748 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:37.408+0000 7fa9a7b08700 1 -- 172.21.15.1:0/2845675907 >> 172.21.15.1:0/2845675907 conn(0x7fa9a0104190 msgr2=0x7fa9a01049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:38.748 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:37.408+0000 7fa9a7b08700 1 -- 172.21.15.1:0/2845675907 shutdown_connections 2024-05-04T03:08:38.748 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:37.408+0000 7fa9a7b08700 1 -- 172.21.15.1:0/2845675907 wait complete. 2024-05-04T03:08:38.748 INFO:teuthology.orchestra.run.smithi001.stdout:Fetching dashboard port number... 2024-05-04T03:08:39.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:39 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3683971830' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-04T03:08:39.959 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 8443 2024-05-04T03:08:39.960 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.163+0000 7fc9381cd700 1 Processor -- start 2024-05-04T03:08:39.960 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.163+0000 7fc9381cd700 1 -- start start 2024-05-04T03:08:39.960 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.163+0000 7fc9381cd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc93010a7c0 0x7fc93010cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:39.960 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.163+0000 7fc9381cd700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc930077110 con 0x7fc93010a7c0 2024-05-04T03:08:39.960 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.163+0000 7fc935f69700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc93010a7c0 0x7fc93010cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:39.960 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.163+0000 7fc935f69700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc93010a7c0 0x7fc93010cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40648/0 (socket says 172.21.15.1:40648) 2024-05-04T03:08:39.960 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.163+0000 7fc935f69700 1 -- 172.21.15.1:0/975076728 learned_addr learned my addr 172.21.15.1:0/975076728 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:39.960 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.164+0000 7fc935f69700 1 -- 172.21.15.1:0/975076728 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93010d120 con 0x7fc93010a7c0 2024-05-04T03:08:39.960 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.164+0000 7fc935f69700 1 --2- 172.21.15.1:0/975076728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc93010a7c0 0x7fc93010cbe0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc92c0098a0 tx=0x7fc92c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b1d60c11f35cb1 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:39.960 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.164+0000 7fc934f67700 1 -- 172.21.15.1:0/975076728 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc92c004160 con 0x7fc93010a7c0 2024-05-04T03:08:39.960 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.164+0000 7fc934f67700 1 -- 172.21.15.1:0/975076728 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc92c004300 con 0x7fc93010a7c0 2024-05-04T03:08:39.960 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.164+0000 7fc934f67700 1 -- 172.21.15.1:0/975076728 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc92c011a20 con 0x7fc93010a7c0 2024-05-04T03:08:39.960 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.165+0000 7fc9381cd700 1 -- 172.21.15.1:0/975076728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc93010a7c0 msgr2=0x7fc93010cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:39.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.165+0000 7fc9381cd700 1 --2- 172.21.15.1:0/975076728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc93010a7c0 0x7fc93010cbe0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc92c0098a0 tx=0x7fc92c009bb0 comp rx=0 tx=0).stop 2024-05-04T03:08:39.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.165+0000 7fc9381cd700 1 -- 172.21.15.1:0/975076728 shutdown_connections 2024-05-04T03:08:39.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.165+0000 7fc9381cd700 1 --2- 172.21.15.1:0/975076728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc93010a7c0 0x7fc93010cbe0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:39.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.165+0000 7fc9381cd700 1 -- 172.21.15.1:0/975076728 >> 172.21.15.1:0/975076728 conn(0x7fc930104240 msgr2=0x7fc930106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:39.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.165+0000 7fc9381cd700 1 -- 172.21.15.1:0/975076728 shutdown_connections 2024-05-04T03:08:39.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.165+0000 7fc9381cd700 1 -- 172.21.15.1:0/975076728 wait complete. 2024-05-04T03:08:39.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.165+0000 7fc9381cd700 1 Processor -- start 2024-05-04T03:08:39.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.166+0000 7fc9381cd700 1 -- start start 2024-05-04T03:08:39.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.166+0000 7fc9381cd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc93010a7c0 0x7fc93019e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:39.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.166+0000 7fc9381cd700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc93010e0c0 con 0x7fc93010a7c0 2024-05-04T03:08:39.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.166+0000 7fc935f69700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc93010a7c0 0x7fc93019e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:39.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.166+0000 7fc935f69700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc93010a7c0 0x7fc93019e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40658/0 (socket says 172.21.15.1:40658) 2024-05-04T03:08:39.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.166+0000 7fc935f69700 1 -- 172.21.15.1:0/3683971830 learned_addr learned my addr 172.21.15.1:0/3683971830 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:39.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.166+0000 7fc935f69700 1 -- 172.21.15.1:0/3683971830 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93019e560 con 0x7fc93010a7c0 2024-05-04T03:08:39.961 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.166+0000 7fc935f69700 1 --2- 172.21.15.1:0/3683971830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc93010a7c0 0x7fc93019e020 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc92c000c00 tx=0x7fc92c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:39.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.166+0000 7fc926ffd700 1 -- 172.21.15.1:0/3683971830 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc92c003aa0 con 0x7fc93010a7c0 2024-05-04T03:08:39.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.167+0000 7fc926ffd700 1 -- 172.21.15.1:0/3683971830 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc92c01b8f0 con 0x7fc93010a7c0 2024-05-04T03:08:39.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.167+0000 7fc9381cd700 1 -- 172.21.15.1:0/3683971830 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc93019e7f0 con 0x7fc93010a7c0 2024-05-04T03:08:39.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.167+0000 7fc926ffd700 1 -- 172.21.15.1:0/3683971830 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc92c023890 con 0x7fc93010a7c0 2024-05-04T03:08:39.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.167+0000 7fc9381cd700 1 -- 172.21.15.1:0/3683971830 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9301a1400 con 0x7fc93010a7c0 2024-05-04T03:08:39.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.167+0000 7fc9381cd700 1 -- 172.21.15.1:0/3683971830 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc930064e20 con 0x7fc93010a7c0 2024-05-04T03:08:39.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.168+0000 7fc926ffd700 1 -- 172.21.15.1:0/3683971830 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fc92c02c3f0 con 0x7fc93010a7c0 2024-05-04T03:08:39.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.168+0000 7fc926ffd700 1 --2- 172.21.15.1:0/3683971830 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc91c03e520 0x7fc91c040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:39.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.168+0000 7fc926ffd700 1 -- 172.21.15.1:0/3683971830 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc92c054110 con 0x7fc93010a7c0 2024-05-04T03:08:39.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.168+0000 7fc935768700 1 --2- 172.21.15.1:0/3683971830 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc91c03e520 0x7fc91c040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:39.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.168+0000 7fc935768700 1 --2- 172.21.15.1:0/3683971830 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc91c03e520 0x7fc91c040a30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc920006fd0 tx=0x7fc920006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:39.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.173+0000 7fc926ffd700 1 -- 172.21.15.1:0/3683971830 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc92c01f030 con 0x7fc93010a7c0 2024-05-04T03:08:39.962 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.386+0000 7fc9381cd700 1 -- 172.21.15.1:0/3683971830 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fc9301a15d0 con 0x7fc93010a7c0 2024-05-04T03:08:39.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.386+0000 7fc926ffd700 1 -- 172.21.15.1:0/3683971830 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fc92c00f2e0 con 0x7fc93010a7c0 2024-05-04T03:08:39.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.388+0000 7fc9381cd700 1 -- 172.21.15.1:0/3683971830 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc91c03e520 msgr2=0x7fc91c040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:39.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.388+0000 7fc9381cd700 1 --2- 172.21.15.1:0/3683971830 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc91c03e520 0x7fc91c040a30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc920006fd0 tx=0x7fc920006e40 comp rx=0 tx=0).stop 2024-05-04T03:08:39.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.388+0000 7fc9381cd700 1 -- 172.21.15.1:0/3683971830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc93010a7c0 msgr2=0x7fc93019e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:39.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.388+0000 7fc9381cd700 1 --2- 172.21.15.1:0/3683971830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc93010a7c0 0x7fc93019e020 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc92c000c00 tx=0x7fc92c011aa0 comp rx=0 tx=0).stop 2024-05-04T03:08:39.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.388+0000 7fc9381cd700 1 -- 172.21.15.1:0/3683971830 shutdown_connections 2024-05-04T03:08:39.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.388+0000 7fc9381cd700 1 --2- 172.21.15.1:0/3683971830 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc91c03e520 0x7fc91c040a30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:39.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.388+0000 7fc9381cd700 1 --2- 172.21.15.1:0/3683971830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc93010a7c0 0x7fc93019e020 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:39.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.388+0000 7fc9381cd700 1 -- 172.21.15.1:0/3683971830 >> 172.21.15.1:0/3683971830 conn(0x7fc930104240 msgr2=0x7fc930105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:39.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.388+0000 7fc9381cd700 1 -- 172.21.15.1:0/3683971830 shutdown_connections 2024-05-04T03:08:39.963 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:39.388+0000 7fc9381cd700 1 -- 172.21.15.1:0/3683971830 wait complete. 2024-05-04T03:08:39.969 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-04T03:08:39.969 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout disabled 2024-05-04T03:08:39.977 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-04T03:08:39.977 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-05-04T03:08:39.977 INFO:teuthology.orchestra.run.smithi001.stdout:firewalld.service is not enabled 2024-05-04T03:08:39.977 INFO:teuthology.orchestra.run.smithi001.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-04T03:08:39.978 INFO:teuthology.orchestra.run.smithi001.stdout:Ceph Dashboard is now available at: 2024-05-04T03:08:39.978 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:08:39.979 INFO:teuthology.orchestra.run.smithi001.stdout: URL: https://smithi001:8443/ 2024-05-04T03:08:39.979 INFO:teuthology.orchestra.run.smithi001.stdout: User: admin 2024-05-04T03:08:39.979 INFO:teuthology.orchestra.run.smithi001.stdout: Password: 39mdhue4st 2024-05-04T03:08:39.979 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:08:39.980 INFO:teuthology.orchestra.run.smithi001.stdout:Saving cluster configuration to /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/config directory 2024-05-04T03:08:39.980 INFO:teuthology.orchestra.run.smithi001.stdout:Enabling autotune for osd_memory_target 2024-05-04T03:08:40.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:40 smithi001 ceph-mon[29731]: mgrmap e13: x(active, since 6s) 2024-05-04T03:08:41.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.441+0000 7f0e8abd5700 1 Processor -- start 2024-05-04T03:08:41.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.441+0000 7f0e8abd5700 1 -- start start 2024-05-04T03:08:41.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.442+0000 7f0e8abd5700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e84107f30 0x7f0e84108340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:41.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.442+0000 7f0e8abd5700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0e84108910 con 0x7f0e84107f30 2024-05-04T03:08:41.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.442+0000 7f0e88971700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e84107f30 0x7f0e84108340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:41.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.442+0000 7f0e88971700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e84107f30 0x7f0e84108340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40660/0 (socket says 172.21.15.1:40660) 2024-05-04T03:08:41.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.442+0000 7f0e88971700 1 -- 172.21.15.1:0/2764168819 learned_addr learned my addr 172.21.15.1:0/2764168819 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:41.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.442+0000 7f0e88971700 1 -- 172.21.15.1:0/2764168819 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e8410af30 con 0x7f0e84107f30 2024-05-04T03:08:41.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.443+0000 7f0e88971700 1 --2- 172.21.15.1:0/2764168819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e84107f30 0x7f0e84108340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0e780098a0 tx=0x7f0e78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c40a40dbea2f4ae server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:41.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.443+0000 7f0e837fe700 1 -- 172.21.15.1:0/2764168819 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e78004160 con 0x7f0e84107f30 2024-05-04T03:08:41.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.443+0000 7f0e837fe700 1 -- 172.21.15.1:0/2764168819 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e78004300 con 0x7f0e84107f30 2024-05-04T03:08:41.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.443+0000 7f0e837fe700 1 -- 172.21.15.1:0/2764168819 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e78011a20 con 0x7f0e84107f30 2024-05-04T03:08:41.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.443+0000 7f0e8abd5700 1 -- 172.21.15.1:0/2764168819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e84107f30 msgr2=0x7f0e84108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:41.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.443+0000 7f0e8abd5700 1 --2- 172.21.15.1:0/2764168819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e84107f30 0x7f0e84108340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0e780098a0 tx=0x7f0e78009bb0 comp rx=0 tx=0).stop 2024-05-04T03:08:41.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.444+0000 7f0e8abd5700 1 -- 172.21.15.1:0/2764168819 shutdown_connections 2024-05-04T03:08:41.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.444+0000 7f0e8abd5700 1 --2- 172.21.15.1:0/2764168819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e84107f30 0x7f0e84108340 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:41.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.444+0000 7f0e8abd5700 1 -- 172.21.15.1:0/2764168819 >> 172.21.15.1:0/2764168819 conn(0x7f0e84103900 msgr2=0x7f0e84105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:41.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.444+0000 7f0e8abd5700 1 -- 172.21.15.1:0/2764168819 shutdown_connections 2024-05-04T03:08:41.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.444+0000 7f0e8abd5700 1 -- 172.21.15.1:0/2764168819 wait complete. 2024-05-04T03:08:41.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.444+0000 7f0e8abd5700 1 Processor -- start 2024-05-04T03:08:41.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.444+0000 7f0e8abd5700 1 -- start start 2024-05-04T03:08:41.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.444+0000 7f0e8abd5700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e84107f30 0x7f0e8419e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:41.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.445+0000 7f0e8abd5700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8410e160 con 0x7f0e84107f30 2024-05-04T03:08:41.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.445+0000 7f0e88971700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e84107f30 0x7f0e8419e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:41.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.445+0000 7f0e88971700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e84107f30 0x7f0e8419e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40672/0 (socket says 172.21.15.1:40672) 2024-05-04T03:08:41.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.445+0000 7f0e88971700 1 -- 172.21.15.1:0/1606059334 learned_addr learned my addr 172.21.15.1:0/1606059334 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:41.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.445+0000 7f0e88971700 1 -- 172.21.15.1:0/1606059334 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e8419e790 con 0x7f0e84107f30 2024-05-04T03:08:41.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.445+0000 7f0e88971700 1 --2- 172.21.15.1:0/1606059334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e84107f30 0x7f0e8419e250 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0e78011a20 tx=0x7f0e78011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:41.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.445+0000 7f0e81ffb700 1 -- 172.21.15.1:0/1606059334 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e780039e0 con 0x7f0e84107f30 2024-05-04T03:08:41.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.445+0000 7f0e81ffb700 1 -- 172.21.15.1:0/1606059334 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e7801b8f0 con 0x7f0e84107f30 2024-05-04T03:08:41.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.445+0000 7f0e8abd5700 1 -- 172.21.15.1:0/1606059334 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e8419ea20 con 0x7f0e84107f30 2024-05-04T03:08:41.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.445+0000 7f0e81ffb700 1 -- 172.21.15.1:0/1606059334 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e78023770 con 0x7f0e84107f30 2024-05-04T03:08:41.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.445+0000 7f0e8abd5700 1 -- 172.21.15.1:0/1606059334 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e841a1630 con 0x7f0e84107f30 2024-05-04T03:08:41.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.446+0000 7f0e8abd5700 1 -- 172.21.15.1:0/1606059334 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e84064e20 con 0x7f0e84107f30 2024-05-04T03:08:41.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.447+0000 7f0e81ffb700 1 -- 172.21.15.1:0/1606059334 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f0e7802c3f0 con 0x7f0e84107f30 2024-05-04T03:08:41.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.447+0000 7f0e81ffb700 1 --2- 172.21.15.1:0/1606059334 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0e7403e570 0x7f0e74040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:41.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.447+0000 7f0e81ffb700 1 -- 172.21.15.1:0/1606059334 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0e78053fd0 con 0x7f0e84107f30 2024-05-04T03:08:41.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.447+0000 7f0e83fff700 1 --2- 172.21.15.1:0/1606059334 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0e7403e570 0x7f0e74040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:41.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.447+0000 7f0e83fff700 1 --2- 172.21.15.1:0/1606059334 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0e7403e570 0x7f0e74040a80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0e70006fd0 tx=0x7f0e70006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:41.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.452+0000 7f0e81ffb700 1 -- 172.21.15.1:0/1606059334 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0e7801f030 con 0x7f0e84107f30 2024-05-04T03:08:41.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.665+0000 7f0e8abd5700 1 -- 172.21.15.1:0/1606059334 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f0e8410e2e0 con 0x7f0e84107f30 2024-05-04T03:08:41.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.666+0000 7f0e81ffb700 1 -- 172.21.15.1:0/1606059334 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f0e7800f2e0 con 0x7f0e84107f30 2024-05-04T03:08:41.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.668+0000 7f0e8abd5700 1 -- 172.21.15.1:0/1606059334 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0e7403e570 msgr2=0x7f0e74040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:41.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.668+0000 7f0e8abd5700 1 --2- 172.21.15.1:0/1606059334 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0e7403e570 0x7f0e74040a80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0e70006fd0 tx=0x7f0e70006e40 comp rx=0 tx=0).stop 2024-05-04T03:08:41.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.668+0000 7f0e8abd5700 1 -- 172.21.15.1:0/1606059334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e84107f30 msgr2=0x7f0e8419e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:41.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.668+0000 7f0e8abd5700 1 --2- 172.21.15.1:0/1606059334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e84107f30 0x7f0e8419e250 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0e78011a20 tx=0x7f0e78011b00 comp rx=0 tx=0).stop 2024-05-04T03:08:41.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.668+0000 7f0e8abd5700 1 -- 172.21.15.1:0/1606059334 shutdown_connections 2024-05-04T03:08:41.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.668+0000 7f0e8abd5700 1 --2- 172.21.15.1:0/1606059334 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0e7403e570 0x7f0e74040a80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:41.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.668+0000 7f0e8abd5700 1 --2- 172.21.15.1:0/1606059334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e84107f30 0x7f0e8419e250 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:41.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.668+0000 7f0e8abd5700 1 -- 172.21.15.1:0/1606059334 >> 172.21.15.1:0/1606059334 conn(0x7f0e84103900 msgr2=0x7f0e84104230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:41.236 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.668+0000 7f0e8abd5700 1 -- 172.21.15.1:0/1606059334 shutdown_connections 2024-05-04T03:08:41.236 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:40.668+0000 7f0e8abd5700 1 -- 172.21.15.1:0/1606059334 wait complete. 2024-05-04T03:08:42.580 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.675+0000 7ffacba35700 1 Processor -- start 2024-05-04T03:08:42.580 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.675+0000 7ffacba35700 1 -- start start 2024-05-04T03:08:42.580 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.675+0000 7ffacba35700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac4108530 0x7ffac4108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:42.580 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.675+0000 7ffacba35700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffac407ce10 con 0x7ffac4108530 2024-05-04T03:08:42.580 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.675+0000 7ffac97d1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac4108530 0x7ffac4108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:42.580 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.676+0000 7ffac97d1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac4108530 0x7ffac4108940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56870/0 (socket says 172.21.15.1:56870) 2024-05-04T03:08:42.581 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.676+0000 7ffac97d1700 1 -- 172.21.15.1:0/3719758740 learned_addr learned my addr 172.21.15.1:0/3719758740 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:42.581 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.676+0000 7ffac97d1700 1 -- 172.21.15.1:0/3719758740 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac4108e80 con 0x7ffac4108530 2024-05-04T03:08:42.581 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.676+0000 7ffac97d1700 1 --2- 172.21.15.1:0/3719758740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac4108530 0x7ffac4108940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ffab80098a0 tx=0x7ffab8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b928fd197f77da5 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:42.581 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.676+0000 7ffab7fff700 1 -- 172.21.15.1:0/3719758740 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffab8004160 con 0x7ffac4108530 2024-05-04T03:08:42.581 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.676+0000 7ffab7fff700 1 -- 172.21.15.1:0/3719758740 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffab8004300 con 0x7ffac4108530 2024-05-04T03:08:42.581 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.676+0000 7ffab7fff700 1 -- 172.21.15.1:0/3719758740 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffab8011a20 con 0x7ffac4108530 2024-05-04T03:08:42.581 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.677+0000 7ffacba35700 1 -- 172.21.15.1:0/3719758740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac4108530 msgr2=0x7ffac4108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:42.581 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.677+0000 7ffacba35700 1 --2- 172.21.15.1:0/3719758740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac4108530 0x7ffac4108940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ffab80098a0 tx=0x7ffab8009bb0 comp rx=0 tx=0).stop 2024-05-04T03:08:42.581 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.677+0000 7ffacba35700 1 -- 172.21.15.1:0/3719758740 shutdown_connections 2024-05-04T03:08:42.581 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.677+0000 7ffacba35700 1 --2- 172.21.15.1:0/3719758740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac4108530 0x7ffac4108940 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:42.581 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.677+0000 7ffacba35700 1 -- 172.21.15.1:0/3719758740 >> 172.21.15.1:0/3719758740 conn(0x7ffac41041e0 msgr2=0x7ffac4106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:42.581 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.677+0000 7ffacba35700 1 -- 172.21.15.1:0/3719758740 shutdown_connections 2024-05-04T03:08:42.581 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.677+0000 7ffacba35700 1 -- 172.21.15.1:0/3719758740 wait complete. 2024-05-04T03:08:42.581 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.678+0000 7ffacba35700 1 Processor -- start 2024-05-04T03:08:42.581 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.678+0000 7ffacba35700 1 -- start start 2024-05-04T03:08:42.582 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.678+0000 7ffacba35700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac4108530 0x7ffac41a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:42.582 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.678+0000 7ffacba35700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffac4109f40 con 0x7ffac4108530 2024-05-04T03:08:42.582 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.678+0000 7ffac97d1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac4108530 0x7ffac41a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:42.582 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.678+0000 7ffac97d1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac4108530 0x7ffac41a0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56880/0 (socket says 172.21.15.1:56880) 2024-05-04T03:08:42.582 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.678+0000 7ffac97d1700 1 -- 172.21.15.1:0/1962439904 learned_addr learned my addr 172.21.15.1:0/1962439904 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:42.582 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.679+0000 7ffac97d1700 1 -- 172.21.15.1:0/1962439904 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac41a07b0 con 0x7ffac4108530 2024-05-04T03:08:42.582 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.679+0000 7ffac97d1700 1 --2- 172.21.15.1:0/1962439904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac4108530 0x7ffac41a0270 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffab8004660 tx=0x7ffab8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:42.582 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.679+0000 7ffab67fc700 1 -- 172.21.15.1:0/1962439904 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffab8003a60 con 0x7ffac4108530 2024-05-04T03:08:42.582 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.679+0000 7ffacba35700 1 -- 172.21.15.1:0/1962439904 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac41a0a40 con 0x7ffac4108530 2024-05-04T03:08:42.582 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.679+0000 7ffab67fc700 1 -- 172.21.15.1:0/1962439904 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffab801b8f0 con 0x7ffac4108530 2024-05-04T03:08:42.582 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.679+0000 7ffab67fc700 1 -- 172.21.15.1:0/1962439904 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffab8023890 con 0x7ffac4108530 2024-05-04T03:08:42.582 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.679+0000 7ffacba35700 1 -- 172.21.15.1:0/1962439904 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac41a3650 con 0x7ffac4108530 2024-05-04T03:08:42.582 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.680+0000 7ffab67fc700 1 -- 172.21.15.1:0/1962439904 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7ffab802c3f0 con 0x7ffac4108530 2024-05-04T03:08:42.583 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.680+0000 7ffacba35700 1 -- 172.21.15.1:0/1962439904 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffac4064e20 con 0x7ffac4108530 2024-05-04T03:08:42.583 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.680+0000 7ffab67fc700 1 --2- 172.21.15.1:0/1962439904 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ffab003e510 0x7ffab0040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:42.583 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.680+0000 7ffab67fc700 1 -- 172.21.15.1:0/1962439904 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ffab80541a0 con 0x7ffac4108530 2024-05-04T03:08:42.583 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.680+0000 7ffac8fd0700 1 --2- 172.21.15.1:0/1962439904 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ffab003e510 0x7ffab0040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:42.583 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.681+0000 7ffac8fd0700 1 --2- 172.21.15.1:0/1962439904 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ffab003e510 0x7ffab0040a20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffac0006fd0 tx=0x7ffac0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:42.583 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:41.686+0000 7ffab67fc700 1 -- 172.21.15.1:0/1962439904 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffab801f030 con 0x7ffac4108530 2024-05-04T03:08:42.583 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:42.018+0000 7ffacba35700 1 -- 172.21.15.1:0/1962439904 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7ffac402e860 con 0x7ffac4108530 2024-05-04T03:08:42.583 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:42.021+0000 7ffab67fc700 1 -- 172.21.15.1:0/1962439904 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7ffab8017e90 con 0x7ffac4108530 2024-05-04T03:08:42.583 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:42.023+0000 7ffacba35700 1 -- 172.21.15.1:0/1962439904 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ffab003e510 msgr2=0x7ffab0040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:42.583 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:42.023+0000 7ffacba35700 1 --2- 172.21.15.1:0/1962439904 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ffab003e510 0x7ffab0040a20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffac0006fd0 tx=0x7ffac0006e40 comp rx=0 tx=0).stop 2024-05-04T03:08:42.583 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:42.023+0000 7ffacba35700 1 -- 172.21.15.1:0/1962439904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac4108530 msgr2=0x7ffac41a0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:42.583 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:42.023+0000 7ffacba35700 1 --2- 172.21.15.1:0/1962439904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac4108530 0x7ffac41a0270 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffab8004660 tx=0x7ffab8011aa0 comp rx=0 tx=0).stop 2024-05-04T03:08:42.583 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:42.023+0000 7ffacba35700 1 -- 172.21.15.1:0/1962439904 shutdown_connections 2024-05-04T03:08:42.583 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:42.023+0000 7ffacba35700 1 --2- 172.21.15.1:0/1962439904 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ffab003e510 0x7ffab0040a20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:42.584 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:42.023+0000 7ffacba35700 1 --2- 172.21.15.1:0/1962439904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffac4108530 0x7ffac41a0270 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:42.584 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:42.023+0000 7ffacba35700 1 -- 172.21.15.1:0/1962439904 >> 172.21.15.1:0/1962439904 conn(0x7ffac41041e0 msgr2=0x7ffac4194220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:42.584 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:42.024+0000 7ffacba35700 1 -- 172.21.15.1:0/1962439904 shutdown_connections 2024-05-04T03:08:42.584 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-05-04T03:08:42.024+0000 7ffacba35700 1 -- 172.21.15.1:0/1962439904 wait complete. 2024-05-04T03:08:42.584 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-04T03:08:42.584 INFO:teuthology.orchestra.run.smithi001.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-04T03:08:42.584 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:08:42.584 INFO:teuthology.orchestra.run.smithi001.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-04T03:08:42.584 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:08:42.584 INFO:teuthology.orchestra.run.smithi001.stdout:Or, if you are only running a single cluster on this host: 2024-05-04T03:08:42.584 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:08:42.584 INFO:teuthology.orchestra.run.smithi001.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-04T03:08:42.584 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:08:42.584 INFO:teuthology.orchestra.run.smithi001.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-04T03:08:42.584 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:08:42.585 INFO:teuthology.orchestra.run.smithi001.stdout: ceph telemetry on 2024-05-04T03:08:42.585 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:08:42.585 INFO:teuthology.orchestra.run.smithi001.stdout:For more information see: 2024-05-04T03:08:42.585 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:08:42.585 INFO:teuthology.orchestra.run.smithi001.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-04T03:08:42.585 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:08:42.585 INFO:teuthology.orchestra.run.smithi001.stdout:Bootstrap complete. 2024-05-04T03:08:42.617 INFO:tasks.cephadm:Fetching config... 2024-05-04T03:08:42.617 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-04T03:08:42.617 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-04T03:08:42.634 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-04T03:08:42.634 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-04T03:08:42.635 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-04T03:08:42.690 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-04T03:08:42.691 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-04T03:08:42.691 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-04T03:08:42.755 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-04T03:08:42.755 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-04T03:08:42.755 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-04T03:08:42.811 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-04T03:08:42.811 DEBUG:teuthology.orchestra.run.smithi001:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGVlXZK1X8uZZfL/hJsK8EueQx2js91wvEokJdSqH5Eimh4brSlWcuYmmf0mxR9mPT1+s5ymBO34rSSYfKFEfu1PiDM9sA3ov2wcME3Ax3lRwZJsK/qvpJ/4mcaOZMX7+Xg55ytxhUCe9pgHAivSdG4ufjDHnH3BKt4OR/89ZCRSBRGpkxiwM+1oMQzlwSKgju6wDWUBFrYBSvmcEIs/Biw1In+/4WhWnNYd63/AjXsGUI/6NPOaaqRFTeHVo7FtLvp50wZXxNeu5Td+NgCSvPdPE+YOAKM6coc9fM4NGSE+K+6rCN7p+XOqS+MF4Xq3g1BNI6gT6/GwXMTN6MX5hPwAIrTCuJJOJFDj32yw8EAGkP/k629ysBee7N8Bp6ORkUByyBbL181/pU/KQ3wRqHl2y8Z7k2DatyDP8Xdp0R25J9mQ65i7Mbxt1+s8GF8G2SNDba/awGwWHVjRRQpRrvoyAVREQWs7JUnkb8INyQjmZd8WXWrkOyTFL9wYgjUtE= ceph-4a572aa8-09c3-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-04T03:08:42.885 INFO:teuthology.orchestra.run.smithi001.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGVlXZK1X8uZZfL/hJsK8EueQx2js91wvEokJdSqH5Eimh4brSlWcuYmmf0mxR9mPT1+s5ymBO34rSSYfKFEfu1PiDM9sA3ov2wcME3Ax3lRwZJsK/qvpJ/4mcaOZMX7+Xg55ytxhUCe9pgHAivSdG4ufjDHnH3BKt4OR/89ZCRSBRGpkxiwM+1oMQzlwSKgju6wDWUBFrYBSvmcEIs/Biw1In+/4WhWnNYd63/AjXsGUI/6NPOaaqRFTeHVo7FtLvp50wZXxNeu5Td+NgCSvPdPE+YOAKM6coc9fM4NGSE+K+6rCN7p+XOqS+MF4Xq3g1BNI6gT6/GwXMTN6MX5hPwAIrTCuJJOJFDj32yw8EAGkP/k629ysBee7N8Bp6ORkUByyBbL181/pU/KQ3wRqHl2y8Z7k2DatyDP8Xdp0R25J9mQ65i7Mbxt1+s8GF8G2SNDba/awGwWHVjRRQpRrvoyAVREQWs7JUnkb8INyQjmZd8WXWrkOyTFL9wYgjUtE= ceph-4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:08:42.896 DEBUG:teuthology.orchestra.run.smithi072:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGVlXZK1X8uZZfL/hJsK8EueQx2js91wvEokJdSqH5Eimh4brSlWcuYmmf0mxR9mPT1+s5ymBO34rSSYfKFEfu1PiDM9sA3ov2wcME3Ax3lRwZJsK/qvpJ/4mcaOZMX7+Xg55ytxhUCe9pgHAivSdG4ufjDHnH3BKt4OR/89ZCRSBRGpkxiwM+1oMQzlwSKgju6wDWUBFrYBSvmcEIs/Biw1In+/4WhWnNYd63/AjXsGUI/6NPOaaqRFTeHVo7FtLvp50wZXxNeu5Td+NgCSvPdPE+YOAKM6coc9fM4NGSE+K+6rCN7p+XOqS+MF4Xq3g1BNI6gT6/GwXMTN6MX5hPwAIrTCuJJOJFDj32yw8EAGkP/k629ysBee7N8Bp6ORkUByyBbL181/pU/KQ3wRqHl2y8Z7k2DatyDP8Xdp0R25J9mQ65i7Mbxt1+s8GF8G2SNDba/awGwWHVjRRQpRrvoyAVREQWs7JUnkb8INyQjmZd8WXWrkOyTFL9wYgjUtE= ceph-4a572aa8-09c3-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-04T03:08:42.935 INFO:teuthology.orchestra.run.smithi072.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGVlXZK1X8uZZfL/hJsK8EueQx2js91wvEokJdSqH5Eimh4brSlWcuYmmf0mxR9mPT1+s5ymBO34rSSYfKFEfu1PiDM9sA3ov2wcME3Ax3lRwZJsK/qvpJ/4mcaOZMX7+Xg55ytxhUCe9pgHAivSdG4ufjDHnH3BKt4OR/89ZCRSBRGpkxiwM+1oMQzlwSKgju6wDWUBFrYBSvmcEIs/Biw1In+/4WhWnNYd63/AjXsGUI/6NPOaaqRFTeHVo7FtLvp50wZXxNeu5Td+NgCSvPdPE+YOAKM6coc9fM4NGSE+K+6rCN7p+XOqS+MF4Xq3g1BNI6gT6/GwXMTN6MX5hPwAIrTCuJJOJFDj32yw8EAGkP/k629ysBee7N8Bp6ORkUByyBbL181/pU/KQ3wRqHl2y8Z7k2DatyDP8Xdp0R25J9mQ65i7Mbxt1+s8GF8G2SNDba/awGwWHVjRRQpRrvoyAVREQWs7JUnkb8INyQjmZd8WXWrkOyTFL9wYgjUtE= ceph-4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:08:42.947 DEBUG:teuthology.orchestra.run.smithi077:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGVlXZK1X8uZZfL/hJsK8EueQx2js91wvEokJdSqH5Eimh4brSlWcuYmmf0mxR9mPT1+s5ymBO34rSSYfKFEfu1PiDM9sA3ov2wcME3Ax3lRwZJsK/qvpJ/4mcaOZMX7+Xg55ytxhUCe9pgHAivSdG4ufjDHnH3BKt4OR/89ZCRSBRGpkxiwM+1oMQzlwSKgju6wDWUBFrYBSvmcEIs/Biw1In+/4WhWnNYd63/AjXsGUI/6NPOaaqRFTeHVo7FtLvp50wZXxNeu5Td+NgCSvPdPE+YOAKM6coc9fM4NGSE+K+6rCN7p+XOqS+MF4Xq3g1BNI6gT6/GwXMTN6MX5hPwAIrTCuJJOJFDj32yw8EAGkP/k629ysBee7N8Bp6ORkUByyBbL181/pU/KQ3wRqHl2y8Z7k2DatyDP8Xdp0R25J9mQ65i7Mbxt1+s8GF8G2SNDba/awGwWHVjRRQpRrvoyAVREQWs7JUnkb8INyQjmZd8WXWrkOyTFL9wYgjUtE= ceph-4a572aa8-09c3-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-04T03:08:42.986 INFO:teuthology.orchestra.run.smithi077.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGVlXZK1X8uZZfL/hJsK8EueQx2js91wvEokJdSqH5Eimh4brSlWcuYmmf0mxR9mPT1+s5ymBO34rSSYfKFEfu1PiDM9sA3ov2wcME3Ax3lRwZJsK/qvpJ/4mcaOZMX7+Xg55ytxhUCe9pgHAivSdG4ufjDHnH3BKt4OR/89ZCRSBRGpkxiwM+1oMQzlwSKgju6wDWUBFrYBSvmcEIs/Biw1In+/4WhWnNYd63/AjXsGUI/6NPOaaqRFTeHVo7FtLvp50wZXxNeu5Td+NgCSvPdPE+YOAKM6coc9fM4NGSE+K+6rCN7p+XOqS+MF4Xq3g1BNI6gT6/GwXMTN6MX5hPwAIrTCuJJOJFDj32yw8EAGkP/k629ysBee7N8Bp6ORkUByyBbL181/pU/KQ3wRqHl2y8Z7k2DatyDP8Xdp0R25J9mQ65i7Mbxt1+s8GF8G2SNDba/awGwWHVjRRQpRrvoyAVREQWs7JUnkb8INyQjmZd8WXWrkOyTFL9wYgjUtE= ceph-4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:08:42.998 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-04T03:08:43.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:43 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/1962439904' entity='client.admin' 2024-05-04T03:08:43.250 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:45.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.368+0000 7f9a37e07700 1 -- 172.21.15.1:0/3619386741 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a301046d0 msgr2=0x7f9a30106af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:45.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.368+0000 7f9a37e07700 1 --2- 172.21.15.1:0/3619386741 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a301046d0 0x7f9a30106af0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c009910 tx=0x7f9a2c009c20 comp rx=0 tx=0).stop 2024-05-04T03:08:45.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.368+0000 7f9a37e07700 1 -- 172.21.15.1:0/3619386741 shutdown_connections 2024-05-04T03:08:45.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.368+0000 7f9a37e07700 1 --2- 172.21.15.1:0/3619386741 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a301046d0 0x7f9a30106af0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:45.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.368+0000 7f9a37e07700 1 -- 172.21.15.1:0/3619386741 >> 172.21.15.1:0/3619386741 conn(0x7f9a300fe110 msgr2=0x7f9a301005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:45.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.368+0000 7f9a37e07700 1 -- 172.21.15.1:0/3619386741 shutdown_connections 2024-05-04T03:08:45.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.368+0000 7f9a37e07700 1 -- 172.21.15.1:0/3619386741 wait complete. 2024-05-04T03:08:45.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.368+0000 7f9a37e07700 1 Processor -- start 2024-05-04T03:08:45.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.368+0000 7f9a37e07700 1 -- start start 2024-05-04T03:08:45.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.369+0000 7f9a37e07700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a301046d0 0x7f9a30103e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:45.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.369+0000 7f9a37e07700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9a30109b90 con 0x7f9a301046d0 2024-05-04T03:08:45.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.369+0000 7f9a35ba3700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a301046d0 0x7f9a30103e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:45.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.369+0000 7f9a35ba3700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a301046d0 0x7f9a30103e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56894/0 (socket says 172.21.15.1:56894) 2024-05-04T03:08:45.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.369+0000 7f9a35ba3700 1 -- 172.21.15.1:0/103303304 learned_addr learned my addr 172.21.15.1:0/103303304 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:45.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.369+0000 7f9a35ba3700 1 -- 172.21.15.1:0/103303304 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a30104390 con 0x7f9a301046d0 2024-05-04T03:08:45.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.369+0000 7f9a35ba3700 1 --2- 172.21.15.1:0/103303304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a301046d0 0x7f9a30103e50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c006010 tx=0x7f9a2c005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:45.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.369+0000 7f9a26ffd700 1 -- 172.21.15.1:0/103303304 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a2c026070 con 0x7f9a301046d0 2024-05-04T03:08:45.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.369+0000 7f9a26ffd700 1 -- 172.21.15.1:0/103303304 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a2c0044a0 con 0x7f9a301046d0 2024-05-04T03:08:45.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.369+0000 7f9a37e07700 1 -- 172.21.15.1:0/103303304 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a301025e0 con 0x7f9a301046d0 2024-05-04T03:08:45.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.369+0000 7f9a26ffd700 1 -- 172.21.15.1:0/103303304 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a2c0039d0 con 0x7f9a301046d0 2024-05-04T03:08:45.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.369+0000 7f9a37e07700 1 -- 172.21.15.1:0/103303304 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a30102ac0 con 0x7f9a301046d0 2024-05-04T03:08:45.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.370+0000 7f9a37e07700 1 -- 172.21.15.1:0/103303304 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a3010a310 con 0x7f9a301046d0 2024-05-04T03:08:45.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.370+0000 7f9a26ffd700 1 -- 172.21.15.1:0/103303304 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f9a2c029070 con 0x7f9a301046d0 2024-05-04T03:08:45.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.371+0000 7f9a26ffd700 1 --2- 172.21.15.1:0/103303304 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9a1c03e510 0x7f9a1c040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:45.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.371+0000 7f9a26ffd700 1 -- 172.21.15.1:0/103303304 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9a2c01e070 con 0x7f9a301046d0 2024-05-04T03:08:45.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.371+0000 7f9a353a2700 1 --2- 172.21.15.1:0/103303304 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9a1c03e510 0x7f9a1c040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:45.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.371+0000 7f9a353a2700 1 --2- 172.21.15.1:0/103303304 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9a1c03e510 0x7f9a1c040a20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9a20006fd0 tx=0x7f9a20006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:45.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.376+0000 7f9a26ffd700 1 -- 172.21.15.1:0/103303304 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9a2c013070 con 0x7f9a301046d0 2024-05-04T03:08:45.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.591+0000 7f9a37e07700 1 -- 172.21.15.1:0/103303304 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f9a30192530 con 0x7f9a301046d0 2024-05-04T03:08:45.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.595+0000 7f9a26ffd700 1 -- 172.21.15.1:0/103303304 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f9a2c0300c0 con 0x7f9a301046d0 2024-05-04T03:08:45.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.597+0000 7f9a37e07700 1 -- 172.21.15.1:0/103303304 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9a1c03e510 msgr2=0x7f9a1c040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:45.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.597+0000 7f9a37e07700 1 --2- 172.21.15.1:0/103303304 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9a1c03e510 0x7f9a1c040a20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9a20006fd0 tx=0x7f9a20006e40 comp rx=0 tx=0).stop 2024-05-04T03:08:45.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.597+0000 7f9a37e07700 1 -- 172.21.15.1:0/103303304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a301046d0 msgr2=0x7f9a30103e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:45.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.597+0000 7f9a37e07700 1 --2- 172.21.15.1:0/103303304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a301046d0 0x7f9a30103e50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c006010 tx=0x7f9a2c005ad0 comp rx=0 tx=0).stop 2024-05-04T03:08:45.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.598+0000 7f9a37e07700 1 -- 172.21.15.1:0/103303304 shutdown_connections 2024-05-04T03:08:45.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.598+0000 7f9a37e07700 1 --2- 172.21.15.1:0/103303304 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9a1c03e510 0x7f9a1c040a20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:45.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.598+0000 7f9a37e07700 1 --2- 172.21.15.1:0/103303304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a301046d0 0x7f9a30103e50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:45.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.598+0000 7f9a37e07700 1 -- 172.21.15.1:0/103303304 >> 172.21.15.1:0/103303304 conn(0x7f9a300fe110 msgr2=0x7f9a300fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:45.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.598+0000 7f9a37e07700 1 -- 172.21.15.1:0/103303304 shutdown_connections 2024-05-04T03:08:45.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:45.598+0000 7f9a37e07700 1 -- 172.21.15.1:0/103303304 wait complete. 2024-05-04T03:08:46.168 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:45 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:08:46.168 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:45 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:08:46.168 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:45 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi001", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:08:46.168 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:45 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:08:46.168 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:45 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:08:46.168 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:45 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:08:46.169 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:45 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/103303304' entity='client.admin' 2024-05-04T03:08:46.169 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:45 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:08:46.169 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:45 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:08:46.194 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-04T03:08:46.194 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-04T03:08:46.436 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:48.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.026+0000 7f7895f06700 1 -- 172.21.15.1:0/2092576982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7890103be0 msgr2=0x7f7890103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:48.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.026+0000 7f7895f06700 1 --2- 172.21.15.1:0/2092576982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7890103be0 0x7f7890103ff0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7880009910 tx=0x7f7880009c20 comp rx=0 tx=0).stop 2024-05-04T03:08:48.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.027+0000 7f7895f06700 1 -- 172.21.15.1:0/2092576982 shutdown_connections 2024-05-04T03:08:48.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.027+0000 7f7895f06700 1 --2- 172.21.15.1:0/2092576982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7890103be0 0x7f7890103ff0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:48.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.027+0000 7f7895f06700 1 -- 172.21.15.1:0/2092576982 >> 172.21.15.1:0/2092576982 conn(0x7f78900ff220 msgr2=0x7f78901016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:48.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.027+0000 7f7895f06700 1 -- 172.21.15.1:0/2092576982 shutdown_connections 2024-05-04T03:08:48.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.027+0000 7f7895f06700 1 -- 172.21.15.1:0/2092576982 wait complete. 2024-05-04T03:08:48.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.027+0000 7f7895f06700 1 Processor -- start 2024-05-04T03:08:48.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.028+0000 7f7895f06700 1 -- start start 2024-05-04T03:08:48.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.028+0000 7f7895f06700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7890103be0 0x7f7890199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:48.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.028+0000 7f7895f06700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7890109ba0 con 0x7f7890103be0 2024-05-04T03:08:48.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.028+0000 7f788f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7890103be0 0x7f7890199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:48.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.028+0000 7f788f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7890103be0 0x7f7890199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56914/0 (socket says 172.21.15.1:56914) 2024-05-04T03:08:48.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.028+0000 7f788f7fe700 1 -- 172.21.15.1:0/2509094802 learned_addr learned my addr 172.21.15.1:0/2509094802 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:48.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.028+0000 7f788f7fe700 1 -- 172.21.15.1:0/2509094802 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7890199c40 con 0x7f7890103be0 2024-05-04T03:08:48.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.029+0000 7f788f7fe700 1 --2- 172.21.15.1:0/2509094802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7890103be0 0x7f7890199700 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f78800057d0 tx=0x7f7880005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:48.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.029+0000 7f788cff9700 1 -- 172.21.15.1:0/2509094802 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7880027070 con 0x7f7890103be0 2024-05-04T03:08:48.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.029+0000 7f788cff9700 1 -- 172.21.15.1:0/2509094802 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f788000be10 con 0x7f7890103be0 2024-05-04T03:08:48.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.029+0000 7f7895f06700 1 -- 172.21.15.1:0/2509094802 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7890199ed0 con 0x7f7890103be0 2024-05-04T03:08:48.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.029+0000 7f788cff9700 1 -- 172.21.15.1:0/2509094802 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7880022dd0 con 0x7f7890103be0 2024-05-04T03:08:48.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.029+0000 7f7895f06700 1 -- 172.21.15.1:0/2509094802 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f789019cae0 con 0x7f7890103be0 2024-05-04T03:08:48.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.030+0000 7f7895f06700 1 -- 172.21.15.1:0/2509094802 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f789010a320 con 0x7f7890103be0 2024-05-04T03:08:48.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.030+0000 7f788cff9700 1 -- 172.21.15.1:0/2509094802 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f788002a070 con 0x7f7890103be0 2024-05-04T03:08:48.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.030+0000 7f788cff9700 1 --2- 172.21.15.1:0/2509094802 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f787803e4c0 0x7f78780409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:48.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.030+0000 7f788cff9700 1 -- 172.21.15.1:0/2509094802 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f788001e030 con 0x7f7890103be0 2024-05-04T03:08:48.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.030+0000 7f788effd700 1 --2- 172.21.15.1:0/2509094802 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f787803e4c0 0x7f78780409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:48.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.031+0000 7f788effd700 1 --2- 172.21.15.1:0/2509094802 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f787803e4c0 0x7f78780409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7884006fd0 tx=0x7f7884006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:48.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.036+0000 7f788cff9700 1 -- 172.21.15.1:0/2509094802 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7880013070 con 0x7f7890103be0 2024-05-04T03:08:48.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.254+0000 7f7895f06700 1 -- 172.21.15.1:0/2509094802 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f789002e700 con 0x7f787803e4c0 2024-05-04T03:08:48.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.257+0000 7f788cff9700 1 -- 172.21.15.1:0/2509094802 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f789002e700 con 0x7f787803e4c0 2024-05-04T03:08:48.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.259+0000 7f7895f06700 1 -- 172.21.15.1:0/2509094802 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f787803e4c0 msgr2=0x7f78780409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:48.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.259+0000 7f7895f06700 1 --2- 172.21.15.1:0/2509094802 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f787803e4c0 0x7f78780409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7884006fd0 tx=0x7f7884006e40 comp rx=0 tx=0).stop 2024-05-04T03:08:48.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.259+0000 7f7895f06700 1 -- 172.21.15.1:0/2509094802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7890103be0 msgr2=0x7f7890199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:48.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.259+0000 7f7895f06700 1 --2- 172.21.15.1:0/2509094802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7890103be0 0x7f7890199700 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f78800057d0 tx=0x7f7880005800 comp rx=0 tx=0).stop 2024-05-04T03:08:48.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.260+0000 7f7895f06700 1 -- 172.21.15.1:0/2509094802 shutdown_connections 2024-05-04T03:08:48.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.260+0000 7f7895f06700 1 --2- 172.21.15.1:0/2509094802 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f787803e4c0 0x7f78780409d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:48.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.260+0000 7f7895f06700 1 --2- 172.21.15.1:0/2509094802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7890103be0 0x7f7890199700 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:48.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.260+0000 7f7895f06700 1 -- 172.21.15.1:0/2509094802 >> 172.21.15.1:0/2509094802 conn(0x7f78900ff220 msgr2=0x7f78900fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:48.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.260+0000 7f7895f06700 1 -- 172.21.15.1:0/2509094802 shutdown_connections 2024-05-04T03:08:48.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:48.260+0000 7f7895f06700 1 -- 172.21.15.1:0/2509094802 wait complete. 2024-05-04T03:08:49.037 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi072 2024-05-04T03:08:49.037 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-04T03:08:49.037 DEBUG:teuthology.orchestra.run.smithi072:> dd of=/etc/ceph/ceph.conf 2024-05-04T03:08:49.055 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-04T03:08:49.055 DEBUG:teuthology.orchestra.run.smithi072:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-04T03:08:49.112 INFO:tasks.cephadm:Adding host smithi072 to orchestrator... 2024-05-04T03:08:49.112 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph orch host add smithi072 2024-05-04T03:08:49.258 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:49 smithi001 ceph-mon[29731]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:08:49.258 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:49 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:08:49.258 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:49 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:08:49.258 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:49 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:08:49.258 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:49 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:08:49.258 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:49 smithi001 ceph-mon[29731]: Updating smithi001:/etc/ceph/ceph.conf 2024-05-04T03:08:49.258 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:49 smithi001 ceph-mon[29731]: Updating smithi001:/var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:08:49.258 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:49 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:08:49.258 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:49 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:08:49.258 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:49 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:08:49.358 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:50.588 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:50 smithi001 ceph-mon[29731]: Updating smithi001:/etc/ceph/ceph.client.admin.keyring 2024-05-04T03:08:50.588 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:50 smithi001 ceph-mon[29731]: Updating smithi001:/var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-04T03:08:50.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:50.866+0000 7f9c0810e700 1 -- 172.21.15.1:0/427500390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c00103dd0 msgr2=0x7f9c001041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:50.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:50.866+0000 7f9c0810e700 1 --2- 172.21.15.1:0/427500390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c00103dd0 0x7f9c001041e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009910 tx=0x7f9bfc009c20 comp rx=0 tx=0).stop 2024-05-04T03:08:50.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:50.866+0000 7f9c0810e700 1 -- 172.21.15.1:0/427500390 shutdown_connections 2024-05-04T03:08:50.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:50.866+0000 7f9c0810e700 1 --2- 172.21.15.1:0/427500390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c00103dd0 0x7f9c001041e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:50.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:50.866+0000 7f9c0810e700 1 -- 172.21.15.1:0/427500390 >> 172.21.15.1:0/427500390 conn(0x7f9c000ff660 msgr2=0x7f9c00101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:50.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:50.866+0000 7f9c0810e700 1 -- 172.21.15.1:0/427500390 shutdown_connections 2024-05-04T03:08:50.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:50.866+0000 7f9c0810e700 1 -- 172.21.15.1:0/427500390 wait complete. 2024-05-04T03:08:50.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:50.867+0000 7f9c0810e700 1 Processor -- start 2024-05-04T03:08:50.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:50.867+0000 7f9c0810e700 1 -- start start 2024-05-04T03:08:50.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:50.867+0000 7f9c0810e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c00103dd0 0x7f9c0019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:50.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:50.867+0000 7f9c0810e700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0010e0c0 con 0x7f9c00103dd0 2024-05-04T03:08:50.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:50.867+0000 7f9c05eaa700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c00103dd0 0x7f9c0019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:50.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:50.867+0000 7f9c05eaa700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c00103dd0 0x7f9c0019f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56946/0 (socket says 172.21.15.1:56946) 2024-05-04T03:08:50.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:50.867+0000 7f9c05eaa700 1 -- 172.21.15.1:0/956172598 learned_addr learned my addr 172.21.15.1:0/956172598 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:50.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:50.867+0000 7f9c05eaa700 1 -- 172.21.15.1:0/956172598 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0019fb40 con 0x7f9c00103dd0 2024-05-04T03:08:50.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:50.868+0000 7f9c05eaa700 1 --2- 172.21.15.1:0/956172598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c00103dd0 0x7f9c0019f600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0056d0 tx=0x7f9bfc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:50.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:50.868+0000 7f9bf6ffd700 1 -- 172.21.15.1:0/956172598 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bfc026070 con 0x7f9c00103dd0 2024-05-04T03:08:50.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:50.868+0000 7f9bf6ffd700 1 -- 172.21.15.1:0/956172598 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bfc00bba0 con 0x7f9c00103dd0 2024-05-04T03:08:50.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:50.868+0000 7f9c0810e700 1 -- 172.21.15.1:0/956172598 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0019fe30 con 0x7f9c00103dd0 2024-05-04T03:08:50.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:50.868+0000 7f9bf6ffd700 1 -- 172.21.15.1:0/956172598 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bfc003920 con 0x7f9c00103dd0 2024-05-04T03:08:50.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:50.868+0000 7f9c0810e700 1 -- 172.21.15.1:0/956172598 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c00199960 con 0x7f9c00103dd0 2024-05-04T03:08:50.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:50.869+0000 7f9bf6ffd700 1 -- 172.21.15.1:0/956172598 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f9bfc029070 con 0x7f9c00103dd0 2024-05-04T03:08:50.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:50.869+0000 7f9c0810e700 1 -- 172.21.15.1:0/956172598 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c00105280 con 0x7f9c00103dd0 2024-05-04T03:08:50.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:50.869+0000 7f9bf6ffd700 1 --2- 172.21.15.1:0/956172598 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9bec03e510 0x7f9bec040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:50.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:50.869+0000 7f9bf6ffd700 1 -- 172.21.15.1:0/956172598 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9bfc01e070 con 0x7f9c00103dd0 2024-05-04T03:08:50.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:50.869+0000 7f9c056a9700 1 --2- 172.21.15.1:0/956172598 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9bec03e510 0x7f9bec040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:50.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:50.870+0000 7f9c056a9700 1 --2- 172.21.15.1:0/956172598 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9bec03e510 0x7f9bec040a20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0006fd0 tx=0x7f9bf0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:50.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:50.875+0000 7f9bf6ffd700 1 -- 172.21.15.1:0/956172598 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9bfc013070 con 0x7f9c00103dd0 2024-05-04T03:08:51.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:51.095+0000 7f9c0810e700 1 -- 172.21.15.1:0/956172598 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi072", "target": ["mon-mgr", ""]}) -- 0x7f9c0019ad10 con 0x7f9bec03e510 2024-05-04T03:08:51.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:51 smithi001 ceph-mon[29731]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi072", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:08:53.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:52 smithi001 ceph-mon[29731]: Deploying cephadm binary to smithi072 2024-05-04T03:08:54.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:54 smithi001 ceph-mon[29731]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:08:55.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:55.332+0000 7f9bf6ffd700 1 -- 172.21.15.1:0/956172598 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f9c0019ad10 con 0x7f9bec03e510 2024-05-04T03:08:55.333 INFO:teuthology.orchestra.run.smithi001.stdout:Added host 'smithi072' with addr '172.21.15.72' 2024-05-04T03:08:55.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:55.335+0000 7f9c0810e700 1 -- 172.21.15.1:0/956172598 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9bec03e510 msgr2=0x7f9bec040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:55.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:55.335+0000 7f9c0810e700 1 --2- 172.21.15.1:0/956172598 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9bec03e510 0x7f9bec040a20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0006fd0 tx=0x7f9bf0006e40 comp rx=0 tx=0).stop 2024-05-04T03:08:55.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:55.335+0000 7f9c0810e700 1 -- 172.21.15.1:0/956172598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c00103dd0 msgr2=0x7f9c0019f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:55.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:55.335+0000 7f9c0810e700 1 --2- 172.21.15.1:0/956172598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c00103dd0 0x7f9c0019f600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0056d0 tx=0x7f9bfc0057b0 comp rx=0 tx=0).stop 2024-05-04T03:08:55.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:55.335+0000 7f9c0810e700 1 -- 172.21.15.1:0/956172598 shutdown_connections 2024-05-04T03:08:55.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:55.335+0000 7f9c0810e700 1 --2- 172.21.15.1:0/956172598 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9bec03e510 0x7f9bec040a20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:55.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:55.335+0000 7f9c0810e700 1 --2- 172.21.15.1:0/956172598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c00103dd0 0x7f9c0019f600 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:55.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:55.335+0000 7f9c0810e700 1 -- 172.21.15.1:0/956172598 >> 172.21.15.1:0/956172598 conn(0x7f9c000ff660 msgr2=0x7f9c00109900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:55.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:55.335+0000 7f9c0810e700 1 -- 172.21.15.1:0/956172598 shutdown_connections 2024-05-04T03:08:55.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:55.335+0000 7f9c0810e700 1 -- 172.21.15.1:0/956172598 wait complete. 2024-05-04T03:08:55.885 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-04T03:08:56.134 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:56.564 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:56 smithi001 ceph-mon[29731]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:08:56.564 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:56 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:08:56.564 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:56 smithi001 ceph-mon[29731]: Added host smithi072 2024-05-04T03:08:56.564 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:56 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:08:56.564 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:56 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:08:57.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:57 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:08:57.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.693+0000 7f9e96102700 1 -- 172.21.15.1:0/480553829 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e90107a60 msgr2=0x7f9e90107e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:57.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.693+0000 7f9e96102700 1 --2- 172.21.15.1:0/480553829 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e90107a60 0x7f9e90107e70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9e88009910 tx=0x7f9e88009c20 comp rx=0 tx=0).stop 2024-05-04T03:08:57.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.693+0000 7f9e96102700 1 -- 172.21.15.1:0/480553829 shutdown_connections 2024-05-04T03:08:57.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.693+0000 7f9e96102700 1 --2- 172.21.15.1:0/480553829 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e90107a60 0x7f9e90107e70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:57.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.693+0000 7f9e96102700 1 -- 172.21.15.1:0/480553829 >> 172.21.15.1:0/480553829 conn(0x7f9e900fde30 msgr2=0x7f9e90100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:57.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.694+0000 7f9e96102700 1 -- 172.21.15.1:0/480553829 shutdown_connections 2024-05-04T03:08:57.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.694+0000 7f9e96102700 1 -- 172.21.15.1:0/480553829 wait complete. 2024-05-04T03:08:57.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.694+0000 7f9e96102700 1 Processor -- start 2024-05-04T03:08:57.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.694+0000 7f9e96102700 1 -- start start 2024-05-04T03:08:57.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.695+0000 7f9e96102700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e90107a60 0x7f9e9019a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:57.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.695+0000 7f9e96102700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9010d100 con 0x7f9e90107a60 2024-05-04T03:08:57.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.695+0000 7f9e8f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e90107a60 0x7f9e9019a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:57.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.695+0000 7f9e8f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e90107a60 0x7f9e9019a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54908/0 (socket says 172.21.15.1:54908) 2024-05-04T03:08:57.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.695+0000 7f9e8f7fe700 1 -- 172.21.15.1:0/2753914259 learned_addr learned my addr 172.21.15.1:0/2753914259 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:08:57.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.695+0000 7f9e8f7fe700 1 -- 172.21.15.1:0/2753914259 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e90102fe0 con 0x7f9e90107a60 2024-05-04T03:08:57.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.695+0000 7f9e8f7fe700 1 --2- 172.21.15.1:0/2753914259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e90107a60 0x7f9e9019a7f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9e880056d0 tx=0x7f9e880057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:57.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.695+0000 7f9e8cff9700 1 -- 172.21.15.1:0/2753914259 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e88026070 con 0x7f9e90107a60 2024-05-04T03:08:57.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.695+0000 7f9e96102700 1 -- 172.21.15.1:0/2753914259 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e9019ad90 con 0x7f9e90107a60 2024-05-04T03:08:57.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.695+0000 7f9e8cff9700 1 -- 172.21.15.1:0/2753914259 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e8800bc00 con 0x7f9e90107a60 2024-05-04T03:08:57.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.695+0000 7f9e8cff9700 1 -- 172.21.15.1:0/2753914259 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e88003920 con 0x7f9e90107a60 2024-05-04T03:08:57.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.695+0000 7f9e96102700 1 -- 172.21.15.1:0/2753914259 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e9019b250 con 0x7f9e90107a60 2024-05-04T03:08:57.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.696+0000 7f9e8cff9700 1 -- 172.21.15.1:0/2753914259 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f9e88029070 con 0x7f9e90107a60 2024-05-04T03:08:57.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.697+0000 7f9e8cff9700 1 --2- 172.21.15.1:0/2753914259 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9e7c03e510 0x7f9e7c040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:08:57.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.697+0000 7f9e8cff9700 1 -- 172.21.15.1:0/2753914259 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9e8801e070 con 0x7f9e90107a60 2024-05-04T03:08:57.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.697+0000 7f9e96102700 1 -- 172.21.15.1:0/2753914259 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e901041f0 con 0x7f9e90107a60 2024-05-04T03:08:57.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.697+0000 7f9e8effd700 1 --2- 172.21.15.1:0/2753914259 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9e7c03e510 0x7f9e7c040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:08:57.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.697+0000 7f9e8effd700 1 --2- 172.21.15.1:0/2753914259 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9e7c03e510 0x7f9e7c040a20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9e78006fd0 tx=0x7f9e78006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:08:57.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.702+0000 7f9e8cff9700 1 -- 172.21.15.1:0/2753914259 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9e88030080 con 0x7f9e90107a60 2024-05-04T03:08:57.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.924+0000 7f9e96102700 1 -- 172.21.15.1:0/2753914259 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9e901026c0 con 0x7f9e7c03e510 2024-05-04T03:08:57.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.925+0000 7f9e8cff9700 1 -- 172.21.15.1:0/2753914259 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+158 (secure 0 0 0) 0x7f9e901026c0 con 0x7f9e7c03e510 2024-05-04T03:08:57.926 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:08:57.926 INFO:teuthology.orchestra.run.smithi001.stdout:[{"addr": "172.21.15.1", "hostname": "smithi001", "labels": [], "status": ""}, {"addr": "172.21.15.72", "hostname": "smithi072", "labels": [], "status": ""}] 2024-05-04T03:08:57.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.927+0000 7f9e96102700 1 -- 172.21.15.1:0/2753914259 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9e7c03e510 msgr2=0x7f9e7c040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:57.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.927+0000 7f9e96102700 1 --2- 172.21.15.1:0/2753914259 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9e7c03e510 0x7f9e7c040a20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9e78006fd0 tx=0x7f9e78006e40 comp rx=0 tx=0).stop 2024-05-04T03:08:57.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.927+0000 7f9e96102700 1 -- 172.21.15.1:0/2753914259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e90107a60 msgr2=0x7f9e9019a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:08:57.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.927+0000 7f9e96102700 1 --2- 172.21.15.1:0/2753914259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e90107a60 0x7f9e9019a7f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9e880056d0 tx=0x7f9e880057b0 comp rx=0 tx=0).stop 2024-05-04T03:08:57.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.928+0000 7f9e96102700 1 -- 172.21.15.1:0/2753914259 shutdown_connections 2024-05-04T03:08:57.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.928+0000 7f9e96102700 1 --2- 172.21.15.1:0/2753914259 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9e7c03e510 0x7f9e7c040a20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:57.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.928+0000 7f9e96102700 1 --2- 172.21.15.1:0/2753914259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e90107a60 0x7f9e9019a7f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:08:57.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.928+0000 7f9e96102700 1 -- 172.21.15.1:0/2753914259 >> 172.21.15.1:0/2753914259 conn(0x7f9e900fde30 msgr2=0x7f9e90104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:08:57.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.928+0000 7f9e96102700 1 -- 172.21.15.1:0/2753914259 shutdown_connections 2024-05-04T03:08:57.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:08:57.928+0000 7f9e96102700 1 -- 172.21.15.1:0/2753914259 wait complete. 2024-05-04T03:08:58.524 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:58 smithi001 ceph-mon[29731]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:08:58.524 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:58 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:08:58.559 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi077 2024-05-04T03:08:58.559 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-04T03:08:58.559 DEBUG:teuthology.orchestra.run.smithi077:> dd of=/etc/ceph/ceph.conf 2024-05-04T03:08:58.579 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-04T03:08:58.579 DEBUG:teuthology.orchestra.run.smithi077:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-04T03:08:58.636 INFO:tasks.cephadm:Adding host smithi077 to orchestrator... 2024-05-04T03:08:58.636 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph orch host add smithi077 2024-05-04T03:08:58.885 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:08:59.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:08:59 smithi001 ceph-mon[29731]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:09:00.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.473+0000 7f5c3ac89700 1 -- 172.21.15.1:0/786862303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c34108a70 msgr2=0x7f5c34108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:00.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.473+0000 7f5c3ac89700 1 --2- 172.21.15.1:0/786862303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c34108a70 0x7f5c34108e80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5c28009910 tx=0x7f5c28009c20 comp rx=0 tx=0).stop 2024-05-04T03:09:00.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.473+0000 7f5c3ac89700 1 -- 172.21.15.1:0/786862303 shutdown_connections 2024-05-04T03:09:00.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.473+0000 7f5c3ac89700 1 --2- 172.21.15.1:0/786862303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c34108a70 0x7f5c34108e80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:00.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.473+0000 7f5c3ac89700 1 -- 172.21.15.1:0/786862303 >> 172.21.15.1:0/786862303 conn(0x7f5c340fe5c0 msgr2=0x7f5c34100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:00.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.473+0000 7f5c3ac89700 1 -- 172.21.15.1:0/786862303 shutdown_connections 2024-05-04T03:09:00.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.473+0000 7f5c3ac89700 1 -- 172.21.15.1:0/786862303 wait complete. 2024-05-04T03:09:00.475 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:00 smithi001 ceph-mon[29731]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:00.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.473+0000 7f5c3ac89700 1 Processor -- start 2024-05-04T03:09:00.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.474+0000 7f5c3ac89700 1 -- start start 2024-05-04T03:09:00.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.474+0000 7f5c3ac89700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c34108a70 0x7f5c340ff4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:00.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.474+0000 7f5c3ac89700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3410cfc0 con 0x7f5c34108a70 2024-05-04T03:09:00.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.474+0000 7f5c38a25700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c34108a70 0x7f5c340ff4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:00.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.474+0000 7f5c38a25700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c34108a70 0x7f5c340ff4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54920/0 (socket says 172.21.15.1:54920) 2024-05-04T03:09:00.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.474+0000 7f5c38a25700 1 -- 172.21.15.1:0/3109146005 learned_addr learned my addr 172.21.15.1:0/3109146005 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:09:00.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.474+0000 7f5c38a25700 1 -- 172.21.15.1:0/3109146005 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c34101300 con 0x7f5c34108a70 2024-05-04T03:09:00.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.474+0000 7f5c38a25700 1 --2- 172.21.15.1:0/3109146005 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c34108a70 0x7f5c340ff4b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5c280056d0 tx=0x7f5c280057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:00.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.475+0000 7f5c31ffb700 1 -- 172.21.15.1:0/3109146005 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c28027070 con 0x7f5c34108a70 2024-05-04T03:09:00.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.475+0000 7f5c31ffb700 1 -- 172.21.15.1:0/3109146005 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c2800be10 con 0x7f5c34108a70 2024-05-04T03:09:00.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.475+0000 7f5c3ac89700 1 -- 172.21.15.1:0/3109146005 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c340ff9f0 con 0x7f5c34108a70 2024-05-04T03:09:00.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.475+0000 7f5c31ffb700 1 -- 172.21.15.1:0/3109146005 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c28022d20 con 0x7f5c34108a70 2024-05-04T03:09:00.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.475+0000 7f5c3ac89700 1 -- 172.21.15.1:0/3109146005 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c340ffeb0 con 0x7f5c34108a70 2024-05-04T03:09:00.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.476+0000 7f5c31ffb700 1 -- 172.21.15.1:0/3109146005 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f5c2802a070 con 0x7f5c34108a70 2024-05-04T03:09:00.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.476+0000 7f5c31ffb700 1 --2- 172.21.15.1:0/3109146005 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5c2403e570 0x7f5c24040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:00.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.476+0000 7f5c3ac89700 1 -- 172.21.15.1:0/3109146005 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c34104180 con 0x7f5c34108a70 2024-05-04T03:09:00.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.476+0000 7f5c31ffb700 1 -- 172.21.15.1:0/3109146005 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5c2801e030 con 0x7f5c34108a70 2024-05-04T03:09:00.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.476+0000 7f5c33fff700 1 --2- 172.21.15.1:0/3109146005 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5c2403e570 0x7f5c24040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:00.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.477+0000 7f5c33fff700 1 --2- 172.21.15.1:0/3109146005 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5c2403e570 0x7f5c24040a80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5c20006fd0 tx=0x7f5c20006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:00.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.482+0000 7f5c31ffb700 1 -- 172.21.15.1:0/3109146005 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5c28013070 con 0x7f5c34108a70 2024-05-04T03:09:00.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:00.698+0000 7f5c3ac89700 1 -- 172.21.15.1:0/3109146005 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi077", "target": ["mon-mgr", ""]}) -- 0x7f5c34100a50 con 0x7f5c2403e570 2024-05-04T03:09:02.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:01 smithi001 ceph-mon[29731]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi077", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:09:02.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:01 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:02.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:01 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:02.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:01 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:02.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:01 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:02.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:01 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi072", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:09:02.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:01 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:02.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:01 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:09:02.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:01 smithi001 ceph-mon[29731]: Updating smithi072:/etc/ceph/ceph.conf 2024-05-04T03:09:02.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:01 smithi001 ceph-mon[29731]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:02.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:01 smithi001 ceph-mon[29731]: Updating smithi072:/var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:09:02.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:01 smithi001 ceph-mon[29731]: Deploying cephadm binary to smithi077 2024-05-04T03:09:02.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:01 smithi001 ceph-mon[29731]: Updating smithi072:/etc/ceph/ceph.client.admin.keyring 2024-05-04T03:09:03.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:02 smithi001 ceph-mon[29731]: Updating smithi072:/var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-04T03:09:04.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:03 smithi001 ceph-mon[29731]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:04.911 INFO:teuthology.orchestra.run.smithi001.stdout:Added host 'smithi077' with addr '172.21.15.77' 2024-05-04T03:09:04.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:04.910+0000 7f5c31ffb700 1 -- 172.21.15.1:0/3109146005 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f5c34100a50 con 0x7f5c2403e570 2024-05-04T03:09:04.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:04.913+0000 7f5c3ac89700 1 -- 172.21.15.1:0/3109146005 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5c2403e570 msgr2=0x7f5c24040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:04.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:04.913+0000 7f5c3ac89700 1 --2- 172.21.15.1:0/3109146005 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5c2403e570 0x7f5c24040a80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5c20006fd0 tx=0x7f5c20006e40 comp rx=0 tx=0).stop 2024-05-04T03:09:04.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:04.913+0000 7f5c3ac89700 1 -- 172.21.15.1:0/3109146005 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c34108a70 msgr2=0x7f5c340ff4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:04.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:04.913+0000 7f5c3ac89700 1 --2- 172.21.15.1:0/3109146005 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c34108a70 0x7f5c340ff4b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5c280056d0 tx=0x7f5c280057b0 comp rx=0 tx=0).stop 2024-05-04T03:09:04.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:04.913+0000 7f5c3ac89700 1 -- 172.21.15.1:0/3109146005 shutdown_connections 2024-05-04T03:09:04.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:04.913+0000 7f5c3ac89700 1 --2- 172.21.15.1:0/3109146005 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5c2403e570 0x7f5c24040a80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:04.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:04.913+0000 7f5c3ac89700 1 --2- 172.21.15.1:0/3109146005 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c34108a70 0x7f5c340ff4b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:04.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:04.913+0000 7f5c3ac89700 1 -- 172.21.15.1:0/3109146005 >> 172.21.15.1:0/3109146005 conn(0x7f5c340fe5c0 msgr2=0x7f5c34106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:04.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:04.913+0000 7f5c3ac89700 1 -- 172.21.15.1:0/3109146005 shutdown_connections 2024-05-04T03:09:04.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:04.913+0000 7f5c3ac89700 1 -- 172.21.15.1:0/3109146005 wait complete. 2024-05-04T03:09:05.553 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:05 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:05.553 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:05 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:05.554 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:05 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:05.554 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:05 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:05.554 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:05 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:09:05.585 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-04T03:09:05.830 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:09:06.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:06 smithi001 ceph-mon[29731]: Added host smithi077 2024-05-04T03:09:06.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:06 smithi001 ceph-mon[29731]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:06.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:06 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:06.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:06 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:07.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.530+0000 7f4be0007700 1 -- 172.21.15.1:0/2507588507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4bd8108a60 msgr2=0x7f4bd8108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:07.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.530+0000 7f4be0007700 1 --2- 172.21.15.1:0/2507588507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4bd8108a60 0x7f4bd8108e70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4bd4009910 tx=0x7f4bd4009c20 comp rx=0 tx=0).stop 2024-05-04T03:09:07.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.530+0000 7f4be0007700 1 -- 172.21.15.1:0/2507588507 shutdown_connections 2024-05-04T03:09:07.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.530+0000 7f4be0007700 1 --2- 172.21.15.1:0/2507588507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4bd8108a60 0x7f4bd8108e70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:07.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.530+0000 7f4be0007700 1 -- 172.21.15.1:0/2507588507 >> 172.21.15.1:0/2507588507 conn(0x7f4bd80fe570 msgr2=0x7f4bd81009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:07.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.530+0000 7f4be0007700 1 -- 172.21.15.1:0/2507588507 shutdown_connections 2024-05-04T03:09:07.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.530+0000 7f4be0007700 1 -- 172.21.15.1:0/2507588507 wait complete. 2024-05-04T03:09:07.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.531+0000 7f4be0007700 1 Processor -- start 2024-05-04T03:09:07.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.531+0000 7f4be0007700 1 -- start start 2024-05-04T03:09:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.531+0000 7f4be0007700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4bd8108a60 0x7f4bd819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.531+0000 7f4be0007700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd810cfb0 con 0x7f4bd8108a60 2024-05-04T03:09:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.532+0000 7f4bddda3700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4bd8108a60 0x7f4bd819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.532+0000 7f4bddda3700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4bd8108a60 0x7f4bd819a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48346/0 (socket says 172.21.15.1:48346) 2024-05-04T03:09:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.532+0000 7f4bddda3700 1 -- 172.21.15.1:0/923885049 learned_addr learned my addr 172.21.15.1:0/923885049 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:09:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.532+0000 7f4bddda3700 1 -- 172.21.15.1:0/923885049 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd806ca70 con 0x7f4bd8108a60 2024-05-04T03:09:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.532+0000 7f4bddda3700 1 --2- 172.21.15.1:0/923885049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4bd8108a60 0x7f4bd819a7e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40056d0 tx=0x7f4bd40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.532+0000 7f4bceffd700 1 -- 172.21.15.1:0/923885049 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bd4027070 con 0x7f4bd8108a60 2024-05-04T03:09:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.532+0000 7f4bceffd700 1 -- 172.21.15.1:0/923885049 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bd400be10 con 0x7f4bd8108a60 2024-05-04T03:09:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.532+0000 7f4bceffd700 1 -- 172.21.15.1:0/923885049 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bd4022d20 con 0x7f4bd8108a60 2024-05-04T03:09:07.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.532+0000 7f4be0007700 1 -- 172.21.15.1:0/923885049 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd819ade0 con 0x7f4bd8108a60 2024-05-04T03:09:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.532+0000 7f4be0007700 1 -- 172.21.15.1:0/923885049 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd819b2a0 con 0x7f4bd8108a60 2024-05-04T03:09:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.533+0000 7f4bceffd700 1 -- 172.21.15.1:0/923885049 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f4bd402a070 con 0x7f4bd8108a60 2024-05-04T03:09:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.533+0000 7f4be0007700 1 -- 172.21.15.1:0/923885049 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bd8104170 con 0x7f4bd8108a60 2024-05-04T03:09:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.534+0000 7f4bceffd700 1 --2- 172.21.15.1:0/923885049 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4bc403e570 0x7f4bc4040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:07.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.534+0000 7f4bceffd700 1 -- 172.21.15.1:0/923885049 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4bd401e030 con 0x7f4bd8108a60 2024-05-04T03:09:07.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.534+0000 7f4bdd5a2700 1 --2- 172.21.15.1:0/923885049 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4bc403e570 0x7f4bc4040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:07.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.534+0000 7f4bdd5a2700 1 --2- 172.21.15.1:0/923885049 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4bc403e570 0x7f4bc4040a80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8006fd0 tx=0x7f4bc8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:07.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.539+0000 7f4bceffd700 1 -- 172.21.15.1:0/923885049 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4bd806ca70 con 0x7f4bd8108a60 2024-05-04T03:09:07.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.757+0000 7f4be0007700 1 -- 172.21.15.1:0/923885049 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4bd806c1f0 con 0x7f4bc403e570 2024-05-04T03:09:07.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.758+0000 7f4bceffd700 1 -- 172.21.15.1:0/923885049 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+237 (secure 0 0 0) 0x7f4bd806c1f0 con 0x7f4bc403e570 2024-05-04T03:09:07.760 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:09:07.760 INFO:teuthology.orchestra.run.smithi001.stdout:[{"addr": "172.21.15.1", "hostname": "smithi001", "labels": [], "status": ""}, {"addr": "172.21.15.72", "hostname": "smithi072", "labels": [], "status": ""}, {"addr": "172.21.15.77", "hostname": "smithi077", "labels": [], "status": ""}] 2024-05-04T03:09:07.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.760+0000 7f4be0007700 1 -- 172.21.15.1:0/923885049 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4bc403e570 msgr2=0x7f4bc4040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:07.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.760+0000 7f4be0007700 1 --2- 172.21.15.1:0/923885049 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4bc403e570 0x7f4bc4040a80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8006fd0 tx=0x7f4bc8006e40 comp rx=0 tx=0).stop 2024-05-04T03:09:07.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.760+0000 7f4be0007700 1 -- 172.21.15.1:0/923885049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4bd8108a60 msgr2=0x7f4bd819a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:07.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.760+0000 7f4be0007700 1 --2- 172.21.15.1:0/923885049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4bd8108a60 0x7f4bd819a7e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40056d0 tx=0x7f4bd40057b0 comp rx=0 tx=0).stop 2024-05-04T03:09:07.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.760+0000 7f4be0007700 1 -- 172.21.15.1:0/923885049 shutdown_connections 2024-05-04T03:09:07.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.760+0000 7f4be0007700 1 --2- 172.21.15.1:0/923885049 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4bc403e570 0x7f4bc4040a80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:07.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.760+0000 7f4be0007700 1 --2- 172.21.15.1:0/923885049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4bd8108a60 0x7f4bd819a7e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:07.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.760+0000 7f4be0007700 1 -- 172.21.15.1:0/923885049 >> 172.21.15.1:0/923885049 conn(0x7f4bd80fe570 msgr2=0x7f4bd80ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:07.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.760+0000 7f4be0007700 1 -- 172.21.15.1:0/923885049 shutdown_connections 2024-05-04T03:09:07.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:07.760+0000 7f4be0007700 1 -- 172.21.15.1:0/923885049 wait complete. 2024-05-04T03:09:08.439 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-04T03:09:08.439 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd crush tunables default 2024-05-04T03:09:08.684 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:09:08.806 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:08 smithi001 ceph-mon[29731]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:08.806 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:08 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:09.705 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:09 smithi001 ceph-mon[29731]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:09:10.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.349+0000 7f44be245700 1 -- 172.21.15.1:0/2473536377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44b8107a60 msgr2=0x7f44b8107e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:10.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.349+0000 7f44be245700 1 --2- 172.21.15.1:0/2473536377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44b8107a60 0x7f44b8107e70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009910 tx=0x7f44a8009c20 comp rx=0 tx=0).stop 2024-05-04T03:09:10.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.349+0000 7f44be245700 1 -- 172.21.15.1:0/2473536377 shutdown_connections 2024-05-04T03:09:10.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.349+0000 7f44be245700 1 --2- 172.21.15.1:0/2473536377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44b8107a60 0x7f44b8107e70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:10.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.349+0000 7f44be245700 1 -- 172.21.15.1:0/2473536377 >> 172.21.15.1:0/2473536377 conn(0x7f44b80fde30 msgr2=0x7f44b8100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:10.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.349+0000 7f44be245700 1 -- 172.21.15.1:0/2473536377 shutdown_connections 2024-05-04T03:09:10.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.349+0000 7f44be245700 1 -- 172.21.15.1:0/2473536377 wait complete. 2024-05-04T03:09:10.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.350+0000 7f44be245700 1 Processor -- start 2024-05-04T03:09:10.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.350+0000 7f44be245700 1 -- start start 2024-05-04T03:09:10.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.350+0000 7f44be245700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44b8107a60 0x7f44b8104750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:10.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.350+0000 7f44be245700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f44b810d100 con 0x7f44b8107a60 2024-05-04T03:09:10.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.350+0000 7f44b77fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44b8107a60 0x7f44b8104750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:10.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.350+0000 7f44b77fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44b8107a60 0x7f44b8104750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48354/0 (socket says 172.21.15.1:48354) 2024-05-04T03:09:10.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.350+0000 7f44b77fe700 1 -- 172.21.15.1:0/175661775 learned_addr learned my addr 172.21.15.1:0/175661775 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:09:10.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.351+0000 7f44b77fe700 1 -- 172.21.15.1:0/175661775 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b81065a0 con 0x7f44b8107a60 2024-05-04T03:09:10.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.351+0000 7f44b77fe700 1 --2- 172.21.15.1:0/175661775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44b8107a60 0x7f44b8104750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f44a80056d0 tx=0x7f44a80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:10.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.351+0000 7f44b4ff9700 1 -- 172.21.15.1:0/175661775 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44a8026070 con 0x7f44b8107a60 2024-05-04T03:09:10.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.351+0000 7f44b4ff9700 1 -- 172.21.15.1:0/175661775 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44a800bc00 con 0x7f44b8107a60 2024-05-04T03:09:10.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.351+0000 7f44be245700 1 -- 172.21.15.1:0/175661775 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b8104c90 con 0x7f44b8107a60 2024-05-04T03:09:10.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.351+0000 7f44b4ff9700 1 -- 172.21.15.1:0/175661775 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44a8003920 con 0x7f44b8107a60 2024-05-04T03:09:10.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.351+0000 7f44be245700 1 -- 172.21.15.1:0/175661775 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b8105090 con 0x7f44b8107a60 2024-05-04T03:09:10.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.352+0000 7f44b4ff9700 1 -- 172.21.15.1:0/175661775 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f44a8029070 con 0x7f44b8107a60 2024-05-04T03:09:10.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.352+0000 7f44be245700 1 -- 172.21.15.1:0/175661775 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44b81041f0 con 0x7f44b8107a60 2024-05-04T03:09:10.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.352+0000 7f44b4ff9700 1 --2- 172.21.15.1:0/175661775 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f44a003e570 0x7f44a0040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:10.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.352+0000 7f44b4ff9700 1 -- 172.21.15.1:0/175661775 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f44a801e070 con 0x7f44b8107a60 2024-05-04T03:09:10.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.353+0000 7f44b6ffd700 1 --2- 172.21.15.1:0/175661775 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f44a003e570 0x7f44a0040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:10.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.353+0000 7f44b6ffd700 1 --2- 172.21.15.1:0/175661775 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f44a003e570 0x7f44a0040a80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f44ac006fd0 tx=0x7f44ac006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:10.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.358+0000 7f44b4ff9700 1 -- 172.21.15.1:0/175661775 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f44a8013070 con 0x7f44b8107a60 2024-05-04T03:09:10.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:10.578+0000 7f44be245700 1 -- 172.21.15.1:0/175661775 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f44b8105d20 con 0x7f44b8107a60 2024-05-04T03:09:10.581 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:10 smithi001 ceph-mon[29731]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:11.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:11.515+0000 7f44b4ff9700 1 -- 172.21.15.1:0/175661775 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f44a8015350 con 0x7f44b8107a60 2024-05-04T03:09:11.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:11.518+0000 7f44be245700 1 -- 172.21.15.1:0/175661775 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f44a003e570 msgr2=0x7f44a0040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:11.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:11.518+0000 7f44be245700 1 --2- 172.21.15.1:0/175661775 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f44a003e570 0x7f44a0040a80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f44ac006fd0 tx=0x7f44ac006e40 comp rx=0 tx=0).stop 2024-05-04T03:09:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:11.518+0000 7f44be245700 1 -- 172.21.15.1:0/175661775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44b8107a60 msgr2=0x7f44b8104750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:11.518+0000 7f44be245700 1 --2- 172.21.15.1:0/175661775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44b8107a60 0x7f44b8104750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f44a80056d0 tx=0x7f44a80057b0 comp rx=0 tx=0).stop 2024-05-04T03:09:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:11.519+0000 7f44be245700 1 -- 172.21.15.1:0/175661775 shutdown_connections 2024-05-04T03:09:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:11.519+0000 7f44be245700 1 --2- 172.21.15.1:0/175661775 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f44a003e570 0x7f44a0040a80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:11.519+0000 7f44be245700 1 --2- 172.21.15.1:0/175661775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44b8107a60 0x7f44b8104750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:11.519+0000 7f44be245700 1 -- 172.21.15.1:0/175661775 >> 172.21.15.1:0/175661775 conn(0x7f44b80fde30 msgr2=0x7f44b81020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:11.519+0000 7f44be245700 1 -- 172.21.15.1:0/175661775 shutdown_connections 2024-05-04T03:09:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:11.519+0000 7f44be245700 1 -- 172.21.15.1:0/175661775 wait complete. 2024-05-04T03:09:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:adjusted tunables profile to default 2024-05-04T03:09:11.802 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:11 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/175661775' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-04T03:09:11.803 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:11 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:11.803 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:11 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:11.803 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:11 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:11.803 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:11 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:11.803 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:11 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi077", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:09:11.803 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:11 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:11.803 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:11 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:09:11.803 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:11 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:11.803 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:11 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:11.803 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:11 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:12.037 INFO:tasks.cephadm:Adding mon.a on smithi001 2024-05-04T03:09:12.038 INFO:tasks.cephadm:Adding mon.b on smithi072 2024-05-04T03:09:12.038 INFO:tasks.cephadm:Adding mon.c on smithi077 2024-05-04T03:09:12.038 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph orch apply mon '3;smithi001:172.21.15.1=a;smithi072:172.21.15.72=b;smithi077:172.21.15.77=c' 2024-05-04T03:09:12.264 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:09:12.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:12 smithi001 ceph-mon[29731]: Updating smithi077:/etc/ceph/ceph.conf 2024-05-04T03:09:12.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:12 smithi001 ceph-mon[29731]: Updating smithi077:/var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:09:12.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:12 smithi001 ceph-mon[29731]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:12.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:12 smithi001 ceph-mon[29731]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-05-04T03:09:12.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:12 smithi001 ceph-mon[29731]: Updating smithi077:/var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-04T03:09:12.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:12 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/175661775' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-04T03:09:12.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:12 smithi001 ceph-mon[29731]: osdmap e4: 0 total, 0 up, 0 in 2024-05-04T03:09:13.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.636+0000 7fa73cfcb700 1 -- 172.21.15.77:0/2271478007 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa738107aa0 msgr2=0x7fa738107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:13.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.636+0000 7fa73cfcb700 1 --2- 172.21.15.77:0/2271478007 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa738107aa0 0x7fa738107eb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa728009910 tx=0x7fa728009c20 comp rx=0 tx=0).stop 2024-05-04T03:09:13.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.637+0000 7fa73cfcb700 1 -- 172.21.15.77:0/2271478007 shutdown_connections 2024-05-04T03:09:13.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.637+0000 7fa73cfcb700 1 --2- 172.21.15.77:0/2271478007 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa738107aa0 0x7fa738107eb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:13.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.637+0000 7fa73cfcb700 1 -- 172.21.15.77:0/2271478007 >> 172.21.15.77:0/2271478007 conn(0x7fa7380fde30 msgr2=0x7fa738100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:13.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.637+0000 7fa73cfcb700 1 -- 172.21.15.77:0/2271478007 shutdown_connections 2024-05-04T03:09:13.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.637+0000 7fa73cfcb700 1 -- 172.21.15.77:0/2271478007 wait complete. 2024-05-04T03:09:13.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.637+0000 7fa73cfcb700 1 Processor -- start 2024-05-04T03:09:13.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.638+0000 7fa73cfcb700 1 -- start start 2024-05-04T03:09:13.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.638+0000 7fa73cfcb700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa738107aa0 0x7fa73819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:13.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.638+0000 7fa73cfcb700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa73810d030 con 0x7fa738107aa0 2024-05-04T03:09:13.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.638+0000 7fa73659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa738107aa0 0x7fa73819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:13.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.638+0000 7fa73659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa738107aa0 0x7fa73819a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.77:53702/0 (socket says 172.21.15.77:53702) 2024-05-04T03:09:13.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.638+0000 7fa73659c700 1 -- 172.21.15.77:0/1631427071 learned_addr learned my addr 172.21.15.77:0/1631427071 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-04T03:09:13.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.639+0000 7fa73659c700 1 -- 172.21.15.77:0/1631427071 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa738102fc0 con 0x7fa738107aa0 2024-05-04T03:09:13.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.639+0000 7fa73659c700 1 --2- 172.21.15.77:0/1631427071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa738107aa0 0x7fa73819a770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa72800b5c0 tx=0x7fa728005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:13.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.639+0000 7fa72ffff700 1 -- 172.21.15.77:0/1631427071 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa728026070 con 0x7fa738107aa0 2024-05-04T03:09:13.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.639+0000 7fa72ffff700 1 -- 172.21.15.77:0/1631427071 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa72800b750 con 0x7fa738107aa0 2024-05-04T03:09:13.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.639+0000 7fa73cfcb700 1 -- 172.21.15.77:0/1631427071 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa73819ad10 con 0x7fa738107aa0 2024-05-04T03:09:13.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.639+0000 7fa72ffff700 1 -- 172.21.15.77:0/1631427071 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7280037d0 con 0x7fa738107aa0 2024-05-04T03:09:13.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.639+0000 7fa73cfcb700 1 -- 172.21.15.77:0/1631427071 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa73819b1d0 con 0x7fa738107aa0 2024-05-04T03:09:13.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.640+0000 7fa72ffff700 1 -- 172.21.15.77:0/1631427071 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fa728029070 con 0x7fa738107aa0 2024-05-04T03:09:13.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.640+0000 7fa73cfcb700 1 -- 172.21.15.77:0/1631427071 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7381041b0 con 0x7fa738107aa0 2024-05-04T03:09:13.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.640+0000 7fa72ffff700 1 --2- 172.21.15.77:0/1631427071 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa72403e5c0 0x7fa724040ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:13.656 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.640+0000 7fa72ffff700 1 -- 172.21.15.77:0/1631427071 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa72801e070 con 0x7fa738107aa0 2024-05-04T03:09:13.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.641+0000 7fa72dd9b700 1 --2- 172.21.15.77:0/1631427071 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa72403e5c0 0x7fa724040ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:13.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.641+0000 7fa72dd9b700 1 --2- 172.21.15.77:0/1631427071 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa72403e5c0 0x7fa724040ad0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa720006fd0 tx=0x7fa720006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:13.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.646+0000 7fa72ffff700 1 -- 172.21.15.77:0/1631427071 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa738102fc0 con 0x7fa738107aa0 2024-05-04T03:09:13.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.859+0000 7fa73cfcb700 1 -- 172.21.15.77:0/1631427071 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi001:172.21.15.1=a;smithi072:172.21.15.72=b;smithi077:172.21.15.77=c", "target": ["mon-mgr", ""]}) -- 0x7fa73819b450 con 0x7fa72403e5c0 2024-05-04T03:09:13.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.866+0000 7fa72ffff700 1 -- 172.21.15.77:0/1631427071 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa73819b450 con 0x7fa72403e5c0 2024-05-04T03:09:13.867 INFO:teuthology.orchestra.run.smithi077.stdout:Scheduled mon update... 2024-05-04T03:09:13.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.868+0000 7fa73cfcb700 1 -- 172.21.15.77:0/1631427071 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa72403e5c0 msgr2=0x7fa724040ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:13.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.868+0000 7fa73cfcb700 1 --2- 172.21.15.77:0/1631427071 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa72403e5c0 0x7fa724040ad0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa720006fd0 tx=0x7fa720006e40 comp rx=0 tx=0).stop 2024-05-04T03:09:13.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.868+0000 7fa73cfcb700 1 -- 172.21.15.77:0/1631427071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa738107aa0 msgr2=0x7fa73819a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:13.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.868+0000 7fa73cfcb700 1 --2- 172.21.15.77:0/1631427071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa738107aa0 0x7fa73819a770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa72800b5c0 tx=0x7fa728005780 comp rx=0 tx=0).stop 2024-05-04T03:09:13.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.868+0000 7fa73cfcb700 1 -- 172.21.15.77:0/1631427071 shutdown_connections 2024-05-04T03:09:13.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.868+0000 7fa73cfcb700 1 --2- 172.21.15.77:0/1631427071 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa72403e5c0 0x7fa724040ad0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:13.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.868+0000 7fa73cfcb700 1 --2- 172.21.15.77:0/1631427071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa738107aa0 0x7fa73819a770 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:13.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.868+0000 7fa73cfcb700 1 -- 172.21.15.77:0/1631427071 >> 172.21.15.77:0/1631427071 conn(0x7fa7380fde30 msgr2=0x7fa7381046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:13.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.868+0000 7fa73cfcb700 1 -- 172.21.15.77:0/1631427071 shutdown_connections 2024-05-04T03:09:13.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:13.868+0000 7fa73cfcb700 1 -- 172.21.15.77:0/1631427071 wait complete. 2024-05-04T03:09:14.457 DEBUG:teuthology.orchestra.run.smithi072:mon.b> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mon.b.service 2024-05-04T03:09:14.460 DEBUG:teuthology.orchestra.run.smithi077:mon.c> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mon.c.service 2024-05-04T03:09:14.462 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-04T03:09:14.462 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-04T03:09:14.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:14 smithi001 ceph-mon[29731]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:14.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:14.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:09:14.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:14.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:09:14.881 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:14.881 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-04T03:09:14.881 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:15.097 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:09:15.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:15 smithi001 ceph-mon[29731]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi001:172.21.15.1=a;smithi072:172.21.15.72=b;smithi077:172.21.15.77=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:09:15.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:15 smithi001 ceph-mon[29731]: Saving service mon spec with placement smithi001:172.21.15.1=a;smithi072:172.21.15.72=b;smithi077:172.21.15.77=c;count:3 2024-05-04T03:09:15.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:15 smithi001 ceph-mon[29731]: Deploying daemon mon.c on smithi077 2024-05-04T03:09:16.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:16 smithi001 ceph-mon[29731]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:17.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.040+0000 7f995b24f700 1 -- 172.21.15.77:0/3502983150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9954103dd0 msgr2=0x7f99541041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:17.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.040+0000 7f995b24f700 1 --2- 172.21.15.77:0/3502983150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9954103dd0 0x7f99541041e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9948009910 tx=0x7f9948009c20 comp rx=0 tx=0).stop 2024-05-04T03:09:17.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.040+0000 7f995b24f700 1 -- 172.21.15.77:0/3502983150 shutdown_connections 2024-05-04T03:09:17.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.040+0000 7f995b24f700 1 --2- 172.21.15.77:0/3502983150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9954103dd0 0x7f99541041e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:17.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.040+0000 7f995b24f700 1 -- 172.21.15.77:0/3502983150 >> 172.21.15.77:0/3502983150 conn(0x7f99540ff660 msgr2=0x7f9954101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:17.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.040+0000 7f995b24f700 1 -- 172.21.15.77:0/3502983150 shutdown_connections 2024-05-04T03:09:17.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.040+0000 7f995b24f700 1 -- 172.21.15.77:0/3502983150 wait complete. 2024-05-04T03:09:17.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.040+0000 7f995b24f700 1 Processor -- start 2024-05-04T03:09:17.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.041+0000 7f995b24f700 1 -- start start 2024-05-04T03:09:17.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.041+0000 7f995b24f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9954103dd0 0x7f995419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:17.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.041+0000 7f995b24f700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f995410e0c0 con 0x7f9954103dd0 2024-05-04T03:09:17.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.041+0000 7f9958feb700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9954103dd0 0x7f995419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:17.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.041+0000 7f9958feb700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9954103dd0 0x7f995419f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.77:53718/0 (socket says 172.21.15.77:53718) 2024-05-04T03:09:17.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.041+0000 7f9958feb700 1 -- 172.21.15.77:0/2980414963 learned_addr learned my addr 172.21.15.77:0/2980414963 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-04T03:09:17.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.041+0000 7f9958feb700 1 -- 172.21.15.77:0/2980414963 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f995419fb40 con 0x7f9954103dd0 2024-05-04T03:09:17.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.042+0000 7f9958feb700 1 --2- 172.21.15.77:0/2980414963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9954103dd0 0x7f995419f600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f99480056d0 tx=0x7f99480057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:17.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.042+0000 7f9951ffb700 1 -- 172.21.15.77:0/2980414963 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9948026070 con 0x7f9954103dd0 2024-05-04T03:09:17.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.042+0000 7f995b24f700 1 -- 172.21.15.77:0/2980414963 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f995419fe30 con 0x7f9954103dd0 2024-05-04T03:09:17.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.042+0000 7f9951ffb700 1 -- 172.21.15.77:0/2980414963 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f994800bba0 con 0x7f9954103dd0 2024-05-04T03:09:17.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.042+0000 7f9951ffb700 1 -- 172.21.15.77:0/2980414963 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9948003920 con 0x7f9954103dd0 2024-05-04T03:09:17.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.042+0000 7f995b24f700 1 -- 172.21.15.77:0/2980414963 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9954199960 con 0x7f9954103dd0 2024-05-04T03:09:17.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.043+0000 7f9951ffb700 1 -- 172.21.15.77:0/2980414963 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f9948029070 con 0x7f9954103dd0 2024-05-04T03:09:17.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.043+0000 7f995b24f700 1 -- 172.21.15.77:0/2980414963 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9954105280 con 0x7f9954103dd0 2024-05-04T03:09:17.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.043+0000 7f9951ffb700 1 --2- 172.21.15.77:0/2980414963 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f994403e510 0x7f9944040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:17.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.043+0000 7f9951ffb700 1 -- 172.21.15.77:0/2980414963 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f994801e070 con 0x7f9954103dd0 2024-05-04T03:09:17.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.043+0000 7f9953fff700 1 --2- 172.21.15.77:0/2980414963 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f994403e510 0x7f9944040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:17.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.044+0000 7f9953fff700 1 --2- 172.21.15.77:0/2980414963 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f994403e510 0x7f9944040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9940006fd0 tx=0x7f9940006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:17.050 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.049+0000 7f9951ffb700 1 -- 172.21.15.77:0/2980414963 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9948013070 con 0x7f9954103dd0 2024-05-04T03:09:17.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.344+0000 7f995b24f700 1 -- 172.21.15.77:0/2980414963 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f99541a0060 con 0x7f9954103dd0 2024-05-04T03:09:17.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.344+0000 7f9951ffb700 1 -- 172.21.15.77:0/2980414963 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7f9948030300 con 0x7f9954103dd0 2024-05-04T03:09:17.346 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-04T03:09:17.346 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":1,"fsid":"4a572aa8-09c3-11ef-bc97-c7b262605968","modified":"2024-05-04T03:07:29.984656Z","created":"2024-05-04T03:07:29.984656Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-04T03:09:17.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.347+0000 7f995b24f700 1 -- 172.21.15.77:0/2980414963 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f994403e510 msgr2=0x7f9944040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:17.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.347+0000 7f995b24f700 1 --2- 172.21.15.77:0/2980414963 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f994403e510 0x7f9944040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9940006fd0 tx=0x7f9940006e40 comp rx=0 tx=0).stop 2024-05-04T03:09:17.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.347+0000 7f995b24f700 1 -- 172.21.15.77:0/2980414963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9954103dd0 msgr2=0x7f995419f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:17.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.347+0000 7f995b24f700 1 --2- 172.21.15.77:0/2980414963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9954103dd0 0x7f995419f600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f99480056d0 tx=0x7f99480057b0 comp rx=0 tx=0).stop 2024-05-04T03:09:17.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.347+0000 7f995b24f700 1 -- 172.21.15.77:0/2980414963 shutdown_connections 2024-05-04T03:09:17.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.347+0000 7f995b24f700 1 --2- 172.21.15.77:0/2980414963 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f994403e510 0x7f9944040a20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:17.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.347+0000 7f995b24f700 1 --2- 172.21.15.77:0/2980414963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9954103dd0 0x7f995419f600 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:17.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.347+0000 7f995b24f700 1 -- 172.21.15.77:0/2980414963 >> 172.21.15.77:0/2980414963 conn(0x7f99540ff660 msgr2=0x7f9954109900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:17.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.347+0000 7f995b24f700 1 -- 172.21.15.77:0/2980414963 shutdown_connections 2024-05-04T03:09:17.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:17.347+0000 7f995b24f700 1 -- 172.21.15.77:0/2980414963 wait complete. 2024-05-04T03:09:17.349 INFO:teuthology.orchestra.run.smithi077.stderr:dumped monmap epoch 1 2024-05-04T03:09:17.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:17 smithi001 ceph-mon[29731]: from='client.? 172.21.15.77:0/2980414963' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-04T03:09:18.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:18 smithi001 ceph-mon[29731]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:18.947 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-04T03:09:18.947 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-04T03:09:19.254 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:09:20.774 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:20 smithi077 systemd[1]: Started Ceph mon.c for 4a572aa8-09c3-11ef-bc97-c7b262605968. 2024-05-04T03:09:20.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:20 smithi001 ceph-mon[29731]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:25.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:09:25.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:09:25.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: mon.a calling monitor election 2024-05-04T03:09:25.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:25.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:09:25.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:09:25.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: mon.c calling monitor election 2024-05-04T03:09:25.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:25.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:09:25.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:09:25.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:25.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:09:25.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-04T03:09:25.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: monmap epoch 2 2024-05-04T03:09:25.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:09:25.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: last_changed 2024-05-04T03:09:20.594989+0000 2024-05-04T03:09:25.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: created 2024-05-04T03:07:29.984656+0000 2024-05-04T03:09:25.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: min_mon_release 19 (squid) 2024-05-04T03:09:25.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: election_strategy: 1 2024-05-04T03:09:25.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-05-04T03:09:25.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: 1: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.c 2024-05-04T03:09:25.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: fsmap 2024-05-04T03:09:25.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: osdmap e4: 0 total, 0 up, 0 in 2024-05-04T03:09:25.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: mgrmap e13: x(active, since 52s) 2024-05-04T03:09:25.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: overall HEALTH_OK 2024-05-04T03:09:25.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:25.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:25.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:25.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-04T03:09:25.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:25 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:26.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:09:26.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:09:26.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: mon.a calling monitor election 2024-05-04T03:09:26.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:26.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:09:26.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:09:26.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: mon.c calling monitor election 2024-05-04T03:09:26.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:26.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:09:26.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:09:26.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:26.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:09:26.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-04T03:09:26.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: monmap epoch 2 2024-05-04T03:09:26.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:09:26.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: last_changed 2024-05-04T03:09:20.594989+0000 2024-05-04T03:09:26.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: created 2024-05-04T03:07:29.984656+0000 2024-05-04T03:09:26.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: min_mon_release 19 (squid) 2024-05-04T03:09:26.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: election_strategy: 1 2024-05-04T03:09:26.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-05-04T03:09:26.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: 1: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.c 2024-05-04T03:09:26.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: fsmap 2024-05-04T03:09:26.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: osdmap e4: 0 total, 0 up, 0 in 2024-05-04T03:09:26.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: mgrmap e13: x(active, since 52s) 2024-05-04T03:09:26.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: overall HEALTH_OK 2024-05-04T03:09:26.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:26.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:26.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:26.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-04T03:09:26.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:25 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:26.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:26 smithi077 ceph-mon[33531]: Deploying daemon mon.b on smithi072 2024-05-04T03:09:26.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:26 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:09:27.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:26 smithi001 ceph-mon[29731]: Deploying daemon mon.b on smithi072 2024-05-04T03:09:27.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:26 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:09:27.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.177+0000 7f32394c1700 1 -- 172.21.15.77:0/2524778786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32200062f0 msgr2=0x7f32200087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:27.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.177+0000 7f32394c1700 1 --2- 172.21.15.77:0/2524778786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32200062f0 0x7f32200087b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3234104d10 tx=0x7f322400fe90 comp rx=0 tx=0).stop 2024-05-04T03:09:27.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.178+0000 7f32394c1700 1 -- 172.21.15.77:0/2524778786 shutdown_connections 2024-05-04T03:09:27.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.178+0000 7f32394c1700 1 --2- 172.21.15.77:0/2524778786 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32200062f0 0x7f32200087b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:27.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.178+0000 7f32394c1700 1 -- 172.21.15.77:0/2524778786 >> 172.21.15.77:0/2524778786 conn(0x7f32340ff750 msgr2=0x7f3234101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:27.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.178+0000 7f32394c1700 1 -- 172.21.15.77:0/2524778786 shutdown_connections 2024-05-04T03:09:27.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.178+0000 7f32394c1700 1 -- 172.21.15.77:0/2524778786 wait complete. 2024-05-04T03:09:27.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.179+0000 7f32394c1700 1 Processor -- start 2024-05-04T03:09:27.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.179+0000 7f32394c1700 1 -- start start 2024-05-04T03:09:27.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.179+0000 7f32394c1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32200062f0 0x7f323419f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:27.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.179+0000 7f32394c1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f323419fd00 0x7f32341999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:27.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.179+0000 7f32394c1700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f323410bc00 con 0x7f323419fd00 2024-05-04T03:09:27.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.179+0000 7f32394c1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f323410ba80 con 0x7f32200062f0 2024-05-04T03:09:27.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.180+0000 7f3232ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32200062f0 0x7f323419f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:27.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.180+0000 7f3232ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32200062f0 0x7f323419f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44660/0 (socket says 172.21.15.77:44660) 2024-05-04T03:09:27.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.180+0000 7f3232ffd700 1 -- 172.21.15.77:0/1573321713 learned_addr learned my addr 172.21.15.77:0/1573321713 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-04T03:09:27.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.180+0000 7f3232ffd700 1 -- 172.21.15.77:0/1573321713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32200062f0 msgr2=0x7f323419f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-04T03:09:27.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.180+0000 7f3232ffd700 1 -- 172.21.15.77:0/1573321713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32200062f0 msgr2=0x7f323419f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-04T03:09:27.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.180+0000 7f3232ffd700 1 --2- 172.21.15.77:0/1573321713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32200062f0 0x7f323419f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-04T03:09:27.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.180+0000 7f322a7fc700 1 --2- 172.21.15.77:0/1573321713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f323419fd00 0x7f32341999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:27.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.180+0000 7f3232ffd700 1 --2- 172.21.15.77:0/1573321713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32200062f0 0x7f323419f7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-04T03:09:27.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.180+0000 7f322a7fc700 1 -- 172.21.15.77:0/1573321713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32200062f0 msgr2=0x7f323419f7c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-04T03:09:27.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.180+0000 7f322a7fc700 1 --2- 172.21.15.77:0/1573321713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32200062f0 0x7f323419f7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:27.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.180+0000 7f322a7fc700 1 -- 172.21.15.77:0/1573321713 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32341a02d0 con 0x7f323419fd00 2024-05-04T03:09:27.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.180+0000 7f322a7fc700 1 --2- 172.21.15.77:0/1573321713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f323419fd00 0x7f32341999e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f321c00b4d0 tx=0x7f321c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:27.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.181+0000 7f3230ff9700 1 -- 172.21.15.77:0/1573321713 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f321c011630 con 0x7f323419fd00 2024-05-04T03:09:27.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.181+0000 7f3230ff9700 1 -- 172.21.15.77:0/1573321713 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f321c011c80 con 0x7f323419fd00 2024-05-04T03:09:27.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.181+0000 7f32394c1700 1 -- 172.21.15.77:0/1573321713 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3234199f80 con 0x7f323419fd00 2024-05-04T03:09:27.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.181+0000 7f3230ff9700 1 -- 172.21.15.77:0/1573321713 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f321c010da0 con 0x7f323419fd00 2024-05-04T03:09:27.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.181+0000 7f32394c1700 1 -- 172.21.15.77:0/1573321713 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f323419a510 con 0x7f323419fd00 2024-05-04T03:09:27.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.182+0000 7f3230ff9700 1 -- 172.21.15.77:0/1573321713 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f321c018020 con 0x7f323419fd00 2024-05-04T03:09:27.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.182+0000 7f32394c1700 1 -- 172.21.15.77:0/1573321713 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f323419a820 con 0x7f323419fd00 2024-05-04T03:09:27.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.182+0000 7f3230ff9700 1 --2- 172.21.15.77:0/1573321713 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f321403e2b0 0x7f32140407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:27.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.182+0000 7f3230ff9700 1 -- 172.21.15.77:0/1573321713 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f321c052480 con 0x7f323419fd00 2024-05-04T03:09:27.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.182+0000 7f3232ffd700 1 --2- 172.21.15.77:0/1573321713 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f321403e2b0 0x7f32140407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:27.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.183+0000 7f3232ffd700 1 --2- 172.21.15.77:0/1573321713 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f321403e2b0 0x7f32140407c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3224005b40 tx=0x7f3224005ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:27.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.188+0000 7f3230ff9700 1 -- 172.21.15.77:0/1573321713 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f321c01f020 con 0x7f323419fd00 2024-05-04T03:09:27.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.474+0000 7f32394c1700 1 -- 172.21.15.77:0/1573321713 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f323402e5f0 con 0x7f323419fd00 2024-05-04T03:09:27.476 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.475+0000 7f3230ff9700 1 -- 172.21.15.77:0/1573321713 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1007 (secure 0 0 0) 0x7f321c0187b0 con 0x7f323419fd00 2024-05-04T03:09:27.476 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-04T03:09:27.476 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":2,"fsid":"4a572aa8-09c3-11ef-bc97-c7b262605968","modified":"2024-05-04T03:09:20.594989Z","created":"2024-05-04T03:07:29.984656Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-04T03:09:27.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.478+0000 7f32394c1700 1 -- 172.21.15.77:0/1573321713 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f321403e2b0 msgr2=0x7f32140407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:27.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.478+0000 7f32394c1700 1 --2- 172.21.15.77:0/1573321713 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f321403e2b0 0x7f32140407c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3224005b40 tx=0x7f3224005ab0 comp rx=0 tx=0).stop 2024-05-04T03:09:27.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.478+0000 7f32394c1700 1 -- 172.21.15.77:0/1573321713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f323419fd00 msgr2=0x7f32341999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:27.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.478+0000 7f32394c1700 1 --2- 172.21.15.77:0/1573321713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f323419fd00 0x7f32341999e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f321c00b4d0 tx=0x7f321c00b890 comp rx=0 tx=0).stop 2024-05-04T03:09:27.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.478+0000 7f32394c1700 1 -- 172.21.15.77:0/1573321713 shutdown_connections 2024-05-04T03:09:27.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.478+0000 7f32394c1700 1 --2- 172.21.15.77:0/1573321713 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f321403e2b0 0x7f32140407c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:27.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.478+0000 7f32394c1700 1 --2- 172.21.15.77:0/1573321713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32200062f0 0x7f323419f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:27.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.478+0000 7f32394c1700 1 --2- 172.21.15.77:0/1573321713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f323419fd00 0x7f32341999e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:27.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.478+0000 7f32394c1700 1 -- 172.21.15.77:0/1573321713 >> 172.21.15.77:0/1573321713 conn(0x7f32340ff750 msgr2=0x7f323410a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:27.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.478+0000 7f32394c1700 1 -- 172.21.15.77:0/1573321713 shutdown_connections 2024-05-04T03:09:27.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:27.478+0000 7f32394c1700 1 -- 172.21.15.77:0/1573321713 wait complete. 2024-05-04T03:09:27.480 INFO:teuthology.orchestra.run.smithi077.stderr:dumped monmap epoch 2 2024-05-04T03:09:27.836 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:27 smithi077 ceph-mon[33531]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:27.836 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:27 smithi077 ceph-mon[33531]: from='client.? 172.21.15.77:0/1573321713' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-04T03:09:28.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:27 smithi001 ceph-mon[29731]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:28.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:27 smithi001 ceph-mon[29731]: from='client.? 172.21.15.77:0/1573321713' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-04T03:09:29.097 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-04T03:09:29.098 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-04T03:09:29.340 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:29 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-04T03:09:29.340 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:29 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:29.340 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:29 smithi072 ceph-mon[33332]: Deploying daemon mon.b on smithi072 2024-05-04T03:09:29.340 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:29 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:09:29.340 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:29 smithi072 ceph-mon[33332]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:29.340 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:29 smithi072 ceph-mon[33332]: from='client.? 172.21.15.77:0/1573321713' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-04T03:09:29.340 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:29 smithi072 ceph-mon[33332]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-04T03:09:29.345 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:09:29.745 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:29 smithi072 bash[33313]: 6bd75c9906c11351c7dc5f277a39c295870634082a7bc8dd70ca98d5b86e571b 2024-05-04T03:09:29.745 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:29 smithi072 systemd[1]: Started Ceph mon.b for 4a572aa8-09c3-11ef-bc97-c7b262605968. 2024-05-04T03:09:34.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.375+0000 7f559cef4700 1 -- 172.21.15.77:0/3216525805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55800057d0 msgr2=0x7f5580007cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:34.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.375+0000 7f559cef4700 1 --2- 172.21.15.77:0/3216525805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55800057d0 0x7f5580007cb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f559806b460 tx=0x7f558c014ec0 comp rx=0 tx=0).stop 2024-05-04T03:09:34.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.375+0000 7f559cef4700 1 -- 172.21.15.77:0/3216525805 shutdown_connections 2024-05-04T03:09:34.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.375+0000 7f559cef4700 1 --2- 172.21.15.77:0/3216525805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55800057d0 0x7f5580007cb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:34.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.375+0000 7f559cef4700 1 -- 172.21.15.77:0/3216525805 >> 172.21.15.77:0/3216525805 conn(0x7f5598077e80 msgr2=0x7f55980782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:34.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.375+0000 7f559cef4700 1 -- 172.21.15.77:0/3216525805 shutdown_connections 2024-05-04T03:09:34.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.375+0000 7f559cef4700 1 -- 172.21.15.77:0/3216525805 wait complete. 2024-05-04T03:09:34.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.376+0000 7f559cef4700 1 Processor -- start 2024-05-04T03:09:34.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.376+0000 7f559cef4700 1 -- start start 2024-05-04T03:09:34.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.377+0000 7f559cef4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55800057d0 0x7f559819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:34.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.377+0000 7f559cef4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f559819dc00 0x7f55981978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:34.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.377+0000 7f559cef4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5598197e20 0x7f5598198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:34.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.377+0000 7f559cef4700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5598109b10 con 0x7f55800057d0 2024-05-04T03:09:34.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.377+0000 7f559cef4700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5598109990 con 0x7f5598197e20 2024-05-04T03:09:34.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.377+0000 7f559cef4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5598109c90 con 0x7f559819dc00 2024-05-04T03:09:34.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.377+0000 7f5595d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f559819dc00 0x7f55981978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:34.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.377+0000 7f559659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55800057d0 0x7f559819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:34.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.377+0000 7f5595d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f559819dc00 0x7f55981978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50364/0 (socket says 172.21.15.77:50364) 2024-05-04T03:09:34.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.377+0000 7f5595d9b700 1 -- 172.21.15.77:0/313486367 learned_addr learned my addr 172.21.15.77:0/313486367 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-04T03:09:34.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.377+0000 7f5596d9d700 1 --2- 172.21.15.77:0/313486367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5598197e20 0x7f5598198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:34.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.377+0000 7f5596d9d700 1 -- 172.21.15.77:0/313486367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5598197e20 msgr2=0x7f5598198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-05-04T03:09:34.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.377+0000 7f5596d9d700 1 -- 172.21.15.77:0/313486367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5598197e20 msgr2=0x7f5598198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-04T03:09:34.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.377+0000 7f5596d9d700 1 --2- 172.21.15.77:0/313486367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5598197e20 0x7f5598198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-04T03:09:34.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.377+0000 7f5595d9b700 1 -- 172.21.15.77:0/313486367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f559819dc00 msgr2=0x7f55981978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-04T03:09:34.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.377+0000 7f5595d9b700 1 -- 172.21.15.77:0/313486367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f559819dc00 msgr2=0x7f55981978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-04T03:09:34.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.377+0000 7f5595d9b700 1 --2- 172.21.15.77:0/313486367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f559819dc00 0x7f55981978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-04T03:09:34.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.377+0000 7f5596d9d700 1 --2- 172.21.15.77:0/313486367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5598197e20 0x7f5598198230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-04T03:09:34.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.377+0000 7f5595d9b700 1 --2- 172.21.15.77:0/313486367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f559819dc00 0x7f55981978e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-04T03:09:34.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.377+0000 7f559659c700 1 -- 172.21.15.77:0/313486367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f559819dc00 msgr2=0x7f55981978e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-04T03:09:34.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.377+0000 7f559659c700 1 --2- 172.21.15.77:0/313486367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f559819dc00 0x7f55981978e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:34.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.377+0000 7f559659c700 1 -- 172.21.15.77:0/313486367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5598197e20 msgr2=0x7f5598198230 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-04T03:09:34.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.377+0000 7f559659c700 1 --2- 172.21.15.77:0/313486367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5598197e20 0x7f5598198230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:34.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.377+0000 7f559659c700 1 -- 172.21.15.77:0/313486367 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55981a4550 con 0x7f55800057d0 2024-05-04T03:09:34.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.378+0000 7f559659c700 1 --2- 172.21.15.77:0/313486367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55800057d0 0x7f559819d6c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f559806b460 tx=0x7f558c0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:34.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.378+0000 7f55877fe700 1 -- 172.21.15.77:0/313486367 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c0056c0 con 0x7f55800057d0 2024-05-04T03:09:34.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.378+0000 7f55877fe700 1 -- 172.21.15.77:0/313486367 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f558c01e470 con 0x7f55800057d0 2024-05-04T03:09:34.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.378+0000 7f55877fe700 1 -- 172.21.15.77:0/313486367 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c012450 con 0x7f55800057d0 2024-05-04T03:09:34.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.378+0000 7f559cef4700 1 -- 172.21.15.77:0/313486367 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55981a4780 con 0x7f55800057d0 2024-05-04T03:09:34.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.378+0000 7f559cef4700 1 -- 172.21.15.77:0/313486367 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55981a4ce0 con 0x7f55800057d0 2024-05-04T03:09:34.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.379+0000 7f55877fe700 1 -- 172.21.15.77:0/313486367 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f558c00fcf0 con 0x7f55800057d0 2024-05-04T03:09:34.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.379+0000 7f559cef4700 1 -- 172.21.15.77:0/313486367 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5598064e20 con 0x7f55800057d0 2024-05-04T03:09:34.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.379+0000 7f55877fe700 1 --2- 172.21.15.77:0/313486367 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f557803e9a0 0x7f5578040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:34.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.380+0000 7f55877fe700 1 -- 172.21.15.77:0/313486367 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f558c054910 con 0x7f55800057d0 2024-05-04T03:09:34.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.380+0000 7f5595d9b700 1 --2- 172.21.15.77:0/313486367 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f557803e9a0 0x7f5578040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:34.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.380+0000 7f5595d9b700 1 --2- 172.21.15.77:0/313486367 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f557803e9a0 0x7f5578040eb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5598101d40 tx=0x7f5588008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:34.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.385+0000 7f55877fe700 1 -- 172.21.15.77:0/313486367 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f558c00de00 con 0x7f55800057d0 2024-05-04T03:09:34.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:34.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:09:34.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: mon.a calling monitor election 2024-05-04T03:09:34.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: mon.c calling monitor election 2024-05-04T03:09:34.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:09:34.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:09:34.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:09:34.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:34.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:09:34.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: mon.b calling monitor election 2024-05-04T03:09:34.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:09:34.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:34.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:09:34.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:09:34.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-04T03:09:34.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: monmap epoch 3 2024-05-04T03:09:34.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:09:34.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: last_changed 2024-05-04T03:09:29.344359+0000 2024-05-04T03:09:34.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: created 2024-05-04T03:07:29.984656+0000 2024-05-04T03:09:34.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: min_mon_release 19 (squid) 2024-05-04T03:09:34.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: election_strategy: 1 2024-05-04T03:09:34.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-05-04T03:09:34.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: 1: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.c 2024-05-04T03:09:34.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: 2: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.b 2024-05-04T03:09:34.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: fsmap 2024-05-04T03:09:34.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: osdmap e4: 0 total, 0 up, 0 in 2024-05-04T03:09:34.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: mgrmap e13: x(active, since 61s) 2024-05-04T03:09:34.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: overall HEALTH_OK 2024-05-04T03:09:34.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:34.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:34.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:34.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:34 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:34.694 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.693+0000 7f559cef4700 1 -- 172.21.15.77:0/313486367 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f55981a50a0 con 0x7f55800057d0 2024-05-04T03:09:34.695 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.693+0000 7f55877fe700 1 -- 172.21.15.77:0/313486367 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1273 (secure 0 0 0) 0x7f558c010ae0 con 0x7f55800057d0 2024-05-04T03:09:34.695 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-04T03:09:34.695 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":3,"fsid":"4a572aa8-09c3-11ef-bc97-c7b262605968","modified":"2024-05-04T03:09:29.344359Z","created":"2024-05-04T03:07:29.984656Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:3300","nonce":0},{"type":"v1","addr":"172.21.15.72:6789","nonce":0}]},"addr":"172.21.15.72:6789/0","public_addr":"172.21.15.72:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-04T03:09:34.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.696+0000 7f559cef4700 1 -- 172.21.15.77:0/313486367 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f557803e9a0 msgr2=0x7f5578040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:34.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.696+0000 7f559cef4700 1 --2- 172.21.15.77:0/313486367 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f557803e9a0 0x7f5578040eb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5598101d40 tx=0x7f5588008040 comp rx=0 tx=0).stop 2024-05-04T03:09:34.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.696+0000 7f559cef4700 1 -- 172.21.15.77:0/313486367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55800057d0 msgr2=0x7f559819d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:34.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.696+0000 7f559cef4700 1 --2- 172.21.15.77:0/313486367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55800057d0 0x7f559819d6c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f559806b460 tx=0x7f558c0049d0 comp rx=0 tx=0).stop 2024-05-04T03:09:34.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.696+0000 7f559cef4700 1 -- 172.21.15.77:0/313486367 shutdown_connections 2024-05-04T03:09:34.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.696+0000 7f559cef4700 1 --2- 172.21.15.77:0/313486367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5598197e20 0x7f5598198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:34.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.696+0000 7f559cef4700 1 --2- 172.21.15.77:0/313486367 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f557803e9a0 0x7f5578040eb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:34.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.696+0000 7f559cef4700 1 --2- 172.21.15.77:0/313486367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55800057d0 0x7f559819d6c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:34.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.696+0000 7f559cef4700 1 --2- 172.21.15.77:0/313486367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f559819dc00 0x7f55981978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:34.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.696+0000 7f559cef4700 1 -- 172.21.15.77:0/313486367 >> 172.21.15.77:0/313486367 conn(0x7f5598077e80 msgr2=0x7f5598100260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:34.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.696+0000 7f559cef4700 1 -- 172.21.15.77:0/313486367 shutdown_connections 2024-05-04T03:09:34.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:34.696+0000 7f559cef4700 1 -- 172.21.15.77:0/313486367 wait complete. 2024-05-04T03:09:34.698 INFO:teuthology.orchestra.run.smithi077.stderr:dumped monmap epoch 3 2024-05-04T03:09:34.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:34.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:09:34.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: mon.a calling monitor election 2024-05-04T03:09:34.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: mon.c calling monitor election 2024-05-04T03:09:34.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:09:34.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:09:34.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:09:34.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:34.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:09:34.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: mon.b calling monitor election 2024-05-04T03:09:34.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:09:34.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:34.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:09:34.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:09:34.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-04T03:09:34.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: monmap epoch 3 2024-05-04T03:09:34.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:09:34.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: last_changed 2024-05-04T03:09:29.344359+0000 2024-05-04T03:09:34.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: created 2024-05-04T03:07:29.984656+0000 2024-05-04T03:09:34.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: min_mon_release 19 (squid) 2024-05-04T03:09:34.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: election_strategy: 1 2024-05-04T03:09:34.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-05-04T03:09:34.748 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: 1: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.c 2024-05-04T03:09:34.748 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: 2: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.b 2024-05-04T03:09:34.748 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: fsmap 2024-05-04T03:09:34.748 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: osdmap e4: 0 total, 0 up, 0 in 2024-05-04T03:09:34.748 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: mgrmap e13: x(active, since 61s) 2024-05-04T03:09:34.748 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: overall HEALTH_OK 2024-05-04T03:09:34.748 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:34.748 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:34.748 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:34.748 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:34 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:34.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:34.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:09:34.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: mon.a calling monitor election 2024-05-04T03:09:34.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: mon.c calling monitor election 2024-05-04T03:09:34.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:09:34.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:09:34.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:09:34.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:34.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:09:34.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: mon.b calling monitor election 2024-05-04T03:09:34.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:09:34.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:34.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:09:34.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:09:34.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-04T03:09:34.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: monmap epoch 3 2024-05-04T03:09:34.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:09:34.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: last_changed 2024-05-04T03:09:29.344359+0000 2024-05-04T03:09:34.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: created 2024-05-04T03:07:29.984656+0000 2024-05-04T03:09:34.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: min_mon_release 19 (squid) 2024-05-04T03:09:34.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: election_strategy: 1 2024-05-04T03:09:34.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: 0: [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon.a 2024-05-04T03:09:34.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: 1: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.c 2024-05-04T03:09:34.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: 2: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.b 2024-05-04T03:09:34.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: fsmap 2024-05-04T03:09:34.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: osdmap e4: 0 total, 0 up, 0 in 2024-05-04T03:09:34.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: mgrmap e13: x(active, since 61s) 2024-05-04T03:09:34.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: overall HEALTH_OK 2024-05-04T03:09:34.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:34.881 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:34.881 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:34.881 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:34 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:35.325 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-04T03:09:35.326 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph config generate-minimal-conf 2024-05-04T03:09:35.583 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:09:35.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:35 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:09:35.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:35 smithi001 ceph-mon[29731]: from='client.? 172.21.15.77:0/313486367' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-04T03:09:35.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:35 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:09:35.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:35 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:09:35.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:35 smithi077 ceph-mon[33531]: from='client.? 172.21.15.77:0/313486367' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-04T03:09:35.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:35 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:09:35.745 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:35 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:09:35.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:35 smithi072 ceph-mon[33332]: from='client.? 172.21.15.77:0/313486367' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-04T03:09:35.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:35 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:09:36.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:36 smithi001 ceph-mon[29731]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:36.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:36.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:36.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:36.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:36.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:36.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:09:36.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:36 smithi077 ceph-mon[33531]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:36.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:36 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:36.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:36 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:36.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:36 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:36.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:36 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:36.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:36 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:36.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:36 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:09:36.998 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:36 smithi072 ceph-mon[33332]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:36.998 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:36 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:36.998 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:36 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:36.998 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:36 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:36.998 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:36 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:36.998 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:36 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:36.998 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:36 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:09:37.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.345+0000 7fe2d6b2f700 1 -- 172.21.15.1:0/3113212155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2d0108b80 msgr2=0x7fe2d0108f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:37.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.345+0000 7fe2d6b2f700 1 --2- 172.21.15.1:0/3113212155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2d0108b80 0x7fe2d0108f90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4009910 tx=0x7fe2c4009c20 comp rx=0 tx=0).stop 2024-05-04T03:09:37.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.345+0000 7fe2d6b2f700 1 -- 172.21.15.1:0/3113212155 shutdown_connections 2024-05-04T03:09:37.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.345+0000 7fe2d6b2f700 1 --2- 172.21.15.1:0/3113212155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2d0108b80 0x7fe2d0108f90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:37.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.345+0000 7fe2d6b2f700 1 -- 172.21.15.1:0/3113212155 >> 172.21.15.1:0/3113212155 conn(0x7fe2d0077d30 msgr2=0x7fe2d0078170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:37.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.345+0000 7fe2d6b2f700 1 -- 172.21.15.1:0/3113212155 shutdown_connections 2024-05-04T03:09:37.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.345+0000 7fe2d6b2f700 1 -- 172.21.15.1:0/3113212155 wait complete. 2024-05-04T03:09:37.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.346+0000 7fe2d6b2f700 1 Processor -- start 2024-05-04T03:09:37.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.346+0000 7fe2d6b2f700 1 -- start start 2024-05-04T03:09:37.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.346+0000 7fe2d6b2f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2d0108b80 0x7fe2d019bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:37.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.346+0000 7fe2d6b2f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2d019c400 0x7fe2d01a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:37.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.346+0000 7fe2d6b2f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2d01a0c30 0x7fe2d01a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:37.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.346+0000 7fe2d6b2f700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d010e220 con 0x7fe2d01a0c30 2024-05-04T03:09:37.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.346+0000 7fe2d6b2f700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d010e0a0 con 0x7fe2d019c400 2024-05-04T03:09:37.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.346+0000 7fe2d6b2f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d010e3a0 con 0x7fe2d0108b80 2024-05-04T03:09:37.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.346+0000 7fe2d50cc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2d01a0c30 0x7fe2d01a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:37.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.346+0000 7fe2d50cc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2d01a0c30 0x7fe2d01a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35162/0 (socket says 172.21.15.1:35162) 2024-05-04T03:09:37.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.346+0000 7fe2d50cc700 1 -- 172.21.15.1:0/2948505489 learned_addr learned my addr 172.21.15.1:0/2948505489 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:09:37.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.346+0000 7fe2d48cb700 1 --2- 172.21.15.1:0/2948505489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2d0108b80 0x7fe2d019bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:37.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.346+0000 7fe2cffff700 1 --2- 172.21.15.1:0/2948505489 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2d019c400 0x7fe2d01a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:37.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.346+0000 7fe2d50cc700 1 -- 172.21.15.1:0/2948505489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2d0108b80 msgr2=0x7fe2d019bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:37.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.346+0000 7fe2d50cc700 1 --2- 172.21.15.1:0/2948505489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2d0108b80 0x7fe2d019bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:37.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.346+0000 7fe2d50cc700 1 -- 172.21.15.1:0/2948505489 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2d019c400 msgr2=0x7fe2d01a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:37.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.346+0000 7fe2d50cc700 1 --2- 172.21.15.1:0/2948505489 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2d019c400 0x7fe2d01a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:37.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.346+0000 7fe2d50cc700 1 -- 172.21.15.1:0/2948505489 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d01a89a0 con 0x7fe2d01a0c30 2024-05-04T03:09:37.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.346+0000 7fe2cffff700 1 --2- 172.21.15.1:0/2948505489 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2d019c400 0x7fe2d01a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:09:37.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.346+0000 7fe2d48cb700 1 --2- 172.21.15.1:0/2948505489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2d0108b80 0x7fe2d019bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:09:37.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.347+0000 7fe2d50cc700 1 --2- 172.21.15.1:0/2948505489 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2d01a0c30 0x7fe2d01a1040 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800ba50 tx=0x7fe2c800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:37.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.347+0000 7fe2cdffb700 1 -- 172.21.15.1:0/2948505489 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8019070 con 0x7fe2d01a0c30 2024-05-04T03:09:37.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.347+0000 7fe2d6b2f700 1 -- 172.21.15.1:0/2948505489 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2d01a8bd0 con 0x7fe2d01a0c30 2024-05-04T03:09:37.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.347+0000 7fe2cdffb700 1 -- 172.21.15.1:0/2948505489 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2c8014440 con 0x7fe2d01a0c30 2024-05-04T03:09:37.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.347+0000 7fe2cdffb700 1 -- 172.21.15.1:0/2948505489 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8007590 con 0x7fe2d01a0c30 2024-05-04T03:09:37.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.347+0000 7fe2d6b2f700 1 -- 172.21.15.1:0/2948505489 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2d01a9110 con 0x7fe2d01a0c30 2024-05-04T03:09:37.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.348+0000 7fe2cdffb700 1 -- 172.21.15.1:0/2948505489 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fe2c801c070 con 0x7fe2d01a0c30 2024-05-04T03:09:37.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.348+0000 7fe2d6b2f700 1 -- 172.21.15.1:0/2948505489 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2d0064e20 con 0x7fe2d01a0c30 2024-05-04T03:09:37.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.348+0000 7fe2cdffb700 1 --2- 172.21.15.1:0/2948505489 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe2c003e9a0 0x7fe2c0040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:37.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.348+0000 7fe2cdffb700 1 -- 172.21.15.1:0/2948505489 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe2c8051ec0 con 0x7fe2d01a0c30 2024-05-04T03:09:37.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.348+0000 7fe2d48cb700 1 --2- 172.21.15.1:0/2948505489 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe2c003e9a0 0x7fe2c0040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:37.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.349+0000 7fe2d48cb700 1 --2- 172.21.15.1:0/2948505489 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe2c003e9a0 0x7fe2c0040eb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4009750 tx=0x7fe2c400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:37.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.354+0000 7fe2cdffb700 1 -- 172.21.15.1:0/2948505489 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe2c8022170 con 0x7fe2d01a0c30 2024-05-04T03:09:37.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.569+0000 7fe2d6b2f700 1 -- 172.21.15.1:0/2948505489 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fe2d01a93d0 con 0x7fe2d01a0c30 2024-05-04T03:09:37.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.570+0000 7fe2cdffb700 1 -- 172.21.15.1:0/2948505489 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+269 (secure 0 0 0) 0x7fe2c8012c40 con 0x7fe2d01a0c30 2024-05-04T03:09:37.571 INFO:teuthology.orchestra.run.smithi001.stdout:# minimal ceph.conf for 4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:09:37.571 INFO:teuthology.orchestra.run.smithi001.stdout:[global] 2024-05-04T03:09:37.571 INFO:teuthology.orchestra.run.smithi001.stdout: fsid = 4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:09:37.571 INFO:teuthology.orchestra.run.smithi001.stdout: mon_host = [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] 2024-05-04T03:09:37.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.572+0000 7fe2d6b2f700 1 -- 172.21.15.1:0/2948505489 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe2c003e9a0 msgr2=0x7fe2c0040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:37.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.572+0000 7fe2d6b2f700 1 --2- 172.21.15.1:0/2948505489 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe2c003e9a0 0x7fe2c0040eb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4009750 tx=0x7fe2c400b540 comp rx=0 tx=0).stop 2024-05-04T03:09:37.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.572+0000 7fe2d6b2f700 1 -- 172.21.15.1:0/2948505489 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2d01a0c30 msgr2=0x7fe2d01a1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:37.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.572+0000 7fe2d6b2f700 1 --2- 172.21.15.1:0/2948505489 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2d01a0c30 0x7fe2d01a1040 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800ba50 tx=0x7fe2c800be10 comp rx=0 tx=0).stop 2024-05-04T03:09:37.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.572+0000 7fe2d6b2f700 1 -- 172.21.15.1:0/2948505489 shutdown_connections 2024-05-04T03:09:37.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.572+0000 7fe2d6b2f700 1 --2- 172.21.15.1:0/2948505489 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe2c003e9a0 0x7fe2c0040eb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:37.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.572+0000 7fe2d6b2f700 1 --2- 172.21.15.1:0/2948505489 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe2d01a0c30 0x7fe2d01a1040 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:37.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.572+0000 7fe2d6b2f700 1 --2- 172.21.15.1:0/2948505489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2d0108b80 0x7fe2d019bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:37.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.572+0000 7fe2d6b2f700 1 --2- 172.21.15.1:0/2948505489 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2d019c400 0x7fe2d01a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:37.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.572+0000 7fe2d6b2f700 1 -- 172.21.15.1:0/2948505489 >> 172.21.15.1:0/2948505489 conn(0x7fe2d0077d30 msgr2=0x7fe2d0100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:37.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.572+0000 7fe2d6b2f700 1 -- 172.21.15.1:0/2948505489 shutdown_connections 2024-05-04T03:09:37.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:37.572+0000 7fe2d6b2f700 1 -- 172.21.15.1:0/2948505489 wait complete. 2024-05-04T03:09:37.826 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:37 smithi001 ceph-mon[29731]: Updating smithi001:/etc/ceph/ceph.conf 2024-05-04T03:09:37.826 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:37 smithi001 ceph-mon[29731]: Updating smithi072:/etc/ceph/ceph.conf 2024-05-04T03:09:37.826 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:37 smithi001 ceph-mon[29731]: Updating smithi077:/etc/ceph/ceph.conf 2024-05-04T03:09:37.826 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:37 smithi001 ceph-mon[29731]: Updating smithi001:/var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:09:37.826 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:37 smithi001 ceph-mon[29731]: Updating smithi072:/var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:09:37.826 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:37 smithi001 ceph-mon[29731]: Updating smithi077:/var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:09:37.826 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:37 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.826 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:37 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.826 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:37 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.826 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:37 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.826 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:37 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.827 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:37 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.827 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:37 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.827 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:37 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.827 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:37 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.827 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:37 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.827 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:37 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.827 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:37 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-04T03:09:37.827 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:37 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-04T03:09:37.827 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:37 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:37.827 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:37 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2948505489' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:37.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:37 smithi077 ceph-mon[33531]: Updating smithi001:/etc/ceph/ceph.conf 2024-05-04T03:09:37.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:37 smithi077 ceph-mon[33531]: Updating smithi072:/etc/ceph/ceph.conf 2024-05-04T03:09:37.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:37 smithi077 ceph-mon[33531]: Updating smithi077:/etc/ceph/ceph.conf 2024-05-04T03:09:37.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:37 smithi077 ceph-mon[33531]: Updating smithi001:/var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:09:37.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:37 smithi077 ceph-mon[33531]: Updating smithi072:/var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:09:37.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:37 smithi077 ceph-mon[33531]: Updating smithi077:/var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:09:37.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:37 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:37 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:37 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:37 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:37 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:37 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:37 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:37 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:37 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:37 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:37 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:37 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-04T03:09:37.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:37 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-04T03:09:37.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:37 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:37.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:37 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/2948505489' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:37.995 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:37 smithi072 ceph-mon[33332]: Updating smithi001:/etc/ceph/ceph.conf 2024-05-04T03:09:37.995 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:37 smithi072 ceph-mon[33332]: Updating smithi072:/etc/ceph/ceph.conf 2024-05-04T03:09:37.995 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:37 smithi072 ceph-mon[33332]: Updating smithi077:/etc/ceph/ceph.conf 2024-05-04T03:09:37.995 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:37 smithi072 ceph-mon[33332]: Updating smithi001:/var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:09:37.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:37 smithi072 ceph-mon[33332]: Updating smithi072:/var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:09:37.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:37 smithi072 ceph-mon[33332]: Updating smithi077:/var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-04T03:09:37.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:37 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:37 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:37 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:37 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:37 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:37 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:37 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:37 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:37 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:37 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:37 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:37.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:37 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-04T03:09:37.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:37 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-04T03:09:37.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:37 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:37.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:37 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/2948505489' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:38.255 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-04T03:09:38.256 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-04T03:09:38.256 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/ceph/ceph.conf 2024-05-04T03:09:38.284 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-04T03:09:38.284 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-04T03:09:38.349 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-04T03:09:38.349 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/ceph/ceph.conf 2024-05-04T03:09:38.381 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-04T03:09:38.381 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-04T03:09:38.448 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-04T03:09:38.449 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.conf 2024-05-04T03:09:38.480 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-04T03:09:38.480 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-04T03:09:38.547 INFO:tasks.cephadm:Adding mgr.x on smithi001 2024-05-04T03:09:38.547 INFO:tasks.cephadm:Adding mgr.y on smithi072 2024-05-04T03:09:38.547 INFO:tasks.cephadm:Adding mgr.z on smithi077 2024-05-04T03:09:38.547 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph orch apply mgr '3;smithi001=x;smithi072=y;smithi077=z' 2024-05-04T03:09:38.745 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:38 smithi072 ceph-mon[33332]: Reconfiguring mon.a (unknown last config time)... 2024-05-04T03:09:38.745 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:38 smithi072 ceph-mon[33332]: Reconfiguring daemon mon.a on smithi001 2024-05-04T03:09:38.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:38 smithi072 ceph-mon[33332]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:38.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:38 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:38.816 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:09:38.853 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:38 smithi001 ceph-mon[29731]: Reconfiguring mon.a (unknown last config time)... 2024-05-04T03:09:38.853 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:38 smithi001 ceph-mon[29731]: Reconfiguring daemon mon.a on smithi001 2024-05-04T03:09:38.853 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:38 smithi001 ceph-mon[29731]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:38.853 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:38 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:38.919 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:38 smithi077 ceph-mon[33531]: Reconfiguring mon.a (unknown last config time)... 2024-05-04T03:09:38.919 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:38 smithi077 ceph-mon[33531]: Reconfiguring daemon mon.a on smithi001 2024-05-04T03:09:38.920 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:38 smithi077 ceph-mon[33531]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:38.920 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:38 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:40.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.198+0000 7fecc3bc1700 1 -- 172.21.15.77:0/943536443 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecbc102c20 msgr2=0x7fecbc103030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:40.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.198+0000 7fecc3bc1700 1 --2- 172.21.15.77:0/943536443 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecbc102c20 0x7fecbc103030 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fecb800b0a0 tx=0x7fecb800b3b0 comp rx=0 tx=0).stop 2024-05-04T03:09:40.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.198+0000 7fecc3bc1700 1 -- 172.21.15.77:0/943536443 shutdown_connections 2024-05-04T03:09:40.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.198+0000 7fecc3bc1700 1 --2- 172.21.15.77:0/943536443 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecbc102c20 0x7fecbc103030 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:40.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.198+0000 7fecc3bc1700 1 -- 172.21.15.77:0/943536443 >> 172.21.15.77:0/943536443 conn(0x7fecbc0fe280 msgr2=0x7fecbc100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:40.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.198+0000 7fecc3bc1700 1 -- 172.21.15.77:0/943536443 shutdown_connections 2024-05-04T03:09:40.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.198+0000 7fecc3bc1700 1 -- 172.21.15.77:0/943536443 wait complete. 2024-05-04T03:09:40.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.198+0000 7fecc3bc1700 1 Processor -- start 2024-05-04T03:09:40.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.198+0000 7fecc3bc1700 1 -- start start 2024-05-04T03:09:40.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.199+0000 7fecc3bc1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecbc102c20 0x7fecbc198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:40.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.199+0000 7fecc3bc1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecbc198db0 0x7fecbc19dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:40.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.199+0000 7fecc3bc1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecbc19e280 0x7fecbc1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:40.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.199+0000 7fecc3bc1700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc108ba0 con 0x7fecbc19e280 2024-05-04T03:09:40.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.199+0000 7fecc3bc1700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc108a20 con 0x7fecbc198db0 2024-05-04T03:09:40.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.199+0000 7fecc3bc1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc108d20 con 0x7fecbc102c20 2024-05-04T03:09:40.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.199+0000 7fecc195d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecbc102c20 0x7fecbc198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:40.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.199+0000 7fecc195d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecbc102c20 0x7fecbc198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42294/0 (socket says 172.21.15.77:42294) 2024-05-04T03:09:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.199+0000 7fecc195d700 1 -- 172.21.15.77:0/462401995 learned_addr learned my addr 172.21.15.77:0/462401995 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-04T03:09:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.199+0000 7fecc115c700 1 --2- 172.21.15.77:0/462401995 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecbc198db0 0x7fecbc19dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.199+0000 7fecc215e700 1 --2- 172.21.15.77:0/462401995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecbc19e280 0x7fecbc1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.199+0000 7fecc195d700 1 -- 172.21.15.77:0/462401995 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecbc198db0 msgr2=0x7fecbc19dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.199+0000 7fecc195d700 1 --2- 172.21.15.77:0/462401995 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecbc198db0 0x7fecbc19dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.199+0000 7fecc195d700 1 -- 172.21.15.77:0/462401995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecbc19e280 msgr2=0x7fecbc1a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.199+0000 7fecc195d700 1 --2- 172.21.15.77:0/462401995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecbc19e280 0x7fecbc1a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.199+0000 7fecc195d700 1 -- 172.21.15.77:0/462401995 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecbc1a0c10 con 0x7fecbc102c20 2024-05-04T03:09:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.199+0000 7fecc215e700 1 --2- 172.21.15.77:0/462401995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecbc19e280 0x7fecbc1a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:09:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.199+0000 7fecc115c700 1 --2- 172.21.15.77:0/462401995 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecbc198db0 0x7fecbc19dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:09:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.199+0000 7fecc195d700 1 --2- 172.21.15.77:0/462401995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecbc102c20 0x7fecbc198870 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fecb8000f80 tx=0x7fecb800bdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.199+0000 7fecb2ffd700 1 -- 172.21.15.77:0/462401995 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8017040 con 0x7fecbc102c20 2024-05-04T03:09:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.200+0000 7fecb2ffd700 1 -- 172.21.15.77:0/462401995 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecb8020070 con 0x7fecbc102c20 2024-05-04T03:09:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.200+0000 7fecc3bc1700 1 -- 172.21.15.77:0/462401995 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecbc1a0ea0 con 0x7fecbc102c20 2024-05-04T03:09:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.200+0000 7fecb2ffd700 1 -- 172.21.15.77:0/462401995 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb80127a0 con 0x7fecbc102c20 2024-05-04T03:09:40.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.200+0000 7fecc3bc1700 1 -- 172.21.15.77:0/462401995 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecbc1a1370 con 0x7fecbc102c20 2024-05-04T03:09:40.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.200+0000 7fecb2ffd700 1 -- 172.21.15.77:0/462401995 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fecb8012940 con 0x7fecbc102c20 2024-05-04T03:09:40.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.200+0000 7fecc3bc1700 1 -- 172.21.15.77:0/462401995 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecbc1a15a0 con 0x7fecbc102c20 2024-05-04T03:09:40.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.201+0000 7fecb2ffd700 1 --2- 172.21.15.77:0/462401995 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7feca803e9a0 0x7feca8040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:40.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.201+0000 7fecb2ffd700 1 -- 172.21.15.77:0/462401995 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fecb800f020 con 0x7fecbc102c20 2024-05-04T03:09:40.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.201+0000 7fecc115c700 1 --2- 172.21.15.77:0/462401995 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7feca803e9a0 0x7feca8040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:40.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.201+0000 7fecc115c700 1 --2- 172.21.15.77:0/462401995 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7feca803e9a0 0x7feca8040eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fecbc199920 tx=0x7fecac008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:40.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.206+0000 7fecb2ffd700 1 -- 172.21.15.77:0/462401995 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fecb8023370 con 0x7fecbc102c20 2024-05-04T03:09:40.245 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:40 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:40.245 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:40 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:40.245 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:40 smithi072 ceph-mon[33332]: Reconfiguring mon.b (monmap changed)... 2024-05-04T03:09:40.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:40 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-04T03:09:40.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:40 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-04T03:09:40.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:40 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:40.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:40 smithi072 ceph-mon[33332]: Reconfiguring daemon mon.b on smithi072 2024-05-04T03:09:40.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:40 smithi072 ceph-mon[33332]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:40.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:40 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:40.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:40 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:40.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:40 smithi001 ceph-mon[29731]: Reconfiguring mon.b (monmap changed)... 2024-05-04T03:09:40.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:40 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-04T03:09:40.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:40 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-04T03:09:40.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:40 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:40.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:40 smithi001 ceph-mon[29731]: Reconfiguring daemon mon.b on smithi072 2024-05-04T03:09:40.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:40 smithi001 ceph-mon[29731]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:40.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.418+0000 7fecc3bc1700 1 -- 172.21.15.77:0/462401995 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi001=x;smithi072=y;smithi077=z", "target": ["mon-mgr", ""]}) -- 0x7fecbc100080 con 0x7feca803e9a0 2024-05-04T03:09:40.420 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:40 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:40.420 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:40 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:40.420 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:40 smithi077 ceph-mon[33531]: Reconfiguring mon.b (monmap changed)... 2024-05-04T03:09:40.420 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:40 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-04T03:09:40.420 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:40 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-04T03:09:40.420 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:40 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:40.421 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:40 smithi077 ceph-mon[33531]: Reconfiguring daemon mon.b on smithi072 2024-05-04T03:09:40.421 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:40 smithi077 ceph-mon[33531]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:40.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.425+0000 7fecb2ffd700 1 -- 172.21.15.77:0/462401995 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fecbc100080 con 0x7feca803e9a0 2024-05-04T03:09:40.426 INFO:teuthology.orchestra.run.smithi077.stdout:Scheduled mgr update... 2024-05-04T03:09:40.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.427+0000 7fecc3bc1700 1 -- 172.21.15.77:0/462401995 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7feca803e9a0 msgr2=0x7feca8040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:40.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.427+0000 7fecc3bc1700 1 --2- 172.21.15.77:0/462401995 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7feca803e9a0 0x7feca8040eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fecbc199920 tx=0x7fecac008040 comp rx=0 tx=0).stop 2024-05-04T03:09:40.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.427+0000 7fecc3bc1700 1 -- 172.21.15.77:0/462401995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecbc102c20 msgr2=0x7fecbc198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:40.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.427+0000 7fecc3bc1700 1 --2- 172.21.15.77:0/462401995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecbc102c20 0x7fecbc198870 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fecb8000f80 tx=0x7fecb800bdc0 comp rx=0 tx=0).stop 2024-05-04T03:09:40.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.427+0000 7fecc3bc1700 1 -- 172.21.15.77:0/462401995 shutdown_connections 2024-05-04T03:09:40.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.427+0000 7fecc3bc1700 1 --2- 172.21.15.77:0/462401995 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7feca803e9a0 0x7feca8040eb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:40.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.427+0000 7fecc3bc1700 1 --2- 172.21.15.77:0/462401995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecbc19e280 0x7fecbc1a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:40.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.427+0000 7fecc3bc1700 1 --2- 172.21.15.77:0/462401995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecbc102c20 0x7fecbc198870 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:40.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.427+0000 7fecc3bc1700 1 --2- 172.21.15.77:0/462401995 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecbc198db0 0x7fecbc19dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:40.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.427+0000 7fecc3bc1700 1 -- 172.21.15.77:0/462401995 >> 172.21.15.77:0/462401995 conn(0x7fecbc0fe280 msgr2=0x7fecbc0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:40.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.427+0000 7fecc3bc1700 1 -- 172.21.15.77:0/462401995 shutdown_connections 2024-05-04T03:09:40.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:09:40.427+0000 7fecc3bc1700 1 -- 172.21.15.77:0/462401995 wait complete. 2024-05-04T03:09:40.972 DEBUG:teuthology.orchestra.run.smithi072:mgr.y> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mgr.y.service 2024-05-04T03:09:40.975 DEBUG:teuthology.orchestra.run.smithi077:mgr.z> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mgr.z.service 2024-05-04T03:09:40.977 INFO:tasks.cephadm:Deploying OSDs... 2024-05-04T03:09:40.977 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-04T03:09:40.977 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/scratch_devs of=/dev/stdout 2024-05-04T03:09:40.997 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-04T03:09:40.997 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_1 2024-05-04T03:09:41.056 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-04T03:09:41.056 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T03:09:41.056 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-04T03:09:41.056 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T03:09:41.056 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-05-04T03:09:41.056 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-05-04 03:08:43.575289293 +0000 2024-05-04T03:09:41.056 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-05-04 03:03:13.467695247 +0000 2024-05-04T03:09:41.056 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-05-04 03:03:13.467695247 +0000 2024-05-04T03:09:41.056 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-05-04 03:03:13.467695247 +0000 2024-05-04T03:09:41.057 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-04T03:09:41.124 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-05-04T03:09:41.125 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-05-04T03:09:41.125 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000130167 s, 3.9 MB/s 2024-05-04T03:09:41.126 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-04T03:09:41.185 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_2 2024-05-04T03:09:41.243 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-04T03:09:41.243 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T03:09:41.243 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-04T03:09:41.244 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T03:09:41.244 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-05-04T03:09:41.244 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-05-04 03:08:43.603288950 +0000 2024-05-04T03:09:41.244 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-05-04 03:03:13.818690724 +0000 2024-05-04T03:09:41.244 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-05-04 03:03:13.818690724 +0000 2024-05-04T03:09:41.244 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-05-04 03:03:13.818690724 +0000 2024-05-04T03:09:41.244 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-04T03:09:41.310 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-05-04T03:09:41.311 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-05-04T03:09:41.311 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.00012593 s, 4.1 MB/s 2024-05-04T03:09:41.312 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-04T03:09:41.371 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_3 2024-05-04T03:09:41.429 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-04T03:09:41.429 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T03:09:41.429 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-04T03:09:41.429 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T03:09:41.429 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-05-04T03:09:41.429 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-05-04 03:08:43.630288619 +0000 2024-05-04T03:09:41.429 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-05-04 03:03:14.154686395 +0000 2024-05-04T03:09:41.429 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-05-04 03:03:14.154686395 +0000 2024-05-04T03:09:41.429 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-05-04 03:03:14.154686395 +0000 2024-05-04T03:09:41.429 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-04T03:09:41.493 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-05-04T03:09:41.493 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-05-04T03:09:41.493 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000122728 s, 4.2 MB/s 2024-05-04T03:09:41.498 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-04T03:09:41.555 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_4 2024-05-04T03:09:41.611 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-04T03:09:41.611 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T03:09:41.611 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-05-04T03:09:41.611 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T03:09:41.611 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-05-04T03:09:41.611 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-05-04 03:08:43.656288301 +0000 2024-05-04T03:09:41.611 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-05-04 03:03:14.505681872 +0000 2024-05-04T03:09:41.611 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-05-04 03:03:14.505681872 +0000 2024-05-04T03:09:41.611 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-05-04 03:03:14.505681872 +0000 2024-05-04T03:09:41.611 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-04T03:09:41.668 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:41 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:41.669 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:41 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:41.669 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:41 smithi077 ceph-mon[33531]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi001=x;smithi072=y;smithi077=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:09:41.669 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:41 smithi077 ceph-mon[33531]: Saving service mgr spec with placement smithi001=x;smithi072=y;smithi077=z;count:3 2024-05-04T03:09:41.669 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:41 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:41.669 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:41 smithi077 ceph-mon[33531]: Reconfiguring mon.c (monmap changed)... 2024-05-04T03:09:41.669 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:41 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-04T03:09:41.669 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:41 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-04T03:09:41.669 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:41 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:41.669 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:41 smithi077 ceph-mon[33531]: Reconfiguring daemon mon.c on smithi077 2024-05-04T03:09:41.672 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:41 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:41.672 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:41 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:41.672 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:41 smithi001 ceph-mon[29731]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi001=x;smithi072=y;smithi077=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:09:41.673 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:41 smithi001 ceph-mon[29731]: Saving service mgr spec with placement smithi001=x;smithi072=y;smithi077=z;count:3 2024-05-04T03:09:41.673 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:41 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:41.673 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:41 smithi001 ceph-mon[29731]: Reconfiguring mon.c (monmap changed)... 2024-05-04T03:09:41.673 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:41 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-04T03:09:41.673 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:41 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-04T03:09:41.673 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:41 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:41.673 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:41 smithi001 ceph-mon[29731]: Reconfiguring daemon mon.c on smithi077 2024-05-04T03:09:41.674 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-05-04T03:09:41.674 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-05-04T03:09:41.675 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000107884 s, 4.7 MB/s 2024-05-04T03:09:41.676 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-04T03:09:41.733 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-04T03:09:41.733 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/scratch_devs of=/dev/stdout 2024-05-04T03:09:41.745 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:41 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:41.745 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:41 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:41.745 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:41 smithi072 ceph-mon[33332]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi001=x;smithi072=y;smithi077=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:09:41.745 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:41 smithi072 ceph-mon[33332]: Saving service mgr spec with placement smithi001=x;smithi072=y;smithi077=z;count:3 2024-05-04T03:09:41.745 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:41 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:41.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:41 smithi072 ceph-mon[33332]: Reconfiguring mon.c (monmap changed)... 2024-05-04T03:09:41.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:41 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-04T03:09:41.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:41 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-04T03:09:41.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:41 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:41.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:41 smithi072 ceph-mon[33332]: Reconfiguring daemon mon.c on smithi077 2024-05-04T03:09:41.752 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-04T03:09:41.752 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_1 2024-05-04T03:09:41.809 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-04T03:09:41.809 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T03:09:41.809 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-05-04T03:09:41.809 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T03:09:41.809 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2024-05-04T03:09:41.809 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-05-04 03:09:28.809402501 +0000 2024-05-04T03:09:41.809 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-05-04 03:03:50.060465764 +0000 2024-05-04T03:09:41.810 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-05-04 03:03:50.060465764 +0000 2024-05-04T03:09:41.810 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-05-04 03:03:50.060465764 +0000 2024-05-04T03:09:41.810 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-04T03:09:41.875 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2024-05-04T03:09:41.875 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2024-05-04T03:09:41.876 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000253769 s, 2.0 MB/s 2024-05-04T03:09:41.877 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-04T03:09:41.934 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_2 2024-05-04T03:09:41.991 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-04T03:09:41.991 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T03:09:41.991 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-05-04T03:09:41.991 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T03:09:41.992 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2024-05-04T03:09:41.992 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-05-04 03:09:28.809402501 +0000 2024-05-04T03:09:41.992 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-05-04 03:03:50.408457745 +0000 2024-05-04T03:09:41.992 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-05-04 03:03:50.408457745 +0000 2024-05-04T03:09:41.992 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-05-04 03:03:50.408457745 +0000 2024-05-04T03:09:41.992 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-04T03:09:42.055 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2024-05-04T03:09:42.056 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2024-05-04T03:09:42.056 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000217098 s, 2.4 MB/s 2024-05-04T03:09:42.057 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-04T03:09:42.113 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_3 2024-05-04T03:09:42.169 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-04T03:09:42.169 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T03:09:42.169 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 700 Links: 1 2024-05-04T03:09:42.170 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T03:09:42.170 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2024-05-04T03:09:42.170 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-05-04 03:09:28.810402480 +0000 2024-05-04T03:09:42.170 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-05-04 03:03:50.751449842 +0000 2024-05-04T03:09:42.170 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-05-04 03:03:50.751449842 +0000 2024-05-04T03:09:42.170 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-05-04 03:03:50.751449842 +0000 2024-05-04T03:09:42.170 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-04T03:09:42.232 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2024-05-04T03:09:42.232 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2024-05-04T03:09:42.233 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000217117 s, 2.4 MB/s 2024-05-04T03:09:42.234 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-04T03:09:42.290 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_4 2024-05-04T03:09:42.346 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-04T03:09:42.346 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T03:09:42.346 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-05-04T03:09:42.346 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T03:09:42.346 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2024-05-04T03:09:42.346 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-05-04 03:09:28.810402480 +0000 2024-05-04T03:09:42.346 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-05-04 03:03:51.096441891 +0000 2024-05-04T03:09:42.346 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-05-04 03:03:51.096441891 +0000 2024-05-04T03:09:42.346 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-05-04 03:03:51.096441891 +0000 2024-05-04T03:09:42.347 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-04T03:09:42.408 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2024-05-04T03:09:42.408 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2024-05-04T03:09:42.408 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000192976 s, 2.7 MB/s 2024-05-04T03:09:42.409 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-04T03:09:42.466 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-04T03:09:42.466 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/scratch_devs of=/dev/stdout 2024-05-04T03:09:42.483 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-04T03:09:42.483 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_1 2024-05-04T03:09:42.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:42 smithi072 ceph-mon[33332]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:42.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:42 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:42.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:42 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:42.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:42 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:09:42.538 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-04T03:09:42.538 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T03:09:42.538 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-04T03:09:42.538 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T03:09:42.538 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-05-04T03:09:42.538 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-05-04 03:09:19.167818695 +0000 2024-05-04T03:09:42.538 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-05-04 03:03:36.165957224 +0000 2024-05-04T03:09:42.538 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-05-04 03:03:36.165957224 +0000 2024-05-04T03:09:42.538 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-05-04 03:03:36.165957224 +0000 2024-05-04T03:09:42.538 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-04T03:09:42.601 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-05-04T03:09:42.601 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-05-04T03:09:42.601 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000346135 s, 1.5 MB/s 2024-05-04T03:09:42.603 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-04T03:09:42.659 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_2 2024-05-04T03:09:42.714 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-04T03:09:42.714 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T03:09:42.714 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-04T03:09:42.714 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T03:09:42.714 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-05-04T03:09:42.714 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-05-04 03:09:19.167818695 +0000 2024-05-04T03:09:42.714 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-05-04 03:03:36.510949925 +0000 2024-05-04T03:09:42.715 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-05-04 03:03:36.510949925 +0000 2024-05-04T03:09:42.715 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-05-04 03:03:36.510949925 +0000 2024-05-04T03:09:42.715 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-04T03:09:42.775 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:42 smithi077 ceph-mon[33531]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:42.775 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:42 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:42.775 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:42 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:42.775 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:42 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:09:42.777 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-05-04T03:09:42.777 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-05-04T03:09:42.777 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000193799 s, 2.6 MB/s 2024-05-04T03:09:42.778 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-04T03:09:42.834 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_3 2024-05-04T03:09:42.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:42 smithi001 ceph-mon[29731]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:42.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:42 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:42.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:42 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:42.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:42 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:09:42.891 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-04T03:09:42.891 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T03:09:42.891 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-04T03:09:42.892 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T03:09:42.892 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-05-04T03:09:42.892 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-05-04 03:09:19.168818676 +0000 2024-05-04T03:09:42.892 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-05-04 03:03:36.834943071 +0000 2024-05-04T03:09:42.892 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-05-04 03:03:36.834943071 +0000 2024-05-04T03:09:42.892 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-05-04 03:03:36.834943071 +0000 2024-05-04T03:09:42.892 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-04T03:09:42.955 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-05-04T03:09:42.955 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-05-04T03:09:42.955 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.00018524 s, 2.8 MB/s 2024-05-04T03:09:42.956 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-04T03:09:43.012 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_4 2024-05-04T03:09:43.066 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-04T03:09:43.066 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-04T03:09:43.066 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-04T03:09:43.066 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T03:09:43.067 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-05-04T03:09:43.067 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-05-04 03:09:19.168818676 +0000 2024-05-04T03:09:43.067 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-05-04 03:03:37.168936005 +0000 2024-05-04T03:09:43.067 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-05-04 03:03:37.168936005 +0000 2024-05-04T03:09:43.067 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-05-04 03:03:37.168936005 +0000 2024-05-04T03:09:43.067 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-04T03:09:43.131 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-05-04T03:09:43.132 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-05-04T03:09:43.132 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000181706 s, 2.8 MB/s 2024-05-04T03:09:43.133 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-04T03:09:43.190 INFO:tasks.cephadm:Deploying osd.0 on smithi001 with /dev/vg_nvme/lv_4... 2024-05-04T03:09:43.190 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-04T03:09:43.442 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:09:43.564 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:43 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:43.565 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:43 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:09:43.565 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:43 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:43.565 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:43 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-04T03:09:43.565 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:43 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-04T03:09:43.565 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:43 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-04T03:09:43.565 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:43 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:43.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:43 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:43.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:43 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:09:43.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:43 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:43.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:43 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-04T03:09:43.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:43 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-04T03:09:43.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:43 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-04T03:09:43.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:43 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:43.745 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:43 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:43.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:43 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:09:43.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:43 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:43.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:43 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-04T03:09:43.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:43 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-04T03:09:43.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:43 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-04T03:09:43.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:43 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:44.563 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:44 smithi077 ceph-mon[33531]: Deploying daemon mgr.z on smithi077 2024-05-04T03:09:44.564 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:44 smithi077 ceph-mon[33531]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:44.613 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:44 smithi001 ceph-mon[29731]: Deploying daemon mgr.z on smithi077 2024-05-04T03:09:44.613 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:44 smithi001 ceph-mon[29731]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:44.745 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:44 smithi072 ceph-mon[33332]: Deploying daemon mgr.z on smithi077 2024-05-04T03:09:44.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:44 smithi072 ceph-mon[33332]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:45.163 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:09:45 smithi077 systemd[1]: Started Ceph mgr.z for 4a572aa8-09c3-11ef-bc97-c7b262605968. 2024-05-04T03:09:45.265 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-04T03:09:45.266 INFO:teuthology.orchestra.run.smithi001.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-04T03:09:45.266 INFO:teuthology.orchestra.run.smithi001.stderr: stderr: 10+0 records in 2024-05-04T03:09:45.266 INFO:teuthology.orchestra.run.smithi001.stderr:10+0 records out 2024-05-04T03:09:45.266 INFO:teuthology.orchestra.run.smithi001.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00871927 s, 1.2 GB/s 2024-05-04T03:09:45.266 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping successful for: 2024-05-04T03:09:45.427 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:09:45 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z[35278]: 2024-05-04T03:09:45.162+0000 7f585fea9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-04T03:09:45.427 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:09:45 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z[35278]: 2024-05-04T03:09:45.346+0000 7f585fea9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-04T03:09:45.754 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:09:45 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z[35278]: 2024-05-04T03:09:45.447+0000 7f585fea9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-04T03:09:45.881 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi001:vg_nvme/lv_4 2024-05-04T03:09:46.124 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:09:46.149 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:09:45 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z[35278]: 2024-05-04T03:09:45.753+0000 7f585fea9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-04T03:09:46.149 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:09:45 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z[35278]: 2024-05-04T03:09:45.836+0000 7f585fea9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-04T03:09:46.404 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:46 smithi072 ceph-mon[33332]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:46.404 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:46 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:46.404 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:46 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:46.404 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:46 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:46.404 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:46 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-04T03:09:46.404 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:46 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-04T03:09:46.404 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:46 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-04T03:09:46.404 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:46 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:46.404 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:46 smithi072 ceph-mon[33332]: Deploying daemon mgr.y on smithi072 2024-05-04T03:09:46.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:46 smithi077 ceph-mon[33531]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:46.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:46 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:46.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:46 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:46.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:46 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:46.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:46 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-04T03:09:46.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:46 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-04T03:09:46.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:46 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-04T03:09:46.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:46 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:46.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:46 smithi077 ceph-mon[33531]: Deploying daemon mgr.y on smithi072 2024-05-04T03:09:46.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:46 smithi001 ceph-mon[29731]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:46.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:46 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:46.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:46 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:46.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:46 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:46.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:46 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-04T03:09:46.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:46 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-04T03:09:46.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:46 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-04T03:09:46.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:46 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:46.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:46 smithi001 ceph-mon[29731]: Deploying daemon mgr.y on smithi072 2024-05-04T03:09:47.246 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:47 smithi072 systemd[1]: Starting Ceph mgr.y for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:09:47.365 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:09:47 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z[35278]: 2024-05-04T03:09:47.023+0000 7f585fea9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-04T03:09:47.365 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:09:47 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z[35278]: 2024-05-04T03:09:47.189+0000 7f585fea9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-04T03:09:47.365 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:09:47 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z[35278]: 2024-05-04T03:09:47.365+0000 7f585fea9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-04T03:09:47.587 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:47 smithi072 podman[34703]: 2024-05-04 03:09:47.239327941 +0000 UTC m=+0.017862136 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:09:47.588 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:47 smithi072 podman[34703]: 2024-05-04 03:09:47.337299533 +0000 UTC m=+0.115833722 container create d698f7723d6eb9b26e068da6cf3fbf4d25f3eeeab09509b2b4d053f745633356 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-04T03:09:47.588 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:47 smithi072 podman[34703]: 2024-05-04 03:09:47.549571175 +0000 UTC m=+0.328105363 container init d698f7723d6eb9b26e068da6cf3fbf4d25f3eeeab09509b2b4d053f745633356 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240430) 2024-05-04T03:09:47.588 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:47 smithi072 podman[34703]: 2024-05-04 03:09:47.552299927 +0000 UTC m=+0.330834118 container start d698f7723d6eb9b26e068da6cf3fbf4d25f3eeeab09509b2b4d053f745633356 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-04T03:09:47.588 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:47 smithi072 ceph-mgr[34722]: -- 172.21.15.72:0/3348999012 <== mon.2 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b3388165a0 con 0x55b33d5a0480 2024-05-04T03:09:47.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.822+0000 7f6ba19c8700 1 -- 172.21.15.1:0/3645660420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b9c103230 msgr2=0x7f6b9c103640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:47.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.822+0000 7f6ba19c8700 1 --2- 172.21.15.1:0/3645660420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b9c103230 0x7f6b9c103640 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c009990 tx=0x7f6b8c009ca0 comp rx=0 tx=0).stop 2024-05-04T03:09:47.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.822+0000 7f6ba19c8700 1 -- 172.21.15.1:0/3645660420 shutdown_connections 2024-05-04T03:09:47.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.822+0000 7f6ba19c8700 1 --2- 172.21.15.1:0/3645660420 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b9c104d00 0x7f6b9c10b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:47.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.822+0000 7f6ba19c8700 1 --2- 172.21.15.1:0/3645660420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b9c103230 0x7f6b9c103640 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:47.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.822+0000 7f6ba19c8700 1 --2- 172.21.15.1:0/3645660420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b9c104360 0x7f6b9c104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:47.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.822+0000 7f6ba19c8700 1 -- 172.21.15.1:0/3645660420 >> 172.21.15.1:0/3645660420 conn(0x7f6b9c0fe890 msgr2=0x7f6b9c100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:09:47.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.822+0000 7f6ba19c8700 1 -- 172.21.15.1:0/3645660420 shutdown_connections 2024-05-04T03:09:47.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.822+0000 7f6ba19c8700 1 -- 172.21.15.1:0/3645660420 wait complete. 2024-05-04T03:09:47.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.823+0000 7f6ba19c8700 1 Processor -- start 2024-05-04T03:09:47.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.823+0000 7f6ba19c8700 1 -- start start 2024-05-04T03:09:47.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.823+0000 7f6ba19c8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b9c103230 0x7f6b9c076450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:47.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.823+0000 7f6ba19c8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b9c104360 0x7f6b9c19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:47.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.823+0000 7f6ba19c8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b9c104d00 0x7f6b9c1a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:47.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.824+0000 7f6ba19c8700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c10ecc0 con 0x7f6b9c103230 2024-05-04T03:09:47.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.824+0000 7f6ba19c8700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c10eb40 con 0x7f6b9c104d00 2024-05-04T03:09:47.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.824+0000 7f6ba19c8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c10ee40 con 0x7f6b9c104360 2024-05-04T03:09:47.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.824+0000 7f6b9affd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b9c103230 0x7f6b9c076450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:47.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.824+0000 7f6b9affd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b9c103230 0x7f6b9c076450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46350/0 (socket says 172.21.15.1:46350) 2024-05-04T03:09:47.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.824+0000 7f6b9affd700 1 -- 172.21.15.1:0/430605470 learned_addr learned my addr 172.21.15.1:0/430605470 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:09:47.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.824+0000 7f6b9a7fc700 1 --2- 172.21.15.1:0/430605470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b9c104360 0x7f6b9c19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:47.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.824+0000 7f6b9b7fe700 1 --2- 172.21.15.1:0/430605470 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b9c104d00 0x7f6b9c1a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:47.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.824+0000 7f6b9affd700 1 -- 172.21.15.1:0/430605470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b9c104360 msgr2=0x7f6b9c19f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:47.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.824+0000 7f6b9affd700 1 --2- 172.21.15.1:0/430605470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b9c104360 0x7f6b9c19f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:47.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.824+0000 7f6b9affd700 1 -- 172.21.15.1:0/430605470 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b9c104d00 msgr2=0x7f6b9c1a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:09:47.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.824+0000 7f6b9affd700 1 --2- 172.21.15.1:0/430605470 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b9c104d00 0x7f6b9c1a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:09:47.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.824+0000 7f6b9affd700 1 -- 172.21.15.1:0/430605470 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9c1a24d0 con 0x7f6b9c103230 2024-05-04T03:09:47.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.824+0000 7f6b9affd700 1 --2- 172.21.15.1:0/430605470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b9c103230 0x7f6b9c076450 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0098e0 tx=0x7f6b8c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:47.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.824+0000 7f6ba09c6700 1 -- 172.21.15.1:0/430605470 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c004830 con 0x7f6b9c103230 2024-05-04T03:09:47.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.824+0000 7f6ba09c6700 1 -- 172.21.15.1:0/430605470 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b8c005020 con 0x7f6b9c103230 2024-05-04T03:09:47.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.824+0000 7f6ba19c8700 1 -- 172.21.15.1:0/430605470 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b9c1a2760 con 0x7f6b9c103230 2024-05-04T03:09:47.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.824+0000 7f6ba09c6700 1 -- 172.21.15.1:0/430605470 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c003db0 con 0x7f6b9c103230 2024-05-04T03:09:47.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.825+0000 7f6ba19c8700 1 -- 172.21.15.1:0/430605470 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b9c1a2b50 con 0x7f6b9c103230 2024-05-04T03:09:47.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.825+0000 7f6ba09c6700 1 -- 172.21.15.1:0/430605470 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f6b8c003f50 con 0x7f6b9c103230 2024-05-04T03:09:47.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.825+0000 7f6ba19c8700 1 -- 172.21.15.1:0/430605470 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b9c0512c0 con 0x7f6b9c103230 2024-05-04T03:09:47.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.826+0000 7f6ba09c6700 1 --2- 172.21.15.1:0/430605470 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6b8403e750 0x7f6b84040c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:09:47.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.826+0000 7f6ba09c6700 1 -- 172.21.15.1:0/430605470 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6b8c053900 con 0x7f6b9c103230 2024-05-04T03:09:47.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.826+0000 7f6b9a7fc700 1 --2- 172.21.15.1:0/430605470 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6b8403e750 0x7f6b84040c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:09:47.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.826+0000 7f6b9a7fc700 1 --2- 172.21.15.1:0/430605470 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6b8403e750 0x7f6b84040c60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c075e00 tx=0x7f6b9000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:09:47.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:47.831+0000 7f6ba09c6700 1 -- 172.21.15.1:0/430605470 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6b8c013070 con 0x7f6b9c103230 2024-05-04T03:09:47.902 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:47 smithi072 bash[34703]: d698f7723d6eb9b26e068da6cf3fbf4d25f3eeeab09509b2b4d053f745633356 2024-05-04T03:09:47.902 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:47 smithi072 systemd[1]: Started Ceph mgr.y for 4a572aa8-09c3-11ef-bc97-c7b262605968. 2024-05-04T03:09:47.902 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:47 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y[34718]: 2024-05-04T03:09:47.715+0000 7fbf6df68200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-04T03:09:47.902 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:47 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y[34718]: 2024-05-04T03:09:47.901+0000 7fbf6df68200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-04T03:09:47.927 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:09:47 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z[35278]: 2024-05-04T03:09:47.649+0000 7f585fea9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-04T03:09:47.927 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:09:47 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z[35278]: 2024-05-04T03:09:47.740+0000 7f585fea9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-04T03:09:48.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:48.053+0000 7f6ba19c8700 1 -- 172.21.15.1:0/430605470 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6b9c063c00 con 0x7f6b8403e750 2024-05-04T03:09:48.246 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:48 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y[34718]: 2024-05-04T03:09:48.007+0000 7fbf6df68200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-04T03:09:48.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:48 smithi072 ceph-mon[33332]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:48.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:48 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:48.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:48 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:48.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:48 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:48.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:48 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:48.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:48 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:09:48.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:48 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:09:48.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:48 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:09:48.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:48 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:48.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:48 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:48.330 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:48 smithi001 ceph-mon[29731]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:48.330 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:48 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:48.330 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:48 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:48.330 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:48 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:48.330 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:48 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:48.330 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:48 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:09:48.330 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:48 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:09:48.330 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:48 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:09:48.330 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:48 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:48.330 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:48 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:48.427 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:09:47 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z[35278]: 2024-05-04T03:09:47.945+0000 7f585fea9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-04T03:09:48.427 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:09:48 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z[35278]: 2024-05-04T03:09:48.144+0000 7f585fea9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-04T03:09:48.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:48 smithi077 ceph-mon[33531]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:48.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:48 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:48.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:48 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:48.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:48 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:48.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:48 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:48.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:48 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:09:48.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:48 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:09:48.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:48 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:09:48.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:48 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:48.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:48 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:48.682 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:48 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y[34718]: 2024-05-04T03:09:48.333+0000 7fbf6df68200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-04T03:09:48.683 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:48 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y[34718]: 2024-05-04T03:09:48.426+0000 7fbf6df68200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-04T03:09:49.012 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:09:48 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z[35278]: 2024-05-04T03:09:48.780+0000 7f585fea9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-04T03:09:49.012 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:09:49 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z[35278]: 2024-05-04T03:09:49.011+0000 7f585fea9200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-04T03:09:49.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:49 smithi001 ceph-mon[29731]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:09:49.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:49 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:49.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:49 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:49.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:49 smithi077 ceph-mon[33531]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:09:49.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:49 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:49.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:49 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:49.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:49 smithi072 ceph-mon[33332]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:09:49.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:49 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:49.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:49 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:49.907 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:09:49 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z[35278]: 2024-05-04T03:09:49.448+0000 7f585fea9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-04T03:09:49.907 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:09:49 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z[35278]: 2024-05-04T03:09:49.543+0000 7f585fea9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-04T03:09:49.907 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:09:49 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z[35278]: 2024-05-04T03:09:49.906+0000 7f585fea9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-04T03:09:49.981 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:49 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y[34718]: 2024-05-04T03:09:49.649+0000 7fbf6df68200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-04T03:09:49.981 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:49 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y[34718]: 2024-05-04T03:09:49.814+0000 7fbf6df68200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-04T03:09:49.981 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:49 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y[34718]: 2024-05-04T03:09:49.980+0000 7fbf6df68200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-04T03:09:50.245 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:50 smithi072 ceph-mon[33332]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:50.245 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:50 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:50.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:50 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:50.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:50 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:50.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:50 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:09:50.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:50 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:50.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:50 smithi072 ceph-mon[33332]: Reconfiguring mgr.x (unknown last config time)... 2024-05-04T03:09:50.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:50 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-04T03:09:50.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:50 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-04T03:09:50.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:50 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:50.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:50 smithi072 ceph-mon[33332]: Reconfiguring daemon mgr.x on smithi001 2024-05-04T03:09:50.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:50 smithi001 ceph-mon[29731]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:50.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:50 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:50.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:50 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:50.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:50 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:50.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:50 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:09:50.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:50 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:50.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:50 smithi001 ceph-mon[29731]: Reconfiguring mgr.x (unknown last config time)... 2024-05-04T03:09:50.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:50 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-04T03:09:50.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:50 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-04T03:09:50.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:50 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:50.464 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:50 smithi001 ceph-mon[29731]: Reconfiguring daemon mgr.x on smithi001 2024-05-04T03:09:50.548 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:50 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y[34718]: 2024-05-04T03:09:50.261+0000 7fbf6df68200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-04T03:09:50.549 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:50 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y[34718]: 2024-05-04T03:09:50.350+0000 7fbf6df68200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-04T03:09:50.549 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:50 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y[34718]: 2024-05-04T03:09:50.547+0000 7fbf6df68200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-04T03:09:50.582 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:09:50 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z[35278]: 2024-05-04T03:09:50.254+0000 7f585fea9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-04T03:09:50.582 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:09:50 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z[35278]: 2024-05-04T03:09:50.344+0000 7f585fea9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-04T03:09:50.582 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:50 smithi077 ceph-mon[33531]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:50.582 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:50 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:50.582 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:50 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:50.583 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:50 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:50.583 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:50 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:09:50.583 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:50 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:50.583 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:50 smithi077 ceph-mon[33531]: Reconfiguring mgr.x (unknown last config time)... 2024-05-04T03:09:50.583 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:50 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-04T03:09:50.583 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:50 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-04T03:09:50.583 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:50 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:50.583 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:50 smithi077 ceph-mon[33531]: Reconfiguring daemon mgr.x on smithi001 2024-05-04T03:09:50.927 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:09:50 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z[35278]: 2024-05-04T03:09:50.581+0000 7f585fea9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-04T03:09:50.927 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:09:50 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z[35278]: 2024-05-04T03:09:50.713+0000 7f585fea9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-04T03:09:50.995 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:50 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y[34718]: 2024-05-04T03:09:50.735+0000 7fbf6df68200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-04T03:09:51.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:51 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/2905775984' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "100fe035-2454-4c95-90c7-759956ec36a7"}]: dispatch 2024-05-04T03:09:51.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:51 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/2905775984' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "100fe035-2454-4c95-90c7-759956ec36a7"}]': finished 2024-05-04T03:09:51.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:51 smithi077 ceph-mon[33531]: osdmap e5: 1 total, 0 up, 1 in 2024-05-04T03:09:51.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:51 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:09:51.427 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:09:51 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z[35278]: 2024-05-04T03:09:51.015+0000 7f585fea9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-04T03:09:51.428 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:09:51 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z[35278]: 2024-05-04T03:09:51.129+0000 7f585fea9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-04T03:09:51.428 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:09:51 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z[35278]: 2024-05-04T03:09:51.222+0000 7f585fea9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-04T03:09:51.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:51 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2905775984' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "100fe035-2454-4c95-90c7-759956ec36a7"}]: dispatch 2024-05-04T03:09:51.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:51 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2905775984' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "100fe035-2454-4c95-90c7-759956ec36a7"}]': finished 2024-05-04T03:09:51.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:51 smithi001 ceph-mon[29731]: osdmap e5: 1 total, 0 up, 1 in 2024-05-04T03:09:51.463 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:51 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:09:51.495 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:51 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y[34718]: 2024-05-04T03:09:51.321+0000 7fbf6df68200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-04T03:09:51.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:51 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/2905775984' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "100fe035-2454-4c95-90c7-759956ec36a7"}]: dispatch 2024-05-04T03:09:51.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:51 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/2905775984' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "100fe035-2454-4c95-90c7-759956ec36a7"}]': finished 2024-05-04T03:09:51.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:51 smithi072 ceph-mon[33332]: osdmap e5: 1 total, 0 up, 1 in 2024-05-04T03:09:51.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:51 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:09:51.957 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:51 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y[34718]: 2024-05-04T03:09:51.537+0000 7fbf6df68200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-04T03:09:52.177 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:09:51 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z[35278]: 2024-05-04T03:09:51.810+0000 7f585fea9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-04T03:09:52.177 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:09:52 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z[35278]: 2024-05-04T03:09:52.022+0000 7f585fea9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-04T03:09:52.245 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:51 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y[34718]: 2024-05-04T03:09:51.956+0000 7fbf6df68200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-04T03:09:52.245 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:52 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y[34718]: 2024-05-04T03:09:52.050+0000 7fbf6df68200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-04T03:09:52.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:52 smithi072 ceph-mon[33332]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:52.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:52 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/3420267135' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:09:52.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:52 smithi072 ceph-mon[33332]: Standby manager daemon z started 2024-05-04T03:09:52.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:52 smithi072 ceph-mon[33332]: from='mgr.? 172.21.15.77:0/1558554327' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-04T03:09:52.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:52 smithi072 ceph-mon[33332]: from='mgr.? 172.21.15.77:0/1558554327' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-04T03:09:52.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:52 smithi072 ceph-mon[33332]: from='mgr.? 172.21.15.77:0/1558554327' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-04T03:09:52.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:52 smithi072 ceph-mon[33332]: from='mgr.? 172.21.15.77:0/1558554327' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-04T03:09:52.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:52 smithi001 ceph-mon[29731]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:52.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:52 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3420267135' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:09:52.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:52 smithi001 ceph-mon[29731]: Standby manager daemon z started 2024-05-04T03:09:52.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:52 smithi001 ceph-mon[29731]: from='mgr.? 172.21.15.77:0/1558554327' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-04T03:09:52.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:52 smithi001 ceph-mon[29731]: from='mgr.? 172.21.15.77:0/1558554327' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-04T03:09:52.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:52 smithi001 ceph-mon[29731]: from='mgr.? 172.21.15.77:0/1558554327' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-04T03:09:52.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:52 smithi001 ceph-mon[29731]: from='mgr.? 172.21.15.77:0/1558554327' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-04T03:09:52.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:52 smithi077 ceph-mon[33531]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:52.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:52 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/3420267135' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:09:52.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:52 smithi077 ceph-mon[33531]: Standby manager daemon z started 2024-05-04T03:09:52.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:52 smithi077 ceph-mon[33531]: from='mgr.? 172.21.15.77:0/1558554327' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-04T03:09:52.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:52 smithi077 ceph-mon[33531]: from='mgr.? 172.21.15.77:0/1558554327' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-04T03:09:52.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:52 smithi077 ceph-mon[33531]: from='mgr.? 172.21.15.77:0/1558554327' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-04T03:09:52.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:52 smithi077 ceph-mon[33531]: from='mgr.? 172.21.15.77:0/1558554327' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-04T03:09:52.745 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:52 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y[34718]: 2024-05-04T03:09:52.425+0000 7fbf6df68200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-04T03:09:52.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:52.819+0000 7f6ba09c6700 1 -- 172.21.15.1:0/430605470 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 14) ==== 99936+0+0 (secure 0 0 0) 0x7f6b8c022470 con 0x7f6b9c103230 2024-05-04T03:09:53.116 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:52 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y[34718]: 2024-05-04T03:09:52.781+0000 7fbf6df68200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-04T03:09:53.116 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:52 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y[34718]: 2024-05-04T03:09:52.873+0000 7fbf6df68200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-04T03:09:53.495 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:53 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y[34718]: 2024-05-04T03:09:53.114+0000 7fbf6df68200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-04T03:09:53.495 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:53 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y[34718]: 2024-05-04T03:09:53.246+0000 7fbf6df68200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-04T03:09:53.824 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:53 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y[34718]: 2024-05-04T03:09:53.550+0000 7fbf6df68200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-04T03:09:53.824 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:53 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y[34718]: 2024-05-04T03:09:53.664+0000 7fbf6df68200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-04T03:09:53.824 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:53 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y[34718]: 2024-05-04T03:09:53.754+0000 7fbf6df68200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-04T03:09:54.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:53 smithi001 ceph-mon[29731]: mgrmap e14: x(active, since 79s), standbys: z 2024-05-04T03:09:54.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:53 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-04T03:09:54.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:53 smithi001 ceph-mon[29731]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:54.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:53 smithi077 ceph-mon[33531]: mgrmap e14: x(active, since 79s), standbys: z 2024-05-04T03:09:54.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:53 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-04T03:09:54.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:53 smithi077 ceph-mon[33531]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:54.245 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:53 smithi072 ceph-mon[33332]: mgrmap e14: x(active, since 79s), standbys: z 2024-05-04T03:09:54.245 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:53 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-04T03:09:54.245 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:53 smithi072 ceph-mon[33332]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:54.745 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:54 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y[34718]: 2024-05-04T03:09:54.349+0000 7fbf6df68200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-04T03:09:54.745 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:09:54 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y[34718]: 2024-05-04T03:09:54.564+0000 7fbf6df68200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-04T03:09:54.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:09:54.836+0000 7f6ba09c6700 1 -- 172.21.15.1:0/430605470 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mgrmap(e 15) ==== 149595+0+0 (secure 0 0 0) 0x7f6b8c020550 con 0x7f6b9c103230 2024-05-04T03:09:55.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:54 smithi001 ceph-mon[29731]: Standby manager daemon y started 2024-05-04T03:09:55.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:54 smithi001 ceph-mon[29731]: from='mgr.? 172.21.15.72:0/2537143279' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-04T03:09:55.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:54 smithi001 ceph-mon[29731]: from='mgr.? 172.21.15.72:0/2537143279' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-04T03:09:55.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:54 smithi001 ceph-mon[29731]: from='mgr.? 172.21.15.72:0/2537143279' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-04T03:09:55.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:54 smithi001 ceph-mon[29731]: from='mgr.? 172.21.15.72:0/2537143279' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-04T03:09:55.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:54 smithi077 ceph-mon[33531]: Standby manager daemon y started 2024-05-04T03:09:55.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:54 smithi077 ceph-mon[33531]: from='mgr.? 172.21.15.72:0/2537143279' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-04T03:09:55.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:54 smithi077 ceph-mon[33531]: from='mgr.? 172.21.15.72:0/2537143279' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-04T03:09:55.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:54 smithi077 ceph-mon[33531]: from='mgr.? 172.21.15.72:0/2537143279' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-04T03:09:55.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:54 smithi077 ceph-mon[33531]: from='mgr.? 172.21.15.72:0/2537143279' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-04T03:09:55.245 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:54 smithi072 ceph-mon[33332]: Standby manager daemon y started 2024-05-04T03:09:55.245 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:54 smithi072 ceph-mon[33332]: from='mgr.? 172.21.15.72:0/2537143279' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-04T03:09:55.245 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:54 smithi072 ceph-mon[33332]: from='mgr.? 172.21.15.72:0/2537143279' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-04T03:09:55.245 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:54 smithi072 ceph-mon[33332]: from='mgr.? 172.21.15.72:0/2537143279' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-04T03:09:55.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:54 smithi072 ceph-mon[33332]: from='mgr.? 172.21.15.72:0/2537143279' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-04T03:09:56.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:55 smithi001 ceph-mon[29731]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-05-04T03:09:56.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:55 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-04T03:09:56.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:55 smithi001 ceph-mon[29731]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:56.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:55 smithi077 ceph-mon[33531]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-05-04T03:09:56.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:55 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-04T03:09:56.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:55 smithi077 ceph-mon[33531]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:56.245 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:55 smithi072 ceph-mon[33332]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-05-04T03:09:56.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:55 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-04T03:09:56.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:55 smithi072 ceph-mon[33332]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:57.251 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:57 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:57.251 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:57 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:57.251 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:57 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:09:57.251 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:57 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:57.251 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:57 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:09:57.251 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:57 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:57.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:57 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:57.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:57 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:57.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:57 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:09:57.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:57 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:57.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:57 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:09:57.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:57 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:57.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:57 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:57.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:57 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:57.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:57 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:09:57.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:57 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:57.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:57 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:09:57.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:57 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:09:58.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:58 smithi001 ceph-mon[29731]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:58.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:58 smithi077 ceph-mon[33531]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:58.745 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:58 smithi072 ceph-mon[33332]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:09:59.558 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:59 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-04T03:09:59.558 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:09:59 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:59.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:59 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-04T03:09:59.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:09:59 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:09:59.745 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:59 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-04T03:09:59.745 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:09:59 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:10:00.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:00 smithi001 ceph-mon[29731]: Deploying daemon osd.0 on smithi001 2024-05-04T03:10:00.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:00 smithi001 ceph-mon[29731]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:10:00.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:00 smithi001 ceph-mon[29731]: overall HEALTH_OK 2024-05-04T03:10:00.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:00 smithi077 ceph-mon[33531]: Deploying daemon osd.0 on smithi001 2024-05-04T03:10:00.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:00 smithi077 ceph-mon[33531]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:10:00.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:00 smithi077 ceph-mon[33531]: overall HEALTH_OK 2024-05-04T03:10:00.745 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:00 smithi072 ceph-mon[33332]: Deploying daemon osd.0 on smithi001 2024-05-04T03:10:00.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:00 smithi072 ceph-mon[33332]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:10:00.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:00 smithi072 ceph-mon[33332]: overall HEALTH_OK 2024-05-04T03:10:02.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:02 smithi001 ceph-mon[29731]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:10:02.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:02 smithi077 ceph-mon[33531]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:10:02.745 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:02 smithi072 ceph-mon[33332]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:10:04.398 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:04 smithi001 ceph-mon[29731]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:10:04.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:04 smithi077 ceph-mon[33531]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:10:04.745 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:04 smithi072 ceph-mon[33332]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:10:05.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:05 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:05.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:05 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:05.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:05 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:10:05.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:05 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:05.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:05 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:05.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:05 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:10:05.995 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:05 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:05.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:05 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:05.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:05 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:10:06.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:06 smithi077 ceph-mon[33531]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:10:06.826 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:06 smithi001 ceph-mon[29731]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:10:06.995 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:06 smithi072 ceph-mon[33332]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:10:07.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:10:07.960+0000 7f6ba09c6700 1 -- 172.21.15.1:0/430605470 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6b9c063c00 con 0x7f6b8403e750 2024-05-04T03:10:07.962 INFO:teuthology.orchestra.run.smithi001.stdout:Created osd(s) 0 on host 'smithi001' 2024-05-04T03:10:07.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:10:07.962+0000 7f6ba19c8700 1 -- 172.21.15.1:0/430605470 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6b8403e750 msgr2=0x7f6b84040c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:07.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:10:07.962+0000 7f6ba19c8700 1 --2- 172.21.15.1:0/430605470 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6b8403e750 0x7f6b84040c60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c075e00 tx=0x7f6b9000b410 comp rx=0 tx=0).stop 2024-05-04T03:10:07.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:10:07.962+0000 7f6ba19c8700 1 -- 172.21.15.1:0/430605470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b9c103230 msgr2=0x7f6b9c076450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:07.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:10:07.962+0000 7f6ba19c8700 1 --2- 172.21.15.1:0/430605470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b9c103230 0x7f6b9c076450 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0098e0 tx=0x7f6b8c0047b0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:10:07.962+0000 7f6ba19c8700 1 -- 172.21.15.1:0/430605470 shutdown_connections 2024-05-04T03:10:07.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:10:07.962+0000 7f6ba19c8700 1 --2- 172.21.15.1:0/430605470 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b9c104d00 0x7f6b9c1a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:10:07.962+0000 7f6ba19c8700 1 --2- 172.21.15.1:0/430605470 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6b8403e750 0x7f6b84040c60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:10:07.962+0000 7f6ba19c8700 1 --2- 172.21.15.1:0/430605470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b9c103230 0x7f6b9c076450 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:10:07.962+0000 7f6ba19c8700 1 --2- 172.21.15.1:0/430605470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b9c104360 0x7f6b9c19f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:07.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:10:07.962+0000 7f6ba19c8700 1 -- 172.21.15.1:0/430605470 >> 172.21.15.1:0/430605470 conn(0x7f6b9c0fe890 msgr2=0x7f6b9c107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:07.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:10:07.962+0000 7f6ba19c8700 1 -- 172.21.15.1:0/430605470 shutdown_connections 2024-05-04T03:10:07.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:10:07.962+0000 7f6ba19c8700 1 -- 172.21.15.1:0/430605470 wait complete. 2024-05-04T03:10:08.616 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:08 smithi001 ceph-mon[29731]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:10:08.616 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:08 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:08.616 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:08 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:08.616 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:08 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:08.616 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:08 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:08.616 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:08 smithi001 ceph-mon[29731]: from='osd.0 [v2:172.21.15.1:6802/3479011484,v1:172.21.15.1:6803/3479011484]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-04T03:10:08.648 DEBUG:teuthology.orchestra.run.smithi001:osd.0> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.0.service 2024-05-04T03:10:08.650 INFO:tasks.cephadm:Deploying osd.1 on smithi072 with /dev/vg_nvme/lv_4... 2024-05-04T03:10:08.650 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-04T03:10:08.897 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.b/config 2024-05-04T03:10:08.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:08 smithi077 ceph-mon[33531]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:10:08.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:08 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:08.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:08 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:08.930 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:08 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:08.930 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:08 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:08.930 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:08 smithi077 ceph-mon[33531]: from='osd.0 [v2:172.21.15.1:6802/3479011484,v1:172.21.15.1:6803/3479011484]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-04T03:10:08.995 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:08 smithi072 ceph-mon[33332]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:10:08.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:08 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:08.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:08 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:08.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:08 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:08.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:08 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:08.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:08 smithi072 ceph-mon[33332]: from='osd.0 [v2:172.21.15.1:6802/3479011484,v1:172.21.15.1:6803/3479011484]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-04T03:10:10.227 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:10 smithi072 ceph-mon[33332]: from='osd.0 [v2:172.21.15.1:6802/3479011484,v1:172.21.15.1:6803/3479011484]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-04T03:10:10.227 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:10 smithi072 ceph-mon[33332]: osdmap e6: 1 total, 0 up, 1 in 2024-05-04T03:10:10.227 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:10 smithi072 ceph-mon[33332]: from='osd.0 [v2:172.21.15.1:6802/3479011484,v1:172.21.15.1:6803/3479011484]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-05-04T03:10:10.227 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:10 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:10:10.227 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:10 smithi072 ceph-mon[33332]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:10:10.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:10 smithi001 ceph-mon[29731]: from='osd.0 [v2:172.21.15.1:6802/3479011484,v1:172.21.15.1:6803/3479011484]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-04T03:10:10.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:10 smithi001 ceph-mon[29731]: osdmap e6: 1 total, 0 up, 1 in 2024-05-04T03:10:10.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:10 smithi001 ceph-mon[29731]: from='osd.0 [v2:172.21.15.1:6802/3479011484,v1:172.21.15.1:6803/3479011484]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-05-04T03:10:10.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:10 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:10:10.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:10 smithi001 ceph-mon[29731]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:10:10.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:10 smithi077 ceph-mon[33531]: from='osd.0 [v2:172.21.15.1:6802/3479011484,v1:172.21.15.1:6803/3479011484]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-04T03:10:10.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:10 smithi077 ceph-mon[33531]: osdmap e6: 1 total, 0 up, 1 in 2024-05-04T03:10:10.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:10 smithi077 ceph-mon[33531]: from='osd.0 [v2:172.21.15.1:6802/3479011484,v1:172.21.15.1:6803/3479011484]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-05-04T03:10:10.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:10 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:10:10.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:10 smithi077 ceph-mon[33531]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:10:10.754 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-04T03:10:10.754 INFO:teuthology.orchestra.run.smithi072.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-04T03:10:10.754 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10+0 records in 2024-05-04T03:10:10.754 INFO:teuthology.orchestra.run.smithi072.stderr:10+0 records out 2024-05-04T03:10:10.754 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112989 s, 928 MB/s 2024-05-04T03:10:10.754 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping successful for: 2024-05-04T03:10:11.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:11 smithi001 ceph-mon[29731]: from='osd.0 [v2:172.21.15.1:6802/3479011484,v1:172.21.15.1:6803/3479011484]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-05-04T03:10:11.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:11 smithi001 ceph-mon[29731]: osdmap e7: 1 total, 0 up, 1 in 2024-05-04T03:10:11.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:11 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:10:11.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:11 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:10:11.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:11 smithi001 ceph-mon[29731]: from='osd.0 [v2:172.21.15.1:6802/3479011484,v1:172.21.15.1:6803/3479011484]' entity='osd.0' 2024-05-04T03:10:11.129 INFO:journalctl@ceph.osd.0.smithi001.stdout:May 04 03:10:10 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-0[41843]: 2024-05-04T03:10:10.830+0000 7f93e4967700 -1 osd.0 0 waiting for initial osdmap 2024-05-04T03:10:11.129 INFO:journalctl@ceph.osd.0.smithi001.stdout:May 04 03:10:10 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-0[41843]: 2024-05-04T03:10:10.834+0000 7f93dc714700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T03:10:11.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:11 smithi072 ceph-mon[33332]: from='osd.0 [v2:172.21.15.1:6802/3479011484,v1:172.21.15.1:6803/3479011484]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-05-04T03:10:11.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:11 smithi072 ceph-mon[33332]: osdmap e7: 1 total, 0 up, 1 in 2024-05-04T03:10:11.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:11 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:10:11.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:11 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:10:11.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:11 smithi072 ceph-mon[33332]: from='osd.0 [v2:172.21.15.1:6802/3479011484,v1:172.21.15.1:6803/3479011484]' entity='osd.0' 2024-05-04T03:10:11.405 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi072:vg_nvme/lv_4 2024-05-04T03:10:11.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:11 smithi077 ceph-mon[33531]: from='osd.0 [v2:172.21.15.1:6802/3479011484,v1:172.21.15.1:6803/3479011484]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-05-04T03:10:11.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:11 smithi077 ceph-mon[33531]: osdmap e7: 1 total, 0 up, 1 in 2024-05-04T03:10:11.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:11 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:10:11.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:11 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:10:11.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:11 smithi077 ceph-mon[33531]: from='osd.0 [v2:172.21.15.1:6802/3479011484,v1:172.21.15.1:6803/3479011484]' entity='osd.0' 2024-05-04T03:10:11.648 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.b/config 2024-05-04T03:10:12.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:12 smithi001 ceph-mon[29731]: purged_snaps scrub starts 2024-05-04T03:10:12.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:12 smithi001 ceph-mon[29731]: purged_snaps scrub ok 2024-05-04T03:10:12.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:12 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:10:12.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:12 smithi001 ceph-mon[29731]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:10:12.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:12 smithi001 ceph-mon[29731]: Detected new or changed devices on smithi001 2024-05-04T03:10:12.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:12 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:12.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:12 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:12.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:12 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:10:12.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:12 smithi001 ceph-mon[29731]: Adjusting osd_memory_target on smithi001 to 17093M 2024-05-04T03:10:12.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:12 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:12.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:12 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:10:12.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:12 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:10:12.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:12 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:12.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:12 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:10:12.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:12 smithi001 ceph-mon[29731]: osd.0 [v2:172.21.15.1:6802/3479011484,v1:172.21.15.1:6803/3479011484] boot 2024-05-04T03:10:12.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:12 smithi001 ceph-mon[29731]: osdmap e8: 1 total, 1 up, 1 in 2024-05-04T03:10:12.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:12 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:10:12.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:12 smithi072 ceph-mon[33332]: purged_snaps scrub starts 2024-05-04T03:10:12.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:12 smithi072 ceph-mon[33332]: purged_snaps scrub ok 2024-05-04T03:10:12.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:12 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:10:12.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:12 smithi072 ceph-mon[33332]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:10:12.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:12 smithi072 ceph-mon[33332]: Detected new or changed devices on smithi001 2024-05-04T03:10:12.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:12 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:12.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:12 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:12.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:12 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:10:12.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:12 smithi072 ceph-mon[33332]: Adjusting osd_memory_target on smithi001 to 17093M 2024-05-04T03:10:12.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:12 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:12.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:12 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:10:12.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:12 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:10:12.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:12 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:12.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:12 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:10:12.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:12 smithi072 ceph-mon[33332]: osd.0 [v2:172.21.15.1:6802/3479011484,v1:172.21.15.1:6803/3479011484] boot 2024-05-04T03:10:12.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:12 smithi072 ceph-mon[33332]: osdmap e8: 1 total, 1 up, 1 in 2024-05-04T03:10:12.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:12 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:10:12.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:12 smithi077 ceph-mon[33531]: purged_snaps scrub starts 2024-05-04T03:10:12.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:12 smithi077 ceph-mon[33531]: purged_snaps scrub ok 2024-05-04T03:10:12.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:12 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:10:12.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:12 smithi077 ceph-mon[33531]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-04T03:10:12.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:12 smithi077 ceph-mon[33531]: Detected new or changed devices on smithi001 2024-05-04T03:10:12.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:12 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:12.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:12 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:12.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:12 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:10:12.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:12 smithi077 ceph-mon[33531]: Adjusting osd_memory_target on smithi001 to 17093M 2024-05-04T03:10:12.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:12 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:12.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:12 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:10:12.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:12 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:10:12.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:12 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:12.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:12 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:10:12.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:12 smithi077 ceph-mon[33531]: osd.0 [v2:172.21.15.1:6802/3479011484,v1:172.21.15.1:6803/3479011484] boot 2024-05-04T03:10:12.429 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:12 smithi077 ceph-mon[33531]: osdmap e8: 1 total, 1 up, 1 in 2024-05-04T03:10:12.429 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:12 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-04T03:10:13.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:13 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:10:13.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:13 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:10:13.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:13 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:13.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.179+0000 7f4fc9704700 1 -- 172.21.15.72:0/1532192938 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4fc41077a0 msgr2=0x7f4fc4109bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:13.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.179+0000 7f4fc9704700 1 --2- 172.21.15.72:0/1532192938 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4fc41077a0 0x7f4fc4109bc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4fac009990 tx=0x7f4fac009ca0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.179+0000 7f4fc9704700 1 -- 172.21.15.72:0/1532192938 shutdown_connections 2024-05-04T03:10:13.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.179+0000 7f4fc9704700 1 --2- 172.21.15.72:0/1532192938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fc410a100 0x7f4fc410c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.179+0000 7f4fc9704700 1 --2- 172.21.15.72:0/1532192938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc4104e40 0x7f4fc4107260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.179+0000 7f4fc9704700 1 --2- 172.21.15.72:0/1532192938 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4fc41077a0 0x7f4fc4109bc0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.179+0000 7f4fc9704700 1 -- 172.21.15.72:0/1532192938 >> 172.21.15.72:0/1532192938 conn(0x7f4fc40fe880 msgr2=0x7f4fc4100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:13.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.179+0000 7f4fc9704700 1 -- 172.21.15.72:0/1532192938 shutdown_connections 2024-05-04T03:10:13.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.179+0000 7f4fc9704700 1 -- 172.21.15.72:0/1532192938 wait complete. 2024-05-04T03:10:13.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.180+0000 7f4fc9704700 1 Processor -- start 2024-05-04T03:10:13.183 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.180+0000 7f4fc9704700 1 -- start start 2024-05-04T03:10:13.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.180+0000 7f4fc9704700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fc4104e40 0x7f4fc419a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:13.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.180+0000 7f4fc9704700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4fc41077a0 0x7f4fc419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:13.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.180+0000 7f4fc9704700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc410a100 0x7f4fc41a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:13.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.180+0000 7f4fc9704700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc410fda0 con 0x7f4fc4104e40 2024-05-04T03:10:13.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.180+0000 7f4fc9704700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc410fc20 con 0x7f4fc41077a0 2024-05-04T03:10:13.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.180+0000 7f4fc9704700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc410ff20 con 0x7f4fc410a100 2024-05-04T03:10:13.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.180+0000 7f4fc27fc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4fc41077a0 0x7f4fc419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:13.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.180+0000 7f4fc27fc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4fc41077a0 0x7f4fc419add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:39908/0 (socket says 172.21.15.72:39908) 2024-05-04T03:10:13.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.180+0000 7f4fc27fc700 1 -- 172.21.15.72:0/2300457342 learned_addr learned my addr 172.21.15.72:0/2300457342 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-05-04T03:10:13.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.180+0000 7f4fc2ffd700 1 --2- 172.21.15.72:0/2300457342 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fc4104e40 0x7f4fc419a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:13.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.181+0000 7f4fc37fe700 1 --2- 172.21.15.72:0/2300457342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc410a100 0x7f4fc41a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:13.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.181+0000 7f4fc27fc700 1 -- 172.21.15.72:0/2300457342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc410a100 msgr2=0x7f4fc41a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:13.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.181+0000 7f4fc27fc700 1 --2- 172.21.15.72:0/2300457342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc410a100 0x7f4fc41a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.184 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.181+0000 7f4fc27fc700 1 -- 172.21.15.72:0/2300457342 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fc4104e40 msgr2=0x7f4fc419a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:13.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.181+0000 7f4fc27fc700 1 --2- 172.21.15.72:0/2300457342 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fc4104e40 0x7f4fc419a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:13.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.181+0000 7f4fc27fc700 1 -- 172.21.15.72:0/2300457342 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc41a2360 con 0x7f4fc41077a0 2024-05-04T03:10:13.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.181+0000 7f4fc27fc700 1 --2- 172.21.15.72:0/2300457342 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4fc41077a0 0x7f4fc419add0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4fac003800 tx=0x7f4fac0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:13.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.181+0000 7f4fbbfff700 1 -- 172.21.15.72:0/2300457342 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac027070 con 0x7f4fc41077a0 2024-05-04T03:10:13.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.181+0000 7f4fc9704700 1 -- 172.21.15.72:0/2300457342 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fc41a25f0 con 0x7f4fc41077a0 2024-05-04T03:10:13.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.181+0000 7f4fbbfff700 1 -- 172.21.15.72:0/2300457342 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fac0041c0 con 0x7f4fc41077a0 2024-05-04T03:10:13.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.181+0000 7f4fbbfff700 1 -- 172.21.15.72:0/2300457342 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac00f460 con 0x7f4fc41077a0 2024-05-04T03:10:13.185 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.181+0000 7f4fc9704700 1 -- 172.21.15.72:0/2300457342 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fc41a2aa0 con 0x7f4fc41077a0 2024-05-04T03:10:13.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.182+0000 7f4fc9704700 1 -- 172.21.15.72:0/2300457342 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fc40512c0 con 0x7f4fc41077a0 2024-05-04T03:10:13.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.183+0000 7f4fbbfff700 1 -- 172.21.15.72:0/2300457342 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 15) ==== 149595+0+0 (secure 0 0 0) 0x7f4fac02a070 con 0x7f4fc41077a0 2024-05-04T03:10:13.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.184+0000 7f4fbbfff700 1 --2- 172.21.15.72:0/2300457342 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4fb00b27f0 0x7f4fb00b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:13.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.185+0000 7f4fc2ffd700 1 --2- 172.21.15.72:0/2300457342 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4fb00b27f0 0x7f4fb00b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:13.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.185+0000 7f4fbbfff700 1 -- 172.21.15.72:0/2300457342 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7f4fac0e18d0 con 0x7f4fc41077a0 2024-05-04T03:10:13.186 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.185+0000 7f4fc2ffd700 1 --2- 172.21.15.72:0/2300457342 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4fb00b27f0 0x7f4fb00b4d00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4fc406c030 tx=0x7f4fb40079e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:13.189 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.189+0000 7f4fbbfff700 1 -- 172.21.15.72:0/2300457342 <== mon.2 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4fac013070 con 0x7f4fc41077a0 2024-05-04T03:10:13.245 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:13 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:10:13.245 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:13 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:10:13.245 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:13 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:13.328 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:13 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:10:13.328 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:13 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:10:13.329 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:13 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:13.404 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:13.403+0000 7f4fc9704700 1 -- 172.21.15.72:0/2300457342 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f4fc4063c00 con 0x7f4fb00b27f0 2024-05-04T03:10:14.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:14 smithi001 ceph-mon[29731]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:14.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:14 smithi001 ceph-mon[29731]: osdmap e9: 1 total, 1 up, 1 in 2024-05-04T03:10:14.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:14 smithi001 ceph-mon[29731]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:10:14.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:10:14.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:10:14.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:10:14.663 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:14 smithi072 ceph-mon[33332]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:14.663 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:14 smithi072 ceph-mon[33332]: osdmap e9: 1 total, 1 up, 1 in 2024-05-04T03:10:14.663 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:14 smithi072 ceph-mon[33332]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:10:14.663 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:14 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:10:14.663 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:14 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:10:14.663 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:14 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:10:14.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:14 smithi077 ceph-mon[33531]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:14.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:14 smithi077 ceph-mon[33531]: osdmap e9: 1 total, 1 up, 1 in 2024-05-04T03:10:14.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:14 smithi077 ceph-mon[33531]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:10:14.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:14 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:10:14.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:14 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:10:14.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:14 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:10:16.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:16 smithi001 ceph-mon[29731]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:16.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:16 smithi001 ceph-mon[29731]: from='client.? 172.21.15.72:0/3383422558' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5826ac76-ef6b-4040-8ad1-305db355af79"}]: dispatch 2024-05-04T03:10:16.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:16 smithi001 ceph-mon[29731]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5826ac76-ef6b-4040-8ad1-305db355af79"}]: dispatch 2024-05-04T03:10:16.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:16 smithi001 ceph-mon[29731]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5826ac76-ef6b-4040-8ad1-305db355af79"}]': finished 2024-05-04T03:10:16.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:16 smithi001 ceph-mon[29731]: osdmap e10: 2 total, 1 up, 2 in 2024-05-04T03:10:16.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:16 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:10:16.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:16 smithi077 ceph-mon[33531]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:16.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:16 smithi077 ceph-mon[33531]: from='client.? 172.21.15.72:0/3383422558' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5826ac76-ef6b-4040-8ad1-305db355af79"}]: dispatch 2024-05-04T03:10:16.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:16 smithi077 ceph-mon[33531]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5826ac76-ef6b-4040-8ad1-305db355af79"}]: dispatch 2024-05-04T03:10:16.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:16 smithi077 ceph-mon[33531]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5826ac76-ef6b-4040-8ad1-305db355af79"}]': finished 2024-05-04T03:10:16.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:16 smithi077 ceph-mon[33531]: osdmap e10: 2 total, 1 up, 2 in 2024-05-04T03:10:16.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:16 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:10:16.745 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:16 smithi072 ceph-mon[33332]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:16.745 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:16 smithi072 ceph-mon[33332]: from='client.? 172.21.15.72:0/3383422558' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5826ac76-ef6b-4040-8ad1-305db355af79"}]: dispatch 2024-05-04T03:10:16.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:16 smithi072 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5826ac76-ef6b-4040-8ad1-305db355af79"}]: dispatch 2024-05-04T03:10:16.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:16 smithi072 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5826ac76-ef6b-4040-8ad1-305db355af79"}]': finished 2024-05-04T03:10:16.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:16 smithi072 ceph-mon[33332]: osdmap e10: 2 total, 1 up, 2 in 2024-05-04T03:10:16.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:16 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:10:17.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:17 smithi001 ceph-mon[29731]: from='client.? 172.21.15.72:0/3989683478' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:10:17.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:17 smithi077 ceph-mon[33531]: from='client.? 172.21.15.72:0/3989683478' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:10:17.745 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:17 smithi072 ceph-mon[33332]: from='client.? 172.21.15.72:0/3989683478' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:10:18.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:18 smithi001 ceph-mon[29731]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:18.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:18 smithi077 ceph-mon[33531]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:18.745 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:18 smithi072 ceph-mon[33332]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:20.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:20 smithi001 ceph-mon[29731]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:20.667 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:20 smithi072 ceph-mon[33332]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:20.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:20 smithi077 ceph-mon[33531]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:22.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:22 smithi001 ceph-mon[29731]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:22.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:22 smithi077 ceph-mon[33531]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:22.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:22 smithi072 ceph-mon[33332]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:24.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:24 smithi001 ceph-mon[29731]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:24.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:24 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-04T03:10:24.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:24 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:10:24.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:24 smithi001 ceph-mon[29731]: Deploying daemon osd.1 on smithi072 2024-05-04T03:10:24.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:24 smithi077 ceph-mon[33531]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:24.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:24 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-04T03:10:24.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:24 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:10:24.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:24 smithi077 ceph-mon[33531]: Deploying daemon osd.1 on smithi072 2024-05-04T03:10:24.682 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:24 smithi072 ceph-mon[33332]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:24.682 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:24 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-04T03:10:24.682 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:24 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:10:24.682 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:24 smithi072 ceph-mon[33332]: Deploying daemon osd.1 on smithi072 2024-05-04T03:10:26.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:26 smithi072 ceph-mon[33332]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:26.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:26 smithi001 ceph-mon[29731]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:26.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:26 smithi077 ceph-mon[33531]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:28.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:28 smithi001 ceph-mon[29731]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:28.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:28 smithi077 ceph-mon[33531]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:28.704 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:28 smithi072 ceph-mon[33332]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:29.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:29 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:29.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:29 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:29.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:29 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:10:29.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:29 smithi072 ceph-mon[33332]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:30.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:29 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:30.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:29 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:30.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:29 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:10:30.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:29 smithi001 ceph-mon[29731]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:30.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:29 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:30.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:29 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:30.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:29 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:10:30.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:29 smithi077 ceph-mon[33531]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:32.278 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:32 smithi072 ceph-mon[33332]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:32.321 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:32.319+0000 7f4fbbfff700 1 -- 172.21.15.72:0/2300457342 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4fc4063c00 con 0x7f4fb00b27f0 2024-05-04T03:10:32.321 INFO:teuthology.orchestra.run.smithi072.stdout:Created osd(s) 1 on host 'smithi072' 2024-05-04T03:10:32.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:32.322+0000 7f4fc9704700 1 -- 172.21.15.72:0/2300457342 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4fb00b27f0 msgr2=0x7f4fb00b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:32.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:32.322+0000 7f4fc9704700 1 --2- 172.21.15.72:0/2300457342 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4fb00b27f0 0x7f4fb00b4d00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4fc406c030 tx=0x7f4fb40079e0 comp rx=0 tx=0).stop 2024-05-04T03:10:32.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:32.322+0000 7f4fc9704700 1 -- 172.21.15.72:0/2300457342 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4fc41077a0 msgr2=0x7f4fc419add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:32.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:32.322+0000 7f4fc9704700 1 --2- 172.21.15.72:0/2300457342 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4fc41077a0 0x7f4fc419add0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4fac003800 tx=0x7f4fac0047b0 comp rx=0 tx=0).stop 2024-05-04T03:10:32.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:32.322+0000 7f4fc9704700 1 -- 172.21.15.72:0/2300457342 shutdown_connections 2024-05-04T03:10:32.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:32.322+0000 7f4fc9704700 1 --2- 172.21.15.72:0/2300457342 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4fb00b27f0 0x7f4fb00b4d00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:32.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:32.322+0000 7f4fc9704700 1 --2- 172.21.15.72:0/2300457342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fc410a100 0x7f4fc41a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:32.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:32.322+0000 7f4fc9704700 1 --2- 172.21.15.72:0/2300457342 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fc4104e40 0x7f4fc419a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:32.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:32.322+0000 7f4fc9704700 1 --2- 172.21.15.72:0/2300457342 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4fc41077a0 0x7f4fc419add0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:32.323 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:32.322+0000 7f4fc9704700 1 -- 172.21.15.72:0/2300457342 >> 172.21.15.72:0/2300457342 conn(0x7f4fc40fe880 msgr2=0x7f4fc4105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:32.324 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:32.322+0000 7f4fc9704700 1 -- 172.21.15.72:0/2300457342 shutdown_connections 2024-05-04T03:10:32.324 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:10:32.322+0000 7f4fc9704700 1 -- 172.21.15.72:0/2300457342 wait complete. 2024-05-04T03:10:32.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:32 smithi001 ceph-mon[29731]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:32.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:32 smithi077 ceph-mon[33531]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:33.023 DEBUG:teuthology.orchestra.run.smithi072:osd.1> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.1.service 2024-05-04T03:10:33.025 INFO:tasks.cephadm:Deploying osd.2 on smithi077 with /dev/vg_nvme/lv_4... 2024-05-04T03:10:33.025 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-04T03:10:33.272 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:10:33.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:33 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:33.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:33 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:33.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:33 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:33.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:33 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:33.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:33 smithi072 ceph-mon[33332]: from='osd.1 [v2:172.21.15.72:6800/3725706149,v1:172.21.15.72:6801/3725706149]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-04T03:10:33.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:33 smithi072 ceph-mon[33332]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-04T03:10:33.592 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:33 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:33.592 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:33 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:33.592 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:33 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:33.592 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:33 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:33.592 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:33 smithi077 ceph-mon[33531]: from='osd.1 [v2:172.21.15.72:6800/3725706149,v1:172.21.15.72:6801/3725706149]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-04T03:10:33.592 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:33 smithi077 ceph-mon[33531]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-04T03:10:33.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:33 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:33.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:33 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:33.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:33 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:33.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:33 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:33.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:33 smithi001 ceph-mon[29731]: from='osd.1 [v2:172.21.15.72:6800/3725706149,v1:172.21.15.72:6801/3725706149]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-04T03:10:33.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:33 smithi001 ceph-mon[29731]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-04T03:10:34.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:34 smithi001 ceph-mon[29731]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:34.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:34 smithi001 ceph-mon[29731]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-04T03:10:34.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:34 smithi001 ceph-mon[29731]: from='osd.1 [v2:172.21.15.72:6800/3725706149,v1:172.21.15.72:6801/3725706149]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-04T03:10:34.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:34 smithi001 ceph-mon[29731]: osdmap e11: 2 total, 1 up, 2 in 2024-05-04T03:10:34.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:34 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:10:34.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:34 smithi001 ceph-mon[29731]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-04T03:10:34.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:34 smithi077 ceph-mon[33531]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:34.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:34 smithi077 ceph-mon[33531]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-04T03:10:34.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:34 smithi077 ceph-mon[33531]: from='osd.1 [v2:172.21.15.72:6800/3725706149,v1:172.21.15.72:6801/3725706149]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-04T03:10:34.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:34 smithi077 ceph-mon[33531]: osdmap e11: 2 total, 1 up, 2 in 2024-05-04T03:10:34.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:34 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:10:34.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:34 smithi077 ceph-mon[33531]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-04T03:10:34.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:34 smithi072 ceph-mon[33332]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:34.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:34 smithi072 ceph-mon[33332]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-04T03:10:34.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:34 smithi072 ceph-mon[33332]: from='osd.1 [v2:172.21.15.72:6800/3725706149,v1:172.21.15.72:6801/3725706149]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-04T03:10:34.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:34 smithi072 ceph-mon[33332]: osdmap e11: 2 total, 1 up, 2 in 2024-05-04T03:10:34.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:34 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:10:34.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:34 smithi072 ceph-mon[33332]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-04T03:10:35.062 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-04T03:10:35.063 INFO:teuthology.orchestra.run.smithi077.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-04T03:10:35.063 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10+0 records in 2024-05-04T03:10:35.063 INFO:teuthology.orchestra.run.smithi077.stderr:10+0 records out 2024-05-04T03:10:35.063 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112784 s, 930 MB/s 2024-05-04T03:10:35.063 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping successful for: 2024-05-04T03:10:35.612 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 04 03:10:35 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-1[37925]: 2024-05-04T03:10:35.530+0000 7f6099f25700 -1 osd.1 0 waiting for initial osdmap 2024-05-04T03:10:35.630 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:35 smithi077 ceph-mon[33531]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-04T03:10:35.631 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:35 smithi077 ceph-mon[33531]: osdmap e12: 2 total, 1 up, 2 in 2024-05-04T03:10:35.631 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:35 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:10:35.631 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:35 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:10:35.631 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:35 smithi077 ceph-mon[33531]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:35.631 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:35 smithi077 ceph-mon[33531]: from='osd.1 ' entity='osd.1' 2024-05-04T03:10:35.632 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_4 2024-05-04T03:10:35.873 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:10:35.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:35 smithi001 ceph-mon[29731]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-04T03:10:35.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:35 smithi001 ceph-mon[29731]: osdmap e12: 2 total, 1 up, 2 in 2024-05-04T03:10:35.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:35 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:10:35.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:35 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:10:35.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:35 smithi001 ceph-mon[29731]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:35.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:35 smithi001 ceph-mon[29731]: from='osd.1 ' entity='osd.1' 2024-05-04T03:10:35.924 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 04 03:10:35 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-1[37925]: 2024-05-04T03:10:35.641+0000 7f6091cd2700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T03:10:35.924 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:35 smithi072 ceph-mon[33332]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-04T03:10:35.924 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:35 smithi072 ceph-mon[33332]: osdmap e12: 2 total, 1 up, 2 in 2024-05-04T03:10:35.925 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:35 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:10:35.925 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:35 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:10:35.925 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:35 smithi072 ceph-mon[33332]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:35.925 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:35 smithi072 ceph-mon[33332]: from='osd.1 ' entity='osd.1' 2024-05-04T03:10:36.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:36 smithi001 ceph-mon[29731]: purged_snaps scrub starts 2024-05-04T03:10:36.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:36 smithi001 ceph-mon[29731]: purged_snaps scrub ok 2024-05-04T03:10:36.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:10:36.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:36 smithi001 ceph-mon[29731]: Detected new or changed devices on smithi072 2024-05-04T03:10:36.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:36.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:36.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:10:36.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:36 smithi001 ceph-mon[29731]: Adjusting osd_memory_target on smithi072 to 17088M 2024-05-04T03:10:36.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:36.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:10:36.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:10:36.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:36.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:10:36.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:10:36.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:10:36.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:36.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:10:36.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:36 smithi077 ceph-mon[33531]: purged_snaps scrub starts 2024-05-04T03:10:36.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:36 smithi077 ceph-mon[33531]: purged_snaps scrub ok 2024-05-04T03:10:36.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:36 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:10:36.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:36 smithi077 ceph-mon[33531]: Detected new or changed devices on smithi072 2024-05-04T03:10:36.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:36 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:36.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:36 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:36.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:36 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:10:36.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:36 smithi077 ceph-mon[33531]: Adjusting osd_memory_target on smithi072 to 17088M 2024-05-04T03:10:36.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:36 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:36.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:36 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:10:36.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:36 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:10:36.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:36 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:36.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:36 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:10:36.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:36 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:10:36.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:36 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:10:36.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:36 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:36.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:36 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:10:36.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:36 smithi072 ceph-mon[33332]: purged_snaps scrub starts 2024-05-04T03:10:36.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:36 smithi072 ceph-mon[33332]: purged_snaps scrub ok 2024-05-04T03:10:36.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:36 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:10:36.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:36 smithi072 ceph-mon[33332]: Detected new or changed devices on smithi072 2024-05-04T03:10:36.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:36 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:36.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:36 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:36.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:36 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:10:36.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:36 smithi072 ceph-mon[33332]: Adjusting osd_memory_target on smithi072 to 17088M 2024-05-04T03:10:36.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:36 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:36.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:36 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:10:36.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:36 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:10:36.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:36 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:36.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:36 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:10:36.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:36 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:10:36.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:36 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:10:36.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:36 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:36.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:36 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:10:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.407+0000 7f14bc93b700 1 -- 172.21.15.77:0/291936086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14b8103240 msgr2=0x7f14b8103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.407+0000 7f14bc93b700 1 --2- 172.21.15.77:0/291936086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14b8103240 0x7f14b8103650 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009990 tx=0x7f14a8009ca0 comp rx=0 tx=0).stop 2024-05-04T03:10:37.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.407+0000 7f14bc93b700 1 -- 172.21.15.77:0/291936086 shutdown_connections 2024-05-04T03:10:37.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.407+0000 7f14bc93b700 1 --2- 172.21.15.77:0/291936086 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14b8104d10 0x7f14b810b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:37.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.407+0000 7f14bc93b700 1 --2- 172.21.15.77:0/291936086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14b8103240 0x7f14b8103650 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:37.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.407+0000 7f14bc93b700 1 --2- 172.21.15.77:0/291936086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14b8104370 0x7f14b8104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:37.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.407+0000 7f14bc93b700 1 -- 172.21.15.77:0/291936086 >> 172.21.15.77:0/291936086 conn(0x7f14b80fe880 msgr2=0x7f14b8100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:37.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.407+0000 7f14bc93b700 1 -- 172.21.15.77:0/291936086 shutdown_connections 2024-05-04T03:10:37.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.407+0000 7f14bc93b700 1 -- 172.21.15.77:0/291936086 wait complete. 2024-05-04T03:10:37.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.408+0000 7f14bc93b700 1 Processor -- start 2024-05-04T03:10:37.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.408+0000 7f14bc93b700 1 -- start start 2024-05-04T03:10:37.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.408+0000 7f14bc93b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14b8103240 0x7f14b81998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:37.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.408+0000 7f14bc93b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14b8104370 0x7f14b8199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:37.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.408+0000 7f14bc93b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14b8104d10 0x7f14b81a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:37.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.408+0000 7f14bc93b700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f14b810ecd0 con 0x7f14b8103240 2024-05-04T03:10:37.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.408+0000 7f14bc93b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f14b810eb50 con 0x7f14b8104d10 2024-05-04T03:10:37.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.408+0000 7f14bc93b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f14b810ee50 con 0x7f14b8104370 2024-05-04T03:10:37.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.408+0000 7f14b5d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14b8104370 0x7f14b8199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:37.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.408+0000 7f14b5d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14b8104370 0x7f14b8199de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36662/0 (socket says 172.21.15.77:36662) 2024-05-04T03:10:37.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.408+0000 7f14b5d9b700 1 -- 172.21.15.77:0/1169004895 learned_addr learned my addr 172.21.15.77:0/1169004895 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-04T03:10:37.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.408+0000 7f14b659c700 1 --2- 172.21.15.77:0/1169004895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14b8103240 0x7f14b81998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:37.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.408+0000 7f14b6d9d700 1 --2- 172.21.15.77:0/1169004895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14b8104d10 0x7f14b81a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:37.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.409+0000 7f14b5d9b700 1 -- 172.21.15.77:0/1169004895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14b8104d10 msgr2=0x7f14b81a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:37.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.409+0000 7f14b5d9b700 1 --2- 172.21.15.77:0/1169004895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14b8104d10 0x7f14b81a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:37.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.409+0000 7f14b5d9b700 1 -- 172.21.15.77:0/1169004895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14b8103240 msgr2=0x7f14b81998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:37.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.409+0000 7f14b5d9b700 1 --2- 172.21.15.77:0/1169004895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14b8103240 0x7f14b81998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:37.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.409+0000 7f14b5d9b700 1 -- 172.21.15.77:0/1169004895 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b81a1400 con 0x7f14b8104370 2024-05-04T03:10:37.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.409+0000 7f14b5d9b700 1 --2- 172.21.15.77:0/1169004895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14b8104370 0x7f14b8199de0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009970 tx=0x7f14ac009c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:37.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.409+0000 7f14a77fe700 1 -- 172.21.15.77:0/1169004895 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac01c070 con 0x7f14b8104370 2024-05-04T03:10:37.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.409+0000 7f14a77fe700 1 -- 172.21.15.77:0/1169004895 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14ac00d260 con 0x7f14b8104370 2024-05-04T03:10:37.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.409+0000 7f14bc93b700 1 -- 172.21.15.77:0/1169004895 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b806b670 con 0x7f14b8104370 2024-05-04T03:10:37.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.409+0000 7f14a77fe700 1 -- 172.21.15.77:0/1169004895 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac017a60 con 0x7f14b8104370 2024-05-04T03:10:37.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.409+0000 7f14bc93b700 1 -- 172.21.15.77:0/1169004895 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b806bbb0 con 0x7f14b8104370 2024-05-04T03:10:37.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.410+0000 7f14bc93b700 1 -- 172.21.15.77:0/1169004895 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14b80512c0 con 0x7f14b8104370 2024-05-04T03:10:37.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.411+0000 7f14a77fe700 1 -- 172.21.15.77:0/1169004895 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 15) ==== 149595+0+0 (secure 0 0 0) 0x7f14ac01f070 con 0x7f14b8104370 2024-05-04T03:10:37.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.412+0000 7f14a77fe700 1 --2- 172.21.15.77:0/1169004895 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f14a00b27f0 0x7f14a00b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:10:37.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.412+0000 7f14a77fe700 1 -- 172.21.15.77:0/1169004895 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7f14ac0e1ae0 con 0x7f14b8104370 2024-05-04T03:10:37.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.412+0000 7f14b659c700 1 --2- 172.21.15.77:0/1169004895 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f14a00b27f0 0x7f14a00b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:10:37.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.413+0000 7f14b659c700 1 --2- 172.21.15.77:0/1169004895 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f14a00b27f0 0x7f14a00b4d00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f14a80038a0 tx=0x7f14a801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:10:37.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.417+0000 7f14a77fe700 1 -- 172.21.15.77:0/1169004895 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f14ac0ab050 con 0x7f14b8104370 2024-05-04T03:10:37.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:37.632+0000 7f14bc93b700 1 -- 172.21.15.77:0/1169004895 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f14b806be70 con 0x7f14a00b27f0 2024-05-04T03:10:37.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:37 smithi001 ceph-mon[29731]: osd.1 [v2:172.21.15.72:6800/3725706149,v1:172.21.15.72:6801/3725706149] boot 2024-05-04T03:10:37.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:37 smithi001 ceph-mon[29731]: osdmap e13: 2 total, 2 up, 2 in 2024-05-04T03:10:37.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:37 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:10:37.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:37 smithi001 ceph-mon[29731]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:37.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:37 smithi077 ceph-mon[33531]: osd.1 [v2:172.21.15.72:6800/3725706149,v1:172.21.15.72:6801/3725706149] boot 2024-05-04T03:10:37.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:37 smithi077 ceph-mon[33531]: osdmap e13: 2 total, 2 up, 2 in 2024-05-04T03:10:37.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:37 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:10:37.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:37 smithi077 ceph-mon[33531]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:37.995 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:37 smithi072 ceph-mon[33332]: osd.1 [v2:172.21.15.72:6800/3725706149,v1:172.21.15.72:6801/3725706149] boot 2024-05-04T03:10:37.995 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:37 smithi072 ceph-mon[33332]: osdmap e13: 2 total, 2 up, 2 in 2024-05-04T03:10:37.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:37 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-04T03:10:37.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:37 smithi072 ceph-mon[33332]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-04T03:10:38.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:38 smithi001 ceph-mon[29731]: osdmap e14: 2 total, 2 up, 2 in 2024-05-04T03:10:38.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:38 smithi001 ceph-mon[29731]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:10:38.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:38 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:10:38.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:38 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:10:38.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:38 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:10:38.891 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:38 smithi077 ceph-mon[33531]: osdmap e14: 2 total, 2 up, 2 in 2024-05-04T03:10:38.892 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:38 smithi077 ceph-mon[33531]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:10:38.892 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:38 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:10:38.892 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:38 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:10:38.892 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:38 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:10:38.995 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:38 smithi072 ceph-mon[33332]: osdmap e14: 2 total, 2 up, 2 in 2024-05-04T03:10:38.995 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:38 smithi072 ceph-mon[33332]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:10:38.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:38 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:10:38.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:38 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:10:38.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:38 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:10:39.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:39 smithi077 ceph-mon[33531]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:39.995 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:39 smithi072 ceph-mon[33332]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:40.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:39 smithi001 ceph-mon[29731]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:40.930 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:40 smithi077 ceph-mon[33531]: from='client.? 172.21.15.77:0/2730254300' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "923d76e5-b032-4190-9e35-546d27722a3c"}]: dispatch 2024-05-04T03:10:40.930 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:40 smithi077 ceph-mon[33531]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "923d76e5-b032-4190-9e35-546d27722a3c"}]: dispatch 2024-05-04T03:10:40.930 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:40 smithi077 ceph-mon[33531]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "923d76e5-b032-4190-9e35-546d27722a3c"}]': finished 2024-05-04T03:10:40.930 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:40 smithi077 ceph-mon[33531]: osdmap e15: 3 total, 2 up, 3 in 2024-05-04T03:10:40.931 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:40 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:10:40.995 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:40 smithi072 ceph-mon[33332]: from='client.? 172.21.15.77:0/2730254300' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "923d76e5-b032-4190-9e35-546d27722a3c"}]: dispatch 2024-05-04T03:10:40.995 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:40 smithi072 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "923d76e5-b032-4190-9e35-546d27722a3c"}]: dispatch 2024-05-04T03:10:40.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:40 smithi072 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "923d76e5-b032-4190-9e35-546d27722a3c"}]': finished 2024-05-04T03:10:40.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:40 smithi072 ceph-mon[33332]: osdmap e15: 3 total, 2 up, 3 in 2024-05-04T03:10:40.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:40 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:10:41.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:40 smithi001 ceph-mon[29731]: from='client.? 172.21.15.77:0/2730254300' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "923d76e5-b032-4190-9e35-546d27722a3c"}]: dispatch 2024-05-04T03:10:41.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:40 smithi001 ceph-mon[29731]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "923d76e5-b032-4190-9e35-546d27722a3c"}]: dispatch 2024-05-04T03:10:41.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:40 smithi001 ceph-mon[29731]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "923d76e5-b032-4190-9e35-546d27722a3c"}]': finished 2024-05-04T03:10:41.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:40 smithi001 ceph-mon[29731]: osdmap e15: 3 total, 2 up, 3 in 2024-05-04T03:10:41.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:40 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:10:41.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:41 smithi077 ceph-mon[33531]: from='client.? 172.21.15.77:0/1949642608' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:10:41.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:41 smithi077 ceph-mon[33531]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:41.995 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:41 smithi072 ceph-mon[33332]: from='client.? 172.21.15.77:0/1949642608' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:10:41.995 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:41 smithi072 ceph-mon[33332]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:42.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:41 smithi001 ceph-mon[29731]: from='client.? 172.21.15.77:0/1949642608' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:10:42.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:41 smithi001 ceph-mon[29731]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:44.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:44 smithi001 ceph-mon[29731]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:44.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:44 smithi077 ceph-mon[33531]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:44.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:44 smithi072 ceph-mon[33332]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:46.396 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:46 smithi077 ceph-mon[33531]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:46.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:46 smithi072 ceph-mon[33332]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:46.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:46 smithi001 ceph-mon[29731]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:48.398 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:48 smithi077 ceph-mon[33531]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:48.398 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:48 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-04T03:10:48.398 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:48 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:10:48.398 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:48 smithi077 ceph-mon[33531]: Deploying daemon osd.2 on smithi077 2024-05-04T03:10:48.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:48 smithi072 ceph-mon[33332]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:48.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:48 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-04T03:10:48.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:48 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:10:48.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:48 smithi072 ceph-mon[33332]: Deploying daemon osd.2 on smithi077 2024-05-04T03:10:48.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:48 smithi001 ceph-mon[29731]: pgmap v72: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:48.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:48 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-04T03:10:48.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:48 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:10:48.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:48 smithi001 ceph-mon[29731]: Deploying daemon osd.2 on smithi077 2024-05-04T03:10:50.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:50 smithi077 ceph-mon[33531]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:50.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:50 smithi072 ceph-mon[33332]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:50.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:50 smithi001 ceph-mon[29731]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:52.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:52 smithi077 ceph-mon[33531]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:52.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:52 smithi072 ceph-mon[33332]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:52.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:52 smithi001 ceph-mon[29731]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:54.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:53 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:54.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:53 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:10:54.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:53 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:54.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:53 smithi001 ceph-mon[29731]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:54.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:53 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:54.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:53 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:10:54.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:53 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:54.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:53 smithi077 ceph-mon[33531]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:54.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:53 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:54.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:53 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:10:54.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:53 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:54.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:53 smithi072 ceph-mon[33332]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:56.004 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 2 on host 'smithi077' 2024-05-04T03:10:56.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:56.002+0000 7f14a77fe700 1 -- 172.21.15.77:0/1169004895 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f14b806be70 con 0x7f14a00b27f0 2024-05-04T03:10:56.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:56.005+0000 7f14bc93b700 1 -- 172.21.15.77:0/1169004895 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f14a00b27f0 msgr2=0x7f14a00b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:56.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:56.005+0000 7f14bc93b700 1 --2- 172.21.15.77:0/1169004895 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f14a00b27f0 0x7f14a00b4d00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f14a80038a0 tx=0x7f14a801a040 comp rx=0 tx=0).stop 2024-05-04T03:10:56.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:56.005+0000 7f14bc93b700 1 -- 172.21.15.77:0/1169004895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14b8104370 msgr2=0x7f14b8199de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:10:56.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:56.005+0000 7f14bc93b700 1 --2- 172.21.15.77:0/1169004895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14b8104370 0x7f14b8199de0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009970 tx=0x7f14ac009c80 comp rx=0 tx=0).stop 2024-05-04T03:10:56.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:56.005+0000 7f14bc93b700 1 -- 172.21.15.77:0/1169004895 shutdown_connections 2024-05-04T03:10:56.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:56.005+0000 7f14bc93b700 1 --2- 172.21.15.77:0/1169004895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14b8104d10 0x7f14b81a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:56.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:56.005+0000 7f14bc93b700 1 --2- 172.21.15.77:0/1169004895 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f14a00b27f0 0x7f14a00b4d00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:56.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:56.005+0000 7f14bc93b700 1 --2- 172.21.15.77:0/1169004895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14b8103240 0x7f14b81998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:56.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:56.005+0000 7f14bc93b700 1 --2- 172.21.15.77:0/1169004895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14b8104370 0x7f14b8199de0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:10:56.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:56.005+0000 7f14bc93b700 1 -- 172.21.15.77:0/1169004895 >> 172.21.15.77:0/1169004895 conn(0x7f14b80fe880 msgr2=0x7f14b81075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:10:56.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:56.005+0000 7f14bc93b700 1 -- 172.21.15.77:0/1169004895 shutdown_connections 2024-05-04T03:10:56.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:10:56.005+0000 7f14bc93b700 1 -- 172.21.15.77:0/1169004895 wait complete. 2024-05-04T03:10:56.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:56 smithi077 ceph-mon[33531]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:56.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:56 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:56.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:56 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:56.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:56 smithi001 ceph-mon[29731]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:56.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:56 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:56.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:56 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:56.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:56 smithi072 ceph-mon[33332]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:56.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:56 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:56.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:56 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:56.608 DEBUG:teuthology.orchestra.run.smithi077:osd.2> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.2.service 2024-05-04T03:10:56.610 INFO:tasks.cephadm:Deploying osd.3 on smithi001 with /dev/vg_nvme/lv_3... 2024-05-04T03:10:56.611 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-04T03:10:56.867 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:10:56.927 INFO:journalctl@ceph.osd.2.smithi077.stdout:May 04 03:10:56 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-2[38578]: 2024-05-04T03:10:56.748+0000 7f8b975ba7c0 -1 osd.2 0 log_to_monitors true 2024-05-04T03:10:57.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:57 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:57.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:57 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:57.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:57 smithi077 ceph-mon[33531]: from='osd.2 [v2:172.21.15.77:6800/515038343,v1:172.21.15.77:6801/515038343]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-04T03:10:57.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:57 smithi077 ceph-mon[33531]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-04T03:10:57.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:57 smithi077 ceph-mon[33531]: pgmap v77: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:57.995 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:57 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:57.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:57 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:57.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:57 smithi072 ceph-mon[33332]: from='osd.2 [v2:172.21.15.77:6800/515038343,v1:172.21.15.77:6801/515038343]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-04T03:10:57.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:57 smithi072 ceph-mon[33332]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-04T03:10:57.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:57 smithi072 ceph-mon[33332]: pgmap v77: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:57.996 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:57 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:57.996 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:57 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:10:57.996 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:57 smithi001 ceph-mon[29731]: from='osd.2 [v2:172.21.15.77:6800/515038343,v1:172.21.15.77:6801/515038343]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-04T03:10:57.997 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:57 smithi001 ceph-mon[29731]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-04T03:10:57.997 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:57 smithi001 ceph-mon[29731]: pgmap v77: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:58.846 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-04T03:10:58.846 INFO:teuthology.orchestra.run.smithi001.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-04T03:10:58.847 INFO:teuthology.orchestra.run.smithi001.stderr: stderr: 10+0 records in 2024-05-04T03:10:58.847 INFO:teuthology.orchestra.run.smithi001.stderr:10+0 records out 2024-05-04T03:10:58.847 INFO:teuthology.orchestra.run.smithi001.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0085732 s, 1.2 GB/s 2024-05-04T03:10:58.847 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping successful for: 2024-05-04T03:10:58.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:58 smithi001 ceph-mon[29731]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-04T03:10:58.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:58 smithi001 ceph-mon[29731]: osdmap e16: 3 total, 2 up, 3 in 2024-05-04T03:10:58.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:58 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:10:58.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:58 smithi001 ceph-mon[29731]: from='osd.2 [v2:172.21.15.77:6800/515038343,v1:172.21.15.77:6801/515038343]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-04T03:10:58.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:58 smithi001 ceph-mon[29731]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-04T03:10:58.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:58 smithi072 ceph-mon[33332]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-04T03:10:58.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:58 smithi072 ceph-mon[33332]: osdmap e16: 3 total, 2 up, 3 in 2024-05-04T03:10:58.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:58 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:10:58.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:58 smithi072 ceph-mon[33332]: from='osd.2 [v2:172.21.15.77:6800/515038343,v1:172.21.15.77:6801/515038343]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-04T03:10:58.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:58 smithi072 ceph-mon[33332]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-04T03:10:59.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:58 smithi077 ceph-mon[33531]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-04T03:10:59.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:58 smithi077 ceph-mon[33531]: osdmap e16: 3 total, 2 up, 3 in 2024-05-04T03:10:59.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:58 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:10:59.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:58 smithi077 ceph-mon[33531]: from='osd.2 [v2:172.21.15.77:6800/515038343,v1:172.21.15.77:6801/515038343]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-04T03:10:59.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:58 smithi077 ceph-mon[33531]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-04T03:10:59.404 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi001:vg_nvme/lv_3 2024-05-04T03:10:59.656 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:10:59.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:59 smithi077 ceph-mon[33531]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-05-04T03:10:59.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:59 smithi077 ceph-mon[33531]: osdmap e17: 3 total, 2 up, 3 in 2024-05-04T03:10:59.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:59 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:10:59.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:59 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:10:59.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:59 smithi077 ceph-mon[33531]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:59.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:10:59 smithi077 ceph-mon[33531]: from='osd.2 ' entity='osd.2' 2024-05-04T03:10:59.928 INFO:journalctl@ceph.osd.2.smithi077.stdout:May 04 03:10:59 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-2[38578]: 2024-05-04T03:10:59.634+0000 7f8b8cc1e700 -1 osd.2 0 waiting for initial osdmap 2024-05-04T03:10:59.928 INFO:journalctl@ceph.osd.2.smithi077.stdout:May 04 03:10:59 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-2[38578]: 2024-05-04T03:10:59.745+0000 7f8b849cb700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T03:10:59.995 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:59 smithi072 ceph-mon[33332]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-05-04T03:10:59.995 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:59 smithi072 ceph-mon[33332]: osdmap e17: 3 total, 2 up, 3 in 2024-05-04T03:10:59.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:59 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:10:59.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:59 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:10:59.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:59 smithi072 ceph-mon[33332]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:10:59.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:10:59 smithi072 ceph-mon[33332]: from='osd.2 ' entity='osd.2' 2024-05-04T03:11:00.008 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:59 smithi001 ceph-mon[29731]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-05-04T03:11:00.008 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:59 smithi001 ceph-mon[29731]: osdmap e17: 3 total, 2 up, 3 in 2024-05-04T03:11:00.008 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:59 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:11:00.008 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:59 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:11:00.008 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:59 smithi001 ceph-mon[29731]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:11:00.009 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:10:59 smithi001 ceph-mon[29731]: from='osd.2 ' entity='osd.2' 2024-05-04T03:11:00.754 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:00 smithi001 ceph-mon[29731]: purged_snaps scrub starts 2024-05-04T03:11:00.754 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:00 smithi001 ceph-mon[29731]: purged_snaps scrub ok 2024-05-04T03:11:00.754 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:00 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:11:00.754 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:00 smithi001 ceph-mon[29731]: Detected new or changed devices on smithi077 2024-05-04T03:11:00.754 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:00 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:00.754 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:00 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:00.754 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:00 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:11:00.754 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:00 smithi001 ceph-mon[29731]: Adjusting osd_memory_target on smithi077 to 17088M 2024-05-04T03:11:00.754 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:00 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:00.754 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:00 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:00.754 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:00 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:11:00.755 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:00 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:00.755 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:00 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:11:00.755 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:00 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:00.755 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:00 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:11:00.755 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:00 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:00.755 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:00 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:11:00.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:00 smithi072 ceph-mon[33332]: purged_snaps scrub starts 2024-05-04T03:11:00.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:00 smithi072 ceph-mon[33332]: purged_snaps scrub ok 2024-05-04T03:11:00.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:00 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:11:00.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:00 smithi072 ceph-mon[33332]: Detected new or changed devices on smithi077 2024-05-04T03:11:00.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:00 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:00.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:00 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:00.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:00 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:11:00.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:00 smithi072 ceph-mon[33332]: Adjusting osd_memory_target on smithi077 to 17088M 2024-05-04T03:11:00.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:00 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:00.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:00 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:00.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:00 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:11:00.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:00 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:00.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:00 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:11:00.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:00 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:00.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:00 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:11:00.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:00 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:00.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:00 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:11:01.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:00 smithi077 ceph-mon[33531]: purged_snaps scrub starts 2024-05-04T03:11:01.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:00 smithi077 ceph-mon[33531]: purged_snaps scrub ok 2024-05-04T03:11:01.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:00 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:11:01.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:00 smithi077 ceph-mon[33531]: Detected new or changed devices on smithi077 2024-05-04T03:11:01.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:00 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:01.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:00 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:01.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:00 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:11:01.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:00 smithi077 ceph-mon[33531]: Adjusting osd_memory_target on smithi077 to 17088M 2024-05-04T03:11:01.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:00 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:01.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:00 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:01.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:00 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:11:01.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:00 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:01.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:00 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:11:01.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:00 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:01.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:00 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:11:01.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:00 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:01.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:00 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:11:01.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.224+0000 7f6eca0c0700 1 -- 172.21.15.1:0/72507723 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ec4103230 msgr2=0x7f6ec4103640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:01.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.224+0000 7f6eca0c0700 1 --2- 172.21.15.1:0/72507723 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ec4103230 0x7f6ec4103640 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009990 tx=0x7f6eb4009ca0 comp rx=0 tx=0).stop 2024-05-04T03:11:01.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.224+0000 7f6eca0c0700 1 -- 172.21.15.1:0/72507723 shutdown_connections 2024-05-04T03:11:01.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.224+0000 7f6eca0c0700 1 --2- 172.21.15.1:0/72507723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ec4104d00 0x7f6ec410b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:01.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.224+0000 7f6eca0c0700 1 --2- 172.21.15.1:0/72507723 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ec4103230 0x7f6ec4103640 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:01.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.224+0000 7f6eca0c0700 1 --2- 172.21.15.1:0/72507723 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ec4104360 0x7f6ec4104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:01.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.224+0000 7f6eca0c0700 1 -- 172.21.15.1:0/72507723 >> 172.21.15.1:0/72507723 conn(0x7f6ec40fe870 msgr2=0x7f6ec4100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:01.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.224+0000 7f6eca0c0700 1 -- 172.21.15.1:0/72507723 shutdown_connections 2024-05-04T03:11:01.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.224+0000 7f6eca0c0700 1 -- 172.21.15.1:0/72507723 wait complete. 2024-05-04T03:11:01.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.225+0000 7f6eca0c0700 1 Processor -- start 2024-05-04T03:11:01.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.225+0000 7f6eca0c0700 1 -- start start 2024-05-04T03:11:01.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.225+0000 7f6eca0c0700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ec4103230 0x7f6ec419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:01.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.225+0000 7f6eca0c0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ec4104360 0x7f6ec419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:01.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.225+0000 7f6eca0c0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ec4104d00 0x7f6ec41a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:01.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.225+0000 7f6eca0c0700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec410fdd0 con 0x7f6ec4104d00 2024-05-04T03:11:01.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.225+0000 7f6eca0c0700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec410fc50 con 0x7f6ec4103230 2024-05-04T03:11:01.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.225+0000 7f6eca0c0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec410ff50 con 0x7f6ec4104360 2024-05-04T03:11:01.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.225+0000 7f6ec3fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ec4104d00 0x7f6ec41a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:01.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.225+0000 7f6ec3fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ec4104d00 0x7f6ec41a1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58298/0 (socket says 172.21.15.1:58298) 2024-05-04T03:11:01.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.225+0000 7f6ec3fff700 1 -- 172.21.15.1:0/3432957319 learned_addr learned my addr 172.21.15.1:0/3432957319 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:11:01.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.225+0000 7f6ec2ffd700 1 --2- 172.21.15.1:0/3432957319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ec4104360 0x7f6ec419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:01.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.226+0000 7f6ec37fe700 1 --2- 172.21.15.1:0/3432957319 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ec4103230 0x7f6ec419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:01.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.226+0000 7f6ec3fff700 1 -- 172.21.15.1:0/3432957319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ec4104360 msgr2=0x7f6ec419ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:01.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.226+0000 7f6ec3fff700 1 --2- 172.21.15.1:0/3432957319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ec4104360 0x7f6ec419ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:01.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.226+0000 7f6ec3fff700 1 -- 172.21.15.1:0/3432957319 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ec4103230 msgr2=0x7f6ec419a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:01.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.226+0000 7f6ec3fff700 1 --2- 172.21.15.1:0/3432957319 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ec4103230 0x7f6ec419a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:01.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.226+0000 7f6ec3fff700 1 -- 172.21.15.1:0/3432957319 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec41a2480 con 0x7f6ec4104d00 2024-05-04T03:11:01.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.226+0000 7f6ec2ffd700 1 --2- 172.21.15.1:0/3432957319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ec4104360 0x7f6ec419ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:11:01.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.226+0000 7f6ec3fff700 1 --2- 172.21.15.1:0/3432957319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ec4104d00 0x7f6ec41a1e00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc007ee0 tx=0x7f6ebc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:01.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.226+0000 7f6ec0ff9700 1 -- 172.21.15.1:0/3432957319 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc00dbd0 con 0x7f6ec4104d00 2024-05-04T03:11:01.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.226+0000 7f6eca0c0700 1 -- 172.21.15.1:0/3432957319 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ec41a2770 con 0x7f6ec4104d00 2024-05-04T03:11:01.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.226+0000 7f6ec0ff9700 1 -- 172.21.15.1:0/3432957319 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ebc00dd70 con 0x7f6ec4104d00 2024-05-04T03:11:01.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.226+0000 7f6ec0ff9700 1 -- 172.21.15.1:0/3432957319 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc005990 con 0x7f6ec4104d00 2024-05-04T03:11:01.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.226+0000 7f6eca0c0700 1 -- 172.21.15.1:0/3432957319 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ec41a2c60 con 0x7f6ec4104d00 2024-05-04T03:11:01.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.227+0000 7f6eca0c0700 1 -- 172.21.15.1:0/3432957319 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ec40512c0 con 0x7f6ec4104d00 2024-05-04T03:11:01.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.229+0000 7f6ec0ff9700 1 -- 172.21.15.1:0/3432957319 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 15) ==== 149595+0+0 (secure 0 0 0) 0x7f6ebc01a020 con 0x7f6ec4104d00 2024-05-04T03:11:01.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.230+0000 7f6ec0ff9700 1 --2- 172.21.15.1:0/3432957319 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6eac0b27f0 0x7f6eac0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:01.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.230+0000 7f6ec37fe700 1 --2- 172.21.15.1:0/3432957319 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6eac0b27f0 0x7f6eac0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:01.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.230+0000 7f6ec0ff9700 1 -- 172.21.15.1:0/3432957319 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2851+0+0 (secure 0 0 0) 0x7f6ebc0e0bb0 con 0x7f6ec4104d00 2024-05-04T03:11:01.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.230+0000 7f6ec37fe700 1 --2- 172.21.15.1:0/3432957319 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6eac0b27f0 0x7f6eac0b4d00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6eb40038a0 tx=0x7f6eb401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:01.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.233+0000 7f6ec0ff9700 1 -- 172.21.15.1:0/3432957319 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6ebc010070 con 0x7f6ec4104d00 2024-05-04T03:11:01.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:01.457+0000 7f6eca0c0700 1 -- 172.21.15.1:0/3432957319 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6ec4063c00 con 0x7f6eac0b27f0 2024-05-04T03:11:01.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:01 smithi077 ceph-mon[33531]: osd.2 [v2:172.21.15.77:6800/515038343,v1:172.21.15.77:6801/515038343] boot 2024-05-04T03:11:01.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:01 smithi077 ceph-mon[33531]: osdmap e18: 3 total, 3 up, 3 in 2024-05-04T03:11:01.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:01 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:11:01.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:01 smithi077 ceph-mon[33531]: pgmap v82: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:11:01.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:01 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-04T03:11:01.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:01 smithi077 ceph-mon[33531]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:11:01.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:01 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:11:01.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:01 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:11:01.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:01 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:01.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:01 smithi072 ceph-mon[33332]: osd.2 [v2:172.21.15.77:6800/515038343,v1:172.21.15.77:6801/515038343] boot 2024-05-04T03:11:01.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:01 smithi072 ceph-mon[33332]: osdmap e18: 3 total, 3 up, 3 in 2024-05-04T03:11:01.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:01 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:11:01.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:01 smithi072 ceph-mon[33332]: pgmap v82: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:11:01.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:01 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-04T03:11:01.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:01 smithi072 ceph-mon[33332]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:11:01.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:01 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:11:01.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:01 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:11:01.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:01 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:02.093 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:01 smithi001 ceph-mon[29731]: osd.2 [v2:172.21.15.77:6800/515038343,v1:172.21.15.77:6801/515038343] boot 2024-05-04T03:11:02.093 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:01 smithi001 ceph-mon[29731]: osdmap e18: 3 total, 3 up, 3 in 2024-05-04T03:11:02.093 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:01 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-04T03:11:02.094 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:01 smithi001 ceph-mon[29731]: pgmap v82: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-05-04T03:11:02.094 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:01 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-04T03:11:02.094 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:01 smithi001 ceph-mon[29731]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:11:02.094 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:01 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:11:02.094 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:01 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:11:02.094 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:01 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:02.995 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:02 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-04T03:11:02.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:02 smithi072 ceph-mon[33332]: osdmap e19: 3 total, 3 up, 3 in 2024-05-04T03:11:02.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:02 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-04T03:11:02.996 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 04 03:11:02 smithi072 sudo[41620]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-04T03:11:02.996 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 04 03:11:02 smithi072 sudo[41620]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-04T03:11:02.996 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 04 03:11:02 smithi072 sudo[41620]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-04T03:11:02.996 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 04 03:11:02 smithi072 sudo[41620]: pam_unix(sudo:session): session closed for user root 2024-05-04T03:11:02.996 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 04 03:11:02 smithi072 sudo[41623]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-04T03:11:02.996 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 04 03:11:02 smithi072 sudo[41623]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-04T03:11:02.996 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 04 03:11:02 smithi072 sudo[41623]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-04T03:11:02.996 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 04 03:11:02 smithi072 sudo[41623]: pam_unix(sudo:session): session closed for user root 2024-05-04T03:11:02.997 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:02 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-04T03:11:02.997 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:02 smithi001 ceph-mon[29731]: osdmap e19: 3 total, 3 up, 3 in 2024-05-04T03:11:02.997 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:02 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-04T03:11:02.997 INFO:journalctl@ceph.osd.0.smithi001.stdout:May 04 03:11:02 smithi001 sudo[46693]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-04T03:11:02.997 INFO:journalctl@ceph.osd.0.smithi001.stdout:May 04 03:11:02 smithi001 sudo[46693]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-04T03:11:02.997 INFO:journalctl@ceph.osd.0.smithi001.stdout:May 04 03:11:02 smithi001 sudo[46693]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-04T03:11:02.997 INFO:journalctl@ceph.osd.0.smithi001.stdout:May 04 03:11:02 smithi001 sudo[46693]: pam_unix(sudo:session): session closed for user root 2024-05-04T03:11:02.998 INFO:journalctl@ceph.osd.0.smithi001.stdout:May 04 03:11:02 smithi001 sudo[46696]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-04T03:11:02.998 INFO:journalctl@ceph.osd.0.smithi001.stdout:May 04 03:11:02 smithi001 sudo[46696]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-04T03:11:02.998 INFO:journalctl@ceph.osd.0.smithi001.stdout:May 04 03:11:02 smithi001 sudo[46696]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-04T03:11:02.998 INFO:journalctl@ceph.osd.0.smithi001.stdout:May 04 03:11:02 smithi001 sudo[46696]: pam_unix(sudo:session): session closed for user root 2024-05-04T03:11:03.036 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:02 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-04T03:11:03.036 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:02 smithi077 ceph-mon[33531]: osdmap e19: 3 total, 3 up, 3 in 2024-05-04T03:11:03.036 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:02 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-04T03:11:03.037 INFO:journalctl@ceph.osd.2.smithi077.stdout:May 04 03:11:02 smithi077 sudo[42200]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-04T03:11:03.037 INFO:journalctl@ceph.osd.2.smithi077.stdout:May 04 03:11:02 smithi077 sudo[42200]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-04T03:11:03.037 INFO:journalctl@ceph.osd.2.smithi077.stdout:May 04 03:11:02 smithi077 sudo[42200]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-04T03:11:03.037 INFO:journalctl@ceph.osd.2.smithi077.stdout:May 04 03:11:02 smithi077 sudo[42200]: pam_unix(sudo:session): session closed for user root 2024-05-04T03:11:03.037 INFO:journalctl@ceph.osd.2.smithi077.stdout:May 04 03:11:02 smithi077 sudo[42203]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-04T03:11:03.037 INFO:journalctl@ceph.osd.2.smithi077.stdout:May 04 03:11:02 smithi077 sudo[42203]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-04T03:11:03.037 INFO:journalctl@ceph.osd.2.smithi077.stdout:May 04 03:11:02 smithi077 sudo[42203]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-04T03:11:03.037 INFO:journalctl@ceph.osd.2.smithi077.stdout:May 04 03:11:02 smithi077 sudo[42203]: pam_unix(sudo:session): session closed for user root 2024-05-04T03:11:03.294 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:02 smithi001 sudo[46699]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-04T03:11:03.295 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:02 smithi001 sudo[46699]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-04T03:11:03.295 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:02 smithi001 sudo[46699]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-04T03:11:03.295 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:03 smithi001 sudo[46699]: pam_unix(sudo:session): session closed for user root 2024-05-04T03:11:03.295 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:03 smithi001 sudo[46702]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-04T03:11:03.295 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:03 smithi001 sudo[46702]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-04T03:11:03.295 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:03 smithi001 sudo[46702]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-04T03:11:03.295 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:03 smithi001 sudo[46702]: pam_unix(sudo:session): session closed for user root 2024-05-04T03:11:03.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 sudo[42206]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-04T03:11:03.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 sudo[42206]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-04T03:11:03.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 sudo[42206]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-04T03:11:03.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 sudo[42206]: pam_unix(sudo:session): session closed for user root 2024-05-04T03:11:03.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 sudo[42209]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-04T03:11:03.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 sudo[42209]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-04T03:11:03.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 sudo[42209]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-04T03:11:03.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 sudo[42209]: pam_unix(sudo:session): session closed for user root 2024-05-04T03:11:03.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 sudo[41626]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-04T03:11:03.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 sudo[41626]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-04T03:11:03.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 sudo[41626]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-04T03:11:03.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 sudo[41626]: pam_unix(sudo:session): session closed for user root 2024-05-04T03:11:03.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 sudo[41629]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-04T03:11:03.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 sudo[41629]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-04T03:11:03.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 sudo[41629]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-04T03:11:03.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 sudo[41629]: pam_unix(sudo:session): session closed for user root 2024-05-04T03:11:03.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-04T03:11:03.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 ceph-mon[33332]: osdmap e20: 3 total, 3 up, 3 in 2024-05-04T03:11:03.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:03.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:03.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:03.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 ceph-mon[33332]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-04T03:11:03.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 ceph-mon[33332]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-04T03:11:03.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:11:03.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:11:03.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:11:03.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 ceph-mon[33332]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-04T03:11:03.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:11:03.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:11:03.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:11:03.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 ceph-mon[33332]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-04T03:11:03.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 ceph-mon[33332]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-04T03:11:03.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:11:03.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:11:03.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:11:03.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 ceph-mon[33332]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:03.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 ceph-mon[33332]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-04T03:11:03.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:03 smithi072 ceph-mon[33332]: osdmap e21: 3 total, 3 up, 3 in 2024-05-04T03:11:04.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:03.999+0000 7f6ec0ff9700 1 -- 172.21.15.1:0/3432957319 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f6ebc0a59f0 con 0x7f6ec4104d00 2024-05-04T03:11:04.002 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-04T03:11:04.002 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:03 smithi001 ceph-mon[29731]: osdmap e20: 3 total, 3 up, 3 in 2024-05-04T03:11:04.002 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:04.002 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:04.003 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:04.003 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:03 smithi001 ceph-mon[29731]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-04T03:11:04.003 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:03 smithi001 ceph-mon[29731]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-04T03:11:04.003 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:11:04.003 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:11:04.003 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:11:04.003 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:03 smithi001 ceph-mon[29731]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-04T03:11:04.003 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:11:04.003 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:11:04.003 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:11:04.003 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:03 smithi001 ceph-mon[29731]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-04T03:11:04.003 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:03 smithi001 ceph-mon[29731]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-04T03:11:04.003 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:11:04.003 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:11:04.004 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:11:04.004 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:03 smithi001 ceph-mon[29731]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:04.004 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:03 smithi001 ceph-mon[29731]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-04T03:11:04.004 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:03 smithi001 ceph-mon[29731]: osdmap e21: 3 total, 3 up, 3 in 2024-05-04T03:11:04.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-04T03:11:04.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 ceph-mon[33531]: osdmap e20: 3 total, 3 up, 3 in 2024-05-04T03:11:04.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:04.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:04.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:04.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 ceph-mon[33531]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-04T03:11:04.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 ceph-mon[33531]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-04T03:11:04.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:11:04.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:11:04.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:11:04.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 ceph-mon[33531]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-04T03:11:04.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:11:04.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:11:04.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:11:04.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 ceph-mon[33531]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-04T03:11:04.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 ceph-mon[33531]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-04T03:11:04.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-04T03:11:04.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-04T03:11:04.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-04T03:11:04.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 ceph-mon[33531]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:04.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 ceph-mon[33531]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-04T03:11:04.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:03 smithi077 ceph-mon[33531]: osdmap e21: 3 total, 3 up, 3 in 2024-05-04T03:11:05.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:04 smithi072 ceph-mon[33332]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-04T03:11:05.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:04 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/3489814461' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "495cc0c5-6a99-481f-ad9e-38f05fb84356"}]: dispatch 2024-05-04T03:11:05.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:04 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/3489814461' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "495cc0c5-6a99-481f-ad9e-38f05fb84356"}]': finished 2024-05-04T03:11:05.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:04 smithi072 ceph-mon[33332]: osdmap e22: 4 total, 3 up, 4 in 2024-05-04T03:11:05.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:04 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:11:05.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:04 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/492085' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:11:05.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:04 smithi001 ceph-mon[29731]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-04T03:11:05.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:04 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3489814461' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "495cc0c5-6a99-481f-ad9e-38f05fb84356"}]: dispatch 2024-05-04T03:11:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:04 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3489814461' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "495cc0c5-6a99-481f-ad9e-38f05fb84356"}]': finished 2024-05-04T03:11:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:04 smithi001 ceph-mon[29731]: osdmap e22: 4 total, 3 up, 4 in 2024-05-04T03:11:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:04 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:11:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:04 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/492085' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:11:05.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:04 smithi077 ceph-mon[33531]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-04T03:11:05.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:04 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/3489814461' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "495cc0c5-6a99-481f-ad9e-38f05fb84356"}]: dispatch 2024-05-04T03:11:05.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:04 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/3489814461' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "495cc0c5-6a99-481f-ad9e-38f05fb84356"}]': finished 2024-05-04T03:11:05.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:04 smithi077 ceph-mon[33531]: osdmap e22: 4 total, 3 up, 4 in 2024-05-04T03:11:05.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:04 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:11:05.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:04 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/492085' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:11:06.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:05 smithi001 ceph-mon[29731]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:06.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:06 smithi077 ceph-mon[33531]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:06.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:06 smithi072 ceph-mon[33332]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:08.278 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:08 smithi001 ceph-mon[29731]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 97 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:08.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:08 smithi077 ceph-mon[33531]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 97 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:08.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:08 smithi072 ceph-mon[33332]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 97 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:10.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:10 smithi001 ceph-mon[29731]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:10.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:10 smithi077 ceph-mon[33531]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:10.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:10 smithi072 ceph-mon[33332]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:12.311 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:12 smithi001 ceph-mon[29731]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:12.311 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:12 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-04T03:11:12.311 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:12 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:12.311 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:12 smithi001 ceph-mon[29731]: Deploying daemon osd.3 on smithi001 2024-05-04T03:11:12.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:12 smithi077 ceph-mon[33531]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:12.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:12 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-04T03:11:12.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:12 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:12.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:12 smithi077 ceph-mon[33531]: Deploying daemon osd.3 on smithi001 2024-05-04T03:11:12.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:12 smithi072 ceph-mon[33332]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:12.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:12 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-04T03:11:12.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:12 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:12.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:12 smithi072 ceph-mon[33332]: Deploying daemon osd.3 on smithi001 2024-05-04T03:11:14.204 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:14 smithi001 ceph-mon[29731]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:14.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:14 smithi077 ceph-mon[33531]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:14.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:14 smithi072 ceph-mon[33332]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:16.295 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:16 smithi001 ceph-mon[29731]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:16.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:16 smithi077 ceph-mon[33531]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:16.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:16 smithi072 ceph-mon[33332]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:18.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:17 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:18.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:17 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:18.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:17 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:11:18.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:17 smithi001 ceph-mon[29731]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:18.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:17 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:18.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:17 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:18.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:17 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:11:18.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:17 smithi077 ceph-mon[33531]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:18.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:17 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:18.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:17 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:18.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:17 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:11:18.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:17 smithi072 ceph-mon[33332]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:20.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:20.217+0000 7f6ec0ff9700 1 -- 172.21.15.1:0/3432957319 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6ec4063c00 con 0x7f6eac0b27f0 2024-05-04T03:11:20.219 INFO:teuthology.orchestra.run.smithi001.stdout:Created osd(s) 3 on host 'smithi001' 2024-05-04T03:11:20.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:20.220+0000 7f6eca0c0700 1 -- 172.21.15.1:0/3432957319 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6eac0b27f0 msgr2=0x7f6eac0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:20.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:20.220+0000 7f6eca0c0700 1 --2- 172.21.15.1:0/3432957319 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6eac0b27f0 0x7f6eac0b4d00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6eb40038a0 tx=0x7f6eb401a040 comp rx=0 tx=0).stop 2024-05-04T03:11:20.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:20.220+0000 7f6eca0c0700 1 -- 172.21.15.1:0/3432957319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ec4104d00 msgr2=0x7f6ec41a1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:20.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:20.220+0000 7f6eca0c0700 1 --2- 172.21.15.1:0/3432957319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ec4104d00 0x7f6ec41a1e00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc007ee0 tx=0x7f6ebc00d3b0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:20.220+0000 7f6eca0c0700 1 -- 172.21.15.1:0/3432957319 shutdown_connections 2024-05-04T03:11:20.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:20.220+0000 7f6eca0c0700 1 --2- 172.21.15.1:0/3432957319 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ec4103230 0x7f6ec419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:20.220+0000 7f6eca0c0700 1 --2- 172.21.15.1:0/3432957319 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6eac0b27f0 0x7f6eac0b4d00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:20.220+0000 7f6eca0c0700 1 --2- 172.21.15.1:0/3432957319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ec4104d00 0x7f6ec41a1e00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:20.220+0000 7f6eca0c0700 1 --2- 172.21.15.1:0/3432957319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ec4104360 0x7f6ec419ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:20.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:20.220+0000 7f6eca0c0700 1 -- 172.21.15.1:0/3432957319 >> 172.21.15.1:0/3432957319 conn(0x7f6ec40fe870 msgr2=0x7f6ec4107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:20.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:20.220+0000 7f6eca0c0700 1 -- 172.21.15.1:0/3432957319 shutdown_connections 2024-05-04T03:11:20.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:11:20.220+0000 7f6eca0c0700 1 -- 172.21.15.1:0/3432957319 wait complete. 2024-05-04T03:11:20.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:20 smithi077 ceph-mon[33531]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:20.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:20 smithi072 ceph-mon[33332]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:20.550 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:20 smithi001 ceph-mon[29731]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:20.896 DEBUG:teuthology.orchestra.run.smithi001:osd.3> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.3.service 2024-05-04T03:11:20.898 INFO:tasks.cephadm:Deploying osd.4 on smithi072 with /dev/vg_nvme/lv_3... 2024-05-04T03:11:20.898 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-04T03:11:21.140 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.b/config 2024-05-04T03:11:21.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:21 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:21.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:21 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:21.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:21 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:21.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:21 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:21.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:21 smithi001 ceph-mon[29731]: from='osd.3 [v2:172.21.15.1:6810/4050439365,v1:172.21.15.1:6811/4050439365]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-04T03:11:21.493 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:21 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:21.493 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:21 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:21.493 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:21 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:21.493 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:21 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:21.493 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:21 smithi072 ceph-mon[33332]: from='osd.3 [v2:172.21.15.1:6810/4050439365,v1:172.21.15.1:6811/4050439365]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-04T03:11:21.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:21 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:21.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:21 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:21.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:21 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:21.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:21 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:21.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:21 smithi077 ceph-mon[33531]: from='osd.3 [v2:172.21.15.1:6810/4050439365,v1:172.21.15.1:6811/4050439365]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-04T03:11:22.347 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:22 smithi001 ceph-mon[29731]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:22.348 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:22 smithi001 ceph-mon[29731]: from='osd.3 [v2:172.21.15.1:6810/4050439365,v1:172.21.15.1:6811/4050439365]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-04T03:11:22.348 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:22 smithi001 ceph-mon[29731]: osdmap e23: 4 total, 3 up, 4 in 2024-05-04T03:11:22.348 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:22 smithi001 ceph-mon[29731]: from='osd.3 [v2:172.21.15.1:6810/4050439365,v1:172.21.15.1:6811/4050439365]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-05-04T03:11:22.348 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:22 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:11:22.481 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:22 smithi072 ceph-mon[33332]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:22.482 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:22 smithi072 ceph-mon[33332]: from='osd.3 [v2:172.21.15.1:6810/4050439365,v1:172.21.15.1:6811/4050439365]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-04T03:11:22.482 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:22 smithi072 ceph-mon[33332]: osdmap e23: 4 total, 3 up, 4 in 2024-05-04T03:11:22.482 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:22 smithi072 ceph-mon[33332]: from='osd.3 [v2:172.21.15.1:6810/4050439365,v1:172.21.15.1:6811/4050439365]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-05-04T03:11:22.482 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:22 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:11:22.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:22 smithi077 ceph-mon[33531]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:22.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:22 smithi077 ceph-mon[33531]: from='osd.3 [v2:172.21.15.1:6810/4050439365,v1:172.21.15.1:6811/4050439365]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-04T03:11:22.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:22 smithi077 ceph-mon[33531]: osdmap e23: 4 total, 3 up, 4 in 2024-05-04T03:11:22.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:22 smithi077 ceph-mon[33531]: from='osd.3 [v2:172.21.15.1:6810/4050439365,v1:172.21.15.1:6811/4050439365]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-05-04T03:11:22.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:22 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:11:23.070 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-04T03:11:23.070 INFO:teuthology.orchestra.run.smithi072.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-04T03:11:23.070 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10+0 records in 2024-05-04T03:11:23.070 INFO:teuthology.orchestra.run.smithi072.stderr:10+0 records out 2024-05-04T03:11:23.070 INFO:teuthology.orchestra.run.smithi072.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011398 s, 920 MB/s 2024-05-04T03:11:23.070 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping successful for: 2024-05-04T03:11:23.761 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi072:vg_nvme/lv_3 2024-05-04T03:11:23.797 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:23 smithi001 ceph-mon[29731]: from='osd.3 [v2:172.21.15.1:6810/4050439365,v1:172.21.15.1:6811/4050439365]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-05-04T03:11:23.797 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:23 smithi001 ceph-mon[29731]: osdmap e24: 4 total, 3 up, 4 in 2024-05-04T03:11:23.797 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:23 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:11:23.797 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:23 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:11:23.797 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:23 smithi001 ceph-mon[29731]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:23.797 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:23 smithi001 ceph-mon[29731]: from='osd.3 [v2:172.21.15.1:6810/4050439365,v1:172.21.15.1:6811/4050439365]' entity='osd.3' 2024-05-04T03:11:23.797 INFO:journalctl@ceph.osd.3.smithi001.stdout:May 04 03:11:23 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-3[48820]: 2024-05-04T03:11:23.552+0000 7f1a52704700 -1 osd.3 0 waiting for initial osdmap 2024-05-04T03:11:23.798 INFO:journalctl@ceph.osd.3.smithi001.stdout:May 04 03:11:23 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-3[48820]: 2024-05-04T03:11:23.561+0000 7f1a49cb0700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T03:11:23.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:23 smithi077 ceph-mon[33531]: from='osd.3 [v2:172.21.15.1:6810/4050439365,v1:172.21.15.1:6811/4050439365]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-05-04T03:11:23.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:23 smithi077 ceph-mon[33531]: osdmap e24: 4 total, 3 up, 4 in 2024-05-04T03:11:23.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:23 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:11:23.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:23 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:11:23.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:23 smithi077 ceph-mon[33531]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:23.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:23 smithi077 ceph-mon[33531]: from='osd.3 [v2:172.21.15.1:6810/4050439365,v1:172.21.15.1:6811/4050439365]' entity='osd.3' 2024-05-04T03:11:23.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:23 smithi072 ceph-mon[33332]: from='osd.3 [v2:172.21.15.1:6810/4050439365,v1:172.21.15.1:6811/4050439365]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-05-04T03:11:23.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:23 smithi072 ceph-mon[33332]: osdmap e24: 4 total, 3 up, 4 in 2024-05-04T03:11:23.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:23 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:11:23.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:23 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:11:23.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:23 smithi072 ceph-mon[33332]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-05-04T03:11:23.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:23 smithi072 ceph-mon[33332]: from='osd.3 [v2:172.21.15.1:6810/4050439365,v1:172.21.15.1:6811/4050439365]' entity='osd.3' 2024-05-04T03:11:24.002 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.b/config 2024-05-04T03:11:24.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:24 smithi072 ceph-mon[33332]: purged_snaps scrub starts 2024-05-04T03:11:24.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:24 smithi072 ceph-mon[33332]: purged_snaps scrub ok 2024-05-04T03:11:24.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:24 smithi072 ceph-mon[33332]: osd.3 [v2:172.21.15.1:6810/4050439365,v1:172.21.15.1:6811/4050439365] boot 2024-05-04T03:11:24.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:24 smithi072 ceph-mon[33332]: osdmap e25: 4 total, 4 up, 4 in 2024-05-04T03:11:24.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:24 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:11:24.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:24 smithi072 ceph-mon[33332]: Detected new or changed devices on smithi001 2024-05-04T03:11:24.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:24 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:24.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:24 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:24.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:24 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:11:24.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:24 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:11:24.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:24 smithi072 ceph-mon[33332]: Adjusting osd_memory_target on smithi001 to 8546M 2024-05-04T03:11:24.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:24 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:24.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:24 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:24.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:24 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:11:24.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:24 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:24.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:24 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:11:24.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:24 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:24.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:24 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:11:24.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:24 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:24.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:24 smithi077 ceph-mon[33531]: purged_snaps scrub starts 2024-05-04T03:11:24.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:24 smithi077 ceph-mon[33531]: purged_snaps scrub ok 2024-05-04T03:11:24.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:24 smithi077 ceph-mon[33531]: osd.3 [v2:172.21.15.1:6810/4050439365,v1:172.21.15.1:6811/4050439365] boot 2024-05-04T03:11:24.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:24 smithi077 ceph-mon[33531]: osdmap e25: 4 total, 4 up, 4 in 2024-05-04T03:11:24.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:24 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:11:24.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:24 smithi077 ceph-mon[33531]: Detected new or changed devices on smithi001 2024-05-04T03:11:24.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:24 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:24.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:24 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:24.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:24 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:11:24.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:24 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:11:24.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:24 smithi077 ceph-mon[33531]: Adjusting osd_memory_target on smithi001 to 8546M 2024-05-04T03:11:24.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:24 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:24.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:24 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:24.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:24 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:11:24.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:24 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:24.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:24 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:11:24.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:24 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:24.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:24 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:11:24.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:24 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:25.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:24 smithi001 ceph-mon[29731]: purged_snaps scrub starts 2024-05-04T03:11:25.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:24 smithi001 ceph-mon[29731]: purged_snaps scrub ok 2024-05-04T03:11:25.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:24 smithi001 ceph-mon[29731]: osd.3 [v2:172.21.15.1:6810/4050439365,v1:172.21.15.1:6811/4050439365] boot 2024-05-04T03:11:25.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:24 smithi001 ceph-mon[29731]: osdmap e25: 4 total, 4 up, 4 in 2024-05-04T03:11:25.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:24 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-04T03:11:25.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:24 smithi001 ceph-mon[29731]: Detected new or changed devices on smithi001 2024-05-04T03:11:25.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:24 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:25.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:24 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:25.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:24 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:11:25.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:24 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:11:25.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:24 smithi001 ceph-mon[29731]: Adjusting osd_memory_target on smithi001 to 8546M 2024-05-04T03:11:25.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:24 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:25.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:24 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:25.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:24 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:11:25.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:24 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:25.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:24 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:11:25.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:24 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:25.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:24 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:11:25.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:24 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:25.545 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.542+0000 7fb40ba6b700 1 -- 172.21.15.72:0/2195772225 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb404104350 msgr2=0x7fb404104760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:25.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.542+0000 7fb40ba6b700 1 --2- 172.21.15.72:0/2195772225 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb404104350 0x7fb404104760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb400009990 tx=0x7fb400009ca0 comp rx=0 tx=0).stop 2024-05-04T03:11:25.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.542+0000 7fb40ba6b700 1 -- 172.21.15.72:0/2195772225 shutdown_connections 2024-05-04T03:11:25.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.542+0000 7fb40ba6b700 1 --2- 172.21.15.72:0/2195772225 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb404104350 0x7fb404104760 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:25.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.542+0000 7fb40ba6b700 1 --2- 172.21.15.72:0/2195772225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb404105e20 0x7fb40410c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:25.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.542+0000 7fb40ba6b700 1 --2- 172.21.15.72:0/2195772225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb404105480 0x7fb404105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:25.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.542+0000 7fb40ba6b700 1 -- 172.21.15.72:0/2195772225 >> 172.21.15.72:0/2195772225 conn(0x7fb4040ff990 msgr2=0x7fb404101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:25.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.542+0000 7fb40ba6b700 1 -- 172.21.15.72:0/2195772225 shutdown_connections 2024-05-04T03:11:25.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.542+0000 7fb40ba6b700 1 -- 172.21.15.72:0/2195772225 wait complete. 2024-05-04T03:11:25.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.543+0000 7fb40ba6b700 1 Processor -- start 2024-05-04T03:11:25.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.543+0000 7fb40ba6b700 1 -- start start 2024-05-04T03:11:25.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.543+0000 7fb40ba6b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb404104350 0x7fb40419a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:25.546 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.543+0000 7fb40ba6b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb404105480 0x7fb40419ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:25.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.543+0000 7fb40ba6b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb404105e20 0x7fb4041a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:25.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.543+0000 7fb40ba6b700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb40410fda0 con 0x7fb404104350 2024-05-04T03:11:25.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.543+0000 7fb40ba6b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb40410fc20 con 0x7fb404105e20 2024-05-04T03:11:25.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.543+0000 7fb40ba6b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb40410ff20 con 0x7fb404105480 2024-05-04T03:11:25.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.543+0000 7fb40a008700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb404105e20 0x7fb4041a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:25.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.543+0000 7fb40a008700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb404105e20 0x7fb4041a1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41982/0 (socket says 172.21.15.72:41982) 2024-05-04T03:11:25.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.543+0000 7fb40a008700 1 -- 172.21.15.72:0/486128994 learned_addr learned my addr 172.21.15.72:0/486128994 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-05-04T03:11:25.547 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.543+0000 7fb409006700 1 --2- 172.21.15.72:0/486128994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb404105480 0x7fb40419ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:25.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.544+0000 7fb409807700 1 --2- 172.21.15.72:0/486128994 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb404104350 0x7fb40419a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:25.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.544+0000 7fb40a008700 1 -- 172.21.15.72:0/486128994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb404105480 msgr2=0x7fb40419ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:25.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.544+0000 7fb40a008700 1 --2- 172.21.15.72:0/486128994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb404105480 0x7fb40419ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:25.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.544+0000 7fb40a008700 1 -- 172.21.15.72:0/486128994 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb404104350 msgr2=0x7fb40419a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:25.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.544+0000 7fb40a008700 1 --2- 172.21.15.72:0/486128994 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb404104350 0x7fb40419a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:25.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.544+0000 7fb40a008700 1 -- 172.21.15.72:0/486128994 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4041a24a0 con 0x7fb404105e20 2024-05-04T03:11:25.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.544+0000 7fb409807700 1 --2- 172.21.15.72:0/486128994 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb404104350 0x7fb40419a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:11:25.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.544+0000 7fb40a008700 1 --2- 172.21.15.72:0/486128994 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb404105e20 0x7fb4041a1e20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc007ee0 tx=0x7fb3fc00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:25.548 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.544+0000 7fb3faffd700 1 -- 172.21.15.72:0/486128994 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc00ddd0 con 0x7fb404105e20 2024-05-04T03:11:25.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.544+0000 7fb40ba6b700 1 -- 172.21.15.72:0/486128994 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4041a2790 con 0x7fb404105e20 2024-05-04T03:11:25.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.544+0000 7fb3faffd700 1 -- 172.21.15.72:0/486128994 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3fc004020 con 0x7fb404105e20 2024-05-04T03:11:25.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.544+0000 7fb3faffd700 1 -- 172.21.15.72:0/486128994 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc015660 con 0x7fb404105e20 2024-05-04T03:11:25.549 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.544+0000 7fb40ba6b700 1 -- 172.21.15.72:0/486128994 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4041a2c80 con 0x7fb404105e20 2024-05-04T03:11:25.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.545+0000 7fb40ba6b700 1 -- 172.21.15.72:0/486128994 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4040512c0 con 0x7fb404105e20 2024-05-04T03:11:25.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.547+0000 7fb3faffd700 1 -- 172.21.15.72:0/486128994 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fb3fc0041d0 con 0x7fb404105e20 2024-05-04T03:11:25.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.547+0000 7fb3faffd700 1 --2- 172.21.15.72:0/486128994 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fb3f00b2880 0x7fb3f00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:25.550 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.547+0000 7fb3faffd700 1 -- 172.21.15.72:0/486128994 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 3669+0+0 (secure 0 0 0) 0x7fb3fc0e12a0 con 0x7fb404105e20 2024-05-04T03:11:25.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.548+0000 7fb409807700 1 --2- 172.21.15.72:0/486128994 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fb3f00b2880 0x7fb3f00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:25.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.548+0000 7fb409807700 1 --2- 172.21.15.72:0/486128994 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fb3f00b2880 0x7fb3f00b4d90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb4000038a0 tx=0x7fb40001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:25.552 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.551+0000 7fb3faffd700 1 -- 172.21.15.72:0/486128994 <== mon.2 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb3fc010070 con 0x7fb404105e20 2024-05-04T03:11:25.745 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:25 smithi072 ceph-mon[33332]: osdmap e26: 4 total, 4 up, 4 in 2024-05-04T03:11:25.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:25 smithi072 ceph-mon[33332]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:11:25.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:25 smithi072 ceph-mon[33332]: osdmap e27: 4 total, 4 up, 4 in 2024-05-04T03:11:25.767 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:25.766+0000 7fb40ba6b700 1 -- 172.21.15.72:0/486128994 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb404063c00 con 0x7fb3f00b2880 2024-05-04T03:11:25.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:25 smithi077 ceph-mon[33531]: osdmap e26: 4 total, 4 up, 4 in 2024-05-04T03:11:25.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:25 smithi077 ceph-mon[33531]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:11:25.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:25 smithi077 ceph-mon[33531]: osdmap e27: 4 total, 4 up, 4 in 2024-05-04T03:11:26.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:25 smithi001 ceph-mon[29731]: osdmap e26: 4 total, 4 up, 4 in 2024-05-04T03:11:26.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:25 smithi001 ceph-mon[29731]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:11:26.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:25 smithi001 ceph-mon[29731]: osdmap e27: 4 total, 4 up, 4 in 2024-05-04T03:11:26.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:26 smithi077 ceph-mon[33531]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:11:26.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:26 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:11:26.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:26 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:11:26.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:26 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:26.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:26 smithi072 ceph-mon[33332]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:11:26.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:26 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:11:26.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:26 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:11:26.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:26 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:27.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:26 smithi001 ceph-mon[29731]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:11:27.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:26 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:11:27.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:26 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:11:27.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:26 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:27.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:27 smithi077 ceph-mon[33531]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:11:27.995 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:27 smithi072 ceph-mon[33332]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:11:28.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:27 smithi001 ceph-mon[29731]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:11:28.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:28 smithi077 ceph-mon[33531]: from='client.? 172.21.15.72:0/236204770' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b86b8026-79c0-4056-9e67-a61f54b6ab5d"}]: dispatch 2024-05-04T03:11:28.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:28 smithi077 ceph-mon[33531]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b86b8026-79c0-4056-9e67-a61f54b6ab5d"}]: dispatch 2024-05-04T03:11:28.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:28 smithi077 ceph-mon[33531]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b86b8026-79c0-4056-9e67-a61f54b6ab5d"}]': finished 2024-05-04T03:11:28.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:28 smithi077 ceph-mon[33531]: osdmap e28: 5 total, 4 up, 5 in 2024-05-04T03:11:28.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:28 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:11:28.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:28 smithi072 ceph-mon[33332]: from='client.? 172.21.15.72:0/236204770' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b86b8026-79c0-4056-9e67-a61f54b6ab5d"}]: dispatch 2024-05-04T03:11:28.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:28 smithi072 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b86b8026-79c0-4056-9e67-a61f54b6ab5d"}]: dispatch 2024-05-04T03:11:28.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:28 smithi072 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b86b8026-79c0-4056-9e67-a61f54b6ab5d"}]': finished 2024-05-04T03:11:28.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:28 smithi072 ceph-mon[33332]: osdmap e28: 5 total, 4 up, 5 in 2024-05-04T03:11:28.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:28 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:11:29.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:28 smithi001 ceph-mon[29731]: from='client.? 172.21.15.72:0/236204770' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b86b8026-79c0-4056-9e67-a61f54b6ab5d"}]: dispatch 2024-05-04T03:11:29.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:28 smithi001 ceph-mon[29731]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b86b8026-79c0-4056-9e67-a61f54b6ab5d"}]: dispatch 2024-05-04T03:11:29.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:28 smithi001 ceph-mon[29731]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b86b8026-79c0-4056-9e67-a61f54b6ab5d"}]': finished 2024-05-04T03:11:29.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:28 smithi001 ceph-mon[29731]: osdmap e28: 5 total, 4 up, 5 in 2024-05-04T03:11:29.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:28 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:11:29.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:29 smithi077 ceph-mon[33531]: from='client.? 172.21.15.72:0/3128472407' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:11:29.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:29 smithi077 ceph-mon[33531]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-05-04T03:11:29.995 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:29 smithi072 ceph-mon[33332]: from='client.? 172.21.15.72:0/3128472407' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:11:29.995 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:29 smithi072 ceph-mon[33332]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-05-04T03:11:30.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:29 smithi001 ceph-mon[29731]: from='client.? 172.21.15.72:0/3128472407' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:11:30.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:29 smithi001 ceph-mon[29731]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-05-04T03:11:32.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:32 smithi077 ceph-mon[33531]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-04T03:11:32.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:32 smithi072 ceph-mon[33332]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-04T03:11:32.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:32 smithi001 ceph-mon[29731]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-04T03:11:34.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:34 smithi072 ceph-mon[33332]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-04T03:11:34.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:34 smithi001 ceph-mon[29731]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-04T03:11:34.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:34 smithi077 ceph-mon[33531]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-04T03:11:36.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:36 smithi077 ceph-mon[33531]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-04T03:11:36.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:36 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-04T03:11:36.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:36 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:36.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:36 smithi072 ceph-mon[33332]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-04T03:11:36.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:36 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-04T03:11:36.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:36 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:36.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:36 smithi001 ceph-mon[29731]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-04T03:11:36.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-04T03:11:36.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:37.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:37 smithi077 ceph-mon[33531]: Deploying daemon osd.4 on smithi072 2024-05-04T03:11:37.472 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:37 smithi072 ceph-mon[33332]: Deploying daemon osd.4 on smithi072 2024-05-04T03:11:37.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:37 smithi001 ceph-mon[29731]: Deploying daemon osd.4 on smithi072 2024-05-04T03:11:38.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:38 smithi077 ceph-mon[33531]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:11:38.442 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:38 smithi072 ceph-mon[33332]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:11:38.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:38 smithi001 ceph-mon[29731]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:11:40.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:40 smithi077 ceph-mon[33531]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:11:40.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:40 smithi072 ceph-mon[33332]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:11:40.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:40 smithi001 ceph-mon[29731]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:11:42.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:42 smithi072 ceph-mon[33332]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:11:42.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:42 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:42.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:42 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:42.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:42 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:11:42.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:42 smithi001 ceph-mon[29731]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:11:42.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:42 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:42.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:42 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:42.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:42 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:11:42.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:42 smithi077 ceph-mon[33531]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:11:42.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:42 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:42.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:42 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:42.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:42 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:11:44.656 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:44 smithi072 ceph-mon[33332]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:11:44.730 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:44 smithi077 ceph-mon[33531]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:11:44.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:44 smithi001 ceph-mon[29731]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:11:44.987 INFO:teuthology.orchestra.run.smithi072.stdout:Created osd(s) 4 on host 'smithi072' 2024-05-04T03:11:44.988 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:44.986+0000 7fb3faffd700 1 -- 172.21.15.72:0/486128994 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb404063c00 con 0x7fb3f00b2880 2024-05-04T03:11:44.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:44.988+0000 7fb40ba6b700 1 -- 172.21.15.72:0/486128994 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fb3f00b2880 msgr2=0x7fb3f00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:44.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:44.988+0000 7fb40ba6b700 1 --2- 172.21.15.72:0/486128994 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fb3f00b2880 0x7fb3f00b4d90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb4000038a0 tx=0x7fb40001a040 comp rx=0 tx=0).stop 2024-05-04T03:11:44.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:44.988+0000 7fb40ba6b700 1 -- 172.21.15.72:0/486128994 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb404105e20 msgr2=0x7fb4041a1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:44.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:44.988+0000 7fb40ba6b700 1 --2- 172.21.15.72:0/486128994 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb404105e20 0x7fb4041a1e20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc007ee0 tx=0x7fb3fc00d3b0 comp rx=0 tx=0).stop 2024-05-04T03:11:44.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:44.988+0000 7fb40ba6b700 1 -- 172.21.15.72:0/486128994 shutdown_connections 2024-05-04T03:11:44.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:44.988+0000 7fb40ba6b700 1 --2- 172.21.15.72:0/486128994 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb404105e20 0x7fb4041a1e20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:44.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:44.988+0000 7fb40ba6b700 1 --2- 172.21.15.72:0/486128994 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fb3f00b2880 0x7fb3f00b4d90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:44.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:44.988+0000 7fb40ba6b700 1 --2- 172.21.15.72:0/486128994 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb404104350 0x7fb40419a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:44.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:44.988+0000 7fb40ba6b700 1 --2- 172.21.15.72:0/486128994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb404105480 0x7fb40419ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:44.990 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:44.988+0000 7fb40ba6b700 1 -- 172.21.15.72:0/486128994 >> 172.21.15.72:0/486128994 conn(0x7fb4040ff990 msgr2=0x7fb4041086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:44.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:44.989+0000 7fb40ba6b700 1 -- 172.21.15.72:0/486128994 shutdown_connections 2024-05-04T03:11:44.991 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:11:44.989+0000 7fb40ba6b700 1 -- 172.21.15.72:0/486128994 wait complete. 2024-05-04T03:11:45.648 DEBUG:teuthology.orchestra.run.smithi072:osd.4> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.4.service 2024-05-04T03:11:45.651 INFO:tasks.cephadm:Deploying osd.5 on smithi077 with /dev/vg_nvme/lv_3... 2024-05-04T03:11:45.651 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-04T03:11:45.896 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:11:46.235 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:45 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:46.235 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:45 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:46.235 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:45 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:46.235 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:45 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:46.235 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:45 smithi077 ceph-mon[33531]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:11:46.235 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:45 smithi077 ceph-mon[33531]: from='osd.4 [v2:172.21.15.72:6808/2501732391,v1:172.21.15.72:6809/2501732391]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-04T03:11:46.235 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:45 smithi077 ceph-mon[33531]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-04T03:11:46.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:45 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:46.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:45 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:46.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:45 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:46.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:45 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:46.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:45 smithi072 ceph-mon[33332]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:11:46.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:45 smithi072 ceph-mon[33332]: from='osd.4 [v2:172.21.15.72:6808/2501732391,v1:172.21.15.72:6809/2501732391]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-04T03:11:46.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:45 smithi072 ceph-mon[33332]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-04T03:11:46.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:45 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:46.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:45 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:46.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:45 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:46.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:45 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:46.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:45 smithi001 ceph-mon[29731]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:11:46.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:45 smithi001 ceph-mon[29731]: from='osd.4 [v2:172.21.15.72:6808/2501732391,v1:172.21.15.72:6809/2501732391]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-04T03:11:46.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:45 smithi001 ceph-mon[29731]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-04T03:11:47.242 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:47 smithi077 ceph-mon[33531]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-04T03:11:47.243 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:47 smithi077 ceph-mon[33531]: osdmap e29: 5 total, 4 up, 5 in 2024-05-04T03:11:47.243 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:47 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:11:47.243 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:47 smithi077 ceph-mon[33531]: from='osd.4 [v2:172.21.15.72:6808/2501732391,v1:172.21.15.72:6809/2501732391]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-04T03:11:47.243 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:47 smithi077 ceph-mon[33531]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-04T03:11:47.245 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:47 smithi072 ceph-mon[33332]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-04T03:11:47.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:47 smithi072 ceph-mon[33332]: osdmap e29: 5 total, 4 up, 5 in 2024-05-04T03:11:47.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:47 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:11:47.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:47 smithi072 ceph-mon[33332]: from='osd.4 [v2:172.21.15.72:6808/2501732391,v1:172.21.15.72:6809/2501732391]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-04T03:11:47.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:47 smithi072 ceph-mon[33332]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-04T03:11:47.246 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 04 03:11:47 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-4[44447]: 2024-05-04T03:11:47.068+0000 7f3de45eb700 -1 osd.4 0 waiting for initial osdmap 2024-05-04T03:11:47.246 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 04 03:11:47 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-4[44447]: 2024-05-04T03:11:47.080+0000 7f3de03a0700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T03:11:47.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:47 smithi001 ceph-mon[29731]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-04T03:11:47.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:47 smithi001 ceph-mon[29731]: osdmap e29: 5 total, 4 up, 5 in 2024-05-04T03:11:47.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:47 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:11:47.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:47 smithi001 ceph-mon[29731]: from='osd.4 [v2:172.21.15.72:6808/2501732391,v1:172.21.15.72:6809/2501732391]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-04T03:11:47.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:47 smithi001 ceph-mon[29731]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-04T03:11:47.800 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-04T03:11:47.801 INFO:teuthology.orchestra.run.smithi077.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-04T03:11:47.801 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10+0 records in 2024-05-04T03:11:47.801 INFO:teuthology.orchestra.run.smithi077.stderr:10+0 records out 2024-05-04T03:11:47.801 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113495 s, 924 MB/s 2024-05-04T03:11:47.801 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping successful for: 2024-05-04T03:11:48.322 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:48 smithi072 ceph-mon[33332]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-04T03:11:48.322 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:48 smithi072 ceph-mon[33332]: osdmap e30: 5 total, 4 up, 5 in 2024-05-04T03:11:48.322 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:48 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:11:48.322 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:48 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:11:48.322 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:48 smithi072 ceph-mon[33332]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:11:48.352 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:48 smithi077 ceph-mon[33531]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-04T03:11:48.353 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:48 smithi077 ceph-mon[33531]: osdmap e30: 5 total, 4 up, 5 in 2024-05-04T03:11:48.353 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:48 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:11:48.353 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:48 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:11:48.353 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:48 smithi077 ceph-mon[33531]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:11:48.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:48 smithi001 ceph-mon[29731]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-04T03:11:48.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:48 smithi001 ceph-mon[29731]: osdmap e30: 5 total, 4 up, 5 in 2024-05-04T03:11:48.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:48 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:11:48.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:48 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:11:48.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:48 smithi001 ceph-mon[29731]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-05-04T03:11:48.380 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_3 2024-05-04T03:11:48.622 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:11:49.267 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:49 smithi077 ceph-mon[33531]: purged_snaps scrub starts 2024-05-04T03:11:49.267 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:49 smithi077 ceph-mon[33531]: purged_snaps scrub ok 2024-05-04T03:11:49.267 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:49 smithi077 ceph-mon[33531]: osd.4 [v2:172.21.15.72:6808/2501732391,v1:172.21.15.72:6809/2501732391] boot 2024-05-04T03:11:49.267 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:49 smithi077 ceph-mon[33531]: osdmap e31: 5 total, 5 up, 5 in 2024-05-04T03:11:49.267 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:49 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:11:49.267 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:49 smithi077 ceph-mon[33531]: Detected new or changed devices on smithi072 2024-05-04T03:11:49.267 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:49 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:49.267 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:49 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:49.267 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:49 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:11:49.267 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:49 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:11:49.267 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:49 smithi077 ceph-mon[33531]: Adjusting osd_memory_target on smithi072 to 8544M 2024-05-04T03:11:49.267 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:49 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:49.268 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:49 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:49.268 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:49 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:11:49.268 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:49 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:49.268 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:49 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:11:49.268 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:49 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:49.268 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:49 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:11:49.268 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:49 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:49.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:49 smithi001 ceph-mon[29731]: purged_snaps scrub starts 2024-05-04T03:11:49.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:49 smithi001 ceph-mon[29731]: purged_snaps scrub ok 2024-05-04T03:11:49.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:49 smithi001 ceph-mon[29731]: osd.4 [v2:172.21.15.72:6808/2501732391,v1:172.21.15.72:6809/2501732391] boot 2024-05-04T03:11:49.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:49 smithi001 ceph-mon[29731]: osdmap e31: 5 total, 5 up, 5 in 2024-05-04T03:11:49.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:49 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:11:49.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:49 smithi001 ceph-mon[29731]: Detected new or changed devices on smithi072 2024-05-04T03:11:49.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:49 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:49.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:49 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:49.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:49 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:11:49.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:49 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:11:49.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:49 smithi001 ceph-mon[29731]: Adjusting osd_memory_target on smithi072 to 8544M 2024-05-04T03:11:49.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:49 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:49.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:49 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:49.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:49 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:11:49.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:49 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:49.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:49 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:11:49.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:49 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:49.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:49 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:11:49.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:49 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:49.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:49 smithi072 ceph-mon[33332]: purged_snaps scrub starts 2024-05-04T03:11:49.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:49 smithi072 ceph-mon[33332]: purged_snaps scrub ok 2024-05-04T03:11:49.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:49 smithi072 ceph-mon[33332]: osd.4 [v2:172.21.15.72:6808/2501732391,v1:172.21.15.72:6809/2501732391] boot 2024-05-04T03:11:49.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:49 smithi072 ceph-mon[33332]: osdmap e31: 5 total, 5 up, 5 in 2024-05-04T03:11:49.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:49 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-04T03:11:49.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:49 smithi072 ceph-mon[33332]: Detected new or changed devices on smithi072 2024-05-04T03:11:49.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:49 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:49.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:49 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:49.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:49 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:11:49.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:49 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:11:49.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:49 smithi072 ceph-mon[33332]: Adjusting osd_memory_target on smithi072 to 8544M 2024-05-04T03:11:49.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:49 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:49.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:49 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:49.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:49 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:11:49.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:49 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:49.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:49 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:11:49.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:49 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:49.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:49 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:11:49.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:49 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:11:50.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.100+0000 7f81fcb6c700 1 -- 172.21.15.77:0/152960151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81f8103230 msgr2=0x7f81f8103640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:50.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.100+0000 7f81fcb6c700 1 --2- 172.21.15.77:0/152960151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81f8103230 0x7f81f8103640 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f81e8009990 tx=0x7f81e8009ca0 comp rx=0 tx=0).stop 2024-05-04T03:11:50.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.100+0000 7f81fcb6c700 1 -- 172.21.15.77:0/152960151 shutdown_connections 2024-05-04T03:11:50.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.100+0000 7f81fcb6c700 1 --2- 172.21.15.77:0/152960151 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81f8104d00 0x7f81f810b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:50.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.100+0000 7f81fcb6c700 1 --2- 172.21.15.77:0/152960151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81f8103230 0x7f81f8103640 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:50.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.100+0000 7f81fcb6c700 1 --2- 172.21.15.77:0/152960151 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81f8104360 0x7f81f8104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:50.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.100+0000 7f81fcb6c700 1 -- 172.21.15.77:0/152960151 >> 172.21.15.77:0/152960151 conn(0x7f81f80fe890 msgr2=0x7f81f8100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:11:50.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.100+0000 7f81fcb6c700 1 -- 172.21.15.77:0/152960151 shutdown_connections 2024-05-04T03:11:50.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.100+0000 7f81fcb6c700 1 -- 172.21.15.77:0/152960151 wait complete. 2024-05-04T03:11:50.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.101+0000 7f81fcb6c700 1 Processor -- start 2024-05-04T03:11:50.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.101+0000 7f81fcb6c700 1 -- start start 2024-05-04T03:11:50.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.101+0000 7f81fcb6c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81f8103230 0x7f81f819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:50.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.101+0000 7f81fcb6c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81f8104360 0x7f81f819aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:50.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.101+0000 7f81fcb6c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81f8104d00 0x7f81f81a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:50.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.101+0000 7f81fcb6c700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f81f810ecf0 con 0x7f81f8104d00 2024-05-04T03:11:50.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.101+0000 7f81fcb6c700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f81f810eb70 con 0x7f81f8103230 2024-05-04T03:11:50.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.101+0000 7f81fcb6c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f81f810ee70 con 0x7f81f8104360 2024-05-04T03:11:50.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.101+0000 7f81edd9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81f8104360 0x7f81f819aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:50.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.101+0000 7f81edd9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81f8104360 0x7f81f819aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53646/0 (socket says 172.21.15.77:53646) 2024-05-04T03:11:50.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.101+0000 7f81edd9b700 1 -- 172.21.15.77:0/3046381799 learned_addr learned my addr 172.21.15.77:0/3046381799 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-04T03:11:50.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.101+0000 7f81f659c700 1 --2- 172.21.15.77:0/3046381799 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81f8103230 0x7f81f819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:50.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.101+0000 7f81f6d9d700 1 --2- 172.21.15.77:0/3046381799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81f8104d00 0x7f81f81a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:50.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.102+0000 7f81edd9b700 1 -- 172.21.15.77:0/3046381799 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81f8103230 msgr2=0x7f81f819a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:50.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.102+0000 7f81edd9b700 1 --2- 172.21.15.77:0/3046381799 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81f8103230 0x7f81f819a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:50.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.102+0000 7f81edd9b700 1 -- 172.21.15.77:0/3046381799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81f8104d00 msgr2=0x7f81f81a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:11:50.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.102+0000 7f81edd9b700 1 --2- 172.21.15.77:0/3046381799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81f8104d00 0x7f81f81a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:11:50.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.102+0000 7f81edd9b700 1 -- 172.21.15.77:0/3046381799 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f81a24e0 con 0x7f81f8104360 2024-05-04T03:11:50.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.102+0000 7f81f6d9d700 1 --2- 172.21.15.77:0/3046381799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81f8104d00 0x7f81f81a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:11:50.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.102+0000 7f81edd9b700 1 --2- 172.21.15.77:0/3046381799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81f8104360 0x7f81f819aec0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f81e000d950 tx=0x7f81e000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:50.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.102+0000 7f81effff700 1 -- 172.21.15.77:0/3046381799 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e0016070 con 0x7f81f8104360 2024-05-04T03:11:50.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.102+0000 7f81fcb6c700 1 -- 172.21.15.77:0/3046381799 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81f81a27d0 con 0x7f81f8104360 2024-05-04T03:11:50.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.102+0000 7f81effff700 1 -- 172.21.15.77:0/3046381799 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81e0009df0 con 0x7f81f8104360 2024-05-04T03:11:50.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.102+0000 7f81effff700 1 -- 172.21.15.77:0/3046381799 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e000f3c0 con 0x7f81f8104360 2024-05-04T03:11:50.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.102+0000 7f81fcb6c700 1 -- 172.21.15.77:0/3046381799 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81f81a2de0 con 0x7f81f8104360 2024-05-04T03:11:50.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.103+0000 7f81fcb6c700 1 -- 172.21.15.77:0/3046381799 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81f80512c0 con 0x7f81f8104360 2024-05-04T03:11:50.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.105+0000 7f81effff700 1 -- 172.21.15.77:0/3046381799 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f81e0019070 con 0x7f81f8104360 2024-05-04T03:11:50.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.105+0000 7f81effff700 1 --2- 172.21.15.77:0/3046381799 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f81e40b2880 0x7f81e40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:11:50.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.105+0000 7f81effff700 1 -- 172.21.15.77:0/3046381799 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7f81e00e0db0 con 0x7f81f8104360 2024-05-04T03:11:50.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.105+0000 7f81f659c700 1 --2- 172.21.15.77:0/3046381799 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f81e40b2880 0x7f81e40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:11:50.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.106+0000 7f81f659c700 1 --2- 172.21.15.77:0/3046381799 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f81e40b2880 0x7f81e40b4d90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f81e80038a0 tx=0x7f81e801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:11:50.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.109+0000 7f81effff700 1 -- 172.21.15.77:0/3046381799 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f81e00a9b40 con 0x7f81f8104360 2024-05-04T03:11:50.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:50 smithi077 ceph-mon[33531]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:11:50.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:50 smithi077 ceph-mon[33531]: osdmap e32: 5 total, 5 up, 5 in 2024-05-04T03:11:50.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:11:50.323+0000 7f81fcb6c700 1 -- 172.21.15.77:0/3046381799 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f81f81a2fb0 con 0x7f81e40b2880 2024-05-04T03:11:50.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:50 smithi001 ceph-mon[29731]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:11:50.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:50 smithi001 ceph-mon[29731]: osdmap e32: 5 total, 5 up, 5 in 2024-05-04T03:11:50.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:50 smithi072 ceph-mon[33332]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:11:50.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:50 smithi072 ceph-mon[33332]: osdmap e32: 5 total, 5 up, 5 in 2024-05-04T03:11:51.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:51 smithi077 ceph-mon[33531]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:11:51.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:51 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:11:51.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:51 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:11:51.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:51 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:51.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:51 smithi001 ceph-mon[29731]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:11:51.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:51 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:11:51.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:51 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:11:51.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:51 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:51.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:51 smithi072 ceph-mon[33332]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:11:51.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:51 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:11:51.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:51 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:11:51.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:51 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:11:52.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:52 smithi001 ceph-mon[29731]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:11:52.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:52 smithi077 ceph-mon[33531]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:11:52.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:52 smithi072 ceph-mon[33332]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:11:53.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:53 smithi001 ceph-mon[29731]: from='client.? 172.21.15.77:0/666117810' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80975cb9-39e0-46b8-a343-9ff69194a14e"}]: dispatch 2024-05-04T03:11:53.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:53 smithi001 ceph-mon[29731]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80975cb9-39e0-46b8-a343-9ff69194a14e"}]: dispatch 2024-05-04T03:11:53.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:53 smithi001 ceph-mon[29731]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80975cb9-39e0-46b8-a343-9ff69194a14e"}]': finished 2024-05-04T03:11:53.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:53 smithi001 ceph-mon[29731]: osdmap e33: 6 total, 5 up, 6 in 2024-05-04T03:11:53.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:53 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:11:53.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:53 smithi077 ceph-mon[33531]: from='client.? 172.21.15.77:0/666117810' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80975cb9-39e0-46b8-a343-9ff69194a14e"}]: dispatch 2024-05-04T03:11:53.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:53 smithi077 ceph-mon[33531]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80975cb9-39e0-46b8-a343-9ff69194a14e"}]: dispatch 2024-05-04T03:11:53.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:53 smithi077 ceph-mon[33531]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80975cb9-39e0-46b8-a343-9ff69194a14e"}]': finished 2024-05-04T03:11:53.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:53 smithi077 ceph-mon[33531]: osdmap e33: 6 total, 5 up, 6 in 2024-05-04T03:11:53.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:53 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:11:53.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:53 smithi072 ceph-mon[33332]: from='client.? 172.21.15.77:0/666117810' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80975cb9-39e0-46b8-a343-9ff69194a14e"}]: dispatch 2024-05-04T03:11:53.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:53 smithi072 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "80975cb9-39e0-46b8-a343-9ff69194a14e"}]: dispatch 2024-05-04T03:11:53.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:53 smithi072 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "80975cb9-39e0-46b8-a343-9ff69194a14e"}]': finished 2024-05-04T03:11:53.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:53 smithi072 ceph-mon[33332]: osdmap e33: 6 total, 5 up, 6 in 2024-05-04T03:11:53.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:53 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:11:54.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:54 smithi001 ceph-mon[29731]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:11:54.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:54 smithi001 ceph-mon[29731]: from='client.? 172.21.15.77:0/2730300506' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:11:54.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:54 smithi077 ceph-mon[33531]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:11:54.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:54 smithi077 ceph-mon[33531]: from='client.? 172.21.15.77:0/2730300506' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:11:54.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:54 smithi072 ceph-mon[33332]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:11:54.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:54 smithi072 ceph-mon[33332]: from='client.? 172.21.15.77:0/2730300506' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:11:56.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:56 smithi001 ceph-mon[29731]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:11:56.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:56 smithi077 ceph-mon[33531]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:11:56.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:56 smithi072 ceph-mon[33332]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:11:58.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:11:58 smithi001 ceph-mon[29731]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:11:58.394 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:11:58 smithi077 ceph-mon[33531]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:11:58.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:11:58 smithi072 ceph-mon[33332]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:12:00.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:00 smithi001 ceph-mon[29731]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:12:00.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:00 smithi077 ceph-mon[33531]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:12:00.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:00 smithi072 ceph-mon[33332]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:12:01.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:01 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-04T03:12:01.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:01 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:01.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:01 smithi001 ceph-mon[29731]: Deploying daemon osd.5 on smithi077 2024-05-04T03:12:01.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:01 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-04T03:12:01.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:01 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:01.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:01 smithi077 ceph-mon[33531]: Deploying daemon osd.5 on smithi077 2024-05-04T03:12:01.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:01 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-04T03:12:01.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:01 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:01.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:01 smithi072 ceph-mon[33332]: Deploying daemon osd.5 on smithi077 2024-05-04T03:12:02.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:02 smithi077 ceph-mon[33531]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:12:02.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:02 smithi001 ceph-mon[29731]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:12:02.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:02 smithi072 ceph-mon[33332]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:12:04.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:04 smithi001 ceph-mon[29731]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:12:04.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:04 smithi077 ceph-mon[33531]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:12:04.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:04 smithi072 ceph-mon[33332]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:12:06.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:06 smithi001 ceph-mon[29731]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:12:06.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:06 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:06.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:06 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:06.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:06 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:12:06.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:06 smithi077 ceph-mon[33531]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:12:06.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:06 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:06.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:06 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:06.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:06 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:12:06.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:06 smithi072 ceph-mon[33332]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:12:06.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:06 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:06.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:06 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:06.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:06 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:12:08.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:08 smithi001 ceph-mon[29731]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:12:08.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:08 smithi077 ceph-mon[33531]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:12:08.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:08 smithi072 ceph-mon[33332]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:12:09.290 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 5 on host 'smithi077' 2024-05-04T03:12:09.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:12:09.289+0000 7f81effff700 1 -- 172.21.15.77:0/3046381799 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f81f81a2fb0 con 0x7f81e40b2880 2024-05-04T03:12:09.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:12:09.291+0000 7f81fcb6c700 1 -- 172.21.15.77:0/3046381799 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f81e40b2880 msgr2=0x7f81e40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:09.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:12:09.291+0000 7f81fcb6c700 1 --2- 172.21.15.77:0/3046381799 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f81e40b2880 0x7f81e40b4d90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f81e80038a0 tx=0x7f81e801a040 comp rx=0 tx=0).stop 2024-05-04T03:12:09.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:12:09.291+0000 7f81fcb6c700 1 -- 172.21.15.77:0/3046381799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81f8104360 msgr2=0x7f81f819aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:09.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:12:09.291+0000 7f81fcb6c700 1 --2- 172.21.15.77:0/3046381799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81f8104360 0x7f81f819aec0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f81e000d950 tx=0x7f81e000dc60 comp rx=0 tx=0).stop 2024-05-04T03:12:09.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:12:09.291+0000 7f81fcb6c700 1 -- 172.21.15.77:0/3046381799 shutdown_connections 2024-05-04T03:12:09.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:12:09.291+0000 7f81fcb6c700 1 --2- 172.21.15.77:0/3046381799 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f81f8103230 0x7f81f819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:09.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:12:09.291+0000 7f81fcb6c700 1 --2- 172.21.15.77:0/3046381799 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f81e40b2880 0x7f81e40b4d90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:09.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:12:09.291+0000 7f81fcb6c700 1 --2- 172.21.15.77:0/3046381799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f81f8104d00 0x7f81f81a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:09.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:12:09.291+0000 7f81fcb6c700 1 --2- 172.21.15.77:0/3046381799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f81f8104360 0x7f81f819aec0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:09.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:12:09.291+0000 7f81fcb6c700 1 -- 172.21.15.77:0/3046381799 >> 172.21.15.77:0/3046381799 conn(0x7f81f80fe890 msgr2=0x7f81f8100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:09.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:12:09.292+0000 7f81fcb6c700 1 -- 172.21.15.77:0/3046381799 shutdown_connections 2024-05-04T03:12:09.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:12:09.292+0000 7f81fcb6c700 1 -- 172.21.15.77:0/3046381799 wait complete. 2024-05-04T03:12:09.619 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:09 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:09.619 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:09 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:09.619 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:09 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:09.619 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:09 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:12:09.619 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:09 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:09.619 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:09 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:09.620 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:09 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:09.620 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:09 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:12:09.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:09 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:09.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:09 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:09.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:09 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:09.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:09 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:12:09.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:09 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:09.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:09 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:09.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:09 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:09.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:09 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:12:09.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:09 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:09.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:09 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:09.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:09 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:09.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:09 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:12:09.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:09 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:09.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:09 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:09.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:09 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:09.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:09 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:12:09.913 DEBUG:teuthology.orchestra.run.smithi077:osd.5> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.5.service 2024-05-04T03:12:09.916 INFO:tasks.cephadm:Deploying osd.6 on smithi001 with /dev/vg_nvme/lv_2... 2024-05-04T03:12:09.916 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-04T03:12:10.172 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:12:10.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:10 smithi001 ceph-mon[29731]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:12:10.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:10 smithi001 ceph-mon[29731]: from='osd.5 [v2:172.21.15.77:6808/3582852165,v1:172.21.15.77:6809/3582852165]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-04T03:12:10.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:10 smithi001 ceph-mon[29731]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-04T03:12:10.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:10 smithi077 ceph-mon[33531]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:12:10.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:10 smithi077 ceph-mon[33531]: from='osd.5 [v2:172.21.15.77:6808/3582852165,v1:172.21.15.77:6809/3582852165]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-04T03:12:10.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:10 smithi077 ceph-mon[33531]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-04T03:12:10.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:10 smithi072 ceph-mon[33332]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:12:10.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:10 smithi072 ceph-mon[33332]: from='osd.5 [v2:172.21.15.77:6808/3582852165,v1:172.21.15.77:6809/3582852165]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-04T03:12:10.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:10 smithi072 ceph-mon[33332]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-04T03:12:11.583 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:11 smithi001 ceph-mon[29731]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-04T03:12:11.584 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:11 smithi001 ceph-mon[29731]: osdmap e34: 6 total, 5 up, 6 in 2024-05-04T03:12:11.584 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:11 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:12:11.584 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:11 smithi001 ceph-mon[29731]: from='osd.5 [v2:172.21.15.77:6808/3582852165,v1:172.21.15.77:6809/3582852165]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-04T03:12:11.584 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:11 smithi001 ceph-mon[29731]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-04T03:12:11.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:11 smithi077 ceph-mon[33531]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-04T03:12:11.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:11 smithi077 ceph-mon[33531]: osdmap e34: 6 total, 5 up, 6 in 2024-05-04T03:12:11.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:11 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:12:11.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:11 smithi077 ceph-mon[33531]: from='osd.5 [v2:172.21.15.77:6808/3582852165,v1:172.21.15.77:6809/3582852165]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-04T03:12:11.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:11 smithi077 ceph-mon[33531]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-04T03:12:11.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:11 smithi072 ceph-mon[33332]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-04T03:12:11.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:11 smithi072 ceph-mon[33332]: osdmap e34: 6 total, 5 up, 6 in 2024-05-04T03:12:11.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:11 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:12:11.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:11 smithi072 ceph-mon[33332]: from='osd.5 [v2:172.21.15.77:6808/3582852165,v1:172.21.15.77:6809/3582852165]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-04T03:12:11.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:11 smithi072 ceph-mon[33332]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-04T03:12:12.224 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-04T03:12:12.225 INFO:teuthology.orchestra.run.smithi001.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-04T03:12:12.225 INFO:teuthology.orchestra.run.smithi001.stderr: stderr: 10+0 records in 2024-05-04T03:12:12.225 INFO:teuthology.orchestra.run.smithi001.stderr:10+0 records out 2024-05-04T03:12:12.225 INFO:teuthology.orchestra.run.smithi001.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00885036 s, 1.2 GB/s 2024-05-04T03:12:12.225 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping successful for: 2024-05-04T03:12:12.543 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:12 smithi001 ceph-mon[29731]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:12:12.543 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:12 smithi001 ceph-mon[29731]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-05-04T03:12:12.543 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:12 smithi001 ceph-mon[29731]: osdmap e35: 6 total, 5 up, 6 in 2024-05-04T03:12:12.543 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:12 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:12:12.543 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:12 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:12:12.543 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:12 smithi001 ceph-mon[29731]: from='osd.5 ' entity='osd.5' 2024-05-04T03:12:12.565 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:12 smithi077 ceph-mon[33531]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:12:12.565 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:12 smithi077 ceph-mon[33531]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-05-04T03:12:12.565 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:12 smithi077 ceph-mon[33531]: osdmap e35: 6 total, 5 up, 6 in 2024-05-04T03:12:12.566 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:12 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:12:12.566 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:12 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:12:12.566 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:12 smithi077 ceph-mon[33531]: from='osd.5 ' entity='osd.5' 2024-05-04T03:12:12.566 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:12:12 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5[45085]: 2024-05-04T03:12:12.344+0000 7f2bdcc82700 -1 osd.5 0 waiting for initial osdmap 2024-05-04T03:12:12.566 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:12:12 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5[45085]: 2024-05-04T03:12:12.465+0000 7f2bd422e700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T03:12:12.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:12 smithi072 ceph-mon[33332]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:12:12.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:12 smithi072 ceph-mon[33332]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-05-04T03:12:12.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:12 smithi072 ceph-mon[33332]: osdmap e35: 6 total, 5 up, 6 in 2024-05-04T03:12:12.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:12 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:12:12.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:12 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:12:12.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:12 smithi072 ceph-mon[33332]: from='osd.5 ' entity='osd.5' 2024-05-04T03:12:12.891 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi001:vg_nvme/lv_2 2024-05-04T03:12:13.146 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:12:13.548 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:13 smithi001 ceph-mon[29731]: purged_snaps scrub starts 2024-05-04T03:12:13.548 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:13 smithi001 ceph-mon[29731]: purged_snaps scrub ok 2024-05-04T03:12:13.548 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:13 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:12:13.548 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:13 smithi001 ceph-mon[29731]: osdmap e36: 6 total, 5 up, 6 in 2024-05-04T03:12:13.548 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:13 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:12:13.548 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:13 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:13.548 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:13 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:13.549 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:13 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:12:13.549 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:13 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:12:13.549 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:13 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:13.549 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:13 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:13.549 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:13 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:12:13.549 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:13 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:13.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:13 smithi072 ceph-mon[33332]: purged_snaps scrub starts 2024-05-04T03:12:13.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:13 smithi072 ceph-mon[33332]: purged_snaps scrub ok 2024-05-04T03:12:13.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:13 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:12:13.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:13 smithi072 ceph-mon[33332]: osdmap e36: 6 total, 5 up, 6 in 2024-05-04T03:12:13.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:13 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:12:13.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:13 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:13.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:13 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:13.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:13 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:12:13.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:13 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:12:13.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:13 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:13.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:13 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:13.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:13 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:12:13.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:13 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:13.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:13 smithi077 ceph-mon[33531]: purged_snaps scrub starts 2024-05-04T03:12:13.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:13 smithi077 ceph-mon[33531]: purged_snaps scrub ok 2024-05-04T03:12:13.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:13 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:12:13.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:13 smithi077 ceph-mon[33531]: osdmap e36: 6 total, 5 up, 6 in 2024-05-04T03:12:13.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:13 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:12:13.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:13 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:13.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:13 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:13.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:13 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:12:13.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:13 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:12:13.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:13 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:13.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:13 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:13.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:13 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:12:13.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:13 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:14.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:14 smithi001 ceph-mon[29731]: Detected new or changed devices on smithi077 2024-05-04T03:12:14.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:14 smithi001 ceph-mon[29731]: Adjusting osd_memory_target on smithi077 to 8544M 2024-05-04T03:12:14.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:14 smithi001 ceph-mon[29731]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:12:14.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:12:14.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:14 smithi001 ceph-mon[29731]: osd.5 [v2:172.21.15.77:6808/3582852165,v1:172.21.15.77:6809/3582852165] boot 2024-05-04T03:12:14.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:14 smithi001 ceph-mon[29731]: osdmap e37: 6 total, 6 up, 6 in 2024-05-04T03:12:14.480 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:12:14.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:14 smithi072 ceph-mon[33332]: Detected new or changed devices on smithi077 2024-05-04T03:12:14.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:14 smithi072 ceph-mon[33332]: Adjusting osd_memory_target on smithi077 to 8544M 2024-05-04T03:12:14.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:14 smithi072 ceph-mon[33332]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:12:14.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:14 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:12:14.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:14 smithi072 ceph-mon[33332]: osd.5 [v2:172.21.15.77:6808/3582852165,v1:172.21.15.77:6809/3582852165] boot 2024-05-04T03:12:14.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:14 smithi072 ceph-mon[33332]: osdmap e37: 6 total, 6 up, 6 in 2024-05-04T03:12:14.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:14 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:12:14.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.751+0000 7f6ef775a700 1 -- 172.21.15.1:0/239218563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ef0107790 msgr2=0x7f6ef0109bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:14.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.751+0000 7f6ef775a700 1 --2- 172.21.15.1:0/239218563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ef0107790 0x7f6ef0109bb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009990 tx=0x7f6ee0009ca0 comp rx=0 tx=0).stop 2024-05-04T03:12:14.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.751+0000 7f6ef775a700 1 -- 172.21.15.1:0/239218563 shutdown_connections 2024-05-04T03:12:14.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.751+0000 7f6ef775a700 1 --2- 172.21.15.1:0/239218563 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ef010a0f0 0x7f6ef010c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:14.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.751+0000 7f6ef775a700 1 --2- 172.21.15.1:0/239218563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ef0104e30 0x7f6ef0107250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:14.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.751+0000 7f6ef775a700 1 --2- 172.21.15.1:0/239218563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ef0107790 0x7f6ef0109bb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:14.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.751+0000 7f6ef775a700 1 -- 172.21.15.1:0/239218563 >> 172.21.15.1:0/239218563 conn(0x7f6ef00fe870 msgr2=0x7f6ef0100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:14.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.751+0000 7f6ef775a700 1 -- 172.21.15.1:0/239218563 shutdown_connections 2024-05-04T03:12:14.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.751+0000 7f6ef775a700 1 -- 172.21.15.1:0/239218563 wait complete. 2024-05-04T03:12:14.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.752+0000 7f6ef775a700 1 Processor -- start 2024-05-04T03:12:14.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.752+0000 7f6ef775a700 1 -- start start 2024-05-04T03:12:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.752+0000 7f6ef775a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ef0104e30 0x7f6ef019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.752+0000 7f6ef775a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ef0107790 0x7f6ef019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.752+0000 7f6ef775a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ef010a0f0 0x7f6ef01a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.752+0000 7f6ef775a700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef010fd90 con 0x7f6ef0104e30 2024-05-04T03:12:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.752+0000 7f6ef775a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef010fc10 con 0x7f6ef010a0f0 2024-05-04T03:12:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.752+0000 7f6ef775a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef010ff10 con 0x7f6ef0107790 2024-05-04T03:12:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.752+0000 7f6ef54f6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ef0104e30 0x7f6ef019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.752+0000 7f6ef54f6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ef0104e30 0x7f6ef019a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56824/0 (socket says 172.21.15.1:56824) 2024-05-04T03:12:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.752+0000 7f6ef4cf5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ef0107790 0x7f6ef019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.752+0000 7f6ef54f6700 1 -- 172.21.15.1:0/890028307 learned_addr learned my addr 172.21.15.1:0/890028307 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:12:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.752+0000 7f6ef5cf7700 1 --2- 172.21.15.1:0/890028307 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ef010a0f0 0x7f6ef01a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.753+0000 7f6ef54f6700 1 -- 172.21.15.1:0/890028307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ef0107790 msgr2=0x7f6ef019ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.753+0000 7f6ef54f6700 1 --2- 172.21.15.1:0/890028307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ef0107790 0x7f6ef019ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:14.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.753+0000 7f6ef54f6700 1 -- 172.21.15.1:0/890028307 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ef010a0f0 msgr2=0x7f6ef01a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.753+0000 7f6ef54f6700 1 --2- 172.21.15.1:0/890028307 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ef010a0f0 0x7f6ef01a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.753+0000 7f6ef54f6700 1 -- 172.21.15.1:0/890028307 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef01a2430 con 0x7f6ef0104e30 2024-05-04T03:12:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.753+0000 7f6ef54f6700 1 --2- 172.21.15.1:0/890028307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ef0104e30 0x7f6ef019a8d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6eec00d950 tx=0x7f6eec00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.753+0000 7f6ee67fc700 1 -- 172.21.15.1:0/890028307 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec016070 con 0x7f6ef0104e30 2024-05-04T03:12:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.753+0000 7f6ee67fc700 1 -- 172.21.15.1:0/890028307 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eec004830 con 0x7f6ef0104e30 2024-05-04T03:12:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.753+0000 7f6ef775a700 1 -- 172.21.15.1:0/890028307 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef01a2720 con 0x7f6ef0104e30 2024-05-04T03:12:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.753+0000 7f6ee67fc700 1 -- 172.21.15.1:0/890028307 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec005520 con 0x7f6ef0104e30 2024-05-04T03:12:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.753+0000 7f6ef775a700 1 -- 172.21.15.1:0/890028307 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef01a2c60 con 0x7f6ef0104e30 2024-05-04T03:12:14.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.754+0000 7f6ef775a700 1 -- 172.21.15.1:0/890028307 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ef00512c0 con 0x7f6ef0104e30 2024-05-04T03:12:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.756+0000 7f6ee67fc700 1 -- 172.21.15.1:0/890028307 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f6eec004220 con 0x7f6ef0104e30 2024-05-04T03:12:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.756+0000 7f6ee67fc700 1 --2- 172.21.15.1:0/890028307 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6edc0b2880 0x7f6edc0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.756+0000 7f6ef4cf5700 1 --2- 172.21.15.1:0/890028307 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6edc0b2880 0x7f6edc0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.756+0000 7f6ee67fc700 1 -- 172.21.15.1:0/890028307 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4533+0+0 (secure 0 0 0) 0x7f6eec0e1600 con 0x7f6ef0104e30 2024-05-04T03:12:14.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.757+0000 7f6ef4cf5700 1 --2- 172.21.15.1:0/890028307 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6edc0b2880 0x7f6edc0b4d90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000b5c0 tx=0x7f6ee001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:14.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.760+0000 7f6ee67fc700 1 -- 172.21.15.1:0/890028307 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6eec01f020 con 0x7f6ef0104e30 2024-05-04T03:12:14.830 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:14 smithi077 ceph-mon[33531]: Detected new or changed devices on smithi077 2024-05-04T03:12:14.830 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:14 smithi077 ceph-mon[33531]: Adjusting osd_memory_target on smithi077 to 8544M 2024-05-04T03:12:14.830 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:14 smithi077 ceph-mon[33531]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-05-04T03:12:14.830 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:14 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:12:14.830 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:14 smithi077 ceph-mon[33531]: osd.5 [v2:172.21.15.77:6808/3582852165,v1:172.21.15.77:6809/3582852165] boot 2024-05-04T03:12:14.830 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:14 smithi077 ceph-mon[33531]: osdmap e37: 6 total, 6 up, 6 in 2024-05-04T03:12:14.830 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:14 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-04T03:12:14.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:14.983+0000 7f6ef775a700 1 -- 172.21.15.1:0/890028307 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f6ef0063c00 con 0x7f6edc0b2880 2024-05-04T03:12:15.723 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:15 smithi001 ceph-mon[29731]: osdmap e38: 6 total, 6 up, 6 in 2024-05-04T03:12:15.723 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:15 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:12:15.724 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:15 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:12:15.724 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:15 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:15.745 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:15 smithi072 ceph-mon[33332]: osdmap e38: 6 total, 6 up, 6 in 2024-05-04T03:12:15.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:15 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:12:15.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:15 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:12:15.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:15 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:15.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:15 smithi077 ceph-mon[33531]: osdmap e38: 6 total, 6 up, 6 in 2024-05-04T03:12:15.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:15 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:12:15.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:15 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:12:15.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:15 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:16.705 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:16 smithi001 ceph-mon[29731]: from='client.14340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:12:16.705 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:16 smithi001 ceph-mon[29731]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:12:16.705 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:16 smithi001 ceph-mon[29731]: osdmap e39: 6 total, 6 up, 6 in 2024-05-04T03:12:16.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:16 smithi072 ceph-mon[33332]: from='client.14340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:12:16.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:16 smithi072 ceph-mon[33332]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:12:16.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:16 smithi072 ceph-mon[33332]: osdmap e39: 6 total, 6 up, 6 in 2024-05-04T03:12:16.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:16 smithi077 ceph-mon[33531]: from='client.14340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:12:16.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:16 smithi077 ceph-mon[33531]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:12:16.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:16 smithi077 ceph-mon[33531]: osdmap e39: 6 total, 6 up, 6 in 2024-05-04T03:12:18.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:18 smithi072 ceph-mon[33332]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:12:18.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:18 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/2437367026' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aca3fac5-fbb5-42de-bf0e-e4a0db7381e2"}]: dispatch 2024-05-04T03:12:18.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:18 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/2437367026' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aca3fac5-fbb5-42de-bf0e-e4a0db7381e2"}]': finished 2024-05-04T03:12:18.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:18 smithi072 ceph-mon[33332]: osdmap e40: 7 total, 6 up, 7 in 2024-05-04T03:12:18.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:18 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:12:18.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:18 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/734241903' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:12:18.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:18 smithi001 ceph-mon[29731]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:12:18.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:18 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2437367026' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aca3fac5-fbb5-42de-bf0e-e4a0db7381e2"}]: dispatch 2024-05-04T03:12:18.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:18 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2437367026' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aca3fac5-fbb5-42de-bf0e-e4a0db7381e2"}]': finished 2024-05-04T03:12:18.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:18 smithi001 ceph-mon[29731]: osdmap e40: 7 total, 6 up, 7 in 2024-05-04T03:12:18.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:18 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:12:18.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:18 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/734241903' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:12:18.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:18 smithi077 ceph-mon[33531]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:12:18.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:18 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/2437367026' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aca3fac5-fbb5-42de-bf0e-e4a0db7381e2"}]: dispatch 2024-05-04T03:12:18.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:18 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/2437367026' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aca3fac5-fbb5-42de-bf0e-e4a0db7381e2"}]': finished 2024-05-04T03:12:18.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:18 smithi077 ceph-mon[33531]: osdmap e40: 7 total, 6 up, 7 in 2024-05-04T03:12:18.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:18 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:12:18.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:18 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/734241903' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:12:20.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:20 smithi072 ceph-mon[33332]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-05-04T03:12:20.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:20 smithi001 ceph-mon[29731]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-05-04T03:12:20.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:20 smithi077 ceph-mon[33531]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-05-04T03:12:22.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:22 smithi001 ceph-mon[29731]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-04T03:12:22.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:22 smithi072 ceph-mon[33332]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-04T03:12:22.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:22 smithi077 ceph-mon[33531]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-05-04T03:12:24.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:24 smithi001 ceph-mon[29731]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-04T03:12:24.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:24 smithi072 ceph-mon[33332]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-04T03:12:24.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:24 smithi077 ceph-mon[33531]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-04T03:12:25.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:25 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-04T03:12:25.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:25 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:25.746 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:25 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-04T03:12:25.746 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:25 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:25.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:25 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-04T03:12:25.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:25 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:26.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:26 smithi072 ceph-mon[33332]: Deploying daemon osd.6 on smithi001 2024-05-04T03:12:26.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:26 smithi072 ceph-mon[33332]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-04T03:12:26.795 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:26 smithi001 ceph-mon[29731]: Deploying daemon osd.6 on smithi001 2024-05-04T03:12:26.795 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:26 smithi001 ceph-mon[29731]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-04T03:12:26.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:26 smithi077 ceph-mon[33531]: Deploying daemon osd.6 on smithi001 2024-05-04T03:12:26.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:26 smithi077 ceph-mon[33531]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-04T03:12:28.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:28 smithi072 ceph-mon[33332]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:12:28.785 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:28 smithi001 ceph-mon[29731]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:12:28.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:28 smithi077 ceph-mon[33531]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:12:30.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:30 smithi072 ceph-mon[33332]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-04T03:12:30.746 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:30 smithi001 ceph-mon[29731]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-04T03:12:30.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:30 smithi077 ceph-mon[33531]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-05-04T03:12:32.027 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:31 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:32.027 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:31 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:32.027 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:31 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:12:32.027 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:31 smithi001 ceph-mon[29731]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:12:32.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:31 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:32.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:31 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:32.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:31 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:12:32.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:31 smithi077 ceph-mon[33531]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:12:32.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:31 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:32.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:31 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:32.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:31 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:12:32.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:31 smithi072 ceph-mon[33332]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:12:34.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:34.259+0000 7f6ee67fc700 1 -- 172.21.15.1:0/890028307 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6ef0063c00 con 0x7f6edc0b2880 2024-05-04T03:12:34.261 INFO:teuthology.orchestra.run.smithi001.stdout:Created osd(s) 6 on host 'smithi001' 2024-05-04T03:12:34.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:34.262+0000 7f6ef775a700 1 -- 172.21.15.1:0/890028307 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6edc0b2880 msgr2=0x7f6edc0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:34.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:34.262+0000 7f6ef775a700 1 --2- 172.21.15.1:0/890028307 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6edc0b2880 0x7f6edc0b4d90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000b5c0 tx=0x7f6ee001a040 comp rx=0 tx=0).stop 2024-05-04T03:12:34.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:34.262+0000 7f6ef775a700 1 -- 172.21.15.1:0/890028307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ef0104e30 msgr2=0x7f6ef019a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:34.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:34.262+0000 7f6ef775a700 1 --2- 172.21.15.1:0/890028307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ef0104e30 0x7f6ef019a8d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6eec00d950 tx=0x7f6eec00dc60 comp rx=0 tx=0).stop 2024-05-04T03:12:34.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:34.262+0000 7f6ef775a700 1 -- 172.21.15.1:0/890028307 shutdown_connections 2024-05-04T03:12:34.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:34.262+0000 7f6ef775a700 1 --2- 172.21.15.1:0/890028307 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ef010a0f0 0x7f6ef01a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:34.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:34.262+0000 7f6ef775a700 1 --2- 172.21.15.1:0/890028307 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6edc0b2880 0x7f6edc0b4d90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:34.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:34.262+0000 7f6ef775a700 1 --2- 172.21.15.1:0/890028307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ef0104e30 0x7f6ef019a8d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:34.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:34.262+0000 7f6ef775a700 1 --2- 172.21.15.1:0/890028307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ef0107790 0x7f6ef019ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:34.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:34.262+0000 7f6ef775a700 1 -- 172.21.15.1:0/890028307 >> 172.21.15.1:0/890028307 conn(0x7f6ef00fe870 msgr2=0x7f6ef0105900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:34.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:34.262+0000 7f6ef775a700 1 -- 172.21.15.1:0/890028307 shutdown_connections 2024-05-04T03:12:34.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:12:34.262+0000 7f6ef775a700 1 -- 172.21.15.1:0/890028307 wait complete. 2024-05-04T03:12:34.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:34 smithi077 ceph-mon[33531]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:12:34.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:34 smithi072 ceph-mon[33332]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:12:34.581 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:34 smithi001 ceph-mon[29731]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:12:34.938 DEBUG:teuthology.orchestra.run.smithi001:osd.6> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.6.service 2024-05-04T03:12:34.941 INFO:tasks.cephadm:Deploying osd.7 on smithi072 with /dev/vg_nvme/lv_2... 2024-05-04T03:12:34.941 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-04T03:12:35.187 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.b/config 2024-05-04T03:12:35.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:35 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:35.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:35 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:35.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:35 smithi072 ceph-mon[33332]: from='osd.6 [v2:172.21.15.1:6818/1799033649,v1:172.21.15.1:6819/1799033649]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-04T03:12:35.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:35 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:35.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:35 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:35.616 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:35 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:35.617 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:35 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:35.617 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:35 smithi001 ceph-mon[29731]: from='osd.6 [v2:172.21.15.1:6818/1799033649,v1:172.21.15.1:6819/1799033649]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-04T03:12:35.617 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:35 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:35.617 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:35 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:35.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:35 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:35.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:35 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:35.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:35 smithi077 ceph-mon[33531]: from='osd.6 [v2:172.21.15.1:6818/1799033649,v1:172.21.15.1:6819/1799033649]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-04T03:12:35.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:35 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:35.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:35 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:36.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:36 smithi001 ceph-mon[29731]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:12:36.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:36 smithi001 ceph-mon[29731]: from='osd.6 [v2:172.21.15.1:6818/1799033649,v1:172.21.15.1:6819/1799033649]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-04T03:12:36.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:36 smithi001 ceph-mon[29731]: osdmap e41: 7 total, 6 up, 7 in 2024-05-04T03:12:36.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:36 smithi001 ceph-mon[29731]: from='osd.6 [v2:172.21.15.1:6818/1799033649,v1:172.21.15.1:6819/1799033649]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-05-04T03:12:36.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:12:36.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:36 smithi072 ceph-mon[33332]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:12:36.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:36 smithi072 ceph-mon[33332]: from='osd.6 [v2:172.21.15.1:6818/1799033649,v1:172.21.15.1:6819/1799033649]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-04T03:12:36.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:36 smithi072 ceph-mon[33332]: osdmap e41: 7 total, 6 up, 7 in 2024-05-04T03:12:36.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:36 smithi072 ceph-mon[33332]: from='osd.6 [v2:172.21.15.1:6818/1799033649,v1:172.21.15.1:6819/1799033649]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-05-04T03:12:36.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:36 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:12:36.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:36 smithi077 ceph-mon[33531]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:12:36.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:36 smithi077 ceph-mon[33531]: from='osd.6 [v2:172.21.15.1:6818/1799033649,v1:172.21.15.1:6819/1799033649]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-04T03:12:36.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:36 smithi077 ceph-mon[33531]: osdmap e41: 7 total, 6 up, 7 in 2024-05-04T03:12:36.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:36 smithi077 ceph-mon[33531]: from='osd.6 [v2:172.21.15.1:6818/1799033649,v1:172.21.15.1:6819/1799033649]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-05-04T03:12:36.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:36 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:12:37.129 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:12:37 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:12:37.029+0000 7faeeffa7700 -1 osd.6 0 waiting for initial osdmap 2024-05-04T03:12:37.129 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:12:37 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:12:37.041+0000 7faee7d54700 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T03:12:37.277 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-04T03:12:37.277 INFO:teuthology.orchestra.run.smithi072.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-04T03:12:37.277 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10+0 records in 2024-05-04T03:12:37.277 INFO:teuthology.orchestra.run.smithi072.stderr:10+0 records out 2024-05-04T03:12:37.277 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113562 s, 923 MB/s 2024-05-04T03:12:37.277 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping successful for: 2024-05-04T03:12:37.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:37 smithi072 ceph-mon[33332]: from='osd.6 [v2:172.21.15.1:6818/1799033649,v1:172.21.15.1:6819/1799033649]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-05-04T03:12:37.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:37 smithi072 ceph-mon[33332]: osdmap e42: 7 total, 6 up, 7 in 2024-05-04T03:12:37.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:37 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:12:37.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:37 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:12:37.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:37 smithi072 ceph-mon[33332]: from='osd.6 [v2:172.21.15.1:6818/1799033649,v1:172.21.15.1:6819/1799033649]' entity='osd.6' 2024-05-04T03:12:37.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:37 smithi001 ceph-mon[29731]: from='osd.6 [v2:172.21.15.1:6818/1799033649,v1:172.21.15.1:6819/1799033649]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-05-04T03:12:37.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:37 smithi001 ceph-mon[29731]: osdmap e42: 7 total, 6 up, 7 in 2024-05-04T03:12:37.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:37 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:12:37.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:37 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:12:37.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:37 smithi001 ceph-mon[29731]: from='osd.6 [v2:172.21.15.1:6818/1799033649,v1:172.21.15.1:6819/1799033649]' entity='osd.6' 2024-05-04T03:12:37.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:37 smithi077 ceph-mon[33531]: from='osd.6 [v2:172.21.15.1:6818/1799033649,v1:172.21.15.1:6819/1799033649]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-05-04T03:12:37.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:37 smithi077 ceph-mon[33531]: osdmap e42: 7 total, 6 up, 7 in 2024-05-04T03:12:37.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:37 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:12:37.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:37 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:12:37.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:37 smithi077 ceph-mon[33531]: from='osd.6 [v2:172.21.15.1:6818/1799033649,v1:172.21.15.1:6819/1799033649]' entity='osd.6' 2024-05-04T03:12:37.968 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi072:vg_nvme/lv_2 2024-05-04T03:12:38.216 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.b/config 2024-05-04T03:12:38.238 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:38 smithi072 ceph-mon[33332]: purged_snaps scrub starts 2024-05-04T03:12:38.238 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:38 smithi072 ceph-mon[33332]: purged_snaps scrub ok 2024-05-04T03:12:38.238 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:38 smithi072 ceph-mon[33332]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:12:38.238 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:38 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:12:38.238 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:38 smithi072 ceph-mon[33332]: osd.6 [v2:172.21.15.1:6818/1799033649,v1:172.21.15.1:6819/1799033649] boot 2024-05-04T03:12:38.238 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:38 smithi072 ceph-mon[33332]: osdmap e43: 7 total, 7 up, 7 in 2024-05-04T03:12:38.239 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:38 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:12:38.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:38 smithi001 ceph-mon[29731]: purged_snaps scrub starts 2024-05-04T03:12:38.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:38 smithi001 ceph-mon[29731]: purged_snaps scrub ok 2024-05-04T03:12:38.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:38 smithi001 ceph-mon[29731]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:12:38.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:38 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:12:38.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:38 smithi001 ceph-mon[29731]: osd.6 [v2:172.21.15.1:6818/1799033649,v1:172.21.15.1:6819/1799033649] boot 2024-05-04T03:12:38.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:38 smithi001 ceph-mon[29731]: osdmap e43: 7 total, 7 up, 7 in 2024-05-04T03:12:38.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:38 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:12:38.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:38 smithi077 ceph-mon[33531]: purged_snaps scrub starts 2024-05-04T03:12:38.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:38 smithi077 ceph-mon[33531]: purged_snaps scrub ok 2024-05-04T03:12:38.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:38 smithi077 ceph-mon[33531]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 191 MiB used, 536 GiB / 536 GiB avail 2024-05-04T03:12:38.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:38 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:12:38.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:38 smithi077 ceph-mon[33531]: osd.6 [v2:172.21.15.1:6818/1799033649,v1:172.21.15.1:6819/1799033649] boot 2024-05-04T03:12:38.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:38 smithi077 ceph-mon[33531]: osdmap e43: 7 total, 7 up, 7 in 2024-05-04T03:12:38.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:38 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-04T03:12:39.694 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:39 smithi072 ceph-mon[33332]: Detected new or changed devices on smithi001 2024-05-04T03:12:39.694 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:39 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:39.694 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:39 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:39.694 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:39 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:12:39.695 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:39 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:12:39.695 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:39 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:12:39.695 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:39 smithi072 ceph-mon[33332]: Adjusting osd_memory_target on smithi001 to 5697M 2024-05-04T03:12:39.695 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:39 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:39.695 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:39 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:39.695 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:39 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:12:39.695 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:39 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:39.695 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:39 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:12:39.695 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:39 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:39.695 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:39 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:12:39.695 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:39 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:39.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.739+0000 7f5835ebc700 1 -- 172.21.15.72:0/395044927 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5830104370 msgr2=0x7f5830104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:39.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.739+0000 7f5835ebc700 1 --2- 172.21.15.72:0/395044927 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5830104370 0x7f5830104780 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5820009990 tx=0x7f5820009ca0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.739+0000 7f5835ebc700 1 -- 172.21.15.72:0/395044927 shutdown_connections 2024-05-04T03:12:39.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.739+0000 7f5835ebc700 1 --2- 172.21.15.72:0/395044927 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5830104370 0x7f5830104780 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.739+0000 7f5835ebc700 1 --2- 172.21.15.72:0/395044927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5830105e40 0x7f583010c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.739+0000 7f5835ebc700 1 --2- 172.21.15.72:0/395044927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f58301054a0 0x7f58301058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.739+0000 7f5835ebc700 1 -- 172.21.15.72:0/395044927 >> 172.21.15.72:0/395044927 conn(0x7f58300ff9d0 msgr2=0x7f5830101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:39.741 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.739+0000 7f5835ebc700 1 -- 172.21.15.72:0/395044927 shutdown_connections 2024-05-04T03:12:39.742 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.740+0000 7f5835ebc700 1 -- 172.21.15.72:0/395044927 wait complete. 2024-05-04T03:12:39.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.740+0000 7f5835ebc700 1 Processor -- start 2024-05-04T03:12:39.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.740+0000 7f5835ebc700 1 -- start start 2024-05-04T03:12:39.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.740+0000 7f5835ebc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5830104370 0x7f583019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:39.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.740+0000 7f5835ebc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58301054a0 0x7f583019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:39.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.741+0000 7f5835ebc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5830105e40 0x7f58301a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:39.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.741+0000 7f5835ebc700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f583010fdb0 con 0x7f5830105e40 2024-05-04T03:12:39.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.741+0000 7f5835ebc700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f583010fc30 con 0x7f58301054a0 2024-05-04T03:12:39.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.741+0000 7f5835ebc700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f583010ff30 con 0x7f5830104370 2024-05-04T03:12:39.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.741+0000 7f582effd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58301054a0 0x7f583019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:39.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.741+0000 7f582effd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58301054a0 0x7f583019ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:60462/0 (socket says 172.21.15.72:60462) 2024-05-04T03:12:39.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.741+0000 7f582effd700 1 -- 172.21.15.72:0/3418242204 learned_addr learned my addr 172.21.15.72:0/3418242204 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-05-04T03:12:39.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.741+0000 7f582f7fe700 1 --2- 172.21.15.72:0/3418242204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5830104370 0x7f583019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:39.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.741+0000 7f582ffff700 1 --2- 172.21.15.72:0/3418242204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5830105e40 0x7f58301a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:39.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.741+0000 7f582effd700 1 -- 172.21.15.72:0/3418242204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5830104370 msgr2=0x7f583019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:39.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.741+0000 7f582effd700 1 --2- 172.21.15.72:0/3418242204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5830104370 0x7f583019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.741+0000 7f582effd700 1 -- 172.21.15.72:0/3418242204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5830105e40 msgr2=0x7f58301a1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:39.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.741+0000 7f582effd700 1 --2- 172.21.15.72:0/3418242204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5830105e40 0x7f58301a1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:39.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.741+0000 7f582effd700 1 -- 172.21.15.72:0/3418242204 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58301a23c0 con 0x7f58301054a0 2024-05-04T03:12:39.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.741+0000 7f582effd700 1 --2- 172.21.15.72:0/3418242204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58301054a0 0x7f583019ae30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f582400c9b0 tx=0x7f582400ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:39.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.741+0000 7f582cff9700 1 -- 172.21.15.72:0/3418242204 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824007b00 con 0x7f58301054a0 2024-05-04T03:12:39.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.741+0000 7f582cff9700 1 -- 172.21.15.72:0/3418242204 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f582400ce50 con 0x7f58301054a0 2024-05-04T03:12:39.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.741+0000 7f5835ebc700 1 -- 172.21.15.72:0/3418242204 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58301a26b0 con 0x7f58301054a0 2024-05-04T03:12:39.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.741+0000 7f582cff9700 1 -- 172.21.15.72:0/3418242204 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824018660 con 0x7f58301054a0 2024-05-04T03:12:39.745 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.742+0000 7f5835ebc700 1 -- 172.21.15.72:0/3418242204 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58301a2bf0 con 0x7f58301054a0 2024-05-04T03:12:39.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.743+0000 7f5835ebc700 1 -- 172.21.15.72:0/3418242204 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58300512c0 con 0x7f58301054a0 2024-05-04T03:12:39.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.744+0000 7f582cff9700 1 -- 172.21.15.72:0/3418242204 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f582400f450 con 0x7f58301054a0 2024-05-04T03:12:39.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.745+0000 7f582cff9700 1 --2- 172.21.15.72:0/3418242204 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f58180b2880 0x7f58180b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:12:39.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.745+0000 7f582cff9700 1 -- 172.21.15.72:0/3418242204 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4965+0+0 (secure 0 0 0) 0x7f58240e1700 con 0x7f58301054a0 2024-05-04T03:12:39.746 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.745+0000 7f582f7fe700 1 --2- 172.21.15.72:0/3418242204 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f58180b2880 0x7f58180b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:12:39.747 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.746+0000 7f582f7fe700 1 --2- 172.21.15.72:0/3418242204 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f58180b2880 0x7f58180b4d90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f58200038a0 tx=0x7f582001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:12:39.750 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.749+0000 7f582cff9700 1 -- 172.21.15.72:0/3418242204 <== mon.2 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5824014030 con 0x7f58301054a0 2024-05-04T03:12:39.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:39 smithi001 ceph-mon[29731]: Detected new or changed devices on smithi001 2024-05-04T03:12:39.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:39 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:39.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:39 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:39.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:39 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:12:39.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:39 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:12:39.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:39 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:12:39.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:39 smithi001 ceph-mon[29731]: Adjusting osd_memory_target on smithi001 to 5697M 2024-05-04T03:12:39.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:39 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:39.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:39 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:39.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:39 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:12:39.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:39 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:39.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:39 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:12:39.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:39 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:39.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:39 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:12:39.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:39 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:39.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:39 smithi077 ceph-mon[33531]: Detected new or changed devices on smithi001 2024-05-04T03:12:39.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:39 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:39.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:39 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:39.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:39 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:12:39.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:39 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:12:39.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:39 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:12:39.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:39 smithi077 ceph-mon[33531]: Adjusting osd_memory_target on smithi001 to 5697M 2024-05-04T03:12:39.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:39 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:39.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:39 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:39.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:39 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:12:39.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:39 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:39.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:39 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:12:39.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:39 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:39.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:39 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:12:39.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:39 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:39.960 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:39.959+0000 7f5835ebc700 1 -- 172.21.15.72:0/3418242204 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5830063c00 con 0x7f58180b2880 2024-05-04T03:12:40.801 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:40 smithi072 ceph-mon[33332]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:40.801 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:40 smithi072 ceph-mon[33332]: osdmap e44: 7 total, 7 up, 7 in 2024-05-04T03:12:40.801 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:40 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:12:40.801 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:40 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:12:40.801 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:40 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:40.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:40 smithi001 ceph-mon[29731]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:40.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:40 smithi001 ceph-mon[29731]: osdmap e44: 7 total, 7 up, 7 in 2024-05-04T03:12:40.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:40 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:12:40.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:40 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:12:40.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:40 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:40.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:40 smithi077 ceph-mon[33531]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:40.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:40 smithi077 ceph-mon[33531]: osdmap e44: 7 total, 7 up, 7 in 2024-05-04T03:12:40.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:40 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:12:40.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:40 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:12:40.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:40 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:41.821 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:41 smithi072 ceph-mon[33332]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:12:41.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:41 smithi001 ceph-mon[29731]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:12:41.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:41 smithi077 ceph-mon[33531]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:12:42.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:42 smithi001 ceph-mon[29731]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:42.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:42 smithi077 ceph-mon[33531]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:42.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:42 smithi072 ceph-mon[33332]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:43.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:43 smithi001 ceph-mon[29731]: from='client.? 172.21.15.72:0/3092152097' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbff6b8a-c463-4249-ae45-fdc6f99a5489"}]: dispatch 2024-05-04T03:12:43.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:43 smithi001 ceph-mon[29731]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbff6b8a-c463-4249-ae45-fdc6f99a5489"}]: dispatch 2024-05-04T03:12:43.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:43 smithi001 ceph-mon[29731]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbff6b8a-c463-4249-ae45-fdc6f99a5489"}]': finished 2024-05-04T03:12:43.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:43 smithi001 ceph-mon[29731]: osdmap e45: 8 total, 7 up, 8 in 2024-05-04T03:12:43.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:43 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:12:43.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:43 smithi001 ceph-mon[29731]: from='client.? 172.21.15.72:0/619471158' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:12:43.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:43 smithi077 ceph-mon[33531]: from='client.? 172.21.15.72:0/3092152097' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbff6b8a-c463-4249-ae45-fdc6f99a5489"}]: dispatch 2024-05-04T03:12:43.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:43 smithi077 ceph-mon[33531]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbff6b8a-c463-4249-ae45-fdc6f99a5489"}]: dispatch 2024-05-04T03:12:43.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:43 smithi077 ceph-mon[33531]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbff6b8a-c463-4249-ae45-fdc6f99a5489"}]': finished 2024-05-04T03:12:43.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:43 smithi077 ceph-mon[33531]: osdmap e45: 8 total, 7 up, 8 in 2024-05-04T03:12:43.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:43 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:12:43.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:43 smithi077 ceph-mon[33531]: from='client.? 172.21.15.72:0/619471158' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:12:43.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:43 smithi072 ceph-mon[33332]: from='client.? 172.21.15.72:0/3092152097' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbff6b8a-c463-4249-ae45-fdc6f99a5489"}]: dispatch 2024-05-04T03:12:43.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:43 smithi072 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbff6b8a-c463-4249-ae45-fdc6f99a5489"}]: dispatch 2024-05-04T03:12:43.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:43 smithi072 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbff6b8a-c463-4249-ae45-fdc6f99a5489"}]': finished 2024-05-04T03:12:43.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:43 smithi072 ceph-mon[33332]: osdmap e45: 8 total, 7 up, 8 in 2024-05-04T03:12:43.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:43 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:12:43.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:43 smithi072 ceph-mon[33332]: from='client.? 172.21.15.72:0/619471158' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:12:44.829 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:44 smithi077 ceph-mon[33531]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:44.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:44 smithi001 ceph-mon[29731]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:44.897 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:44 smithi072 ceph-mon[33332]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:46.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:46 smithi001 ceph-mon[29731]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:46.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:46 smithi077 ceph-mon[33531]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:46.958 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:46 smithi072 ceph-mon[33332]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:48.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:48 smithi001 ceph-mon[29731]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:48.884 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:48 smithi072 ceph-mon[33332]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:48.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:48 smithi077 ceph-mon[33531]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:50.827 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:50 smithi072 ceph-mon[33332]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:50.827 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:50 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-04T03:12:50.828 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:50 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:50.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:50 smithi001 ceph-mon[29731]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:50.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:50 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-04T03:12:50.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:50 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:50.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:50 smithi077 ceph-mon[33531]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:50.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:50 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-04T03:12:50.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:50 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:12:51.579 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:51 smithi072 ceph-mon[33332]: Deploying daemon osd.7 on smithi072 2024-05-04T03:12:51.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:51 smithi001 ceph-mon[29731]: Deploying daemon osd.7 on smithi072 2024-05-04T03:12:51.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:51 smithi077 ceph-mon[33531]: Deploying daemon osd.7 on smithi072 2024-05-04T03:12:52.770 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:52 smithi072 ceph-mon[33332]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:52.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:52 smithi001 ceph-mon[29731]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:52.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:52 smithi077 ceph-mon[33531]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:54.627 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:54 smithi072 ceph-mon[33332]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:54.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:54 smithi001 ceph-mon[29731]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:54.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:54 smithi077 ceph-mon[33531]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:56.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:56 smithi072 ceph-mon[33332]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:56.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:56 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:56.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:56 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:56.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:56 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:12:56.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:56 smithi001 ceph-mon[29731]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:56.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:56 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:56.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:56 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:56.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:56 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:12:56.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:56 smithi077 ceph-mon[33531]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:56.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:56 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:56.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:56 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:12:56.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:56 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:12:58.732 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:58 smithi072 ceph-mon[33332]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:58.779 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:58.777+0000 7f582cff9700 1 -- 172.21.15.72:0/3418242204 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5830063c00 con 0x7f58180b2880 2024-05-04T03:12:58.780 INFO:teuthology.orchestra.run.smithi072.stdout:Created osd(s) 7 on host 'smithi072' 2024-05-04T03:12:58.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:58.780+0000 7f5835ebc700 1 -- 172.21.15.72:0/3418242204 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f58180b2880 msgr2=0x7f58180b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:58.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:58.780+0000 7f5835ebc700 1 --2- 172.21.15.72:0/3418242204 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f58180b2880 0x7f58180b4d90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f58200038a0 tx=0x7f582001a040 comp rx=0 tx=0).stop 2024-05-04T03:12:58.781 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:58.780+0000 7f5835ebc700 1 -- 172.21.15.72:0/3418242204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58301054a0 msgr2=0x7f583019ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:12:58.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:58.780+0000 7f5835ebc700 1 --2- 172.21.15.72:0/3418242204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58301054a0 0x7f583019ae30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f582400c9b0 tx=0x7f582400ccc0 comp rx=0 tx=0).stop 2024-05-04T03:12:58.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:58.780+0000 7f5835ebc700 1 -- 172.21.15.72:0/3418242204 shutdown_connections 2024-05-04T03:12:58.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:58.780+0000 7f5835ebc700 1 --2- 172.21.15.72:0/3418242204 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f58180b2880 0x7f58180b4d90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:58.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:58.780+0000 7f5835ebc700 1 --2- 172.21.15.72:0/3418242204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5830105e40 0x7f58301a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:58.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:58.780+0000 7f5835ebc700 1 --2- 172.21.15.72:0/3418242204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5830104370 0x7f583019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:58.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:58.780+0000 7f5835ebc700 1 --2- 172.21.15.72:0/3418242204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f58301054a0 0x7f583019ae30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:12:58.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:58.780+0000 7f5835ebc700 1 -- 172.21.15.72:0/3418242204 >> 172.21.15.72:0/3418242204 conn(0x7f58300ff9d0 msgr2=0x7f58301086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:12:58.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:58.780+0000 7f5835ebc700 1 -- 172.21.15.72:0/3418242204 shutdown_connections 2024-05-04T03:12:58.782 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:12:58.780+0000 7f5835ebc700 1 -- 172.21.15.72:0/3418242204 wait complete. 2024-05-04T03:12:58.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:58 smithi001 ceph-mon[29731]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:58.930 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:58 smithi077 ceph-mon[33531]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:12:59.390 DEBUG:teuthology.orchestra.run.smithi072:osd.7> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.7.service 2024-05-04T03:12:59.393 INFO:tasks.cephadm:Deploying osd.8 on smithi077 with /dev/vg_nvme/lv_2... 2024-05-04T03:12:59.393 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-04T03:12:59.642 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:13:00.086 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:59 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:00.086 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:59 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:00.086 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:59 smithi077 ceph-mon[33531]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:13:00.086 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:59 smithi077 ceph-mon[33531]: from='osd.7 [v2:172.21.15.72:6816/3460835447,v1:172.21.15.72:6817/3460835447]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-04T03:13:00.086 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:59 smithi077 ceph-mon[33531]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-04T03:13:00.086 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:59 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:00.086 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:12:59 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:00.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:59 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:00.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:59 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:00.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:59 smithi001 ceph-mon[29731]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:13:00.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:59 smithi001 ceph-mon[29731]: from='osd.7 [v2:172.21.15.72:6816/3460835447,v1:172.21.15.72:6817/3460835447]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-04T03:13:00.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:59 smithi001 ceph-mon[29731]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-04T03:13:00.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:59 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:00.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:12:59 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:00.144 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:59 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:00.144 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:59 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:00.144 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:59 smithi072 ceph-mon[33332]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:13:00.144 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:59 smithi072 ceph-mon[33332]: from='osd.7 [v2:172.21.15.72:6816/3460835447,v1:172.21.15.72:6817/3460835447]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-04T03:13:00.144 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:59 smithi072 ceph-mon[33332]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-04T03:13:00.145 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:59 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:00.145 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:12:59 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:01.035 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:00 smithi077 ceph-mon[33531]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-04T03:13:01.035 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:00 smithi077 ceph-mon[33531]: osdmap e46: 8 total, 7 up, 8 in 2024-05-04T03:13:01.035 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:00 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:13:01.035 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:00 smithi077 ceph-mon[33531]: from='osd.7 [v2:172.21.15.72:6816/3460835447,v1:172.21.15.72:6817/3460835447]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-04T03:13:01.035 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:00 smithi077 ceph-mon[33531]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-04T03:13:01.120 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:00 smithi072 ceph-mon[33332]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-04T03:13:01.120 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:00 smithi072 ceph-mon[33332]: osdmap e46: 8 total, 7 up, 8 in 2024-05-04T03:13:01.121 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:00 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:13:01.121 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:00 smithi072 ceph-mon[33332]: from='osd.7 [v2:172.21.15.72:6816/3460835447,v1:172.21.15.72:6817/3460835447]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-04T03:13:01.121 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:00 smithi072 ceph-mon[33332]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-04T03:13:01.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:00 smithi001 ceph-mon[29731]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-04T03:13:01.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:00 smithi001 ceph-mon[29731]: osdmap e46: 8 total, 7 up, 8 in 2024-05-04T03:13:01.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:00 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:13:01.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:00 smithi001 ceph-mon[29731]: from='osd.7 [v2:172.21.15.72:6816/3460835447,v1:172.21.15.72:6817/3460835447]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-04T03:13:01.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:00 smithi001 ceph-mon[29731]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-04T03:13:01.578 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-04T03:13:01.582 INFO:teuthology.orchestra.run.smithi077.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-04T03:13:01.582 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10+0 records in 2024-05-04T03:13:01.583 INFO:teuthology.orchestra.run.smithi077.stderr:10+0 records out 2024-05-04T03:13:01.583 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130439 s, 804 MB/s 2024-05-04T03:13:01.583 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping successful for: 2024-05-04T03:13:01.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:01 smithi072 ceph-mon[33332]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-04T03:13:01.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:01 smithi072 ceph-mon[33332]: osdmap e47: 8 total, 7 up, 8 in 2024-05-04T03:13:01.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:01 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:13:01.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:01 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:13:01.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:01 smithi072 ceph-mon[33332]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:13:01.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:01 smithi072 ceph-mon[33332]: from='osd.7 ' entity='osd.7' 2024-05-04T03:13:01.997 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:13:01 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:13:01.704+0000 7f20a4bdb700 -1 osd.7 0 waiting for initial osdmap 2024-05-04T03:13:01.997 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:13:01 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:13:01.873+0000 7f209c988700 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T03:13:02.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:01 smithi001 ceph-mon[29731]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-04T03:13:02.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:01 smithi001 ceph-mon[29731]: osdmap e47: 8 total, 7 up, 8 in 2024-05-04T03:13:02.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:01 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:13:02.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:01 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:13:02.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:01 smithi001 ceph-mon[29731]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:13:02.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:01 smithi001 ceph-mon[29731]: from='osd.7 ' entity='osd.7' 2024-05-04T03:13:02.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:01 smithi077 ceph-mon[33531]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-04T03:13:02.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:01 smithi077 ceph-mon[33531]: osdmap e47: 8 total, 7 up, 8 in 2024-05-04T03:13:02.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:01 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:13:02.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:01 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:13:02.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:01 smithi077 ceph-mon[33531]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:13:02.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:01 smithi077 ceph-mon[33531]: from='osd.7 ' entity='osd.7' 2024-05-04T03:13:02.205 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_2 2024-05-04T03:13:02.445 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:13:02.911 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:02 smithi072 ceph-mon[33332]: purged_snaps scrub starts 2024-05-04T03:13:02.912 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:02 smithi072 ceph-mon[33332]: purged_snaps scrub ok 2024-05-04T03:13:02.912 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:02 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:13:02.912 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:02 smithi072 ceph-mon[33332]: osdmap e48: 8 total, 7 up, 8 in 2024-05-04T03:13:02.912 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:02 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:13:02.912 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:02 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:13:03.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:02 smithi001 ceph-mon[29731]: purged_snaps scrub starts 2024-05-04T03:13:03.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:02 smithi001 ceph-mon[29731]: purged_snaps scrub ok 2024-05-04T03:13:03.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:02 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:13:03.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:02 smithi001 ceph-mon[29731]: osdmap e48: 8 total, 7 up, 8 in 2024-05-04T03:13:03.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:02 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:13:03.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:02 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:13:03.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:02 smithi077 ceph-mon[33531]: purged_snaps scrub starts 2024-05-04T03:13:03.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:02 smithi077 ceph-mon[33531]: purged_snaps scrub ok 2024-05-04T03:13:03.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:02 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:13:03.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:02 smithi077 ceph-mon[33531]: osdmap e48: 8 total, 7 up, 8 in 2024-05-04T03:13:03.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:02 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:13:03.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:02 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:13:03.872 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:03 smithi077 ceph-mon[33531]: osd.7 [v2:172.21.15.72:6816/3460835447,v1:172.21.15.72:6817/3460835447] boot 2024-05-04T03:13:03.873 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:03 smithi077 ceph-mon[33531]: osdmap e49: 8 total, 8 up, 8 in 2024-05-04T03:13:03.873 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:13:03.873 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:03 smithi077 ceph-mon[33531]: Detected new or changed devices on smithi072 2024-05-04T03:13:03.873 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:03.873 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:03.873 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:03.873 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:03.873 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:03.873 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:03 smithi077 ceph-mon[33531]: Adjusting osd_memory_target on smithi072 to 5696M 2024-05-04T03:13:03.873 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:03.873 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:03.873 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:13:03.874 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:03.874 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:03 smithi077 ceph-mon[33531]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:13:03.874 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:13:03.874 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:03.874 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:13:03.874 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:04.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.108+0000 7f35ee57e700 1 -- 172.21.15.77:0/3334561504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e8104d20 msgr2=0x7f35e810b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:04.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.108+0000 7f35ee57e700 1 --2- 172.21.15.77:0/3334561504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e8104d20 0x7f35e810b5d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f35e000b0a0 tx=0x7f35e000b3b0 comp rx=0 tx=0).stop 2024-05-04T03:13:04.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.108+0000 7f35ee57e700 1 -- 172.21.15.77:0/3334561504 shutdown_connections 2024-05-04T03:13:04.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.108+0000 7f35ee57e700 1 --2- 172.21.15.77:0/3334561504 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35e8103250 0x7f35e8103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:04.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.108+0000 7f35ee57e700 1 --2- 172.21.15.77:0/3334561504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e8104d20 0x7f35e810b5d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:04.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.108+0000 7f35ee57e700 1 --2- 172.21.15.77:0/3334561504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35e8104380 0x7f35e8104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:04.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.109+0000 7f35ee57e700 1 -- 172.21.15.77:0/3334561504 >> 172.21.15.77:0/3334561504 conn(0x7f35e80fe890 msgr2=0x7f35e8100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:04.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.109+0000 7f35ee57e700 1 -- 172.21.15.77:0/3334561504 shutdown_connections 2024-05-04T03:13:04.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.109+0000 7f35ee57e700 1 -- 172.21.15.77:0/3334561504 wait complete. 2024-05-04T03:13:04.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.109+0000 7f35ee57e700 1 Processor -- start 2024-05-04T03:13:04.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.109+0000 7f35ee57e700 1 -- start start 2024-05-04T03:13:04.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.110+0000 7f35ee57e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35e8103250 0x7f35e8076450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:04.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.110+0000 7f35ee57e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e8104380 0x7f35e819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:04.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.110+0000 7f35ee57e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35e8104d20 0x7f35e81a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:04.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.110+0000 7f35ee57e700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f35e810ed30 con 0x7f35e8103250 2024-05-04T03:13:04.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.110+0000 7f35ee57e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f35e810ebb0 con 0x7f35e8104d20 2024-05-04T03:13:04.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.110+0000 7f35ee57e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f35e810eeb0 con 0x7f35e8104380 2024-05-04T03:13:04.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.110+0000 7f35e77fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e8104380 0x7f35e819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:04.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.110+0000 7f35e77fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e8104380 0x7f35e819f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:35904/0 (socket says 172.21.15.77:35904) 2024-05-04T03:13:04.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.110+0000 7f35e77fe700 1 -- 172.21.15.77:0/162011855 learned_addr learned my addr 172.21.15.77:0/162011855 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-04T03:13:04.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.110+0000 7f35e7fff700 1 --2- 172.21.15.77:0/162011855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35e8103250 0x7f35e8076450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:04.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.110+0000 7f35ecb1b700 1 --2- 172.21.15.77:0/162011855 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35e8104d20 0x7f35e81a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:04.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.110+0000 7f35e77fe700 1 -- 172.21.15.77:0/162011855 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35e8104d20 msgr2=0x7f35e81a1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:04.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.110+0000 7f35e77fe700 1 --2- 172.21.15.77:0/162011855 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35e8104d20 0x7f35e81a1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:04.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.110+0000 7f35e77fe700 1 -- 172.21.15.77:0/162011855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35e8103250 msgr2=0x7f35e8076450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:04.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.110+0000 7f35e77fe700 1 --2- 172.21.15.77:0/162011855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35e8103250 0x7f35e8076450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:04.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.110+0000 7f35e77fe700 1 -- 172.21.15.77:0/162011855 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e81a2440 con 0x7f35e8104380 2024-05-04T03:13:04.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.110+0000 7f35ecb1b700 1 --2- 172.21.15.77:0/162011855 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35e8104d20 0x7f35e81a1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:13:04.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.110+0000 7f35e77fe700 1 --2- 172.21.15.77:0/162011855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e8104380 0x7f35e819f6a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f35d800c920 tx=0x7f35d800cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:04.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.110+0000 7f35e57fa700 1 -- 172.21.15.77:0/162011855 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8007b00 con 0x7f35e8104380 2024-05-04T03:13:04.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.110+0000 7f35e57fa700 1 -- 172.21.15.77:0/162011855 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35d8004ce0 con 0x7f35e8104380 2024-05-04T03:13:04.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.110+0000 7f35ee57e700 1 -- 172.21.15.77:0/162011855 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e81a2730 con 0x7f35e8104380 2024-05-04T03:13:04.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.110+0000 7f35e57fa700 1 -- 172.21.15.77:0/162011855 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8005550 con 0x7f35e8104380 2024-05-04T03:13:04.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.110+0000 7f35ee57e700 1 -- 172.21.15.77:0/162011855 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e81a2cf0 con 0x7f35e8104380 2024-05-04T03:13:04.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.111+0000 7f35ee57e700 1 -- 172.21.15.77:0/162011855 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e80512c0 con 0x7f35e8104380 2024-05-04T03:13:04.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.113+0000 7f35e57fa700 1 -- 172.21.15.77:0/162011855 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f35d8007ca0 con 0x7f35e8104380 2024-05-04T03:13:04.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.113+0000 7f35e57fa700 1 --2- 172.21.15.77:0/162011855 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f35d40b2880 0x7f35d40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:04.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.114+0000 7f35e57fa700 1 -- 172.21.15.77:0/162011855 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5397+0+0 (secure 0 0 0) 0x7f35d80e1620 con 0x7f35e8104380 2024-05-04T03:13:04.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.114+0000 7f35e7fff700 1 --2- 172.21.15.77:0/162011855 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f35d40b2880 0x7f35d40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:04.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.114+0000 7f35e7fff700 1 --2- 172.21.15.77:0/162011855 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f35d40b2880 0x7f35d40b4d90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f35e8104280 tx=0x7f35d0009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:04.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.117+0000 7f35e57fa700 1 -- 172.21.15.77:0/162011855 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f35d8014030 con 0x7f35e8104380 2024-05-04T03:13:04.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:03 smithi001 ceph-mon[29731]: osd.7 [v2:172.21.15.72:6816/3460835447,v1:172.21.15.72:6817/3460835447] boot 2024-05-04T03:13:04.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:03 smithi001 ceph-mon[29731]: osdmap e49: 8 total, 8 up, 8 in 2024-05-04T03:13:04.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:13:04.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:03 smithi001 ceph-mon[29731]: Detected new or changed devices on smithi072 2024-05-04T03:13:04.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:04.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:04.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:04.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:04.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:04.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:03 smithi001 ceph-mon[29731]: Adjusting osd_memory_target on smithi072 to 5696M 2024-05-04T03:13:04.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:04.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:04.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:13:04.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:04.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:03 smithi001 ceph-mon[29731]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:13:04.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:13:04.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:04.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:13:04.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:04.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:03 smithi072 ceph-mon[33332]: osd.7 [v2:172.21.15.72:6816/3460835447,v1:172.21.15.72:6817/3460835447] boot 2024-05-04T03:13:04.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:03 smithi072 ceph-mon[33332]: osdmap e49: 8 total, 8 up, 8 in 2024-05-04T03:13:04.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-04T03:13:04.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:03 smithi072 ceph-mon[33332]: Detected new or changed devices on smithi072 2024-05-04T03:13:04.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:04.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:04.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:04.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:04.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:04.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:03 smithi072 ceph-mon[33332]: Adjusting osd_memory_target on smithi072 to 5696M 2024-05-04T03:13:04.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:04.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:04.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:13:04.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:04.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:03 smithi072 ceph-mon[33332]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 626 GiB / 626 GiB avail 2024-05-04T03:13:04.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:13:04.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:04.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:13:04.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:04.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:04.327+0000 7f35ee57e700 1 -- 172.21.15.77:0/162011855 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f35e8063c00 con 0x7f35d40b2880 2024-05-04T03:13:05.073 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:04 smithi077 ceph-mon[33531]: osdmap e50: 8 total, 8 up, 8 in 2024-05-04T03:13:05.074 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:04 smithi077 ceph-mon[33531]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:13:05.074 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:04 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:13:05.074 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:04 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:13:05.074 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:04 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:05.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:04 smithi001 ceph-mon[29731]: osdmap e50: 8 total, 8 up, 8 in 2024-05-04T03:13:05.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:04 smithi001 ceph-mon[29731]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:13:05.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:04 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:13:05.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:04 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:13:05.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:04 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:05.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:04 smithi072 ceph-mon[33332]: osdmap e50: 8 total, 8 up, 8 in 2024-05-04T03:13:05.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:04 smithi072 ceph-mon[33332]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:13:05.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:04 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:13:05.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:04 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:13:05.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:04 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:06.055 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:05 smithi077 ceph-mon[33531]: osdmap e51: 8 total, 8 up, 8 in 2024-05-04T03:13:06.055 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:05 smithi077 ceph-mon[33531]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:13:06.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:05 smithi001 ceph-mon[29731]: osdmap e51: 8 total, 8 up, 8 in 2024-05-04T03:13:06.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:05 smithi001 ceph-mon[29731]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:13:06.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:05 smithi072 ceph-mon[33332]: osdmap e51: 8 total, 8 up, 8 in 2024-05-04T03:13:06.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:05 smithi072 ceph-mon[33332]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:13:07.344 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:07 smithi077 ceph-mon[33531]: from='client.? 172.21.15.77:0/3447937493' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e140bf13-4887-4d68-9373-3d4fc1c4584a"}]: dispatch 2024-05-04T03:13:07.344 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:07 smithi077 ceph-mon[33531]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e140bf13-4887-4d68-9373-3d4fc1c4584a"}]: dispatch 2024-05-04T03:13:07.344 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:07 smithi077 ceph-mon[33531]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e140bf13-4887-4d68-9373-3d4fc1c4584a"}]': finished 2024-05-04T03:13:07.344 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:07 smithi077 ceph-mon[33531]: osdmap e52: 9 total, 8 up, 9 in 2024-05-04T03:13:07.344 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:07 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:13:07.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:07 smithi001 ceph-mon[29731]: from='client.? 172.21.15.77:0/3447937493' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e140bf13-4887-4d68-9373-3d4fc1c4584a"}]: dispatch 2024-05-04T03:13:07.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:07 smithi001 ceph-mon[29731]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e140bf13-4887-4d68-9373-3d4fc1c4584a"}]: dispatch 2024-05-04T03:13:07.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:07 smithi001 ceph-mon[29731]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e140bf13-4887-4d68-9373-3d4fc1c4584a"}]': finished 2024-05-04T03:13:07.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:07 smithi001 ceph-mon[29731]: osdmap e52: 9 total, 8 up, 9 in 2024-05-04T03:13:07.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:07 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:13:07.498 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:07 smithi072 ceph-mon[33332]: from='client.? 172.21.15.77:0/3447937493' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e140bf13-4887-4d68-9373-3d4fc1c4584a"}]: dispatch 2024-05-04T03:13:07.499 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:07 smithi072 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e140bf13-4887-4d68-9373-3d4fc1c4584a"}]: dispatch 2024-05-04T03:13:07.499 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:07 smithi072 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e140bf13-4887-4d68-9373-3d4fc1c4584a"}]': finished 2024-05-04T03:13:07.499 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:07 smithi072 ceph-mon[33332]: osdmap e52: 9 total, 8 up, 9 in 2024-05-04T03:13:07.499 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:07 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:13:08.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:08 smithi001 ceph-mon[29731]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:13:08.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:08 smithi001 ceph-mon[29731]: from='client.? 172.21.15.77:0/1356196820' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:13:08.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:08 smithi077 ceph-mon[33531]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:13:08.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:08 smithi077 ceph-mon[33531]: from='client.? 172.21.15.77:0/1356196820' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:13:08.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:08 smithi072 ceph-mon[33332]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:13:08.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:08 smithi072 ceph-mon[33332]: from='client.? 172.21.15.77:0/1356196820' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:13:10.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:10 smithi077 ceph-mon[33531]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-04T03:13:10.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:10 smithi072 ceph-mon[33332]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-04T03:13:10.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:10 smithi001 ceph-mon[29731]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-04T03:13:12.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:12 smithi077 ceph-mon[33531]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-04T03:13:12.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:12 smithi072 ceph-mon[33332]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-04T03:13:12.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:12 smithi001 ceph-mon[29731]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-04T03:13:14.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:14 smithi077 ceph-mon[33531]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-04T03:13:14.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:14 smithi072 ceph-mon[33332]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-04T03:13:14.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:14 smithi001 ceph-mon[29731]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-04T03:13:15.308 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:15 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-04T03:13:15.308 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:15 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:15.308 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:15 smithi077 ceph-mon[33531]: Deploying daemon osd.8 on smithi077 2024-05-04T03:13:15.492 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:15 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-04T03:13:15.492 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:15 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:15.492 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:15 smithi072 ceph-mon[33332]: Deploying daemon osd.8 on smithi077 2024-05-04T03:13:15.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:15 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-04T03:13:15.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:15 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:15.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:15 smithi001 ceph-mon[29731]: Deploying daemon osd.8 on smithi077 2024-05-04T03:13:16.204 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:16 smithi077 ceph-mon[33531]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:13:16.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:16 smithi072 ceph-mon[33332]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:13:16.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:16 smithi001 ceph-mon[29731]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:13:18.314 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:18 smithi077 ceph-mon[33531]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-05-04T03:13:18.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:18 smithi072 ceph-mon[33332]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-05-04T03:13:18.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:18 smithi001 ceph-mon[29731]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-05-04T03:13:20.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:20 smithi077 ceph-mon[33531]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-04T03:13:20.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:20 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:20.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:20 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:20.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:20 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:13:20.431 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:20 smithi001 ceph-mon[29731]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-04T03:13:20.431 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:20 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:20.431 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:20 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:20.431 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:20 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:13:20.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:20 smithi072 ceph-mon[33332]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-04T03:13:20.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:20 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:20.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:20 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:20.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:20 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:13:22.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:22 smithi077 ceph-mon[33531]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:13:22.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:22 smithi072 ceph-mon[33332]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:13:22.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:22 smithi001 ceph-mon[29731]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:13:23.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:23.948+0000 7f35e57fa700 1 -- 172.21.15.77:0/162011855 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f35e8063c00 con 0x7f35d40b2880 2024-05-04T03:13:23.949 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 8 on host 'smithi077' 2024-05-04T03:13:23.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:23.950+0000 7f35ee57e700 1 -- 172.21.15.77:0/162011855 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f35d40b2880 msgr2=0x7f35d40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:23.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:23.951+0000 7f35ee57e700 1 --2- 172.21.15.77:0/162011855 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f35d40b2880 0x7f35d40b4d90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f35e8104280 tx=0x7f35d0009190 comp rx=0 tx=0).stop 2024-05-04T03:13:23.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:23.951+0000 7f35ee57e700 1 -- 172.21.15.77:0/162011855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e8104380 msgr2=0x7f35e819f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:23.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:23.951+0000 7f35ee57e700 1 --2- 172.21.15.77:0/162011855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e8104380 0x7f35e819f6a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f35d800c920 tx=0x7f35d800cc30 comp rx=0 tx=0).stop 2024-05-04T03:13:23.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:23.951+0000 7f35ee57e700 1 -- 172.21.15.77:0/162011855 shutdown_connections 2024-05-04T03:13:23.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:23.951+0000 7f35ee57e700 1 --2- 172.21.15.77:0/162011855 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35e8104d20 0x7f35e81a1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:23.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:23.951+0000 7f35ee57e700 1 --2- 172.21.15.77:0/162011855 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f35d40b2880 0x7f35d40b4d90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:23.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:23.951+0000 7f35ee57e700 1 --2- 172.21.15.77:0/162011855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f35e8103250 0x7f35e8076450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:23.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:23.951+0000 7f35ee57e700 1 --2- 172.21.15.77:0/162011855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35e8104380 0x7f35e819f6a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:23.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:23.951+0000 7f35ee57e700 1 -- 172.21.15.77:0/162011855 >> 172.21.15.77:0/162011855 conn(0x7f35e80fe890 msgr2=0x7f35e81075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:23.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:23.951+0000 7f35ee57e700 1 -- 172.21.15.77:0/162011855 shutdown_connections 2024-05-04T03:13:23.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:13:23.951+0000 7f35ee57e700 1 -- 172.21.15.77:0/162011855 wait complete. 2024-05-04T03:13:24.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:23 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:24.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:23 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:24.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:23 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:24.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:23 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:13:24.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:23 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:24.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:23 smithi077 ceph-mon[33531]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:13:24.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:23 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:24.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:23 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:24.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:23 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:24.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:23 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:13:24.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:23 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:24.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:23 smithi072 ceph-mon[33332]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:13:24.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:23 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:24.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:23 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:24.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:23 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:24.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:23 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:13:24.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:23 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:24.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:23 smithi001 ceph-mon[29731]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:13:24.574 DEBUG:teuthology.orchestra.run.smithi077:osd.8> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.8.service 2024-05-04T03:13:24.576 INFO:tasks.cephadm:Deploying osd.9 on smithi001 with /dev/vg_nvme/lv_1... 2024-05-04T03:13:24.576 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-04T03:13:24.835 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:13:25.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:24 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:25.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:24 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:25.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:24 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:13:25.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:24 smithi077 ceph-mon[33531]: from='osd.8 [v2:172.21.15.77:6816/1124479323,v1:172.21.15.77:6817/1124479323]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-04T03:13:25.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:24 smithi077 ceph-mon[33531]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-04T03:13:25.205 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:24 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:25.206 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:24 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:25.206 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:24 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:13:25.206 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:24 smithi001 ceph-mon[29731]: from='osd.8 [v2:172.21.15.77:6816/1124479323,v1:172.21.15.77:6817/1124479323]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-04T03:13:25.206 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:24 smithi001 ceph-mon[29731]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-04T03:13:25.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:24 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:25.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:24 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:25.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:24 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:13:25.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:24 smithi072 ceph-mon[33332]: from='osd.8 [v2:172.21.15.77:6816/1124479323,v1:172.21.15.77:6817/1124479323]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-04T03:13:25.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:24 smithi072 ceph-mon[33332]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-04T03:13:26.057 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:25 smithi077 ceph-mon[33531]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-04T03:13:26.057 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:25 smithi077 ceph-mon[33531]: osdmap e53: 9 total, 8 up, 9 in 2024-05-04T03:13:26.057 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:25 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:13:26.057 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:25 smithi077 ceph-mon[33531]: from='osd.8 [v2:172.21.15.77:6816/1124479323,v1:172.21.15.77:6817/1124479323]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-04T03:13:26.057 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:25 smithi077 ceph-mon[33531]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-04T03:13:26.058 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:25 smithi077 ceph-mon[33531]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:13:26.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:25 smithi072 ceph-mon[33332]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-04T03:13:26.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:25 smithi072 ceph-mon[33332]: osdmap e53: 9 total, 8 up, 9 in 2024-05-04T03:13:26.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:25 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:13:26.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:25 smithi072 ceph-mon[33332]: from='osd.8 [v2:172.21.15.77:6816/1124479323,v1:172.21.15.77:6817/1124479323]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-04T03:13:26.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:25 smithi072 ceph-mon[33332]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-04T03:13:26.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:25 smithi072 ceph-mon[33332]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:13:26.273 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:25 smithi001 ceph-mon[29731]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-04T03:13:26.273 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:25 smithi001 ceph-mon[29731]: osdmap e53: 9 total, 8 up, 9 in 2024-05-04T03:13:26.274 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:25 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:13:26.274 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:25 smithi001 ceph-mon[29731]: from='osd.8 [v2:172.21.15.77:6816/1124479323,v1:172.21.15.77:6817/1124479323]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-04T03:13:26.274 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:25 smithi001 ceph-mon[29731]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-04T03:13:26.274 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:25 smithi001 ceph-mon[29731]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:13:27.069 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-04T03:13:27.069 INFO:teuthology.orchestra.run.smithi001.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-04T03:13:27.069 INFO:teuthology.orchestra.run.smithi001.stderr: stderr: 10+0 records in 2024-05-04T03:13:27.069 INFO:teuthology.orchestra.run.smithi001.stderr:10+0 records out 2024-05-04T03:13:27.069 INFO:teuthology.orchestra.run.smithi001.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00858938 s, 1.2 GB/s 2024-05-04T03:13:27.069 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping successful for: 2024-05-04T03:13:27.177 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:26 smithi077 ceph-mon[33531]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-05-04T03:13:27.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:26 smithi077 ceph-mon[33531]: osdmap e54: 9 total, 8 up, 9 in 2024-05-04T03:13:27.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:26 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:13:27.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:26 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:13:27.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:26 smithi077 ceph-mon[33531]: from='osd.8 ' entity='osd.8' 2024-05-04T03:13:27.178 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:13:26 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:13:26.827+0000 7feeed5b5700 -1 osd.8 0 waiting for initial osdmap 2024-05-04T03:13:27.178 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:13:26 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:13:26.959+0000 7feee5362700 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T03:13:27.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:26 smithi072 ceph-mon[33332]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-05-04T03:13:27.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:26 smithi072 ceph-mon[33332]: osdmap e54: 9 total, 8 up, 9 in 2024-05-04T03:13:27.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:26 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:13:27.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:26 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:13:27.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:26 smithi072 ceph-mon[33332]: from='osd.8 ' entity='osd.8' 2024-05-04T03:13:27.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:26 smithi001 ceph-mon[29731]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-05-04T03:13:27.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:26 smithi001 ceph-mon[29731]: osdmap e54: 9 total, 8 up, 9 in 2024-05-04T03:13:27.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:26 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:13:27.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:26 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:13:27.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:26 smithi001 ceph-mon[29731]: from='osd.8 ' entity='osd.8' 2024-05-04T03:13:27.919 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi001:vg_nvme/lv_1 2024-05-04T03:13:28.170 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:13:28.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:27 smithi077 ceph-mon[33531]: purged_snaps scrub starts 2024-05-04T03:13:28.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:27 smithi077 ceph-mon[33531]: purged_snaps scrub ok 2024-05-04T03:13:28.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:27 smithi077 ceph-mon[33531]: osdmap e55: 9 total, 8 up, 9 in 2024-05-04T03:13:28.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:27 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:13:28.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:27 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:13:28.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:27 smithi077 ceph-mon[33531]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:13:28.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:27 smithi077 ceph-mon[33531]: Detected new or changed devices on smithi077 2024-05-04T03:13:28.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:27 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:28.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:27 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:28.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:27 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:28.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:27 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:28.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:27 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:28.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:27 smithi077 ceph-mon[33531]: Adjusting osd_memory_target on smithi077 to 5696M 2024-05-04T03:13:28.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:27 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:28.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:27 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:28.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:27 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:13:28.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:27 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:28.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:27 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:13:28.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:27 smithi072 ceph-mon[33332]: purged_snaps scrub starts 2024-05-04T03:13:28.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:27 smithi072 ceph-mon[33332]: purged_snaps scrub ok 2024-05-04T03:13:28.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:27 smithi072 ceph-mon[33332]: osdmap e55: 9 total, 8 up, 9 in 2024-05-04T03:13:28.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:27 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:13:28.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:27 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:13:28.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:27 smithi072 ceph-mon[33332]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:13:28.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:27 smithi072 ceph-mon[33332]: Detected new or changed devices on smithi077 2024-05-04T03:13:28.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:27 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:28.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:27 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:28.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:27 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:28.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:27 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:28.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:27 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:28.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:27 smithi072 ceph-mon[33332]: Adjusting osd_memory_target on smithi077 to 5696M 2024-05-04T03:13:28.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:27 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:28.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:27 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:28.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:27 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:13:28.248 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:27 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:28.248 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:27 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:13:28.292 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:27 smithi001 ceph-mon[29731]: purged_snaps scrub starts 2024-05-04T03:13:28.292 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:27 smithi001 ceph-mon[29731]: purged_snaps scrub ok 2024-05-04T03:13:28.292 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:27 smithi001 ceph-mon[29731]: osdmap e55: 9 total, 8 up, 9 in 2024-05-04T03:13:28.292 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:27 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:13:28.292 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:27 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:13:28.292 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:27 smithi001 ceph-mon[29731]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-05-04T03:13:28.292 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:27 smithi001 ceph-mon[29731]: Detected new or changed devices on smithi077 2024-05-04T03:13:28.293 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:27 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:28.293 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:27 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:28.293 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:27 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:28.293 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:27 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:28.293 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:27 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:28.293 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:27 smithi001 ceph-mon[29731]: Adjusting osd_memory_target on smithi077 to 5696M 2024-05-04T03:13:28.293 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:27 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:28.293 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:27 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:28.293 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:27 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:13:28.293 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:27 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:28.293 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:27 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:13:29.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:28 smithi001 ceph-mon[29731]: osd.8 [v2:172.21.15.77:6816/1124479323,v1:172.21.15.77:6817/1124479323] boot 2024-05-04T03:13:29.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:28 smithi001 ceph-mon[29731]: osdmap e56: 9 total, 9 up, 9 in 2024-05-04T03:13:29.197 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:28 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:13:29.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:28 smithi072 ceph-mon[33332]: osd.8 [v2:172.21.15.77:6816/1124479323,v1:172.21.15.77:6817/1124479323] boot 2024-05-04T03:13:29.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:28 smithi072 ceph-mon[33332]: osdmap e56: 9 total, 9 up, 9 in 2024-05-04T03:13:29.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:28 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:13:29.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:28 smithi077 ceph-mon[33531]: osd.8 [v2:172.21.15.77:6816/1124479323,v1:172.21.15.77:6817/1124479323] boot 2024-05-04T03:13:29.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:28 smithi077 ceph-mon[33531]: osdmap e56: 9 total, 9 up, 9 in 2024-05-04T03:13:29.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:28 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-04T03:13:29.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.668+0000 7f14dfb96700 1 -- 172.21.15.1:0/1512752462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14d80787f0 msgr2=0x7f14d8076cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:29.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.668+0000 7f14dfb96700 1 --2- 172.21.15.1:0/1512752462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14d80787f0 0x7f14d8076cc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009990 tx=0x7f14d4009ca0 comp rx=0 tx=0).stop 2024-05-04T03:13:29.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.668+0000 7f14dfb96700 1 -- 172.21.15.1:0/1512752462 shutdown_connections 2024-05-04T03:13:29.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.668+0000 7f14dfb96700 1 --2- 172.21.15.1:0/1512752462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14d810c220 0x7f14d810e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:29.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.668+0000 7f14dfb96700 1 --2- 172.21.15.1:0/1512752462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14d80787f0 0x7f14d8076cc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:29.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.668+0000 7f14dfb96700 1 --2- 172.21.15.1:0/1512752462 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14d8077200 0x7f14d8077690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:29.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.668+0000 7f14dfb96700 1 -- 172.21.15.1:0/1512752462 >> 172.21.15.1:0/1512752462 conn(0x7f14d80ff9d0 msgr2=0x7f14d8101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:29.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.668+0000 7f14dfb96700 1 -- 172.21.15.1:0/1512752462 shutdown_connections 2024-05-04T03:13:29.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.668+0000 7f14dfb96700 1 -- 172.21.15.1:0/1512752462 wait complete. 2024-05-04T03:13:29.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.669+0000 7f14dfb96700 1 Processor -- start 2024-05-04T03:13:29.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.669+0000 7f14dfb96700 1 -- start start 2024-05-04T03:13:29.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.669+0000 7f14dfb96700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14d8077200 0x7f14d819cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:29.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.669+0000 7f14dfb96700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14d80787f0 0x7f14d819d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:29.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.669+0000 7f14dfb96700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14d810c220 0x7f14d81a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:29.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.669+0000 7f14dfb96700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f14d8112000 con 0x7f14d80787f0 2024-05-04T03:13:29.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.669+0000 7f14dfb96700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f14d8111e80 con 0x7f14d810c220 2024-05-04T03:13:29.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.669+0000 7f14dfb96700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f14d8112180 con 0x7f14d8077200 2024-05-04T03:13:29.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.670+0000 7f14dd131700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14d80787f0 0x7f14d819d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:29.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.670+0000 7f14dd131700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14d80787f0 0x7f14d819d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43478/0 (socket says 172.21.15.1:43478) 2024-05-04T03:13:29.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.670+0000 7f14dd131700 1 -- 172.21.15.1:0/1348556545 learned_addr learned my addr 172.21.15.1:0/1348556545 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:13:29.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.670+0000 7f14dd932700 1 --2- 172.21.15.1:0/1348556545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14d8077200 0x7f14d819cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:29.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.670+0000 7f14de133700 1 --2- 172.21.15.1:0/1348556545 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14d810c220 0x7f14d81a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:29.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.670+0000 7f14dd131700 1 -- 172.21.15.1:0/1348556545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14d8077200 msgr2=0x7f14d819cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:29.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.670+0000 7f14dd131700 1 --2- 172.21.15.1:0/1348556545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14d8077200 0x7f14d819cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:29.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.670+0000 7f14dd131700 1 -- 172.21.15.1:0/1348556545 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14d810c220 msgr2=0x7f14d81a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:29.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.670+0000 7f14dd131700 1 --2- 172.21.15.1:0/1348556545 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14d810c220 0x7f14d81a4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:29.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.670+0000 7f14dd131700 1 -- 172.21.15.1:0/1348556545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d81a46a0 con 0x7f14d80787f0 2024-05-04T03:13:29.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.670+0000 7f14dd131700 1 --2- 172.21.15.1:0/1348556545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14d80787f0 0x7f14d819d080 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f14c800c920 tx=0x7f14c800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:29.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.670+0000 7f14ceffd700 1 -- 172.21.15.1:0/1348556545 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8004a30 con 0x7f14d80787f0 2024-05-04T03:13:29.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.670+0000 7f14dfb96700 1 -- 172.21.15.1:0/1348556545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d81a4990 con 0x7f14d80787f0 2024-05-04T03:13:29.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.670+0000 7f14ceffd700 1 -- 172.21.15.1:0/1348556545 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14c80054d0 con 0x7f14d80787f0 2024-05-04T03:13:29.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.670+0000 7f14ceffd700 1 -- 172.21.15.1:0/1348556545 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8007c70 con 0x7f14d80787f0 2024-05-04T03:13:29.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.670+0000 7f14dfb96700 1 -- 172.21.15.1:0/1348556545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d81a4ed0 con 0x7f14d80787f0 2024-05-04T03:13:29.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.672+0000 7f14dfb96700 1 -- 172.21.15.1:0/1348556545 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14d80512c0 con 0x7f14d80787f0 2024-05-04T03:13:29.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.673+0000 7f14ceffd700 1 -- 172.21.15.1:0/1348556545 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f14c8004c60 con 0x7f14d80787f0 2024-05-04T03:13:29.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.674+0000 7f14ceffd700 1 --2- 172.21.15.1:0/1348556545 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f14c40b2880 0x7f14c40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:29.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.674+0000 7f14dd932700 1 --2- 172.21.15.1:0/1348556545 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f14c40b2880 0x7f14c40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:29.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.674+0000 7f14ceffd700 1 -- 172.21.15.1:0/1348556545 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5829+0+0 (secure 0 0 0) 0x7f14c80e1820 con 0x7f14d80787f0 2024-05-04T03:13:29.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.674+0000 7f14dd932700 1 --2- 172.21.15.1:0/1348556545 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f14c40b2880 0x7f14c40b4d90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f14d40038a0 tx=0x7f14d401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:29.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.677+0000 7f14ceffd700 1 -- 172.21.15.1:0/1348556545 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f14c8014030 con 0x7f14d80787f0 2024-05-04T03:13:29.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:29.901+0000 7f14dfb96700 1 -- 172.21.15.1:0/1348556545 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f14d8063c00 con 0x7f14c40b2880 2024-05-04T03:13:30.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:29 smithi072 ceph-mon[33332]: osdmap e57: 9 total, 9 up, 9 in 2024-05-04T03:13:30.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:29 smithi072 ceph-mon[33332]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:13:30.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:29 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:13:30.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:29 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:13:30.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:29 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:30.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:29 smithi001 ceph-mon[29731]: osdmap e57: 9 total, 9 up, 9 in 2024-05-04T03:13:30.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:29 smithi001 ceph-mon[29731]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:13:30.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:29 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:13:30.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:29 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:13:30.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:29 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:30.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:29 smithi077 ceph-mon[33531]: osdmap e57: 9 total, 9 up, 9 in 2024-05-04T03:13:30.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:29 smithi077 ceph-mon[33531]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:13:30.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:29 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:13:30.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:29 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:13:30.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:29 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:31.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:30 smithi001 ceph-mon[29731]: from='client.14397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:13:31.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:30 smithi001 ceph-mon[29731]: osdmap e58: 9 total, 9 up, 9 in 2024-05-04T03:13:31.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:30 smithi072 ceph-mon[33332]: from='client.14397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:13:31.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:30 smithi072 ceph-mon[33332]: osdmap e58: 9 total, 9 up, 9 in 2024-05-04T03:13:31.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:30 smithi077 ceph-mon[33531]: from='client.14397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:13:31.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:30 smithi077 ceph-mon[33531]: osdmap e58: 9 total, 9 up, 9 in 2024-05-04T03:13:32.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:31 smithi001 ceph-mon[29731]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:13:32.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:31 smithi072 ceph-mon[33332]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:13:32.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:31 smithi077 ceph-mon[33531]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 289 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:13:33.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:32 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/4007610468' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1e8a232-cb11-4b9d-8789-e1637fa5aaa8"}]: dispatch 2024-05-04T03:13:33.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:32 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/4007610468' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1e8a232-cb11-4b9d-8789-e1637fa5aaa8"}]': finished 2024-05-04T03:13:33.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:32 smithi072 ceph-mon[33332]: osdmap e59: 10 total, 9 up, 10 in 2024-05-04T03:13:33.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:32 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:13:33.378 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:32 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/4007610468' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1e8a232-cb11-4b9d-8789-e1637fa5aaa8"}]: dispatch 2024-05-04T03:13:33.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:32 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/4007610468' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1e8a232-cb11-4b9d-8789-e1637fa5aaa8"}]': finished 2024-05-04T03:13:33.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:32 smithi001 ceph-mon[29731]: osdmap e59: 10 total, 9 up, 10 in 2024-05-04T03:13:33.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:32 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:13:33.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:32 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/4007610468' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1e8a232-cb11-4b9d-8789-e1637fa5aaa8"}]: dispatch 2024-05-04T03:13:33.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:32 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/4007610468' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1e8a232-cb11-4b9d-8789-e1637fa5aaa8"}]': finished 2024-05-04T03:13:33.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:32 smithi077 ceph-mon[33531]: osdmap e59: 10 total, 9 up, 10 in 2024-05-04T03:13:33.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:32 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:13:34.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:33 smithi072 ceph-mon[33332]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 292 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:13:34.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:33 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/993128199' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:13:34.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:33 smithi001 ceph-mon[29731]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 292 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:13:34.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:33 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/993128199' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:13:34.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:33 smithi077 ceph-mon[33531]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 292 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:13:34.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:33 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/993128199' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:13:36.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:36 smithi077 ceph-mon[33531]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-04T03:13:36.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:36 smithi072 ceph-mon[33332]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-04T03:13:36.607 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:36 smithi001 ceph-mon[29731]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-04T03:13:38.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:38 smithi001 ceph-mon[29731]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-04T03:13:38.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:38 smithi077 ceph-mon[33531]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-04T03:13:38.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:38 smithi072 ceph-mon[33332]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-04T03:13:40.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:40 smithi077 ceph-mon[33531]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-04T03:13:40.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:40 smithi072 ceph-mon[33332]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-04T03:13:40.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:40 smithi001 ceph-mon[29731]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-05-04T03:13:41.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:41 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-04T03:13:41.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:41 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:41.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:41 smithi077 ceph-mon[33531]: Deploying daemon osd.9 on smithi001 2024-05-04T03:13:41.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:41 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-04T03:13:41.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:41 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:41.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:41 smithi072 ceph-mon[33332]: Deploying daemon osd.9 on smithi001 2024-05-04T03:13:41.572 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:41 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-04T03:13:41.572 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:41 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:41.572 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:41 smithi001 ceph-mon[29731]: Deploying daemon osd.9 on smithi001 2024-05-04T03:13:42.276 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:42 smithi001 ceph-mon[29731]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:13:42.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:42 smithi077 ceph-mon[33531]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:13:42.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:42 smithi072 ceph-mon[33332]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:13:44.427 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:44 smithi077 ceph-mon[33531]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-04T03:13:44.476 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:44 smithi001 ceph-mon[29731]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-04T03:13:44.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:44 smithi072 ceph-mon[33332]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-04T03:13:46.250 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:46 smithi001 ceph-mon[29731]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-04T03:13:46.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:46 smithi072 ceph-mon[33332]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-04T03:13:46.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:46 smithi077 ceph-mon[33531]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-04T03:13:47.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:47 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:47.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:47 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:47.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:47 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:13:47.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:47 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:47.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:47 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:47.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:47 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:13:47.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:47 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:47.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:47 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:47.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:47 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:13:48.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:48 smithi001 ceph-mon[29731]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:13:48.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:48 smithi077 ceph-mon[33531]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:13:48.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:48 smithi072 ceph-mon[33332]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:13:50.210 INFO:teuthology.orchestra.run.smithi001.stdout:Created osd(s) 9 on host 'smithi001' 2024-05-04T03:13:50.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:50.208+0000 7f14ceffd700 1 -- 172.21.15.1:0/1348556545 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f14d8063c00 con 0x7f14c40b2880 2024-05-04T03:13:50.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:50.211+0000 7f14dfb96700 1 -- 172.21.15.1:0/1348556545 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f14c40b2880 msgr2=0x7f14c40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:50.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:50.211+0000 7f14dfb96700 1 --2- 172.21.15.1:0/1348556545 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f14c40b2880 0x7f14c40b4d90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f14d40038a0 tx=0x7f14d401a040 comp rx=0 tx=0).stop 2024-05-04T03:13:50.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:50.211+0000 7f14dfb96700 1 -- 172.21.15.1:0/1348556545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14d80787f0 msgr2=0x7f14d819d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:50.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:50.211+0000 7f14dfb96700 1 --2- 172.21.15.1:0/1348556545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14d80787f0 0x7f14d819d080 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f14c800c920 tx=0x7f14c800cc30 comp rx=0 tx=0).stop 2024-05-04T03:13:50.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:50.211+0000 7f14dfb96700 1 -- 172.21.15.1:0/1348556545 shutdown_connections 2024-05-04T03:13:50.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:50.211+0000 7f14dfb96700 1 --2- 172.21.15.1:0/1348556545 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f14d810c220 0x7f14d81a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:50.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:50.211+0000 7f14dfb96700 1 --2- 172.21.15.1:0/1348556545 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f14c40b2880 0x7f14c40b4d90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:50.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:50.211+0000 7f14dfb96700 1 --2- 172.21.15.1:0/1348556545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14d8077200 0x7f14d819cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:50.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:50.211+0000 7f14dfb96700 1 --2- 172.21.15.1:0/1348556545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f14d80787f0 0x7f14d819d080 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:50.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:50.211+0000 7f14dfb96700 1 -- 172.21.15.1:0/1348556545 >> 172.21.15.1:0/1348556545 conn(0x7f14d80ff9d0 msgr2=0x7f14d81081b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:50.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:50.211+0000 7f14dfb96700 1 -- 172.21.15.1:0/1348556545 shutdown_connections 2024-05-04T03:13:50.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:13:50.211+0000 7f14dfb96700 1 -- 172.21.15.1:0/1348556545 wait complete. 2024-05-04T03:13:50.496 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:50 smithi001 ceph-mon[29731]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:13:50.496 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:50 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:50.496 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:50 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:50.496 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:50 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:50.496 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:50 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:13:50.496 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:50 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:50.496 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:50 smithi001 ceph-mon[29731]: from='osd.9 [v2:172.21.15.1:6826/2681322272,v1:172.21.15.1:6827/2681322272]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-04T03:13:50.496 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:50 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:13:50.497 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:50 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:50.497 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:50 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:50.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:50 smithi077 ceph-mon[33531]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:13:50.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:50 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:50.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:50 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:50.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:50 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:50.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:50 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:13:50.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:50 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:50.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:50 smithi077 ceph-mon[33531]: from='osd.9 [v2:172.21.15.1:6826/2681322272,v1:172.21.15.1:6827/2681322272]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-04T03:13:50.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:50 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:13:50.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:50 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:50.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:50 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:50.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:50 smithi072 ceph-mon[33332]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:13:50.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:50 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:50.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:50 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:50.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:50 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:50.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:50 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:13:50.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:50 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:50.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:50 smithi072 ceph-mon[33332]: from='osd.9 [v2:172.21.15.1:6826/2681322272,v1:172.21.15.1:6827/2681322272]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-04T03:13:50.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:50 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:13:50.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:50 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:50.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:50 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:50.805 DEBUG:teuthology.orchestra.run.smithi001:osd.9> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.9.service 2024-05-04T03:13:50.807 INFO:tasks.cephadm:Deploying osd.10 on smithi072 with /dev/vg_nvme/lv_1... 2024-05-04T03:13:50.807 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-04T03:13:51.052 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.b/config 2024-05-04T03:13:51.481 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:51 smithi072 ceph-mon[33332]: from='osd.9 [v2:172.21.15.1:6826/2681322272,v1:172.21.15.1:6827/2681322272]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-04T03:13:51.482 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:51 smithi072 ceph-mon[33332]: osdmap e60: 10 total, 9 up, 10 in 2024-05-04T03:13:51.482 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:51 smithi072 ceph-mon[33332]: from='osd.9 [v2:172.21.15.1:6826/2681322272,v1:172.21.15.1:6827/2681322272]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-05-04T03:13:51.482 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:51 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:13:51.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:51 smithi077 ceph-mon[33531]: from='osd.9 [v2:172.21.15.1:6826/2681322272,v1:172.21.15.1:6827/2681322272]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-04T03:13:51.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:51 smithi077 ceph-mon[33531]: osdmap e60: 10 total, 9 up, 10 in 2024-05-04T03:13:51.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:51 smithi077 ceph-mon[33531]: from='osd.9 [v2:172.21.15.1:6826/2681322272,v1:172.21.15.1:6827/2681322272]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-05-04T03:13:51.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:51 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:13:51.811 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:51 smithi001 ceph-mon[29731]: from='osd.9 [v2:172.21.15.1:6826/2681322272,v1:172.21.15.1:6827/2681322272]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-04T03:13:51.811 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:51 smithi001 ceph-mon[29731]: osdmap e60: 10 total, 9 up, 10 in 2024-05-04T03:13:51.811 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:51 smithi001 ceph-mon[29731]: from='osd.9 [v2:172.21.15.1:6826/2681322272,v1:172.21.15.1:6827/2681322272]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-05-04T03:13:51.811 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:51 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:13:51.812 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:13:51 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:13:51.386+0000 7fc0d82e5700 -1 osd.9 0 waiting for initial osdmap 2024-05-04T03:13:51.812 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:13:51 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:13:51.399+0000 7fc0d3899700 -1 osd.9 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T03:13:52.508 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:52 smithi001 ceph-mon[29731]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:13:52.509 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:52 smithi001 ceph-mon[29731]: from='osd.9 [v2:172.21.15.1:6826/2681322272,v1:172.21.15.1:6827/2681322272]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-05-04T03:13:52.509 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:52 smithi001 ceph-mon[29731]: osdmap e61: 10 total, 9 up, 10 in 2024-05-04T03:13:52.509 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:52 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:13:52.509 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:52 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:13:52.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:52 smithi077 ceph-mon[33531]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:13:52.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:52 smithi077 ceph-mon[33531]: from='osd.9 [v2:172.21.15.1:6826/2681322272,v1:172.21.15.1:6827/2681322272]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-05-04T03:13:52.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:52 smithi077 ceph-mon[33531]: osdmap e61: 10 total, 9 up, 10 in 2024-05-04T03:13:52.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:52 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:13:52.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:52 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:13:52.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:52 smithi072 ceph-mon[33332]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 804 GiB / 805 GiB avail 2024-05-04T03:13:52.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:52 smithi072 ceph-mon[33332]: from='osd.9 [v2:172.21.15.1:6826/2681322272,v1:172.21.15.1:6827/2681322272]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-05-04T03:13:52.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:52 smithi072 ceph-mon[33332]: osdmap e61: 10 total, 9 up, 10 in 2024-05-04T03:13:52.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:52 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:13:52.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:52 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:13:52.898 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-04T03:13:52.898 INFO:teuthology.orchestra.run.smithi072.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-04T03:13:52.898 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10+0 records in 2024-05-04T03:13:52.899 INFO:teuthology.orchestra.run.smithi072.stderr:10+0 records out 2024-05-04T03:13:52.899 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113206 s, 926 MB/s 2024-05-04T03:13:52.899 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping successful for: 2024-05-04T03:13:53.494 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:53 smithi072 ceph-mon[33332]: purged_snaps scrub starts 2024-05-04T03:13:53.494 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:53 smithi072 ceph-mon[33332]: purged_snaps scrub ok 2024-05-04T03:13:53.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:53 smithi072 ceph-mon[33332]: osd.9 [v2:172.21.15.1:6826/2681322272,v1:172.21.15.1:6827/2681322272] boot 2024-05-04T03:13:53.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:53 smithi072 ceph-mon[33332]: osdmap e62: 10 total, 10 up, 10 in 2024-05-04T03:13:53.495 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:53 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:13:53.521 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi072:vg_nvme/lv_1 2024-05-04T03:13:53.522 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:53 smithi001 ceph-mon[29731]: purged_snaps scrub starts 2024-05-04T03:13:53.522 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:53 smithi001 ceph-mon[29731]: purged_snaps scrub ok 2024-05-04T03:13:53.522 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:53 smithi001 ceph-mon[29731]: osd.9 [v2:172.21.15.1:6826/2681322272,v1:172.21.15.1:6827/2681322272] boot 2024-05-04T03:13:53.522 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:53 smithi001 ceph-mon[29731]: osdmap e62: 10 total, 10 up, 10 in 2024-05-04T03:13:53.522 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:53 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:13:53.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:53 smithi077 ceph-mon[33531]: purged_snaps scrub starts 2024-05-04T03:13:53.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:53 smithi077 ceph-mon[33531]: purged_snaps scrub ok 2024-05-04T03:13:53.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:53 smithi077 ceph-mon[33531]: osd.9 [v2:172.21.15.1:6826/2681322272,v1:172.21.15.1:6827/2681322272] boot 2024-05-04T03:13:53.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:53 smithi077 ceph-mon[33531]: osdmap e62: 10 total, 10 up, 10 in 2024-05-04T03:13:53.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:53 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-04T03:13:53.766 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.b/config 2024-05-04T03:13:54.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:54 smithi077 ceph-mon[33531]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:13:54.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:54 smithi077 ceph-mon[33531]: osdmap e63: 10 total, 10 up, 10 in 2024-05-04T03:13:54.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:54 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:54.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:54 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:54.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:54 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:54.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:54 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:54.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:54 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:54.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:54 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:54.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:54 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:54.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:54 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:54.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:54 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:13:54.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:54 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:54.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:54 smithi072 ceph-mon[33332]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:13:54.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:54 smithi072 ceph-mon[33332]: osdmap e63: 10 total, 10 up, 10 in 2024-05-04T03:13:54.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:54 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:54.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:54 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:54.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:54 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:54.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:54 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:54.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:54 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:54.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:54 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:54.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:54 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:54.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:54 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:54.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:54 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:13:54.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:54 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:54.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:54 smithi001 ceph-mon[29731]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:13:54.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:54 smithi001 ceph-mon[29731]: osdmap e63: 10 total, 10 up, 10 in 2024-05-04T03:13:54.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:54 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:54.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:54 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:54.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:54 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:54.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:54 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:54.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:54 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:54.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:54 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:13:54.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:54 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:54.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:54 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:54.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:54 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:13:54.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:54 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:13:55.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.327+0000 7f1631b6c700 1 -- 172.21.15.72:0/3425167179 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f162c105e40 msgr2=0x7f162c10c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:55.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.327+0000 7f1631b6c700 1 --2- 172.21.15.72:0/3425167179 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f162c105e40 0x7f162c10c6f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f162400b0a0 tx=0x7f162400b3b0 comp rx=0 tx=0).stop 2024-05-04T03:13:55.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.327+0000 7f1631b6c700 1 -- 172.21.15.72:0/3425167179 shutdown_connections 2024-05-04T03:13:55.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.327+0000 7f1631b6c700 1 --2- 172.21.15.72:0/3425167179 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f162c105e40 0x7f162c10c6f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:55.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.327+0000 7f1631b6c700 1 --2- 172.21.15.72:0/3425167179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f162c104370 0x7f162c104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:55.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.327+0000 7f1631b6c700 1 --2- 172.21.15.72:0/3425167179 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f162c1054a0 0x7f162c1058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:55.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.327+0000 7f1631b6c700 1 -- 172.21.15.72:0/3425167179 >> 172.21.15.72:0/3425167179 conn(0x7f162c0ff9d0 msgr2=0x7f162c101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:13:55.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.327+0000 7f1631b6c700 1 -- 172.21.15.72:0/3425167179 shutdown_connections 2024-05-04T03:13:55.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.327+0000 7f1631b6c700 1 -- 172.21.15.72:0/3425167179 wait complete. 2024-05-04T03:13:55.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.327+0000 7f1631b6c700 1 Processor -- start 2024-05-04T03:13:55.332 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.328+0000 7f1631b6c700 1 -- start start 2024-05-04T03:13:55.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.328+0000 7f1631b6c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f162c104370 0x7f162c19a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:55.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.328+0000 7f1631b6c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f162c1054a0 0x7f162c19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:55.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.328+0000 7f1631b6c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f162c105e40 0x7f162c1a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:55.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.328+0000 7f1631b6c700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f162c10fe70 con 0x7f162c1054a0 2024-05-04T03:13:55.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.328+0000 7f1631b6c700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f162c10fcf0 con 0x7f162c104370 2024-05-04T03:13:55.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.328+0000 7f1631b6c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f162c10fff0 con 0x7f162c105e40 2024-05-04T03:13:55.333 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.328+0000 7f162b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f162c104370 0x7f162c19a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:55.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.328+0000 7f162b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f162c104370 0x7f162c19a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:46292/0 (socket says 172.21.15.72:46292) 2024-05-04T03:13:55.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.328+0000 7f162b7fe700 1 -- 172.21.15.72:0/1226021598 learned_addr learned my addr 172.21.15.72:0/1226021598 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-05-04T03:13:55.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.328+0000 7f162bfff700 1 --2- 172.21.15.72:0/1226021598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f162c105e40 0x7f162c1a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:55.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.328+0000 7f162affd700 1 --2- 172.21.15.72:0/1226021598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f162c1054a0 0x7f162c19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:55.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.328+0000 7f162b7fe700 1 -- 172.21.15.72:0/1226021598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f162c105e40 msgr2=0x7f162c1a1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:55.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.328+0000 7f162b7fe700 1 --2- 172.21.15.72:0/1226021598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f162c105e40 0x7f162c1a1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:55.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.328+0000 7f162b7fe700 1 -- 172.21.15.72:0/1226021598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f162c1054a0 msgr2=0x7f162c19ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:13:55.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.328+0000 7f162b7fe700 1 --2- 172.21.15.72:0/1226021598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f162c1054a0 0x7f162c19ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:13:55.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.328+0000 7f162b7fe700 1 -- 172.21.15.72:0/1226021598 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162c1a2420 con 0x7f162c104370 2024-05-04T03:13:55.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.329+0000 7f162b7fe700 1 --2- 172.21.15.72:0/1226021598 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f162c104370 0x7f162c19a8c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f161c00d950 tx=0x7f161c00dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:55.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.329+0000 7f1628ff9700 1 -- 172.21.15.72:0/1226021598 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c017070 con 0x7f162c104370 2024-05-04T03:13:55.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.329+0000 7f1628ff9700 1 -- 172.21.15.72:0/1226021598 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f161c00de30 con 0x7f162c104370 2024-05-04T03:13:55.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.329+0000 7f1631b6c700 1 -- 172.21.15.72:0/1226021598 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f162c1a2710 con 0x7f162c104370 2024-05-04T03:13:55.334 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.329+0000 7f1628ff9700 1 -- 172.21.15.72:0/1226021598 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c00f3c0 con 0x7f162c104370 2024-05-04T03:13:55.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.329+0000 7f1631b6c700 1 -- 172.21.15.72:0/1226021598 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f162c1a2cd0 con 0x7f162c104370 2024-05-04T03:13:55.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.330+0000 7f1631b6c700 1 -- 172.21.15.72:0/1226021598 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f162c0512c0 con 0x7f162c104370 2024-05-04T03:13:55.335 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.331+0000 7f1628ff9700 1 -- 172.21.15.72:0/1226021598 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f161c01a070 con 0x7f162c104370 2024-05-04T03:13:55.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.332+0000 7f1628ff9700 1 --2- 172.21.15.72:0/1226021598 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f16140b2900 0x7f16140b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:13:55.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.332+0000 7f1628ff9700 1 -- 172.21.15.72:0/1226021598 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6261+0+0 (secure 0 0 0) 0x7f161c0e1310 con 0x7f162c104370 2024-05-04T03:13:55.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.332+0000 7f162affd700 1 --2- 172.21.15.72:0/1226021598 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f16140b2900 0x7f16140b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:13:55.336 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.333+0000 7f162affd700 1 --2- 172.21.15.72:0/1226021598 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f16140b2900 0x7f16140b4e10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1620006fd0 tx=0x7f1620008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:13:55.338 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.337+0000 7f1628ff9700 1 -- 172.21.15.72:0/1226021598 <== mon.2 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f161c010460 con 0x7f162c104370 2024-05-04T03:13:55.551 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:13:55.550+0000 7f1631b6c700 1 -- 172.21.15.72:0/1226021598 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f162c063c00 con 0x7f16140b2900 2024-05-04T03:13:55.551 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:55 smithi072 ceph-mon[33332]: Detected new or changed devices on smithi001 2024-05-04T03:13:55.552 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:55 smithi072 ceph-mon[33332]: Adjusting osd_memory_target on smithi001 to 4273M 2024-05-04T03:13:55.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:55 smithi077 ceph-mon[33531]: Detected new or changed devices on smithi001 2024-05-04T03:13:55.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:55 smithi077 ceph-mon[33531]: Adjusting osd_memory_target on smithi001 to 4273M 2024-05-04T03:13:55.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:55 smithi001 ceph-mon[29731]: Detected new or changed devices on smithi001 2024-05-04T03:13:55.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:55 smithi001 ceph-mon[29731]: Adjusting osd_memory_target on smithi001 to 4273M 2024-05-04T03:13:56.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:56 smithi077 ceph-mon[33531]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:13:56.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:56 smithi077 ceph-mon[33531]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:13:56.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:56 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:13:56.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:56 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:13:56.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:56 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:56.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:56 smithi072 ceph-mon[33332]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:13:56.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:56 smithi072 ceph-mon[33332]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:13:56.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:56 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:13:56.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:56 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:13:56.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:56 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:56.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:56 smithi001 ceph-mon[29731]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:13:56.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:56 smithi001 ceph-mon[29731]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:13:56.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:56 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:13:56.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:56 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:13:56.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:56 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:13:58.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:58 smithi077 ceph-mon[33531]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:13:58.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:58 smithi077 ceph-mon[33531]: from='client.? 172.21.15.72:0/2132284340' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79b570af-3d1e-439d-a328-9e6bc7d67839"}]: dispatch 2024-05-04T03:13:58.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:58 smithi077 ceph-mon[33531]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79b570af-3d1e-439d-a328-9e6bc7d67839"}]: dispatch 2024-05-04T03:13:58.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:58 smithi077 ceph-mon[33531]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79b570af-3d1e-439d-a328-9e6bc7d67839"}]': finished 2024-05-04T03:13:58.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:58 smithi077 ceph-mon[33531]: osdmap e64: 11 total, 10 up, 11 in 2024-05-04T03:13:58.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:58 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:13:58.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:58 smithi072 ceph-mon[33332]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:13:58.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:58 smithi072 ceph-mon[33332]: from='client.? 172.21.15.72:0/2132284340' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79b570af-3d1e-439d-a328-9e6bc7d67839"}]: dispatch 2024-05-04T03:13:58.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:58 smithi072 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79b570af-3d1e-439d-a328-9e6bc7d67839"}]: dispatch 2024-05-04T03:13:58.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:58 smithi072 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79b570af-3d1e-439d-a328-9e6bc7d67839"}]': finished 2024-05-04T03:13:58.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:58 smithi072 ceph-mon[33332]: osdmap e64: 11 total, 10 up, 11 in 2024-05-04T03:13:58.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:58 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:13:58.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:58 smithi001 ceph-mon[29731]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:13:58.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:58 smithi001 ceph-mon[29731]: from='client.? 172.21.15.72:0/2132284340' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79b570af-3d1e-439d-a328-9e6bc7d67839"}]: dispatch 2024-05-04T03:13:58.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:58 smithi001 ceph-mon[29731]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79b570af-3d1e-439d-a328-9e6bc7d67839"}]: dispatch 2024-05-04T03:13:58.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:58 smithi001 ceph-mon[29731]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79b570af-3d1e-439d-a328-9e6bc7d67839"}]': finished 2024-05-04T03:13:58.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:58 smithi001 ceph-mon[29731]: osdmap e64: 11 total, 10 up, 11 in 2024-05-04T03:13:58.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:58 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:13:59.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:13:59 smithi077 ceph-mon[33531]: from='client.? 172.21.15.72:0/4272798569' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:13:59.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:13:59 smithi072 ceph-mon[33332]: from='client.? 172.21.15.72:0/4272798569' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:13:59.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:13:59 smithi001 ceph-mon[29731]: from='client.? 172.21.15.72:0/4272798569' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:14:00.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:00 smithi077 ceph-mon[33531]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:00.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:00 smithi072 ceph-mon[33332]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:00.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:00 smithi001 ceph-mon[29731]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:02.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:02 smithi072 ceph-mon[33332]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:02.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:02 smithi077 ceph-mon[33531]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:02.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:02 smithi001 ceph-mon[29731]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:04.540 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:04 smithi072 ceph-mon[33332]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:04.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:04 smithi077 ceph-mon[33531]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:04.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:04 smithi001 ceph-mon[29731]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:06.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:06 smithi077 ceph-mon[33531]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:06.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:06 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-04T03:14:06.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:06 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:06.744 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:06 smithi072 ceph-mon[33332]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:06.745 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:06 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-04T03:14:06.745 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:06 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:06.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:06 smithi001 ceph-mon[29731]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:06.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:06 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-04T03:14:06.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:06 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:07.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:07 smithi072 ceph-mon[33332]: Deploying daemon osd.10 on smithi072 2024-05-04T03:14:07.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:07 smithi001 ceph-mon[29731]: Deploying daemon osd.10 on smithi072 2024-05-04T03:14:07.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:07 smithi077 ceph-mon[33531]: Deploying daemon osd.10 on smithi072 2024-05-04T03:14:08.692 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:08 smithi072 ceph-mon[33332]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:08.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:08 smithi001 ceph-mon[29731]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:08.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:08 smithi077 ceph-mon[33531]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:10.584 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:10 smithi072 ceph-mon[33332]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:10.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:10 smithi001 ceph-mon[29731]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:10.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:10 smithi077 ceph-mon[33531]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:12.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:12 smithi072 ceph-mon[33332]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:12.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:12 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:12.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:12 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:12.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:12 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:14:12.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:12 smithi001 ceph-mon[29731]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:12.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:12 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:12.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:12 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:12.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:12 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:14:12.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:12 smithi077 ceph-mon[33531]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:12.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:12 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:12.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:12 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:12.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:12 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:14:14.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:14 smithi072 ceph-mon[33332]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:14.870 INFO:teuthology.orchestra.run.smithi072.stdout:Created osd(s) 10 on host 'smithi072' 2024-05-04T03:14:14.870 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:14:14.868+0000 7f1628ff9700 1 -- 172.21.15.72:0/1226021598 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f162c063c00 con 0x7f16140b2900 2024-05-04T03:14:14.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:14:14.870+0000 7f1631b6c700 1 -- 172.21.15.72:0/1226021598 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f16140b2900 msgr2=0x7f16140b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:14.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:14:14.870+0000 7f1631b6c700 1 --2- 172.21.15.72:0/1226021598 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f16140b2900 0x7f16140b4e10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1620006fd0 tx=0x7f1620008040 comp rx=0 tx=0).stop 2024-05-04T03:14:14.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:14:14.870+0000 7f1631b6c700 1 -- 172.21.15.72:0/1226021598 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f162c104370 msgr2=0x7f162c19a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:14.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:14:14.870+0000 7f1631b6c700 1 --2- 172.21.15.72:0/1226021598 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f162c104370 0x7f162c19a8c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f161c00d950 tx=0x7f161c00dd10 comp rx=0 tx=0).stop 2024-05-04T03:14:14.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:14:14.870+0000 7f1631b6c700 1 -- 172.21.15.72:0/1226021598 shutdown_connections 2024-05-04T03:14:14.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:14:14.870+0000 7f1631b6c700 1 --2- 172.21.15.72:0/1226021598 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f162c104370 0x7f162c19a8c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:14.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:14:14.870+0000 7f1631b6c700 1 --2- 172.21.15.72:0/1226021598 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f16140b2900 0x7f16140b4e10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:14.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:14:14.870+0000 7f1631b6c700 1 --2- 172.21.15.72:0/1226021598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f162c105e40 0x7f162c1a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:14.872 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:14:14.870+0000 7f1631b6c700 1 --2- 172.21.15.72:0/1226021598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f162c1054a0 0x7f162c19ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:14.873 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:14:14.870+0000 7f1631b6c700 1 -- 172.21.15.72:0/1226021598 >> 172.21.15.72:0/1226021598 conn(0x7f162c0ff9d0 msgr2=0x7f162c1086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:14.873 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:14:14.871+0000 7f1631b6c700 1 -- 172.21.15.72:0/1226021598 shutdown_connections 2024-05-04T03:14:14.873 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-04T03:14:14.871+0000 7f1631b6c700 1 -- 172.21.15.72:0/1226021598 wait complete. 2024-05-04T03:14:14.878 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:14 smithi001 ceph-mon[29731]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:14.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:14 smithi077 ceph-mon[33531]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:15.535 DEBUG:teuthology.orchestra.run.smithi072:osd.10> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.10.service 2024-05-04T03:14:15.537 INFO:tasks.cephadm:Deploying osd.11 on smithi077 with /dev/vg_nvme/lv_1... 2024-05-04T03:14:15.537 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-04T03:14:15.784 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:14:16.070 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:15 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:16.070 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:15 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:16.070 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:15 smithi077 ceph-mon[33531]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:16.070 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:15 smithi077 ceph-mon[33531]: from='osd.10 [v2:172.21.15.72:6824/3293703569,v1:172.21.15.72:6825/3293703569]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-04T03:14:16.070 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:15 smithi077 ceph-mon[33531]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-04T03:14:16.070 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:15 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:16.071 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:15 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:16.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:15 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:16.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:15 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:16.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:15 smithi001 ceph-mon[29731]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:16.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:15 smithi001 ceph-mon[29731]: from='osd.10 [v2:172.21.15.72:6824/3293703569,v1:172.21.15.72:6825/3293703569]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-04T03:14:16.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:15 smithi001 ceph-mon[29731]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-04T03:14:16.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:15 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:16.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:15 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:16.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:15 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:16.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:15 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:16.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:15 smithi072 ceph-mon[33332]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:16.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:15 smithi072 ceph-mon[33332]: from='osd.10 [v2:172.21.15.72:6824/3293703569,v1:172.21.15.72:6825/3293703569]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-04T03:14:16.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:15 smithi072 ceph-mon[33332]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-04T03:14:16.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:15 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:16.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:15 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:17.091 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:16 smithi077 ceph-mon[33531]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-04T03:14:17.091 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:16 smithi077 ceph-mon[33531]: osdmap e65: 11 total, 10 up, 11 in 2024-05-04T03:14:17.091 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:16 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:14:17.091 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:16 smithi077 ceph-mon[33531]: from='osd.10 [v2:172.21.15.72:6824/3293703569,v1:172.21.15.72:6825/3293703569]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-04T03:14:17.091 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:16 smithi077 ceph-mon[33531]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-04T03:14:17.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:16 smithi001 ceph-mon[29731]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-04T03:14:17.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:16 smithi001 ceph-mon[29731]: osdmap e65: 11 total, 10 up, 11 in 2024-05-04T03:14:17.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:16 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:14:17.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:16 smithi001 ceph-mon[29731]: from='osd.10 [v2:172.21.15.72:6824/3293703569,v1:172.21.15.72:6825/3293703569]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-04T03:14:17.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:16 smithi001 ceph-mon[29731]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-04T03:14:17.215 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:16 smithi072 ceph-mon[33332]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-04T03:14:17.215 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:16 smithi072 ceph-mon[33332]: osdmap e65: 11 total, 10 up, 11 in 2024-05-04T03:14:17.215 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:16 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:14:17.216 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:16 smithi072 ceph-mon[33332]: from='osd.10 [v2:172.21.15.72:6824/3293703569,v1:172.21.15.72:6825/3293703569]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-04T03:14:17.216 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:16 smithi072 ceph-mon[33332]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-04T03:14:17.216 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:14:16 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:14:16.834+0000 7fe552ba7700 -1 osd.10 0 waiting for initial osdmap 2024-05-04T03:14:17.216 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:14:16 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:14:16.850+0000 7fe54e95c700 -1 osd.10 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T03:14:17.760 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-04T03:14:17.760 INFO:teuthology.orchestra.run.smithi077.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-04T03:14:17.760 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10+0 records in 2024-05-04T03:14:17.760 INFO:teuthology.orchestra.run.smithi077.stderr:10+0 records out 2024-05-04T03:14:17.760 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114401 s, 917 MB/s 2024-05-04T03:14:17.760 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping successful for: 2024-05-04T03:14:18.121 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:17 smithi077 ceph-mon[33531]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-04T03:14:18.122 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:17 smithi077 ceph-mon[33531]: osdmap e66: 11 total, 10 up, 11 in 2024-05-04T03:14:18.122 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:17 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:14:18.122 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:17 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:14:18.122 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:17 smithi077 ceph-mon[33531]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:18.122 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:17 smithi077 ceph-mon[33531]: osd.10 [v2:172.21.15.72:6824/3293703569,v1:172.21.15.72:6825/3293703569] boot 2024-05-04T03:14:18.122 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:17 smithi077 ceph-mon[33531]: osdmap e67: 11 total, 11 up, 11 in 2024-05-04T03:14:18.122 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:17 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:14:18.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:17 smithi001 ceph-mon[29731]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-04T03:14:18.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:17 smithi001 ceph-mon[29731]: osdmap e66: 11 total, 10 up, 11 in 2024-05-04T03:14:18.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:17 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:14:18.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:17 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:14:18.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:17 smithi001 ceph-mon[29731]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:18.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:17 smithi001 ceph-mon[29731]: osd.10 [v2:172.21.15.72:6824/3293703569,v1:172.21.15.72:6825/3293703569] boot 2024-05-04T03:14:18.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:17 smithi001 ceph-mon[29731]: osdmap e67: 11 total, 11 up, 11 in 2024-05-04T03:14:18.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:17 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:14:18.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:17 smithi072 ceph-mon[33332]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-04T03:14:18.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:17 smithi072 ceph-mon[33332]: osdmap e66: 11 total, 10 up, 11 in 2024-05-04T03:14:18.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:17 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:14:18.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:17 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:14:18.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:17 smithi072 ceph-mon[33332]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 894 GiB / 894 GiB avail 2024-05-04T03:14:18.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:17 smithi072 ceph-mon[33332]: osd.10 [v2:172.21.15.72:6824/3293703569,v1:172.21.15.72:6825/3293703569] boot 2024-05-04T03:14:18.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:17 smithi072 ceph-mon[33332]: osdmap e67: 11 total, 11 up, 11 in 2024-05-04T03:14:18.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:17 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-04T03:14:18.462 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_1 2024-05-04T03:14:18.718 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:14:18.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:18 smithi072 ceph-mon[33332]: purged_snaps scrub starts 2024-05-04T03:14:18.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:18 smithi072 ceph-mon[33332]: purged_snaps scrub ok 2024-05-04T03:14:18.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:18 smithi072 ceph-mon[33332]: osdmap e68: 11 total, 11 up, 11 in 2024-05-04T03:14:19.087 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:18 smithi077 ceph-mon[33531]: purged_snaps scrub starts 2024-05-04T03:14:19.087 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:18 smithi077 ceph-mon[33531]: purged_snaps scrub ok 2024-05-04T03:14:19.087 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:18 smithi077 ceph-mon[33531]: osdmap e68: 11 total, 11 up, 11 in 2024-05-04T03:14:19.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:18 smithi001 ceph-mon[29731]: purged_snaps scrub starts 2024-05-04T03:14:19.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:18 smithi001 ceph-mon[29731]: purged_snaps scrub ok 2024-05-04T03:14:19.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:18 smithi001 ceph-mon[29731]: osdmap e68: 11 total, 11 up, 11 in 2024-05-04T03:14:20.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:20 smithi077 ceph-mon[33531]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:14:20.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:20 smithi077 ceph-mon[33531]: Detected new or changed devices on smithi072 2024-05-04T03:14:20.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:20 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:20.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:20 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:20.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:20 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:14:20.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:20 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:14:20.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:20 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:14:20.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:20 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:14:20.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:20 smithi077 ceph-mon[33531]: Adjusting osd_memory_target on smithi072 to 4272M 2024-05-04T03:14:20.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:20 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:20.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:20 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:20.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:20 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:14:20.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:20 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:20.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:20 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:14:20.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:20 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:20.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:20 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:14:20.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:20 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:20.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:20 smithi077 ceph-mon[33531]: osdmap e69: 11 total, 11 up, 11 in 2024-05-04T03:14:20.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.458+0000 7f10d88d7700 1 -- 172.21.15.77:0/1169679575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10d4104d10 msgr2=0x7f10d410b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:20.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.458+0000 7f10d88d7700 1 --2- 172.21.15.77:0/1169679575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10d4104d10 0x7f10d410b5c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00b0a0 tx=0x7f10cc00b3b0 comp rx=0 tx=0).stop 2024-05-04T03:14:20.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.458+0000 7f10d88d7700 1 -- 172.21.15.77:0/1169679575 shutdown_connections 2024-05-04T03:14:20.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.458+0000 7f10d88d7700 1 --2- 172.21.15.77:0/1169679575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10d4104d10 0x7f10d410b5c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:20.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.458+0000 7f10d88d7700 1 --2- 172.21.15.77:0/1169679575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10d4103240 0x7f10d4103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:20.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.458+0000 7f10d88d7700 1 --2- 172.21.15.77:0/1169679575 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10d4104370 0x7f10d4104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:20.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.458+0000 7f10d88d7700 1 -- 172.21.15.77:0/1169679575 >> 172.21.15.77:0/1169679575 conn(0x7f10d40fe880 msgr2=0x7f10d4100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:20.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.458+0000 7f10d88d7700 1 -- 172.21.15.77:0/1169679575 shutdown_connections 2024-05-04T03:14:20.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.458+0000 7f10d88d7700 1 -- 172.21.15.77:0/1169679575 wait complete. 2024-05-04T03:14:20.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.459+0000 7f10d88d7700 1 Processor -- start 2024-05-04T03:14:20.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.459+0000 7f10d88d7700 1 -- start start 2024-05-04T03:14:20.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.459+0000 7f10d88d7700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10d4103240 0x7f10d406c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:20.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.459+0000 7f10d88d7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10d4104370 0x7f10d419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:20.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.459+0000 7f10d88d7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10d4104d10 0x7f10d41a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:20.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.459+0000 7f10d88d7700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f10d410eca0 con 0x7f10d4103240 2024-05-04T03:14:20.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.459+0000 7f10d88d7700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f10d410eb20 con 0x7f10d4104d10 2024-05-04T03:14:20.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.459+0000 7f10d88d7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f10d410ee20 con 0x7f10d4104370 2024-05-04T03:14:20.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.459+0000 7f10d1d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10d4104370 0x7f10d419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:20.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.459+0000 7f10d1d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10d4104370 0x7f10d419f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57568/0 (socket says 172.21.15.77:57568) 2024-05-04T03:14:20.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.459+0000 7f10d1d9b700 1 -- 172.21.15.77:0/1811589892 learned_addr learned my addr 172.21.15.77:0/1811589892 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-04T03:14:20.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.459+0000 7f10d2d9d700 1 --2- 172.21.15.77:0/1811589892 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10d4104d10 0x7f10d41a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:20.463 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.459+0000 7f10d259c700 1 --2- 172.21.15.77:0/1811589892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10d4103240 0x7f10d406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:20.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.459+0000 7f10d1d9b700 1 -- 172.21.15.77:0/1811589892 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10d4104d10 msgr2=0x7f10d41a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:20.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.459+0000 7f10d1d9b700 1 --2- 172.21.15.77:0/1811589892 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10d4104d10 0x7f10d41a1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:20.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.459+0000 7f10d1d9b700 1 -- 172.21.15.77:0/1811589892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10d4103240 msgr2=0x7f10d406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:20.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.459+0000 7f10d1d9b700 1 --2- 172.21.15.77:0/1811589892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10d4103240 0x7f10d406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:20.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.459+0000 7f10d1d9b700 1 -- 172.21.15.77:0/1811589892 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d41a23f0 con 0x7f10d4104370 2024-05-04T03:14:20.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.460+0000 7f10d1d9b700 1 --2- 172.21.15.77:0/1811589892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10d4104370 0x7f10d419f630 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f10c400e9a0 tx=0x7f10c400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:20.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.460+0000 7f10cb7fe700 1 -- 172.21.15.77:0/1811589892 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c400cbd0 con 0x7f10d4104370 2024-05-04T03:14:20.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.460+0000 7f10cb7fe700 1 -- 172.21.15.77:0/1811589892 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10c400cd70 con 0x7f10d4104370 2024-05-04T03:14:20.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.460+0000 7f10d88d7700 1 -- 172.21.15.77:0/1811589892 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d41a26e0 con 0x7f10d4104370 2024-05-04T03:14:20.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.460+0000 7f10cb7fe700 1 -- 172.21.15.77:0/1811589892 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c4018680 con 0x7f10d4104370 2024-05-04T03:14:20.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.460+0000 7f10d88d7700 1 -- 172.21.15.77:0/1811589892 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d41a2c20 con 0x7f10d4104370 2024-05-04T03:14:20.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.461+0000 7f10d88d7700 1 -- 172.21.15.77:0/1811589892 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d40512c0 con 0x7f10d4104370 2024-05-04T03:14:20.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.462+0000 7f10cb7fe700 1 -- 172.21.15.77:0/1811589892 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f10c4010430 con 0x7f10d4104370 2024-05-04T03:14:20.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.463+0000 7f10cb7fe700 1 --2- 172.21.15.77:0/1811589892 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f10c00b2880 0x7f10c00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:20.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.463+0000 7f10cb7fe700 1 -- 172.21.15.77:0/1811589892 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6694+0+0 (secure 0 0 0) 0x7f10c40e2340 con 0x7f10d4104370 2024-05-04T03:14:20.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.463+0000 7f10d259c700 1 --2- 172.21.15.77:0/1811589892 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f10c00b2880 0x7f10c00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:20.465 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.464+0000 7f10d259c700 1 --2- 172.21.15.77:0/1811589892 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f10c00b2880 0x7f10c00b4d90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f10bc005950 tx=0x7f10bc0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:20.468 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.466+0000 7f10cb7fe700 1 -- 172.21.15.77:0/1811589892 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f10c40aa6b0 con 0x7f10d4104370 2024-05-04T03:14:20.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:20 smithi072 ceph-mon[33332]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:14:20.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:20 smithi072 ceph-mon[33332]: Detected new or changed devices on smithi072 2024-05-04T03:14:20.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:20 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:20.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:20 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:20.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:20 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:14:20.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:20 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:14:20.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:20 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:14:20.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:20 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:14:20.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:20 smithi072 ceph-mon[33332]: Adjusting osd_memory_target on smithi072 to 4272M 2024-05-04T03:14:20.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:20 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:20.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:20 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:20.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:20 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:14:20.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:20 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:20.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:20 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:14:20.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:20 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:20.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:20 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:14:20.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:20 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:20.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:20 smithi072 ceph-mon[33332]: osdmap e69: 11 total, 11 up, 11 in 2024-05-04T03:14:20.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:20 smithi001 ceph-mon[29731]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:14:20.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:20 smithi001 ceph-mon[29731]: Detected new or changed devices on smithi072 2024-05-04T03:14:20.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:20 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:20.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:20 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:20.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:20 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:14:20.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:20 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:14:20.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:20 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:14:20.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:20 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:14:20.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:20 smithi001 ceph-mon[29731]: Adjusting osd_memory_target on smithi072 to 4272M 2024-05-04T03:14:20.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:20 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:20.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:20 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:20.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:20 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:14:20.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:20 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:20.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:20 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:14:20.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:20 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:20.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:20 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:14:20.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:20 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:20.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:20 smithi001 ceph-mon[29731]: osdmap e69: 11 total, 11 up, 11 in 2024-05-04T03:14:20.679 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:20.677+0000 7f10d88d7700 1 -- 172.21.15.77:0/1811589892 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f10d4063c00 con 0x7f10c00b2880 2024-05-04T03:14:21.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:21 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:14:21.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:21 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:14:21.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:21 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:21.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:21 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:14:21.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:21 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:14:21.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:21 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:21.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:21 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-04T03:14:21.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:21 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-04T03:14:21.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:21 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:22.472 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:22 smithi077 ceph-mon[33531]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:14:22.472 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:22 smithi077 ceph-mon[33531]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:14:22.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:22 smithi072 ceph-mon[33332]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:14:22.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:22 smithi072 ceph-mon[33332]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:14:22.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:22 smithi001 ceph-mon[29731]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:14:22.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:22 smithi001 ceph-mon[29731]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:14:24.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:24 smithi072 ceph-mon[33332]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:14:24.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:24 smithi072 ceph-mon[33332]: from='client.? 172.21.15.77:0/3544202472' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e35bf83d-9ee4-4745-96c2-1c35cc8eefba"}]: dispatch 2024-05-04T03:14:24.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:24 smithi072 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e35bf83d-9ee4-4745-96c2-1c35cc8eefba"}]: dispatch 2024-05-04T03:14:24.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:24 smithi072 ceph-mon[33332]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e35bf83d-9ee4-4745-96c2-1c35cc8eefba"}]': finished 2024-05-04T03:14:24.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:24 smithi072 ceph-mon[33332]: osdmap e70: 12 total, 11 up, 12 in 2024-05-04T03:14:24.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:24 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:14:24.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:24 smithi072 ceph-mon[33332]: from='client.? 172.21.15.77:0/1745435831' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:14:24.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:24 smithi001 ceph-mon[29731]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:14:24.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:24 smithi001 ceph-mon[29731]: from='client.? 172.21.15.77:0/3544202472' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e35bf83d-9ee4-4745-96c2-1c35cc8eefba"}]: dispatch 2024-05-04T03:14:24.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:24 smithi001 ceph-mon[29731]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e35bf83d-9ee4-4745-96c2-1c35cc8eefba"}]: dispatch 2024-05-04T03:14:24.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:24 smithi001 ceph-mon[29731]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e35bf83d-9ee4-4745-96c2-1c35cc8eefba"}]': finished 2024-05-04T03:14:24.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:24 smithi001 ceph-mon[29731]: osdmap e70: 12 total, 11 up, 12 in 2024-05-04T03:14:24.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:24 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:14:24.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:24 smithi001 ceph-mon[29731]: from='client.? 172.21.15.77:0/1745435831' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:14:24.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:24 smithi077 ceph-mon[33531]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:14:24.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:24 smithi077 ceph-mon[33531]: from='client.? 172.21.15.77:0/3544202472' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e35bf83d-9ee4-4745-96c2-1c35cc8eefba"}]: dispatch 2024-05-04T03:14:24.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:24 smithi077 ceph-mon[33531]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e35bf83d-9ee4-4745-96c2-1c35cc8eefba"}]: dispatch 2024-05-04T03:14:24.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:24 smithi077 ceph-mon[33531]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e35bf83d-9ee4-4745-96c2-1c35cc8eefba"}]': finished 2024-05-04T03:14:24.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:24 smithi077 ceph-mon[33531]: osdmap e70: 12 total, 11 up, 12 in 2024-05-04T03:14:24.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:24 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:14:24.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:24 smithi077 ceph-mon[33531]: from='client.? 172.21.15.77:0/1745435831' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-04T03:14:26.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:26 smithi072 ceph-mon[33332]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-04T03:14:26.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:26 smithi001 ceph-mon[29731]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-04T03:14:26.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:26 smithi077 ceph-mon[33531]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-04T03:14:28.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:28 smithi077 ceph-mon[33531]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-04T03:14:28.498 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:28 smithi072 ceph-mon[33332]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-04T03:14:28.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:28 smithi001 ceph-mon[29731]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-04T03:14:30.306 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:30 smithi077 ceph-mon[33531]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-04T03:14:30.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:30 smithi072 ceph-mon[33332]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-04T03:14:30.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:30 smithi001 ceph-mon[29731]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-04T03:14:31.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:31 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-04T03:14:31.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:31 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:31.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:31 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-04T03:14:31.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:31 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:31.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:31 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-04T03:14:31.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:31 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:32.409 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:32 smithi077 ceph-mon[33531]: Deploying daemon osd.11 on smithi077 2024-05-04T03:14:32.409 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:32 smithi077 ceph-mon[33531]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:14:32.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:32 smithi072 ceph-mon[33332]: Deploying daemon osd.11 on smithi077 2024-05-04T03:14:32.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:32 smithi072 ceph-mon[33332]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:14:32.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:32 smithi001 ceph-mon[29731]: Deploying daemon osd.11 on smithi077 2024-05-04T03:14:32.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:32 smithi001 ceph-mon[29731]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:14:34.495 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:34 smithi077 ceph-mon[33531]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:14:34.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:34 smithi072 ceph-mon[33332]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:14:34.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:34 smithi001 ceph-mon[29731]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-04T03:14:36.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:36 smithi072 ceph-mon[33332]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-05-04T03:14:36.548 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:36 smithi077 ceph-mon[33531]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-05-04T03:14:36.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:36 smithi001 ceph-mon[29731]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-05-04T03:14:38.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:37 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:38.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:37 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:38.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:37 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:14:38.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:37 smithi001 ceph-mon[29731]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:14:38.144 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:37 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:38.144 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:37 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:38.144 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:37 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:14:38.144 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:37 smithi077 ceph-mon[33531]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:14:38.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:37 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:38.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:37 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:38.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:37 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:14:38.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:37 smithi072 ceph-mon[33332]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:14:40.279 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:40 smithi077 ceph-mon[33531]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:14:40.279 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:40 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:40.280 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:40 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:40.280 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:40 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:40.280 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:40 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:14:40.280 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:40 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:40.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:40 smithi072 ceph-mon[33332]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:14:40.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:40 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:40.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:40 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:40.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:40 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:40.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:40 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:14:40.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:40 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:40.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:40 smithi001 ceph-mon[29731]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:14:40.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:40 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:40.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:40 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:40.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:40 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:40.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:40 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:14:40.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:40 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:40.757 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 11 on host 'smithi077' 2024-05-04T03:14:40.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:40.755+0000 7f10cb7fe700 1 -- 172.21.15.77:0/1811589892 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f10d4063c00 con 0x7f10c00b2880 2024-05-04T03:14:40.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:40.757+0000 7f10d88d7700 1 -- 172.21.15.77:0/1811589892 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f10c00b2880 msgr2=0x7f10c00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:40.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:40.757+0000 7f10d88d7700 1 --2- 172.21.15.77:0/1811589892 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f10c00b2880 0x7f10c00b4d90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f10bc005950 tx=0x7f10bc0058e0 comp rx=0 tx=0).stop 2024-05-04T03:14:40.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:40.757+0000 7f10d88d7700 1 -- 172.21.15.77:0/1811589892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10d4104370 msgr2=0x7f10d419f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:40.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:40.757+0000 7f10d88d7700 1 --2- 172.21.15.77:0/1811589892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10d4104370 0x7f10d419f630 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f10c400e9a0 tx=0x7f10c400ecb0 comp rx=0 tx=0).stop 2024-05-04T03:14:40.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:40.757+0000 7f10d88d7700 1 -- 172.21.15.77:0/1811589892 shutdown_connections 2024-05-04T03:14:40.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:40.757+0000 7f10d88d7700 1 --2- 172.21.15.77:0/1811589892 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f10d4104d10 0x7f10d41a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:40.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:40.757+0000 7f10d88d7700 1 --2- 172.21.15.77:0/1811589892 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f10c00b2880 0x7f10c00b4d90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:40.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:40.757+0000 7f10d88d7700 1 --2- 172.21.15.77:0/1811589892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10d4103240 0x7f10d406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:40.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:40.757+0000 7f10d88d7700 1 --2- 172.21.15.77:0/1811589892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f10d4104370 0x7f10d419f630 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:40.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:40.757+0000 7f10d88d7700 1 -- 172.21.15.77:0/1811589892 >> 172.21.15.77:0/1811589892 conn(0x7f10d40fe880 msgr2=0x7f10d41075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:40.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:40.757+0000 7f10d88d7700 1 -- 172.21.15.77:0/1811589892 shutdown_connections 2024-05-04T03:14:40.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:40.757+0000 7f10d88d7700 1 -- 172.21.15.77:0/1811589892 wait complete. 2024-05-04T03:14:41.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:41 smithi077 ceph-mon[33531]: from='osd.11 [v2:172.21.15.77:6824/4155947548,v1:172.21.15.77:6825/4155947548]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-04T03:14:41.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:41 smithi077 ceph-mon[33531]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-04T03:14:41.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:41 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:41.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:41 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:41.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:41 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:14:41.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:41 smithi072 ceph-mon[33332]: from='osd.11 [v2:172.21.15.77:6824/4155947548,v1:172.21.15.77:6825/4155947548]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-04T03:14:41.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:41 smithi072 ceph-mon[33332]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-04T03:14:41.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:41 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:41.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:41 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:41.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:41 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:14:41.536 DEBUG:teuthology.orchestra.run.smithi077:osd.11> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.11.service 2024-05-04T03:14:41.540 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-04T03:14:41.540 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-04T03:14:41.569 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:41 smithi001 ceph-mon[29731]: from='osd.11 [v2:172.21.15.77:6824/4155947548,v1:172.21.15.77:6825/4155947548]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-04T03:14:41.569 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:41 smithi001 ceph-mon[29731]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-04T03:14:41.569 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:41 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:41.569 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:41 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:41.569 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:41 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:14:41.800 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:14:42.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:42 smithi072 ceph-mon[33332]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:14:42.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:42 smithi072 ceph-mon[33332]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-04T03:14:42.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:42 smithi072 ceph-mon[33332]: osdmap e71: 12 total, 11 up, 12 in 2024-05-04T03:14:42.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:42 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:14:42.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:42 smithi072 ceph-mon[33332]: from='osd.11 [v2:172.21.15.77:6824/4155947548,v1:172.21.15.77:6825/4155947548]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-04T03:14:42.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:42 smithi072 ceph-mon[33332]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-04T03:14:42.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:42 smithi001 ceph-mon[29731]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:14:42.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:42 smithi001 ceph-mon[29731]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-04T03:14:42.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:42 smithi001 ceph-mon[29731]: osdmap e71: 12 total, 11 up, 12 in 2024-05-04T03:14:42.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:42 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:14:42.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:42 smithi001 ceph-mon[29731]: from='osd.11 [v2:172.21.15.77:6824/4155947548,v1:172.21.15.77:6825/4155947548]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-04T03:14:42.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:42 smithi001 ceph-mon[29731]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-04T03:14:42.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:42 smithi077 ceph-mon[33531]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:14:42.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:42 smithi077 ceph-mon[33531]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-04T03:14:42.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:42 smithi077 ceph-mon[33531]: osdmap e71: 12 total, 11 up, 12 in 2024-05-04T03:14:42.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:42 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:14:42.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:42 smithi077 ceph-mon[33531]: from='osd.11 [v2:172.21.15.77:6824/4155947548,v1:172.21.15.77:6825/4155947548]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-04T03:14:42.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:42 smithi077 ceph-mon[33531]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-04T03:14:43.368 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:43 smithi001 ceph-mon[29731]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-05-04T03:14:43.368 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:43 smithi001 ceph-mon[29731]: osdmap e72: 12 total, 11 up, 12 in 2024-05-04T03:14:43.368 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:43 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:14:43.368 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:43 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:14:43.368 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:43 smithi001 ceph-mon[29731]: from='osd.11 ' entity='osd.11' 2024-05-04T03:14:43.429 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:43 smithi077 ceph-mon[33531]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-05-04T03:14:43.430 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:43 smithi077 ceph-mon[33531]: osdmap e72: 12 total, 11 up, 12 in 2024-05-04T03:14:43.430 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:43 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:14:43.430 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:43 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:14:43.430 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:43 smithi077 ceph-mon[33531]: from='osd.11 ' entity='osd.11' 2024-05-04T03:14:43.430 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:14:43 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:14:43.109+0000 7fa0b5b68700 -1 osd.11 0 waiting for initial osdmap 2024-05-04T03:14:43.430 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:14:43 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:14:43.235+0000 7fa0ad915700 -1 osd.11 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-04T03:14:43.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:43 smithi072 ceph-mon[33332]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-05-04T03:14:43.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:43 smithi072 ceph-mon[33332]: osdmap e72: 12 total, 11 up, 12 in 2024-05-04T03:14:43.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:43 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:14:43.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:43 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:14:43.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:43 smithi072 ceph-mon[33332]: from='osd.11 ' entity='osd.11' 2024-05-04T03:14:43.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.607+0000 7f4fe2552700 1 -- 172.21.15.1:0/3099575541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fdc104d10 msgr2=0x7f4fdc10b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:43.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.607+0000 7f4fe2552700 1 --2- 172.21.15.1:0/3099575541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fdc104d10 0x7f4fdc10b5c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400b0a0 tx=0x7f4fd400b3b0 comp rx=0 tx=0).stop 2024-05-04T03:14:43.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.607+0000 7f4fe2552700 1 -- 172.21.15.1:0/3099575541 shutdown_connections 2024-05-04T03:14:43.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.607+0000 7f4fe2552700 1 --2- 172.21.15.1:0/3099575541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fdc104d10 0x7f4fdc10b5c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:43.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.607+0000 7f4fe2552700 1 --2- 172.21.15.1:0/3099575541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fdc103240 0x7f4fdc103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:43.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.607+0000 7f4fe2552700 1 --2- 172.21.15.1:0/3099575541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4fdc104370 0x7f4fdc104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:43.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.607+0000 7f4fe2552700 1 -- 172.21.15.1:0/3099575541 >> 172.21.15.1:0/3099575541 conn(0x7f4fdc0fe880 msgr2=0x7f4fdc100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:43.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.607+0000 7f4fe2552700 1 -- 172.21.15.1:0/3099575541 shutdown_connections 2024-05-04T03:14:43.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.607+0000 7f4fe2552700 1 -- 172.21.15.1:0/3099575541 wait complete. 2024-05-04T03:14:43.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.608+0000 7f4fe2552700 1 Processor -- start 2024-05-04T03:14:43.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.608+0000 7f4fe2552700 1 -- start start 2024-05-04T03:14:43.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.608+0000 7f4fe2552700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4fdc103240 0x7f4fdc19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:43.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.608+0000 7f4fe2552700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fdc104370 0x7f4fdc19af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:43.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.608+0000 7f4fe2552700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fdc104d10 0x7f4fdc1a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:43.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.608+0000 7f4fe2552700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4fdc10eda0 con 0x7f4fdc104370 2024-05-04T03:14:43.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.608+0000 7f4fe2552700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4fdc10ec20 con 0x7f4fdc103240 2024-05-04T03:14:43.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.608+0000 7f4fe2552700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4fdc10ef20 con 0x7f4fdc104d10 2024-05-04T03:14:43.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.609+0000 7f4fdb7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fdc104370 0x7f4fdc19af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:43.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.609+0000 7f4fdb7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fdc104370 0x7f4fdc19af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41966/0 (socket says 172.21.15.1:41966) 2024-05-04T03:14:43.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.609+0000 7f4fe0aef700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fdc104d10 0x7f4fdc1a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:43.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.609+0000 7f4fdb7fe700 1 -- 172.21.15.1:0/1603472169 learned_addr learned my addr 172.21.15.1:0/1603472169 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:14:43.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.609+0000 7f4fdbfff700 1 --2- 172.21.15.1:0/1603472169 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4fdc103240 0x7f4fdc19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:43.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.609+0000 7f4fdb7fe700 1 -- 172.21.15.1:0/1603472169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fdc104d10 msgr2=0x7f4fdc1a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:43.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.609+0000 7f4fdb7fe700 1 --2- 172.21.15.1:0/1603472169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fdc104d10 0x7f4fdc1a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:43.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.609+0000 7f4fdb7fe700 1 -- 172.21.15.1:0/1603472169 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4fdc103240 msgr2=0x7f4fdc19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:43.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.609+0000 7f4fdb7fe700 1 --2- 172.21.15.1:0/1603472169 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4fdc103240 0x7f4fdc19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:43.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.609+0000 7f4fdb7fe700 1 -- 172.21.15.1:0/1603472169 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fdc1a2590 con 0x7f4fdc104370 2024-05-04T03:14:43.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.609+0000 7f4fdb7fe700 1 --2- 172.21.15.1:0/1603472169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fdc104370 0x7f4fdc19af70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4fd000b530 tx=0x7f4fd000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:43.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.609+0000 7f4fd97fa700 1 -- 172.21.15.1:0/1603472169 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0015020 con 0x7f4fdc104370 2024-05-04T03:14:43.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.609+0000 7f4fd97fa700 1 -- 172.21.15.1:0/1603472169 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fd0011ce0 con 0x7f4fdc104370 2024-05-04T03:14:43.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.609+0000 7f4fe2552700 1 -- 172.21.15.1:0/1603472169 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fdc1a2880 con 0x7f4fdc104370 2024-05-04T03:14:43.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.609+0000 7f4fd97fa700 1 -- 172.21.15.1:0/1603472169 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0010e40 con 0x7f4fdc104370 2024-05-04T03:14:43.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.609+0000 7f4fe2552700 1 -- 172.21.15.1:0/1603472169 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fdc1a2e40 con 0x7f4fdc104370 2024-05-04T03:14:43.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.610+0000 7f4fe2552700 1 -- 172.21.15.1:0/1603472169 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fdc0512c0 con 0x7f4fdc104370 2024-05-04T03:14:43.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.613+0000 7f4fd97fa700 1 -- 172.21.15.1:0/1603472169 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f4fd0018020 con 0x7f4fdc104370 2024-05-04T03:14:43.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.613+0000 7f4fd97fa700 1 --2- 172.21.15.1:0/1603472169 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4fc40b2880 0x7f4fc40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:43.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.614+0000 7f4fdbfff700 1 --2- 172.21.15.1:0/1603472169 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4fc40b2880 0x7f4fc40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:43.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.614+0000 7f4fd97fa700 1 -- 172.21.15.1:0/1603472169 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6847+0+0 (secure 0 0 0) 0x7f4fd00e1ae0 con 0x7f4fdc104370 2024-05-04T03:14:43.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.614+0000 7f4fdbfff700 1 --2- 172.21.15.1:0/1603472169 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4fc40b2880 0x7f4fc40b4d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc006fd0 tx=0x7f4fcc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:43.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.617+0000 7f4fd97fa700 1 -- 172.21.15.1:0/1603472169 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4fd001f020 con 0x7f4fdc104370 2024-05-04T03:14:43.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.832+0000 7f4fe2552700 1 -- 172.21.15.1:0/1603472169 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f4fdc1a31e0 con 0x7f4fdc104370 2024-05-04T03:14:43.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.832+0000 7f4fd97fa700 1 -- 172.21.15.1:0/1603472169 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7f4fd00a9d20 con 0x7f4fdc104370 2024-05-04T03:14:43.835 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:14:43.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.834+0000 7f4fe2552700 1 -- 172.21.15.1:0/1603472169 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4fc40b2880 msgr2=0x7f4fc40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:43.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.834+0000 7f4fe2552700 1 --2- 172.21.15.1:0/1603472169 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4fc40b2880 0x7f4fc40b4d90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc006fd0 tx=0x7f4fcc008040 comp rx=0 tx=0).stop 2024-05-04T03:14:43.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.834+0000 7f4fe2552700 1 -- 172.21.15.1:0/1603472169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fdc104370 msgr2=0x7f4fdc19af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:43.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.834+0000 7f4fe2552700 1 --2- 172.21.15.1:0/1603472169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fdc104370 0x7f4fdc19af70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4fd000b530 tx=0x7f4fd000b840 comp rx=0 tx=0).stop 2024-05-04T03:14:43.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.834+0000 7f4fe2552700 1 -- 172.21.15.1:0/1603472169 shutdown_connections 2024-05-04T03:14:43.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.834+0000 7f4fe2552700 1 --2- 172.21.15.1:0/1603472169 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4fdc103240 0x7f4fdc19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:43.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.834+0000 7f4fe2552700 1 --2- 172.21.15.1:0/1603472169 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4fc40b2880 0x7f4fc40b4d90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:43.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.834+0000 7f4fe2552700 1 --2- 172.21.15.1:0/1603472169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4fdc104d10 0x7f4fdc1a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:43.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.834+0000 7f4fe2552700 1 --2- 172.21.15.1:0/1603472169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4fdc104370 0x7f4fdc19af70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:43.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.834+0000 7f4fe2552700 1 -- 172.21.15.1:0/1603472169 >> 172.21.15.1:0/1603472169 conn(0x7f4fdc0fe880 msgr2=0x7f4fdc1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:43.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.834+0000 7f4fe2552700 1 -- 172.21.15.1:0/1603472169 shutdown_connections 2024-05-04T03:14:43.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:43.834+0000 7f4fe2552700 1 -- 172.21.15.1:0/1603472169 wait complete. 2024-05-04T03:14:44.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:44 smithi072 ceph-mon[33332]: purged_snaps scrub starts 2024-05-04T03:14:44.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:44 smithi072 ceph-mon[33332]: purged_snaps scrub ok 2024-05-04T03:14:44.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:44 smithi072 ceph-mon[33332]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:14:44.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:44 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:14:44.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:44 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/1603472169' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-04T03:14:44.573 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:44 smithi001 ceph-mon[29731]: purged_snaps scrub starts 2024-05-04T03:14:44.573 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:44 smithi001 ceph-mon[29731]: purged_snaps scrub ok 2024-05-04T03:14:44.573 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:44 smithi001 ceph-mon[29731]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:14:44.573 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:44 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:14:44.573 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:44 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/1603472169' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-04T03:14:44.603 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":72,"num_osds":12,"num_up_osds":11,"osd_up_since":1714792457,"num_in_osds":12,"osd_in_since":1714792463,"num_remapped_pgs":0} 2024-05-04T03:14:44.625 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:44 smithi077 ceph-mon[33531]: purged_snaps scrub starts 2024-05-04T03:14:44.626 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:44 smithi077 ceph-mon[33531]: purged_snaps scrub ok 2024-05-04T03:14:44.626 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:44 smithi077 ceph-mon[33531]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:14:44.626 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:44 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:14:44.626 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:44 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/1603472169' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-04T03:14:45.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:45 smithi072 ceph-mon[33332]: osd.11 [v2:172.21.15.77:6824/4155947548,v1:172.21.15.77:6825/4155947548] boot 2024-05-04T03:14:45.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:45 smithi072 ceph-mon[33332]: osdmap e73: 12 total, 12 up, 12 in 2024-05-04T03:14:45.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:45 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:14:45.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:45 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:45.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:45 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:45.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:45 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:14:45.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:45 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:14:45.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:45 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:14:45.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:45 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:14:45.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:45 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:45.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:45 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:45.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:45 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:14:45.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:45 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:45.604 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-04T03:14:45.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:45 smithi001 ceph-mon[29731]: osd.11 [v2:172.21.15.77:6824/4155947548,v1:172.21.15.77:6825/4155947548] boot 2024-05-04T03:14:45.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:45 smithi001 ceph-mon[29731]: osdmap e73: 12 total, 12 up, 12 in 2024-05-04T03:14:45.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:45 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:14:45.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:45 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:45.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:45 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:45.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:45 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:14:45.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:45 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:14:45.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:45 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:14:45.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:45 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:14:45.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:45 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:45.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:45 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:45.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:45 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:14:45.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:45 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:45.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:45 smithi077 ceph-mon[33531]: osd.11 [v2:172.21.15.77:6824/4155947548,v1:172.21.15.77:6825/4155947548] boot 2024-05-04T03:14:45.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:45 smithi077 ceph-mon[33531]: osdmap e73: 12 total, 12 up, 12 in 2024-05-04T03:14:45.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:45 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-04T03:14:45.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:45 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:45.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:45 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:45.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:45 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:14:45.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:45 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:14:45.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:45 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:14:45.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:45 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:14:45.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:45 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:45.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:45 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:45.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:45 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:14:45.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:45 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:45.852 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:14:46.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:46 smithi077 ceph-mon[33531]: Detected new or changed devices on smithi077 2024-05-04T03:14:46.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:46 smithi077 ceph-mon[33531]: Adjusting osd_memory_target on smithi077 to 4272M 2024-05-04T03:14:46.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:46 smithi077 ceph-mon[33531]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:14:46.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:46 smithi077 ceph-mon[33531]: osdmap e74: 12 total, 12 up, 12 in 2024-05-04T03:14:46.498 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:46 smithi072 ceph-mon[33332]: Detected new or changed devices on smithi077 2024-05-04T03:14:46.498 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:46 smithi072 ceph-mon[33332]: Adjusting osd_memory_target on smithi077 to 4272M 2024-05-04T03:14:46.498 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:46 smithi072 ceph-mon[33332]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:14:46.498 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:46 smithi072 ceph-mon[33332]: osdmap e74: 12 total, 12 up, 12 in 2024-05-04T03:14:46.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:46 smithi001 ceph-mon[29731]: Detected new or changed devices on smithi077 2024-05-04T03:14:46.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:46 smithi001 ceph-mon[29731]: Adjusting osd_memory_target on smithi077 to 4272M 2024-05-04T03:14:46.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:46 smithi001 ceph-mon[29731]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-05-04T03:14:46.631 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:46 smithi001 ceph-mon[29731]: osdmap e74: 12 total, 12 up, 12 in 2024-05-04T03:14:47.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.589+0000 7f4b05a6b700 1 -- 172.21.15.1:0/3305565102 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b00104370 msgr2=0x7f4b00104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:47.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.589+0000 7f4b05a6b700 1 --2- 172.21.15.1:0/3305565102 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b00104370 0x7f4b00104780 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4af0009990 tx=0x7f4af0009ca0 comp rx=0 tx=0).stop 2024-05-04T03:14:47.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.589+0000 7f4b05a6b700 1 -- 172.21.15.1:0/3305565102 shutdown_connections 2024-05-04T03:14:47.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.589+0000 7f4b05a6b700 1 --2- 172.21.15.1:0/3305565102 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b00104d10 0x7f4b0010b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:47.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.589+0000 7f4b05a6b700 1 --2- 172.21.15.1:0/3305565102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b00103240 0x7f4b00103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:47.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.589+0000 7f4b05a6b700 1 --2- 172.21.15.1:0/3305565102 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b00104370 0x7f4b00104780 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:47.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.589+0000 7f4b05a6b700 1 -- 172.21.15.1:0/3305565102 >> 172.21.15.1:0/3305565102 conn(0x7f4b000fe880 msgr2=0x7f4b00100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:47.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.589+0000 7f4b05a6b700 1 -- 172.21.15.1:0/3305565102 shutdown_connections 2024-05-04T03:14:47.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.589+0000 7f4b05a6b700 1 -- 172.21.15.1:0/3305565102 wait complete. 2024-05-04T03:14:47.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.590+0000 7f4b05a6b700 1 Processor -- start 2024-05-04T03:14:47.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.590+0000 7f4b05a6b700 1 -- start start 2024-05-04T03:14:47.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.590+0000 7f4b05a6b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b00103240 0x7f4b0006c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:47.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.590+0000 7f4b05a6b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b00104370 0x7f4b0019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:47.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.590+0000 7f4b05a6b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b00104d10 0x7f4b001a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:47.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.590+0000 7f4b05a6b700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0010eca0 con 0x7f4b00104d10 2024-05-04T03:14:47.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.590+0000 7f4b05a6b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0010eb20 con 0x7f4b00103240 2024-05-04T03:14:47.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.590+0000 7f4b05a6b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0010ee20 con 0x7f4b00104370 2024-05-04T03:14:47.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.591+0000 7f4afffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b00104d10 0x7f4b001a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:47.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.591+0000 7f4afffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b00104d10 0x7f4b001a1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41982/0 (socket says 172.21.15.1:41982) 2024-05-04T03:14:47.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.591+0000 7f4afffff700 1 -- 172.21.15.1:0/1286961281 learned_addr learned my addr 172.21.15.1:0/1286961281 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:14:47.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.591+0000 7f4aff7fe700 1 --2- 172.21.15.1:0/1286961281 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b00103240 0x7f4b0006c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:47.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.591+0000 7f4afeffd700 1 --2- 172.21.15.1:0/1286961281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b00104370 0x7f4b0019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:47.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.591+0000 7f4afffff700 1 -- 172.21.15.1:0/1286961281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b00104370 msgr2=0x7f4b0019f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:47.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.591+0000 7f4afffff700 1 --2- 172.21.15.1:0/1286961281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b00104370 0x7f4b0019f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:47.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.591+0000 7f4afffff700 1 -- 172.21.15.1:0/1286961281 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b00103240 msgr2=0x7f4b0006c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:47.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.591+0000 7f4afffff700 1 --2- 172.21.15.1:0/1286961281 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b00103240 0x7f4b0006c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:47.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.591+0000 7f4afffff700 1 -- 172.21.15.1:0/1286961281 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b001a2450 con 0x7f4b00104d10 2024-05-04T03:14:47.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.591+0000 7f4aff7fe700 1 --2- 172.21.15.1:0/1286961281 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b00103240 0x7f4b0006c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:14:47.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.591+0000 7f4afffff700 1 --2- 172.21.15.1:0/1286961281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b00104d10 0x7f4b001a1c20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4af800ef30 tx=0x7f4af800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:47.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.591+0000 7f4afcff9700 1 -- 172.21.15.1:0/1286961281 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af80092e0 con 0x7f4b00104d10 2024-05-04T03:14:47.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.591+0000 7f4afcff9700 1 -- 172.21.15.1:0/1286961281 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4af8004bb0 con 0x7f4b00104d10 2024-05-04T03:14:47.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.591+0000 7f4b05a6b700 1 -- 172.21.15.1:0/1286961281 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b001a26e0 con 0x7f4b00104d10 2024-05-04T03:14:47.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.591+0000 7f4afcff9700 1 -- 172.21.15.1:0/1286961281 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af8004020 con 0x7f4b00104d10 2024-05-04T03:14:47.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.592+0000 7f4b05a6b700 1 -- 172.21.15.1:0/1286961281 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b001a2c20 con 0x7f4b00104d10 2024-05-04T03:14:47.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.593+0000 7f4b05a6b700 1 -- 172.21.15.1:0/1286961281 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b000512c0 con 0x7f4b00104d10 2024-05-04T03:14:47.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.594+0000 7f4afcff9700 1 -- 172.21.15.1:0/1286961281 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f4af801b030 con 0x7f4b00104d10 2024-05-04T03:14:47.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.595+0000 7f4afcff9700 1 --2- 172.21.15.1:0/1286961281 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4ae80b2900 0x7f4ae80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:47.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.595+0000 7f4aff7fe700 1 --2- 172.21.15.1:0/1286961281 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4ae80b2900 0x7f4ae80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:47.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.595+0000 7f4afcff9700 1 -- 172.21.15.1:0/1286961281 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f4af80e2270 con 0x7f4b00104d10 2024-05-04T03:14:47.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.596+0000 7f4aff7fe700 1 --2- 172.21.15.1:0/1286961281 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4ae80b2900 0x7f4ae80b4e10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4b00104270 tx=0x7f4af400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:47.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.599+0000 7f4afcff9700 1 -- 172.21.15.1:0/1286961281 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4af8022020 con 0x7f4b00104d10 2024-05-04T03:14:47.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.815+0000 7f4b05a6b700 1 -- 172.21.15.1:0/1286961281 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f4b000643d0 con 0x7f4b00104d10 2024-05-04T03:14:47.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.815+0000 7f4afcff9700 1 -- 172.21.15.1:0/1286961281 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7f4af8010040 con 0x7f4b00104d10 2024-05-04T03:14:47.818 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:14:47.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.818+0000 7f4b05a6b700 1 -- 172.21.15.1:0/1286961281 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4ae80b2900 msgr2=0x7f4ae80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:47.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.818+0000 7f4b05a6b700 1 --2- 172.21.15.1:0/1286961281 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4ae80b2900 0x7f4ae80b4e10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4b00104270 tx=0x7f4af400b410 comp rx=0 tx=0).stop 2024-05-04T03:14:47.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.818+0000 7f4b05a6b700 1 -- 172.21.15.1:0/1286961281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b00104d10 msgr2=0x7f4b001a1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:47.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.818+0000 7f4b05a6b700 1 --2- 172.21.15.1:0/1286961281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b00104d10 0x7f4b001a1c20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4af800ef30 tx=0x7f4af800c3a0 comp rx=0 tx=0).stop 2024-05-04T03:14:47.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.818+0000 7f4b05a6b700 1 -- 172.21.15.1:0/1286961281 shutdown_connections 2024-05-04T03:14:47.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.818+0000 7f4b05a6b700 1 --2- 172.21.15.1:0/1286961281 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b00103240 0x7f4b0006c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:47.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.818+0000 7f4b05a6b700 1 --2- 172.21.15.1:0/1286961281 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4ae80b2900 0x7f4ae80b4e10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:47.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.818+0000 7f4b05a6b700 1 --2- 172.21.15.1:0/1286961281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b00104d10 0x7f4b001a1c20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:47.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.818+0000 7f4b05a6b700 1 --2- 172.21.15.1:0/1286961281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b00104370 0x7f4b0019f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:47.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.818+0000 7f4b05a6b700 1 -- 172.21.15.1:0/1286961281 >> 172.21.15.1:0/1286961281 conn(0x7f4b000fe880 msgr2=0x7f4b001075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:47.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.818+0000 7f4b05a6b700 1 -- 172.21.15.1:0/1286961281 shutdown_connections 2024-05-04T03:14:47.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:47.818+0000 7f4b05a6b700 1 -- 172.21.15.1:0/1286961281 wait complete. 2024-05-04T03:14:48.315 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":74,"num_osds":12,"num_up_osds":12,"osd_up_since":1714792484,"num_in_osds":12,"osd_in_since":1714792463,"num_remapped_pgs":0} 2024-05-04T03:14:48.316 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-04T03:14:48.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:48 smithi072 ceph-mon[33332]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:14:48.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:48 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/1286961281' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-04T03:14:48.603 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:14:48.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:48 smithi001 ceph-mon[29731]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:14:48.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:48 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/1286961281' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-04T03:14:48.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:48 smithi077 ceph-mon[33531]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:14:48.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:48 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/1286961281' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-04T03:14:50.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.407+0000 7f678da16700 1 -- 172.21.15.1:0/3772302175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6788105dd0 msgr2=0x7f678810e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:50.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.407+0000 7f678da16700 1 --2- 172.21.15.1:0/3772302175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6788105dd0 0x7f678810e320 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f677c009990 tx=0x7f677c009ca0 comp rx=0 tx=0).stop 2024-05-04T03:14:50.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.407+0000 7f678da16700 1 -- 172.21.15.1:0/3772302175 shutdown_connections 2024-05-04T03:14:50.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.407+0000 7f678da16700 1 --2- 172.21.15.1:0/3772302175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f678810e8f0 0x7f6788110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:50.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.407+0000 7f678da16700 1 --2- 172.21.15.1:0/3772302175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6788105480 0x7f6788105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:50.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.407+0000 7f678da16700 1 --2- 172.21.15.1:0/3772302175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6788105dd0 0x7f678810e320 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:50.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.407+0000 7f678da16700 1 -- 172.21.15.1:0/3772302175 >> 172.21.15.1:0/3772302175 conn(0x7f67880fef30 msgr2=0x7f6788101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:50.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.408+0000 7f678da16700 1 -- 172.21.15.1:0/3772302175 shutdown_connections 2024-05-04T03:14:50.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.408+0000 7f678da16700 1 -- 172.21.15.1:0/3772302175 wait complete. 2024-05-04T03:14:50.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.408+0000 7f678da16700 1 Processor -- start 2024-05-04T03:14:50.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.408+0000 7f678da16700 1 -- start start 2024-05-04T03:14:50.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.409+0000 7f678da16700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6788105480 0x7f6788103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:50.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.409+0000 7f678da16700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6788105dd0 0x7f67881036b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:50.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.409+0000 7f678da16700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f678810e8f0 0x7f6788103bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:50.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.409+0000 7f678da16700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6788114450 con 0x7f6788105dd0 2024-05-04T03:14:50.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.409+0000 7f678da16700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f67881142d0 con 0x7f678810e8f0 2024-05-04T03:14:50.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.409+0000 7f678da16700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f67881145d0 con 0x7f6788105480 2024-05-04T03:14:50.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.409+0000 7f67867fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6788105dd0 0x7f67881036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:50.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.409+0000 7f67867fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6788105dd0 0x7f67881036b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41992/0 (socket says 172.21.15.1:41992) 2024-05-04T03:14:50.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.409+0000 7f67867fc700 1 -- 172.21.15.1:0/195600660 learned_addr learned my addr 172.21.15.1:0/195600660 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:14:50.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.409+0000 7f6786ffd700 1 --2- 172.21.15.1:0/195600660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6788105480 0x7f6788103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:50.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.409+0000 7f67877fe700 1 --2- 172.21.15.1:0/195600660 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f678810e8f0 0x7f6788103bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:50.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.409+0000 7f67867fc700 1 -- 172.21.15.1:0/195600660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6788105480 msgr2=0x7f6788103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:50.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.409+0000 7f67867fc700 1 --2- 172.21.15.1:0/195600660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6788105480 0x7f6788103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:50.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.409+0000 7f67867fc700 1 -- 172.21.15.1:0/195600660 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f678810e8f0 msgr2=0x7f6788103bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:50.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.409+0000 7f67867fc700 1 --2- 172.21.15.1:0/195600660 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f678810e8f0 0x7f6788103bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:50.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.409+0000 7f67867fc700 1 -- 172.21.15.1:0/195600660 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67881a9180 con 0x7f6788105dd0 2024-05-04T03:14:50.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.409+0000 7f67867fc700 1 --2- 172.21.15.1:0/195600660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6788105dd0 0x7f67881036b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f677c000c00 tx=0x7f677c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:50.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.410+0000 7f678ca14700 1 -- 172.21.15.1:0/195600660 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c027070 con 0x7f6788105dd0 2024-05-04T03:14:50.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.410+0000 7f678ca14700 1 -- 172.21.15.1:0/195600660 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f677c003f20 con 0x7f6788105dd0 2024-05-04T03:14:50.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.410+0000 7f678da16700 1 -- 172.21.15.1:0/195600660 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67881a9410 con 0x7f6788105dd0 2024-05-04T03:14:50.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.410+0000 7f678ca14700 1 -- 172.21.15.1:0/195600660 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c022ca0 con 0x7f6788105dd0 2024-05-04T03:14:50.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.410+0000 7f678da16700 1 -- 172.21.15.1:0/195600660 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67881a9860 con 0x7f6788105dd0 2024-05-04T03:14:50.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.411+0000 7f678da16700 1 -- 172.21.15.1:0/195600660 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67880512c0 con 0x7f6788105dd0 2024-05-04T03:14:50.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.412+0000 7f678ca14700 1 -- 172.21.15.1:0/195600660 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f677c02a070 con 0x7f6788105dd0 2024-05-04T03:14:50.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.413+0000 7f678ca14700 1 --2- 172.21.15.1:0/195600660 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f67700b2880 0x7f67700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:50.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.413+0000 7f6786ffd700 1 --2- 172.21.15.1:0/195600660 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f67700b2880 0x7f67700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:50.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.413+0000 7f678ca14700 1 -- 172.21.15.1:0/195600660 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f677c0e2fc0 con 0x7f6788105dd0 2024-05-04T03:14:50.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.413+0000 7f6786ffd700 1 --2- 172.21.15.1:0/195600660 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f67700b2880 0x7f67700b4d90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6778009550 tx=0x7f67780091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:50.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.417+0000 7f678ca14700 1 -- 172.21.15.1:0/195600660 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f677c031080 con 0x7f6788105dd0 2024-05-04T03:14:50.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:50 smithi072 ceph-mon[33332]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:14:50.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:50 smithi001 ceph-mon[29731]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:14:50.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.636+0000 7f678da16700 1 -- 172.21.15.1:0/195600660 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f678802e7b0 con 0x7f6788105dd0 2024-05-04T03:14:50.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.637+0000 7f678ca14700 1 -- 172.21.15.1:0/195600660 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+18325 (secure 0 0 0) 0x7f677c0b3070 con 0x7f6788105dd0 2024-05-04T03:14:50.639 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:14:50.639 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":74,"fsid":"4a572aa8-09c3-11ef-bc97-c7b262605968","created":"2024-05-04T03:07:32.845547+0000","modified":"2024-05-04T03:14:45.201229+0000","last_up_change":"2024-05-04T03:14:44.192571+0000","last_in_change":"2024-05-04T03:14:23.353342+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T03:11:01.139189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"100fe035-2454-4c95-90c7-759956ec36a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6803","nonce":3479011484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6805","nonce":3479011484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6809","nonce":3479011484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6807","nonce":3479011484}]},"public_addr":"172.21.15.1:6803/3479011484","cluster_addr":"172.21.15.1:6805/3479011484","heartbeat_back_addr":"172.21.15.1:6809/3479011484","heartbeat_front_addr":"172.21.15.1:6807/3479011484","state":["exists","up"]},{"osd":1,"uuid":"5826ac76-ef6b-4040-8ad1-305db355af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6801","nonce":3725706149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6803","nonce":3725706149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6807","nonce":3725706149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6805","nonce":3725706149}]},"public_addr":"172.21.15.72:6801/3725706149","cluster_addr":"172.21.15.72:6803/3725706149","heartbeat_back_addr":"172.21.15.72:6807/3725706149","heartbeat_front_addr":"172.21.15.72:6805/3725706149","state":["exists","up"]},{"osd":2,"uuid":"923d76e5-b032-4190-9e35-546d27722a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6801","nonce":515038343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6803","nonce":515038343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6807","nonce":515038343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6805","nonce":515038343}]},"public_addr":"172.21.15.77:6801/515038343","cluster_addr":"172.21.15.77:6803/515038343","heartbeat_back_addr":"172.21.15.77:6807/515038343","heartbeat_front_addr":"172.21.15.77:6805/515038343","state":["exists","up"]},{"osd":3,"uuid":"495cc0c5-6a99-481f-ad9e-38f05fb84356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6811","nonce":4050439365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6813","nonce":4050439365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6817","nonce":4050439365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6815","nonce":4050439365}]},"public_addr":"172.21.15.1:6811/4050439365","cluster_addr":"172.21.15.1:6813/4050439365","heartbeat_back_addr":"172.21.15.1:6817/4050439365","heartbeat_front_addr":"172.21.15.1:6815/4050439365","state":["exists","up"]},{"osd":4,"uuid":"b86b8026-79c0-4056-9e67-a61f54b6ab5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6809","nonce":2501732391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6811","nonce":2501732391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6815","nonce":2501732391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6813","nonce":2501732391}]},"public_addr":"172.21.15.72:6809/2501732391","cluster_addr":"172.21.15.72:6811/2501732391","heartbeat_back_addr":"172.21.15.72:6815/2501732391","heartbeat_front_addr":"172.21.15.72:6813/2501732391","state":["exists","up"]},{"osd":5,"uuid":"80975cb9-39e0-46b8-a343-9ff69194a14e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6809","nonce":3582852165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6811","nonce":3582852165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6815","nonce":3582852165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6813","nonce":3582852165}]},"public_addr":"172.21.15.77:6809/3582852165","cluster_addr":"172.21.15.77:6811/3582852165","heartbeat_back_addr":"172.21.15.77:6815/3582852165","heartbeat_front_addr":"172.21.15.77:6813/3582852165","state":["exists","up"]},{"osd":6,"uuid":"aca3fac5-fbb5-42de-bf0e-e4a0db7381e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6819","nonce":1799033649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6821","nonce":1799033649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6825","nonce":1799033649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6823","nonce":1799033649}]},"public_addr":"172.21.15.1:6819/1799033649","cluster_addr":"172.21.15.1:6821/1799033649","heartbeat_back_addr":"172.21.15.1:6825/1799033649","heartbeat_front_addr":"172.21.15.1:6823/1799033649","state":["exists","up"]},{"osd":7,"uuid":"cbff6b8a-c463-4249-ae45-fdc6f99a5489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6817","nonce":3460835447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6819","nonce":3460835447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6823","nonce":3460835447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6821","nonce":3460835447}]},"public_addr":"172.21.15.72:6817/3460835447","cluster_addr":"172.21.15.72:6819/3460835447","heartbeat_back_addr":"172.21.15.72:6823/3460835447","heartbeat_front_addr":"172.21.15.72:6821/3460835447","state":["exists","up"]},{"osd":8,"uuid":"e140bf13-4887-4d68-9373-3d4fc1c4584a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6817","nonce":1124479323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6819","nonce":1124479323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6823","nonce":1124479323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6821","nonce":1124479323}]},"public_addr":"172.21.15.77:6817/1124479323","cluster_addr":"172.21.15.77:6819/1124479323","heartbeat_back_addr":"172.21.15.77:6823/1124479323","heartbeat_front_addr":"172.21.15.77:6821/1124479323","state":["exists","up"]},{"osd":9,"uuid":"f1e8a232-cb11-4b9d-8789-e1637fa5aaa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6827","nonce":2681322272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6829","nonce":2681322272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6833","nonce":2681322272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6831","nonce":2681322272}]},"public_addr":"172.21.15.1:6827/2681322272","cluster_addr":"172.21.15.1:6829/2681322272","heartbeat_back_addr":"172.21.15.1:6833/2681322272","heartbeat_front_addr":"172.21.15.1:6831/2681322272","state":["exists","up"]},{"osd":10,"uuid":"79b570af-3d1e-439d-a328-9e6bc7d67839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6825","nonce":3293703569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6827","nonce":3293703569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6831","nonce":3293703569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6829","nonce":3293703569}]},"public_addr":"172.21.15.72:6825/3293703569","cluster_addr":"172.21.15.72:6827/3293703569","heartbeat_back_addr":"172.21.15.72:6831/3293703569","heartbeat_front_addr":"172.21.15.72:6829/3293703569","state":["exists","up"]},{"osd":11,"uuid":"e35bf83d-9ee4-4745-96c2-1c35cc8eefba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6825","nonce":4155947548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6827","nonce":4155947548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6831","nonce":4155947548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6829","nonce":4155947548}]},"public_addr":"172.21.15.77:6825/4155947548","cluster_addr":"172.21.15.77:6827/4155947548","heartbeat_back_addr":"172.21.15.77:6831/4155947548","heartbeat_front_addr":"172.21.15.77:6829/4155947548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:09.467126+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:33.861139+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:57.741962+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:11:21.746021+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:11:46.323252+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:12:10.640180+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:12:35.750591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:00.306549+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:25.012495+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:51.185229+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:14:16.403850+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:14:41.653316+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/2029897106":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/402488450":"2024-05-05T03:08:33.045884+0000","172.21.15.1:6801/128793132":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/2233526606":"2024-05-05T03:08:06.135783+0000","172.21.15.1:0/2715541855":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/2415697325":"2024-05-05T03:08:06.135783+0000","172.21.15.1:0/2305298106":"2024-05-05T03:08:06.135783+0000","172.21.15.1:6800/128793132":"2024-05-05T03:08:33.045884+0000","172.21.15.1:6800/4200529863":"2024-05-05T03:08:06.135783+0000","172.21.15.1:6801/4200529863":"2024-05-05T03:08:06.135783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T03:14:50.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.639+0000 7f678da16700 1 -- 172.21.15.1:0/195600660 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f67700b2880 msgr2=0x7f67700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:50.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.639+0000 7f678da16700 1 --2- 172.21.15.1:0/195600660 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f67700b2880 0x7f67700b4d90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6778009550 tx=0x7f67780091b0 comp rx=0 tx=0).stop 2024-05-04T03:14:50.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.640+0000 7f678da16700 1 -- 172.21.15.1:0/195600660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6788105dd0 msgr2=0x7f67881036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:50.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.640+0000 7f678da16700 1 --2- 172.21.15.1:0/195600660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6788105dd0 0x7f67881036b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f677c000c00 tx=0x7f677c0047b0 comp rx=0 tx=0).stop 2024-05-04T03:14:50.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.640+0000 7f678da16700 1 -- 172.21.15.1:0/195600660 shutdown_connections 2024-05-04T03:14:50.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.640+0000 7f678da16700 1 --2- 172.21.15.1:0/195600660 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f678810e8f0 0x7f6788103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:50.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.640+0000 7f678da16700 1 --2- 172.21.15.1:0/195600660 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f67700b2880 0x7f67700b4d90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:50.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.640+0000 7f678da16700 1 --2- 172.21.15.1:0/195600660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6788105480 0x7f6788103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:50.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.640+0000 7f678da16700 1 --2- 172.21.15.1:0/195600660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6788105dd0 0x7f67881036b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:50.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.640+0000 7f678da16700 1 -- 172.21.15.1:0/195600660 >> 172.21.15.1:0/195600660 conn(0x7f67880fef30 msgr2=0x7f67880ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:50.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.640+0000 7f678da16700 1 -- 172.21.15.1:0/195600660 shutdown_connections 2024-05-04T03:14:50.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:50.640+0000 7f678da16700 1 -- 172.21.15.1:0/195600660 wait complete. 2024-05-04T03:14:50.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:50 smithi077 ceph-mon[33531]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:14:51.260 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-04T03:11:01.139189+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-04T03:14:51.261 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-04T03:14:51.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:51 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/195600660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:14:51.506 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:14:51.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:51 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/195600660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:14:51.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:51 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/195600660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:14:52.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:52 smithi072 ceph-mon[33332]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:14:52.595 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:52 smithi001 ceph-mon[29731]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:14:52.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:52 smithi077 ceph-mon[33531]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:14:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.099+0000 7fafd238f700 1 -- 172.21.15.1:0/2718791896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafcc108760 msgr2=0x7fafcc1100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.099+0000 7fafd238f700 1 --2- 172.21.15.1:0/2718791896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafcc108760 0x7fafcc1100c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fafc400b0d0 tx=0x7fafc400b3e0 comp rx=0 tx=0).stop 2024-05-04T03:14:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.099+0000 7fafd238f700 1 -- 172.21.15.1:0/2718791896 shutdown_connections 2024-05-04T03:14:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.099+0000 7fafd238f700 1 --2- 172.21.15.1:0/2718791896 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fafcc1034e0 0x7fafcc1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.099+0000 7fafd238f700 1 --2- 172.21.15.1:0/2718791896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafcc108760 0x7fafcc1100c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.099+0000 7fafd238f700 1 --2- 172.21.15.1:0/2718791896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafcc107e10 0x7fafcc108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.099+0000 7fafd238f700 1 -- 172.21.15.1:0/2718791896 >> 172.21.15.1:0/2718791896 conn(0x7fafcc0fef30 msgr2=0x7fafcc101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.099+0000 7fafd238f700 1 -- 172.21.15.1:0/2718791896 shutdown_connections 2024-05-04T03:14:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.099+0000 7fafd238f700 1 -- 172.21.15.1:0/2718791896 wait complete. 2024-05-04T03:14:53.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.100+0000 7fafd238f700 1 Processor -- start 2024-05-04T03:14:53.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.100+0000 7fafd238f700 1 -- start start 2024-05-04T03:14:53.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.100+0000 7fafd238f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafcc1034e0 0x7fafcc0ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:53.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.100+0000 7fafd238f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafcc107e10 0x7fafcc0ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:53.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.100+0000 7fafd238f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fafcc108760 0x7fafcc0ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:53.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.100+0000 7fafd238f700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc113460 con 0x7fafcc107e10 2024-05-04T03:14:53.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.100+0000 7fafd238f700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc1132e0 con 0x7fafcc108760 2024-05-04T03:14:53.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.100+0000 7fafd238f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc1135e0 con 0x7fafcc1034e0 2024-05-04T03:14:53.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.101+0000 7fafcb7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafcc107e10 0x7fafcc0ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:53.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.101+0000 7fafcb7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafcc107e10 0x7fafcc0ff950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45274/0 (socket says 172.21.15.1:45274) 2024-05-04T03:14:53.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.101+0000 7fafcb7fe700 1 -- 172.21.15.1:0/2677196421 learned_addr learned my addr 172.21.15.1:0/2677196421 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:14:53.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.101+0000 7fafcbfff700 1 --2- 172.21.15.1:0/2677196421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafcc1034e0 0x7fafcc0ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:53.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.101+0000 7fafd092c700 1 --2- 172.21.15.1:0/2677196421 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fafcc108760 0x7fafcc0ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:53.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.101+0000 7fafcb7fe700 1 -- 172.21.15.1:0/2677196421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafcc1034e0 msgr2=0x7fafcc0ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:53.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.101+0000 7fafcb7fe700 1 --2- 172.21.15.1:0/2677196421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafcc1034e0 0x7fafcc0ff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:53.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.101+0000 7fafcb7fe700 1 -- 172.21.15.1:0/2677196421 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fafcc108760 msgr2=0x7fafcc0ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:53.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.101+0000 7fafcb7fe700 1 --2- 172.21.15.1:0/2677196421 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fafcc108760 0x7fafcc0ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:53.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.101+0000 7fafcb7fe700 1 -- 172.21.15.1:0/2677196421 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafcc1a9240 con 0x7fafcc107e10 2024-05-04T03:14:53.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.101+0000 7fafd092c700 1 --2- 172.21.15.1:0/2677196421 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fafcc108760 0x7fafcc0ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:14:53.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.101+0000 7fafcb7fe700 1 --2- 172.21.15.1:0/2677196421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafcc107e10 0x7fafcc0ff950 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00c8f0 tx=0x7fafbc00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:53.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.101+0000 7fafc97fa700 1 -- 172.21.15.1:0/2677196421 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc007940 con 0x7fafcc107e10 2024-05-04T03:14:53.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.101+0000 7fafc97fa700 1 -- 172.21.15.1:0/2677196421 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafbc00f450 con 0x7fafcc107e10 2024-05-04T03:14:53.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.101+0000 7fafd238f700 1 -- 172.21.15.1:0/2677196421 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafcc1a9530 con 0x7fafcc107e10 2024-05-04T03:14:53.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.101+0000 7fafc97fa700 1 -- 172.21.15.1:0/2677196421 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc018660 con 0x7fafcc107e10 2024-05-04T03:14:53.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.101+0000 7fafd238f700 1 -- 172.21.15.1:0/2677196421 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafcc1a9a70 con 0x7fafcc107e10 2024-05-04T03:14:53.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.103+0000 7fafd238f700 1 -- 172.21.15.1:0/2677196421 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafcc0512c0 con 0x7fafcc107e10 2024-05-04T03:14:53.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.104+0000 7fafc97fa700 1 -- 172.21.15.1:0/2677196421 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fafbc00f890 con 0x7fafcc107e10 2024-05-04T03:14:53.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.105+0000 7fafc97fa700 1 --2- 172.21.15.1:0/2677196421 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fafb80b2880 0x7fafb80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:53.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.105+0000 7fafc97fa700 1 -- 172.21.15.1:0/2677196421 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fafbc0e1cd0 con 0x7fafcc107e10 2024-05-04T03:14:53.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.105+0000 7fafcbfff700 1 --2- 172.21.15.1:0/2677196421 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fafb80b2880 0x7fafb80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:53.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.105+0000 7fafcbfff700 1 --2- 172.21.15.1:0/2677196421 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fafb80b2880 0x7fafb80b4d90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fafb4009a10 tx=0x7fafb4009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:53.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.109+0000 7fafc97fa700 1 -- 172.21.15.1:0/2677196421 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fafbc014030 con 0x7fafcc107e10 2024-05-04T03:14:53.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.321+0000 7fafd238f700 1 -- 172.21.15.1:0/2677196421 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fafcc02e5f0 con 0x7fafcc107e10 2024-05-04T03:14:53.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.322+0000 7fafc97fa700 1 -- 172.21.15.1:0/2677196421 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7fafbc0a9f20 con 0x7fafcc107e10 2024-05-04T03:14:53.324 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-04T03:14:53.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.324+0000 7fafd238f700 1 -- 172.21.15.1:0/2677196421 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fafb80b2880 msgr2=0x7fafb80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:53.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.324+0000 7fafd238f700 1 --2- 172.21.15.1:0/2677196421 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fafb80b2880 0x7fafb80b4d90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fafb4009a10 tx=0x7fafb4009190 comp rx=0 tx=0).stop 2024-05-04T03:14:53.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.324+0000 7fafd238f700 1 -- 172.21.15.1:0/2677196421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafcc107e10 msgr2=0x7fafcc0ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:53.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.324+0000 7fafd238f700 1 --2- 172.21.15.1:0/2677196421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafcc107e10 0x7fafcc0ff950 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00c8f0 tx=0x7fafbc00cc00 comp rx=0 tx=0).stop 2024-05-04T03:14:53.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.324+0000 7fafd238f700 1 -- 172.21.15.1:0/2677196421 shutdown_connections 2024-05-04T03:14:53.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.324+0000 7fafd238f700 1 --2- 172.21.15.1:0/2677196421 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fafcc108760 0x7fafcc0ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:53.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.324+0000 7fafd238f700 1 --2- 172.21.15.1:0/2677196421 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fafb80b2880 0x7fafb80b4d90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:53.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.324+0000 7fafd238f700 1 --2- 172.21.15.1:0/2677196421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fafcc1034e0 0x7fafcc0ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:53.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.324+0000 7fafd238f700 1 --2- 172.21.15.1:0/2677196421 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fafcc107e10 0x7fafcc0ff950 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:53.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.324+0000 7fafd238f700 1 -- 172.21.15.1:0/2677196421 >> 172.21.15.1:0/2677196421 conn(0x7fafcc0fef30 msgr2=0x7fafcc06c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:53.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.324+0000 7fafd238f700 1 -- 172.21.15.1:0/2677196421 shutdown_connections 2024-05-04T03:14:53.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:53.324+0000 7fafd238f700 1 -- 172.21.15.1:0/2677196421 wait complete. 2024-05-04T03:14:53.949 INFO:tasks.cephadm:Adding mds.a on smithi001 2024-05-04T03:14:53.949 INFO:tasks.cephadm:Adding mds.d on smithi001 2024-05-04T03:14:53.949 INFO:tasks.cephadm:Adding mds.g on smithi001 2024-05-04T03:14:53.950 INFO:tasks.cephadm:Adding mds.j on smithi001 2024-05-04T03:14:53.950 INFO:tasks.cephadm:Adding mds.b on smithi072 2024-05-04T03:14:53.950 INFO:tasks.cephadm:Adding mds.e on smithi072 2024-05-04T03:14:53.950 INFO:tasks.cephadm:Adding mds.h on smithi072 2024-05-04T03:14:53.950 INFO:tasks.cephadm:Adding mds.k on smithi072 2024-05-04T03:14:53.950 INFO:tasks.cephadm:Adding mds.c on smithi077 2024-05-04T03:14:53.950 INFO:tasks.cephadm:Adding mds.f on smithi077 2024-05-04T03:14:53.950 INFO:tasks.cephadm:Adding mds.i on smithi077 2024-05-04T03:14:53.950 INFO:tasks.cephadm:Adding mds.l on smithi077 2024-05-04T03:14:53.950 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph orch apply mds all '12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi072=b;smithi072=e;smithi072=h;smithi072=k;smithi077=c;smithi077=f;smithi077=i;smithi077=l' 2024-05-04T03:14:54.201 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:14:54.322 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:54 smithi077 ceph-mon[33531]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:14:54.323 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:54 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/2677196421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-04T03:14:54.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:54 smithi072 ceph-mon[33332]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:14:54.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:54 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/2677196421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-04T03:14:54.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:54 smithi001 ceph-mon[29731]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:14:54.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:54 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2677196421' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-04T03:14:55.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.813+0000 7f6f13094700 1 -- 172.21.15.77:0/2746185978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f0c10a0e0 msgr2=0x7f6f0c10c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:55.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.813+0000 7f6f13094700 1 --2- 172.21.15.77:0/2746185978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f0c10a0e0 0x7f6f0c10c500 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009990 tx=0x7f6f04009ca0 comp rx=0 tx=0).stop 2024-05-04T03:14:55.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.813+0000 7f6f13094700 1 -- 172.21.15.77:0/2746185978 shutdown_connections 2024-05-04T03:14:55.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.813+0000 7f6f13094700 1 --2- 172.21.15.77:0/2746185978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f0c10a0e0 0x7f6f0c10c500 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:55.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.813+0000 7f6f13094700 1 --2- 172.21.15.77:0/2746185978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f0c102690 0x7f6f0c102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:55.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.813+0000 7f6f13094700 1 --2- 172.21.15.77:0/2746185978 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f0c107780 0x7f6f0c109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:55.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.813+0000 7f6f13094700 1 -- 172.21.15.77:0/2746185978 >> 172.21.15.77:0/2746185978 conn(0x7f6f0c0fe010 msgr2=0x7f6f0c1004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:55.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.813+0000 7f6f13094700 1 -- 172.21.15.77:0/2746185978 shutdown_connections 2024-05-04T03:14:55.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.813+0000 7f6f13094700 1 -- 172.21.15.77:0/2746185978 wait complete. 2024-05-04T03:14:55.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.814+0000 7f6f13094700 1 Processor -- start 2024-05-04T03:14:55.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.814+0000 7f6f13094700 1 -- start start 2024-05-04T03:14:55.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.814+0000 7f6f13094700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f0c102690 0x7f6f0c104f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:55.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.814+0000 7f6f13094700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f0c107780 0x7f6f0c103630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:55.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.814+0000 7f6f13094700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f0c10a0e0 0x7f6f0c103b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:55.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.814+0000 7f6f13094700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c10fd70 con 0x7f6f0c10a0e0 2024-05-04T03:14:55.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.814+0000 7f6f13094700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c10fbf0 con 0x7f6f0c102690 2024-05-04T03:14:55.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.814+0000 7f6f13094700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c10fef0 con 0x7f6f0c107780 2024-05-04T03:14:55.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.814+0000 7f6f0bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f0c107780 0x7f6f0c103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:55.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.814+0000 7f6f0bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f0c107780 0x7f6f0c103630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55414/0 (socket says 172.21.15.77:55414) 2024-05-04T03:14:55.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.814+0000 7f6f0bfff700 1 -- 172.21.15.77:0/339558422 learned_addr learned my addr 172.21.15.77:0/339558422 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-04T03:14:55.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.814+0000 7f6f10e30700 1 --2- 172.21.15.77:0/339558422 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f0c102690 0x7f6f0c104f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:55.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.814+0000 7f6f11631700 1 --2- 172.21.15.77:0/339558422 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f0c10a0e0 0x7f6f0c103b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:55.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.814+0000 7f6f0bfff700 1 -- 172.21.15.77:0/339558422 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f0c102690 msgr2=0x7f6f0c104f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:55.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.814+0000 7f6f0bfff700 1 --2- 172.21.15.77:0/339558422 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f0c102690 0x7f6f0c104f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:55.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.814+0000 7f6f0bfff700 1 -- 172.21.15.77:0/339558422 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f0c10a0e0 msgr2=0x7f6f0c103b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:55.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.814+0000 7f6f0bfff700 1 --2- 172.21.15.77:0/339558422 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f0c10a0e0 0x7f6f0c103b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:55.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.815+0000 7f6f0bfff700 1 -- 172.21.15.77:0/339558422 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0c104240 con 0x7f6f0c107780 2024-05-04T03:14:55.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.815+0000 7f6f0bfff700 1 --2- 172.21.15.77:0/339558422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f0c107780 0x7f6f0c103630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400c9b0 tx=0x7f6ef400cd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:55.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.815+0000 7f6f09ffb700 1 -- 172.21.15.77:0/339558422 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4007b00 con 0x7f6f0c107780 2024-05-04T03:14:55.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.815+0000 7f6f09ffb700 1 -- 172.21.15.77:0/339558422 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ef4007ca0 con 0x7f6f0c107780 2024-05-04T03:14:55.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.815+0000 7f6f09ffb700 1 -- 172.21.15.77:0/339558422 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4018730 con 0x7f6f0c107780 2024-05-04T03:14:55.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.815+0000 7f6f13094700 1 -- 172.21.15.77:0/339558422 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f0c1a48b0 con 0x7f6f0c107780 2024-05-04T03:14:55.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.815+0000 7f6f13094700 1 -- 172.21.15.77:0/339558422 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f0c1a4cb0 con 0x7f6f0c107780 2024-05-04T03:14:55.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.816+0000 7f6f13094700 1 -- 172.21.15.77:0/339558422 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f0c0512c0 con 0x7f6f0c107780 2024-05-04T03:14:55.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.818+0000 7f6f09ffb700 1 -- 172.21.15.77:0/339558422 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f6ef400f450 con 0x7f6f0c107780 2024-05-04T03:14:55.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.818+0000 7f6f09ffb700 1 --2- 172.21.15.77:0/339558422 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6ef80b2880 0x7f6ef80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:55.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.818+0000 7f6f09ffb700 1 -- 172.21.15.77:0/339558422 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f6ef40e2100 con 0x7f6f0c107780 2024-05-04T03:14:55.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.818+0000 7f6f10e30700 1 --2- 172.21.15.77:0/339558422 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6ef80b2880 0x7f6ef80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:55.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.819+0000 7f6f10e30700 1 --2- 172.21.15.77:0/339558422 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6ef80b2880 0x7f6ef80b4d90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0060b0 tx=0x7f6efc007b80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:55.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:55.822+0000 7f6f09ffb700 1 -- 172.21.15.77:0/339558422 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6ef4014030 con 0x7f6f0c107780 2024-05-04T03:14:56.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:56.039+0000 7f6f13094700 1 -- 172.21.15.77:0/339558422 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi072=b;smithi072=e;smithi072=h;smithi072=k;smithi077=c;smithi077=f;smithi077=i;smithi077=l", "target": ["mon-mgr", ""]}) -- 0x7f6f0c063e00 con 0x7f6ef80b2880 2024-05-04T03:14:56.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:56.048+0000 7f6f09ffb700 1 -- 172.21.15.77:0/339558422 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f6f0c063e00 con 0x7f6ef80b2880 2024-05-04T03:14:56.049 INFO:teuthology.orchestra.run.smithi077.stdout:Scheduled mds.all update... 2024-05-04T03:14:56.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:56.050+0000 7f6f13094700 1 -- 172.21.15.77:0/339558422 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6ef80b2880 msgr2=0x7f6ef80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:56.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:56.050+0000 7f6f13094700 1 --2- 172.21.15.77:0/339558422 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6ef80b2880 0x7f6ef80b4d90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0060b0 tx=0x7f6efc007b80 comp rx=0 tx=0).stop 2024-05-04T03:14:56.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:56.050+0000 7f6f13094700 1 -- 172.21.15.77:0/339558422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f0c107780 msgr2=0x7f6f0c103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:56.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:56.050+0000 7f6f13094700 1 --2- 172.21.15.77:0/339558422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f0c107780 0x7f6f0c103630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400c9b0 tx=0x7f6ef400cd70 comp rx=0 tx=0).stop 2024-05-04T03:14:56.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:56.050+0000 7f6f13094700 1 -- 172.21.15.77:0/339558422 shutdown_connections 2024-05-04T03:14:56.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:56.050+0000 7f6f13094700 1 --2- 172.21.15.77:0/339558422 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6f0c102690 0x7f6f0c104f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:56.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:56.050+0000 7f6f13094700 1 --2- 172.21.15.77:0/339558422 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6ef80b2880 0x7f6ef80b4d90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:56.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:56.050+0000 7f6f13094700 1 --2- 172.21.15.77:0/339558422 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f0c10a0e0 0x7f6f0c103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:56.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:56.050+0000 7f6f13094700 1 --2- 172.21.15.77:0/339558422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6f0c107780 0x7f6f0c103630 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:56.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:56.050+0000 7f6f13094700 1 -- 172.21.15.77:0/339558422 >> 172.21.15.77:0/339558422 conn(0x7f6f0c0fe010 msgr2=0x7f6f0c1085c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:56.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:56.050+0000 7f6f13094700 1 -- 172.21.15.77:0/339558422 shutdown_connections 2024-05-04T03:14:56.052 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-04T03:14:56.050+0000 7f6f13094700 1 -- 172.21.15.77:0/339558422 wait complete. 2024-05-04T03:14:56.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:56 smithi072 ceph-mon[33332]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:14:56.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:56 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:56.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:56 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:14:56.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:56 smithi001 ceph-mon[29731]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:14:56.829 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:56 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:56.830 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:56 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:14:56.831 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:56 smithi077 ceph-mon[33531]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:14:56.831 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:56 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:56.831 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:56 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:14:56.832 DEBUG:teuthology.orchestra.run.smithi001:mds.a> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.a.service 2024-05-04T03:14:56.837 DEBUG:teuthology.orchestra.run.smithi001:mds.d> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.d.service 2024-05-04T03:14:56.882 DEBUG:teuthology.orchestra.run.smithi001:mds.g> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.g.service 2024-05-04T03:14:56.927 DEBUG:teuthology.orchestra.run.smithi001:mds.j> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.j.service 2024-05-04T03:14:56.970 DEBUG:teuthology.orchestra.run.smithi072:mds.b> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.b.service 2024-05-04T03:14:56.973 DEBUG:teuthology.orchestra.run.smithi072:mds.e> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.e.service 2024-05-04T03:14:57.017 DEBUG:teuthology.orchestra.run.smithi072:mds.h> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.h.service 2024-05-04T03:14:57.061 DEBUG:teuthology.orchestra.run.smithi072:mds.k> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.k.service 2024-05-04T03:14:57.104 DEBUG:teuthology.orchestra.run.smithi077:mds.c> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.c.service 2024-05-04T03:14:57.107 DEBUG:teuthology.orchestra.run.smithi077:mds.f> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.f.service 2024-05-04T03:14:57.150 DEBUG:teuthology.orchestra.run.smithi077:mds.i> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.i.service 2024-05-04T03:14:57.193 DEBUG:teuthology.orchestra.run.smithi077:mds.l> sudo journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.l.service 2024-05-04T03:14:57.237 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-04T03:14:57.238 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:14:57.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:57 smithi072 ceph-mon[33332]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi072=b;smithi072=e;smithi072=h;smithi072=k;smithi077=c;smithi077=f;smithi077=i;smithi077=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:14:57.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:57 smithi072 ceph-mon[33332]: Saving service mds.all spec with placement smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi072=b;smithi072=e;smithi072=h;smithi072=k;smithi077=c;smithi077=f;smithi077=i;smithi077=l;count:12 2024-05-04T03:14:57.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:57 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:57.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:57 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:14:57.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:57 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:57.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:57 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:14:57.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:57 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:14:57.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:57 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:57.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:57 smithi077 ceph-mon[33531]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi072=b;smithi072=e;smithi072=h;smithi072=k;smithi077=c;smithi077=f;smithi077=i;smithi077=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:14:57.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:57 smithi077 ceph-mon[33531]: Saving service mds.all spec with placement smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi072=b;smithi072=e;smithi072=h;smithi072=k;smithi077=c;smithi077=f;smithi077=i;smithi077=l;count:12 2024-05-04T03:14:57.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:57 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:57.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:57 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:14:57.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:57 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:57.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:57 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:14:57.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:57 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:14:57.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:57 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:57.483 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:14:57.605 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:57 smithi001 ceph-mon[29731]: from='client.24412 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi072=b;smithi072=e;smithi072=h;smithi072=k;smithi077=c;smithi077=f;smithi077=i;smithi077=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:14:57.605 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:57 smithi001 ceph-mon[29731]: Saving service mds.all spec with placement smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi072=b;smithi072=e;smithi072=h;smithi072=k;smithi077=c;smithi077=f;smithi077=i;smithi077=l;count:12 2024-05-04T03:14:57.605 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:57 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:57.606 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:57 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:14:57.606 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:57 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:14:57.606 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:57 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:14:57.606 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:57 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:14:57.606 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:57 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:14:58.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:58 smithi072 ceph-mon[33332]: Deploying daemon mds.l on smithi077 2024-05-04T03:14:58.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:14:58 smithi072 ceph-mon[33332]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:14:58.526 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:58 smithi077 ceph-mon[33531]: Deploying daemon mds.l on smithi077 2024-05-04T03:14:58.526 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:14:58 smithi077 ceph-mon[33531]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:14:58.617 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:58 smithi001 ceph-mon[29731]: Deploying daemon mds.l on smithi077 2024-05-04T03:14:58.617 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:14:58 smithi001 ceph-mon[29731]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:14:59.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.116+0000 7f2bacc6b700 1 -- 172.21.15.1:0/2902353248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ba8105e40 msgr2=0x7f2ba810e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:59.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.116+0000 7f2bacc6b700 1 --2- 172.21.15.1:0/2902353248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ba8105e40 0x7f2ba810e390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c009990 tx=0x7f2b9c009ca0 comp rx=0 tx=0).stop 2024-05-04T03:14:59.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.116+0000 7f2bacc6b700 1 -- 172.21.15.1:0/2902353248 shutdown_connections 2024-05-04T03:14:59.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.116+0000 7f2bacc6b700 1 --2- 172.21.15.1:0/2902353248 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ba810e920 0x7f2ba8110e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:59.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.116+0000 7f2bacc6b700 1 --2- 172.21.15.1:0/2902353248 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ba81054f0 0x7f2ba8105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:59.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.116+0000 7f2bacc6b700 1 --2- 172.21.15.1:0/2902353248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ba8105e40 0x7f2ba810e390 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:59.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.116+0000 7f2bacc6b700 1 -- 172.21.15.1:0/2902353248 >> 172.21.15.1:0/2902353248 conn(0x7f2ba8078e00 msgr2=0x7f2ba8079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:59.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.116+0000 7f2bacc6b700 1 -- 172.21.15.1:0/2902353248 shutdown_connections 2024-05-04T03:14:59.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.116+0000 7f2bacc6b700 1 -- 172.21.15.1:0/2902353248 wait complete. 2024-05-04T03:14:59.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.117+0000 7f2bacc6b700 1 Processor -- start 2024-05-04T03:14:59.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.117+0000 7f2bacc6b700 1 -- start start 2024-05-04T03:14:59.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.117+0000 7f2bacc6b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ba81054f0 0x7f2ba819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:59.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.117+0000 7f2bacc6b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ba8105e40 0x7f2ba819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:59.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.117+0000 7f2bacc6b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ba810e920 0x7f2ba819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:59.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.118+0000 7f2bacc6b700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba8114480 con 0x7f2ba810e920 2024-05-04T03:14:59.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.118+0000 7f2bacc6b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba8114300 con 0x7f2ba81054f0 2024-05-04T03:14:59.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.118+0000 7f2bacc6b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba8114600 con 0x7f2ba8105e40 2024-05-04T03:14:59.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.118+0000 7f2ba6d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ba810e920 0x7f2ba819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:59.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.118+0000 7f2ba6d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ba810e920 0x7f2ba819f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45296/0 (socket says 172.21.15.1:45296) 2024-05-04T03:14:59.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.118+0000 7f2ba6d9d700 1 -- 172.21.15.1:0/2124196391 learned_addr learned my addr 172.21.15.1:0/2124196391 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:14:59.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.118+0000 7f2ba5d9b700 1 --2- 172.21.15.1:0/2124196391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ba8105e40 0x7f2ba819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:59.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.118+0000 7f2ba659c700 1 --2- 172.21.15.1:0/2124196391 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ba81054f0 0x7f2ba819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:59.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.118+0000 7f2ba6d9d700 1 -- 172.21.15.1:0/2124196391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ba8105e40 msgr2=0x7f2ba819b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:59.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.118+0000 7f2ba6d9d700 1 --2- 172.21.15.1:0/2124196391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ba8105e40 0x7f2ba819b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:59.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.118+0000 7f2ba6d9d700 1 -- 172.21.15.1:0/2124196391 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ba81054f0 msgr2=0x7f2ba819ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:59.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.118+0000 7f2ba6d9d700 1 --2- 172.21.15.1:0/2124196391 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ba81054f0 0x7f2ba819ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:59.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.118+0000 7f2ba6d9d700 1 -- 172.21.15.1:0/2124196391 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba819fce0 con 0x7f2ba810e920 2024-05-04T03:14:59.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.118+0000 7f2ba6d9d700 1 --2- 172.21.15.1:0/2124196391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ba810e920 0x7f2ba819f640 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000ef30 tx=0x7f2ba000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:59.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.118+0000 7f2b977fe700 1 -- 172.21.15.1:0/2124196391 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba00092e0 con 0x7f2ba810e920 2024-05-04T03:14:59.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.118+0000 7f2bacc6b700 1 -- 172.21.15.1:0/2124196391 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba819ff70 con 0x7f2ba810e920 2024-05-04T03:14:59.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.118+0000 7f2b977fe700 1 -- 172.21.15.1:0/2124196391 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ba0004bb0 con 0x7f2ba810e920 2024-05-04T03:14:59.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.118+0000 7f2b977fe700 1 -- 172.21.15.1:0/2124196391 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0004020 con 0x7f2ba810e920 2024-05-04T03:14:59.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.118+0000 7f2bacc6b700 1 -- 172.21.15.1:0/2124196391 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba81a0430 con 0x7f2ba810e920 2024-05-04T03:14:59.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.119+0000 7f2bacc6b700 1 -- 172.21.15.1:0/2124196391 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ba80512c0 con 0x7f2ba810e920 2024-05-04T03:14:59.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.121+0000 7f2b977fe700 1 -- 172.21.15.1:0/2124196391 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f2ba001b030 con 0x7f2ba810e920 2024-05-04T03:14:59.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.122+0000 7f2b977fe700 1 --2- 172.21.15.1:0/2124196391 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f2b900b2880 0x7f2b900b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:14:59.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.122+0000 7f2ba659c700 1 --2- 172.21.15.1:0/2124196391 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f2b900b2880 0x7f2b900b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:14:59.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.122+0000 7f2b977fe700 1 -- 172.21.15.1:0/2124196391 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f2ba00e2280 con 0x7f2ba810e920 2024-05-04T03:14:59.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.122+0000 7f2ba659c700 1 --2- 172.21.15.1:0/2124196391 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f2b900b2880 0x7f2b900b4d90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2b98005950 tx=0x7f2b9800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:14:59.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.125+0000 7f2b977fe700 1 -- 172.21.15.1:0/2124196391 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2ba0022020 con 0x7f2ba810e920 2024-05-04T03:14:59.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.393+0000 7f2bacc6b700 1 -- 172.21.15.1:0/2124196391 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ba802e5f0 con 0x7f2ba810e920 2024-05-04T03:14:59.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.394+0000 7f2b977fe700 1 -- 172.21.15.1:0/2124196391 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f2ba0010040 con 0x7f2ba810e920 2024-05-04T03:14:59.395 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:14:59.395 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-04T03:14:59.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.396+0000 7f2bacc6b700 1 -- 172.21.15.1:0/2124196391 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f2b900b2880 msgr2=0x7f2b900b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:59.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.396+0000 7f2bacc6b700 1 --2- 172.21.15.1:0/2124196391 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f2b900b2880 0x7f2b900b4d90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2b98005950 tx=0x7f2b9800b410 comp rx=0 tx=0).stop 2024-05-04T03:14:59.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.396+0000 7f2bacc6b700 1 -- 172.21.15.1:0/2124196391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ba810e920 msgr2=0x7f2ba819f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:14:59.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.396+0000 7f2bacc6b700 1 --2- 172.21.15.1:0/2124196391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ba810e920 0x7f2ba819f640 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000ef30 tx=0x7f2ba000c3a0 comp rx=0 tx=0).stop 2024-05-04T03:14:59.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.396+0000 7f2bacc6b700 1 -- 172.21.15.1:0/2124196391 shutdown_connections 2024-05-04T03:14:59.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.396+0000 7f2bacc6b700 1 --2- 172.21.15.1:0/2124196391 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2ba81054f0 0x7f2ba819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:59.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.396+0000 7f2bacc6b700 1 --2- 172.21.15.1:0/2124196391 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f2b900b2880 0x7f2b900b4d90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:59.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.396+0000 7f2bacc6b700 1 --2- 172.21.15.1:0/2124196391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ba810e920 0x7f2ba819f640 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:59.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.396+0000 7f2bacc6b700 1 --2- 172.21.15.1:0/2124196391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2ba8105e40 0x7f2ba819b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:14:59.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.396+0000 7f2bacc6b700 1 -- 172.21.15.1:0/2124196391 >> 172.21.15.1:0/2124196391 conn(0x7f2ba8078e00 msgr2=0x7f2ba81000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:14:59.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.396+0000 7f2bacc6b700 1 -- 172.21.15.1:0/2124196391 shutdown_connections 2024-05-04T03:14:59.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:14:59.396+0000 7f2bacc6b700 1 -- 172.21.15.1:0/2124196391 wait complete. 2024-05-04T03:14:59.399 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 1 2024-05-04T03:14:59.678 INFO:journalctl@ceph.mds.l.smithi077.stdout:May 04 03:14:59 smithi077 systemd[1]: Started Ceph mds.l for 4a572aa8-09c3-11ef-bc97-c7b262605968. 2024-05-04T03:15:00.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:00 smithi001 ceph-mon[29731]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:00.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:00 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2124196391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:15:00.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:00 smithi001 ceph-mon[29731]: mds.? [v2:172.21.15.77:6832/379026128,v1:172.21.15.77:6833/379026128] up:boot 2024-05-04T03:15:00.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:00 smithi001 ceph-mon[29731]: fsmap 1 up:standby 2024-05-04T03:15:00.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:00 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-04T03:15:00.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:00 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:00.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:00 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:00.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:00 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:00.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:00 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:00.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:00 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:00.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:00 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:00.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:00 smithi001 ceph-mon[29731]: Deploying daemon mds.k on smithi072 2024-05-04T03:15:00.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:00 smithi072 ceph-mon[33332]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:00.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:00 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/2124196391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:15:00.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:00 smithi072 ceph-mon[33332]: mds.? [v2:172.21.15.77:6832/379026128,v1:172.21.15.77:6833/379026128] up:boot 2024-05-04T03:15:00.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:00 smithi072 ceph-mon[33332]: fsmap 1 up:standby 2024-05-04T03:15:00.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:00 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-04T03:15:00.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:00 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:00.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:00 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:00.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:00 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:00.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:00 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:00.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:00 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:00.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:00 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:00.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:00 smithi072 ceph-mon[33332]: Deploying daemon mds.k on smithi072 2024-05-04T03:15:00.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:00 smithi077 ceph-mon[33531]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:00.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:00 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/2124196391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:15:00.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:00 smithi077 ceph-mon[33531]: mds.? [v2:172.21.15.77:6832/379026128,v1:172.21.15.77:6833/379026128] up:boot 2024-05-04T03:15:00.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:00 smithi077 ceph-mon[33531]: fsmap 1 up:standby 2024-05-04T03:15:00.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:00 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-04T03:15:00.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:00 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:00.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:00 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:00.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:00 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:00.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:00 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:00.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:00 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:00.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:00 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:00.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:00 smithi077 ceph-mon[33531]: Deploying daemon mds.k on smithi072 2024-05-04T03:15:01.996 INFO:journalctl@ceph.mds.k.smithi072.stdout:May 04 03:15:01 smithi072 systemd[1]: Starting Ceph mds.k for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:15:02.070 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:15:02.275 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:02 smithi072 ceph-mon[33332]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:02.275 INFO:journalctl@ceph.mds.k.smithi072.stdout:May 04 03:15:02 smithi072 podman[62366]: 2024-05-04 03:15:01.912476395 +0000 UTC m=+0.018085287 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:15:02.276 INFO:journalctl@ceph.mds.k.smithi072.stdout:May 04 03:15:02 smithi072 podman[62366]: 2024-05-04 03:15:02.009314562 +0000 UTC m=+0.114923454 container create 040b9c4f8036c656a3a1bc8f4da9832d8d74ccdd7c61c4bc5d6c2ed6d38231b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240430) 2024-05-04T03:15:02.276 INFO:journalctl@ceph.mds.k.smithi072.stdout:May 04 03:15:02 smithi072 podman[62366]: 2024-05-04 03:15:02.238765033 +0000 UTC m=+0.344373925 container init 040b9c4f8036c656a3a1bc8f4da9832d8d74ccdd7c61c4bc5d6c2ed6d38231b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-k, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-04T03:15:02.276 INFO:journalctl@ceph.mds.k.smithi072.stdout:May 04 03:15:02 smithi072 podman[62366]: 2024-05-04 03:15:02.241757265 +0000 UTC m=+0.347366166 container start 040b9c4f8036c656a3a1bc8f4da9832d8d74ccdd7c61c4bc5d6c2ed6d38231b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-k, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-04T03:15:02.276 INFO:journalctl@ceph.mds.k.smithi072.stdout:May 04 03:15:02 smithi072 ceph-mds[62385]: -- 172.21.15.72:0/870135772 <== mon.2 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55edb7b8fa40 con 0x55edb7bd7a80 2024-05-04T03:15:02.333 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:15:02.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:02 smithi001 ceph-mon[29731]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:02.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:02 smithi077 ceph-mon[33531]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:02.747 INFO:journalctl@ceph.mds.k.smithi072.stdout:May 04 03:15:02 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-k[62381]: starting mds.k at 2024-05-04T03:15:02.747 INFO:journalctl@ceph.mds.k.smithi072.stdout:May 04 03:15:02 smithi072 bash[62366]: 040b9c4f8036c656a3a1bc8f4da9832d8d74ccdd7c61c4bc5d6c2ed6d38231b8 2024-05-04T03:15:02.747 INFO:journalctl@ceph.mds.k.smithi072.stdout:May 04 03:15:02 smithi072 systemd[1]: Started Ceph mds.k for 4a572aa8-09c3-11ef-bc97-c7b262605968. 2024-05-04T03:15:03.486 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:03.486 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:03.486 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:03.486 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:03.486 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:03.486 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:03.487 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:03 smithi001 ceph-mon[29731]: Deploying daemon mds.j on smithi001 2024-05-04T03:15:03.487 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:03 smithi001 ceph-mon[29731]: mds.? [v2:172.21.15.72:6832/3816098761,v1:172.21.15.72:6833/3816098761] up:boot 2024-05-04T03:15:03.487 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:03 smithi001 ceph-mon[29731]: fsmap 2 up:standby 2024-05-04T03:15:03.487 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:03 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-04T03:15:03.653 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:03.653 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:03.653 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:03.653 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:03.653 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:03.653 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:03.653 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:03 smithi072 ceph-mon[33332]: Deploying daemon mds.j on smithi001 2024-05-04T03:15:03.654 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:03 smithi072 ceph-mon[33332]: mds.? [v2:172.21.15.72:6832/3816098761,v1:172.21.15.72:6833/3816098761] up:boot 2024-05-04T03:15:03.654 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:03 smithi072 ceph-mon[33332]: fsmap 2 up:standby 2024-05-04T03:15:03.654 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:03 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-04T03:15:03.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:03.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:03.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:03.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:03.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:03.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:03.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:03 smithi077 ceph-mon[33531]: Deploying daemon mds.j on smithi001 2024-05-04T03:15:03.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:03 smithi077 ceph-mon[33531]: mds.? [v2:172.21.15.72:6832/3816098761,v1:172.21.15.72:6833/3816098761] up:boot 2024-05-04T03:15:03.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:03 smithi077 ceph-mon[33531]: fsmap 2 up:standby 2024-05-04T03:15:03.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:03 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-04T03:15:04.504 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:04 smithi001 ceph-mon[29731]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:04.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.527+0000 7fbf4c47b700 1 -- 172.21.15.1:0/1775148424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf44105480 msgr2=0x7fbf44105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:04.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.527+0000 7fbf4c47b700 1 --2- 172.21.15.1:0/1775148424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf44105480 0x7fbf44105890 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbf40009990 tx=0x7fbf40009ca0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.527+0000 7fbf4c47b700 1 -- 172.21.15.1:0/1775148424 shutdown_connections 2024-05-04T03:15:04.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.527+0000 7fbf4c47b700 1 --2- 172.21.15.1:0/1775148424 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf4410e8f0 0x7fbf44110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.527+0000 7fbf4c47b700 1 --2- 172.21.15.1:0/1775148424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf44105480 0x7fbf44105890 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.527+0000 7fbf4c47b700 1 --2- 172.21.15.1:0/1775148424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf44105dd0 0x7fbf4410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.527+0000 7fbf4c47b700 1 -- 172.21.15.1:0/1775148424 >> 172.21.15.1:0/1775148424 conn(0x7fbf440fef30 msgr2=0x7fbf44101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:04.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.527+0000 7fbf4c47b700 1 -- 172.21.15.1:0/1775148424 shutdown_connections 2024-05-04T03:15:04.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.527+0000 7fbf4c47b700 1 -- 172.21.15.1:0/1775148424 wait complete. 2024-05-04T03:15:04.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.528+0000 7fbf4c47b700 1 Processor -- start 2024-05-04T03:15:04.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.528+0000 7fbf4c47b700 1 -- start start 2024-05-04T03:15:04.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.528+0000 7fbf4c47b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf44105480 0x7fbf44103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:04.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.528+0000 7fbf4c47b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf44105dd0 0x7fbf441036b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:04.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.528+0000 7fbf4c47b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf4410e8f0 0x7fbf44103bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:04.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.528+0000 7fbf4c47b700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbf44114440 con 0x7fbf44105480 2024-05-04T03:15:04.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.528+0000 7fbf4c47b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fbf441142c0 con 0x7fbf44105dd0 2024-05-04T03:15:04.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.528+0000 7fbf4c47b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbf441145c0 con 0x7fbf4410e8f0 2024-05-04T03:15:04.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.528+0000 7fbf4a217700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf44105480 0x7fbf44103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:04.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.528+0000 7fbf4a217700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf44105480 0x7fbf44103170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48160/0 (socket says 172.21.15.1:48160) 2024-05-04T03:15:04.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.528+0000 7fbf4a217700 1 -- 172.21.15.1:0/3278017020 learned_addr learned my addr 172.21.15.1:0/3278017020 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:15:04.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.528+0000 7fbf49a16700 1 --2- 172.21.15.1:0/3278017020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf44105dd0 0x7fbf441036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:04.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.529+0000 7fbf4aa18700 1 --2- 172.21.15.1:0/3278017020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf4410e8f0 0x7fbf44103bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:04.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.529+0000 7fbf4a217700 1 -- 172.21.15.1:0/3278017020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf4410e8f0 msgr2=0x7fbf44103bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:04.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.529+0000 7fbf4a217700 1 --2- 172.21.15.1:0/3278017020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf4410e8f0 0x7fbf44103bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.529+0000 7fbf4a217700 1 -- 172.21.15.1:0/3278017020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf44105dd0 msgr2=0x7fbf441036b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.529+0000 7fbf4a217700 1 --2- 172.21.15.1:0/3278017020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf44105dd0 0x7fbf441036b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.529+0000 7fbf4a217700 1 -- 172.21.15.1:0/3278017020 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf441a9130 con 0x7fbf44105480 2024-05-04T03:15:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.529+0000 7fbf49a16700 1 --2- 172.21.15.1:0/3278017020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf44105dd0 0x7fbf441036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:15:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.529+0000 7fbf4a217700 1 --2- 172.21.15.1:0/3278017020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf44105480 0x7fbf44103170 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbf400057c0 tx=0x7fbf40009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.529+0000 7fbf3b7fe700 1 -- 172.21.15.1:0/3278017020 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf40026070 con 0x7fbf44105480 2024-05-04T03:15:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.529+0000 7fbf4c47b700 1 -- 172.21.15.1:0/3278017020 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf441a93c0 con 0x7fbf44105480 2024-05-04T03:15:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.529+0000 7fbf3b7fe700 1 -- 172.21.15.1:0/3278017020 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf40021410 con 0x7fbf44105480 2024-05-04T03:15:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.529+0000 7fbf4c47b700 1 -- 172.21.15.1:0/3278017020 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf441a98f0 con 0x7fbf44105480 2024-05-04T03:15:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.529+0000 7fbf3b7fe700 1 -- 172.21.15.1:0/3278017020 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf40017650 con 0x7fbf44105480 2024-05-04T03:15:04.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.530+0000 7fbf4c47b700 1 -- 172.21.15.1:0/3278017020 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf28005330 con 0x7fbf44105480 2024-05-04T03:15:04.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.532+0000 7fbf3b7fe700 1 -- 172.21.15.1:0/3278017020 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fbf40029070 con 0x7fbf44105480 2024-05-04T03:15:04.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.532+0000 7fbf3b7fe700 1 --2- 172.21.15.1:0/3278017020 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fbf300b2880 0x7fbf300b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:04.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.533+0000 7fbf49a16700 1 --2- 172.21.15.1:0/3278017020 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fbf300b2880 0x7fbf300b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:04.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.533+0000 7fbf3b7fe700 1 -- 172.21.15.1:0/3278017020 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fbf400e2b60 con 0x7fbf44105480 2024-05-04T03:15:04.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.533+0000 7fbf49a16700 1 --2- 172.21.15.1:0/3278017020 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fbf300b2880 0x7fbf300b4d90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbf44102450 tx=0x7fbf3400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:04.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.536+0000 7fbf3b7fe700 1 -- 172.21.15.1:0/3278017020 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbf40013070 con 0x7fbf44105480 2024-05-04T03:15:04.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:04 smithi072 ceph-mon[33332]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:04.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.800+0000 7fbf4c47b700 1 -- 172.21.15.1:0/3278017020 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf28005620 con 0x7fbf44105480 2024-05-04T03:15:04.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.801+0000 7fbf3b7fe700 1 -- 172.21.15.1:0/3278017020 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2215 (secure 0 0 0) 0x7fbf400aae40 con 0x7fbf44105480 2024-05-04T03:15:04.803 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:15:04.803 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/3816098761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":3816098761},{"type":"v1","addr":"172.21.15.72:6833","nonce":3816098761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/379026128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":379026128},{"type":"v1","addr":"172.21.15.77:6833","nonce":379026128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-04T03:15:04.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.803+0000 7fbf4c47b700 1 -- 172.21.15.1:0/3278017020 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fbf300b2880 msgr2=0x7fbf300b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:04.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.803+0000 7fbf4c47b700 1 --2- 172.21.15.1:0/3278017020 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fbf300b2880 0x7fbf300b4d90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbf44102450 tx=0x7fbf3400a380 comp rx=0 tx=0).stop 2024-05-04T03:15:04.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.803+0000 7fbf4c47b700 1 -- 172.21.15.1:0/3278017020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf44105480 msgr2=0x7fbf44103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:04.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.803+0000 7fbf4c47b700 1 --2- 172.21.15.1:0/3278017020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf44105480 0x7fbf44103170 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbf400057c0 tx=0x7fbf40009580 comp rx=0 tx=0).stop 2024-05-04T03:15:04.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.803+0000 7fbf4c47b700 1 -- 172.21.15.1:0/3278017020 shutdown_connections 2024-05-04T03:15:04.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.803+0000 7fbf4c47b700 1 --2- 172.21.15.1:0/3278017020 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fbf300b2880 0x7fbf300b4d90 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.803+0000 7fbf4c47b700 1 --2- 172.21.15.1:0/3278017020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf4410e8f0 0x7fbf44103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.803+0000 7fbf4c47b700 1 --2- 172.21.15.1:0/3278017020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf44105480 0x7fbf44103170 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.803+0000 7fbf4c47b700 1 --2- 172.21.15.1:0/3278017020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf44105dd0 0x7fbf441036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:04.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.803+0000 7fbf4c47b700 1 -- 172.21.15.1:0/3278017020 >> 172.21.15.1:0/3278017020 conn(0x7fbf440fef30 msgr2=0x7fbf440ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:04.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.803+0000 7fbf4c47b700 1 -- 172.21.15.1:0/3278017020 shutdown_connections 2024-05-04T03:15:04.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:04.803+0000 7fbf4c47b700 1 -- 172.21.15.1:0/3278017020 wait complete. 2024-05-04T03:15:04.806 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 3 2024-05-04T03:15:04.927 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:04 smithi077 ceph-mon[33531]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:05.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:05 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/3278017020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:15:05.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:05 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3278017020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:15:05.879 INFO:journalctl@ceph.mds.j.smithi001.stdout:May 04 03:15:05 smithi001 systemd[1]: Starting Ceph mds.j for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:15:05.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:05 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/3278017020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:15:06.173 INFO:journalctl@ceph.mds.j.smithi001.stdout:May 04 03:15:05 smithi001 podman[67862]: 2024-05-04 03:15:05.805092038 +0000 UTC m=+0.018628525 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:15:06.173 INFO:journalctl@ceph.mds.j.smithi001.stdout:May 04 03:15:05 smithi001 podman[67862]: 2024-05-04 03:15:05.906638363 +0000 UTC m=+0.120174837 container create fe7d3006db2c9062f758434814b5cf789c2a4fc87af5902b4500facf362095e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-j, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:15:06.173 INFO:journalctl@ceph.mds.j.smithi001.stdout:May 04 03:15:06 smithi001 podman[67862]: 2024-05-04 03:15:06.097418912 +0000 UTC m=+0.310955383 container init fe7d3006db2c9062f758434814b5cf789c2a4fc87af5902b4500facf362095e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-j, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430) 2024-05-04T03:15:06.173 INFO:journalctl@ceph.mds.j.smithi001.stdout:May 04 03:15:06 smithi001 podman[67862]: 2024-05-04 03:15:06.10007367 +0000 UTC m=+0.313610142 container start fe7d3006db2c9062f758434814b5cf789c2a4fc87af5902b4500facf362095e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-04T03:15:06.173 INFO:journalctl@ceph.mds.j.smithi001.stdout:May 04 03:15:06 smithi001 ceph-mds[67881]: -- 172.21.15.1:0/82924610 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a25e381a40 con 0x55a25e3c9180 2024-05-04T03:15:06.173 INFO:journalctl@ceph.mds.j.smithi001.stdout:May 04 03:15:06 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-j[67877]: starting mds.j at 2024-05-04T03:15:06.173 INFO:journalctl@ceph.mds.j.smithi001.stdout:May 04 03:15:06 smithi001 bash[67862]: fe7d3006db2c9062f758434814b5cf789c2a4fc87af5902b4500facf362095e4 2024-05-04T03:15:06.452 INFO:journalctl@ceph.mds.j.smithi001.stdout:May 04 03:15:06 smithi001 systemd[1]: Started Ceph mds.j for 4a572aa8-09c3-11ef-bc97-c7b262605968. 2024-05-04T03:15:06.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:06 smithi072 ceph-mon[33332]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:06.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:06 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:06.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:06 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:06.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:06 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:06.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:06 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:06.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:06 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:06.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:06 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:06.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:06 smithi001 ceph-mon[29731]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:06.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:06 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:06.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:06 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:06.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:06 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:06.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:06 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:06.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:06 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:06.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:06 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:06.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:06 smithi077 ceph-mon[33531]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:06.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:06 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:06.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:06 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:06.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:06 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:06.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:06 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:06.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:06 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:06.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:06 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:07.612 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:07 smithi001 ceph-mon[29731]: Deploying daemon mds.d on smithi001 2024-05-04T03:15:07.612 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:07 smithi001 ceph-mon[29731]: mds.? [v2:172.21.15.1:6834/2186681910,v1:172.21.15.1:6835/2186681910] up:boot 2024-05-04T03:15:07.613 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:07 smithi001 ceph-mon[29731]: fsmap 3 up:standby 2024-05-04T03:15:07.613 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:07 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-04T03:15:07.631 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:15:07.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:07 smithi072 ceph-mon[33332]: Deploying daemon mds.d on smithi001 2024-05-04T03:15:07.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:07 smithi072 ceph-mon[33332]: mds.? [v2:172.21.15.1:6834/2186681910,v1:172.21.15.1:6835/2186681910] up:boot 2024-05-04T03:15:07.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:07 smithi072 ceph-mon[33332]: fsmap 3 up:standby 2024-05-04T03:15:07.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:07 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-04T03:15:07.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:07 smithi077 ceph-mon[33531]: Deploying daemon mds.d on smithi001 2024-05-04T03:15:07.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:07 smithi077 ceph-mon[33531]: mds.? [v2:172.21.15.1:6834/2186681910,v1:172.21.15.1:6835/2186681910] up:boot 2024-05-04T03:15:07.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:07 smithi077 ceph-mon[33531]: fsmap 3 up:standby 2024-05-04T03:15:07.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:07 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-04T03:15:08.226 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:15:08.594 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:08 smithi001 ceph-mon[29731]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:08.594 INFO:journalctl@ceph.mds.d.smithi001.stdout:May 04 03:15:08 smithi001 systemd[1]: Starting Ceph mds.d for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:15:08.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:08 smithi072 ceph-mon[33332]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:08.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:08 smithi077 ceph-mon[33531]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:09.840 INFO:journalctl@ceph.mds.d.smithi001.stdout:May 04 03:15:09 smithi001 podman[68253]: 2024-05-04 03:15:09.588723902 +0000 UTC m=+0.549323316 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:15:10.129 INFO:journalctl@ceph.mds.d.smithi001.stdout:May 04 03:15:09 smithi001 podman[68253]: 2024-05-04 03:15:09.932084284 +0000 UTC m=+0.892683692 container create efad207677c1b861078dbc90370ca477a7475957600b32031493eb4462f16d19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240430) 2024-05-04T03:15:10.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.266+0000 7fa7cc258700 1 -- 172.21.15.1:0/1607615322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7c41045f0 msgr2=0x7fa7c4104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:10.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.266+0000 7fa7cc258700 1 --2- 172.21.15.1:0/1607615322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7c41045f0 0x7fa7c4104a00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00099c0 tx=0x7fa7c0009cd0 comp rx=0 tx=0).stop 2024-05-04T03:15:10.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.266+0000 7fa7cc258700 1 -- 172.21.15.1:0/1607615322 shutdown_connections 2024-05-04T03:15:10.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.266+0000 7fa7cc258700 1 --2- 172.21.15.1:0/1607615322 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa7c4109870 0x7fa7c41111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:10.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.266+0000 7fa7cc258700 1 --2- 172.21.15.1:0/1607615322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7c41045f0 0x7fa7c4104a00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:10.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.266+0000 7fa7cc258700 1 --2- 172.21.15.1:0/1607615322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7c4108f20 0x7fa7c4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:10.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.266+0000 7fa7cc258700 1 -- 172.21.15.1:0/1607615322 >> 172.21.15.1:0/1607615322 conn(0x7fa7c4100040 msgr2=0x7fa7c4102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:10.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.266+0000 7fa7cc258700 1 -- 172.21.15.1:0/1607615322 shutdown_connections 2024-05-04T03:15:10.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.266+0000 7fa7cc258700 1 -- 172.21.15.1:0/1607615322 wait complete. 2024-05-04T03:15:10.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.267+0000 7fa7cc258700 1 Processor -- start 2024-05-04T03:15:10.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.267+0000 7fa7cc258700 1 -- start start 2024-05-04T03:15:10.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.267+0000 7fa7cc258700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7c41045f0 0x7fa7c419aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:10.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.267+0000 7fa7cc258700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa7c4108f20 0x7fa7c419b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:10.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.267+0000 7fa7cc258700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7c4109870 0x7fa7c419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:10.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.267+0000 7fa7cc258700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41144f0 con 0x7fa7c41045f0 2024-05-04T03:15:10.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.267+0000 7fa7cc258700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c4114370 con 0x7fa7c4108f20 2024-05-04T03:15:10.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.267+0000 7fa7cc258700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c4114670 con 0x7fa7c4109870 2024-05-04T03:15:10.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.267+0000 7fa7c9ff4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7c41045f0 0x7fa7c419aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:10.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.267+0000 7fa7c9ff4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7c41045f0 0x7fa7c419aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48198/0 (socket says 172.21.15.1:48198) 2024-05-04T03:15:10.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.267+0000 7fa7c9ff4700 1 -- 172.21.15.1:0/3414234363 learned_addr learned my addr 172.21.15.1:0/3414234363 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:15:10.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.267+0000 7fa7ca7f5700 1 --2- 172.21.15.1:0/3414234363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7c4109870 0x7fa7c419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:10.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.267+0000 7fa7c97f3700 1 --2- 172.21.15.1:0/3414234363 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa7c4108f20 0x7fa7c419b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:10.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.268+0000 7fa7c9ff4700 1 -- 172.21.15.1:0/3414234363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7c4109870 msgr2=0x7fa7c419f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:10.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.268+0000 7fa7c9ff4700 1 --2- 172.21.15.1:0/3414234363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7c4109870 0x7fa7c419f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:10.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.268+0000 7fa7c9ff4700 1 -- 172.21.15.1:0/3414234363 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa7c4108f20 msgr2=0x7fa7c419b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:10.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.268+0000 7fa7c9ff4700 1 --2- 172.21.15.1:0/3414234363 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa7c4108f20 0x7fa7c419b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:10.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.268+0000 7fa7c9ff4700 1 -- 172.21.15.1:0/3414234363 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c419fd90 con 0x7fa7c41045f0 2024-05-04T03:15:10.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.268+0000 7fa7c9ff4700 1 --2- 172.21.15.1:0/3414234363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7c41045f0 0x7fa7c419aec0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0005910 tx=0x7fa7c00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:10.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.268+0000 7fa7baffd700 1 -- 172.21.15.1:0/3414234363 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0026070 con 0x7fa7c41045f0 2024-05-04T03:15:10.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.268+0000 7fa7baffd700 1 -- 172.21.15.1:0/3414234363 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7c000f460 con 0x7fa7c41045f0 2024-05-04T03:15:10.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.268+0000 7fa7cc258700 1 -- 172.21.15.1:0/3414234363 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c41a0020 con 0x7fa7c41045f0 2024-05-04T03:15:10.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.268+0000 7fa7baffd700 1 -- 172.21.15.1:0/3414234363 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0009e60 con 0x7fa7c41045f0 2024-05-04T03:15:10.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.268+0000 7fa7cc258700 1 -- 172.21.15.1:0/3414234363 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c41a0500 con 0x7fa7c41045f0 2024-05-04T03:15:10.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.269+0000 7fa7cc258700 1 -- 172.21.15.1:0/3414234363 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7c40512c0 con 0x7fa7c41045f0 2024-05-04T03:15:10.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.272+0000 7fa7baffd700 1 -- 172.21.15.1:0/3414234363 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fa7c0029070 con 0x7fa7c41045f0 2024-05-04T03:15:10.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.272+0000 7fa7baffd700 1 --2- 172.21.15.1:0/3414234363 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa7b00b2880 0x7fa7b00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:10.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.272+0000 7fa7c97f3700 1 --2- 172.21.15.1:0/3414234363 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa7b00b2880 0x7fa7b00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:10.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.273+0000 7fa7baffd700 1 -- 172.21.15.1:0/3414234363 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fa7c00e2780 con 0x7fa7c41045f0 2024-05-04T03:15:10.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.273+0000 7fa7c97f3700 1 --2- 172.21.15.1:0/3414234363 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa7b00b2880 0x7fa7b00b4d90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4103650 tx=0x7fa7b400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:10.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.276+0000 7fa7baffd700 1 -- 172.21.15.1:0/3414234363 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa7c0013070 con 0x7fa7c41045f0 2024-05-04T03:15:10.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.547+0000 7fa7cc258700 1 -- 172.21.15.1:0/3414234363 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7c402e5f0 con 0x7fa7c41045f0 2024-05-04T03:15:10.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.548+0000 7fa7baffd700 1 -- 172.21.15.1:0/3414234363 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3859 (secure 0 0 0) 0x7fa7c00aa9d0 con 0x7fa7c41045f0 2024-05-04T03:15:10.550 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:15:10.550 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2186681910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2186681910},{"type":"v1","addr":"172.21.15.1:6835","nonce":2186681910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2747191047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2747191047},{"type":"v1","addr":"172.21.15.1:6837","nonce":2747191047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/3816098761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":3816098761},{"type":"v1","addr":"172.21.15.72:6833","nonce":3816098761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/379026128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":379026128},{"type":"v1","addr":"172.21.15.77:6833","nonce":379026128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-04T03:15:10.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.551+0000 7fa7cc258700 1 -- 172.21.15.1:0/3414234363 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa7b00b2880 msgr2=0x7fa7b00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:10.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.551+0000 7fa7cc258700 1 --2- 172.21.15.1:0/3414234363 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa7b00b2880 0x7fa7b00b4d90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4103650 tx=0x7fa7b400b410 comp rx=0 tx=0).stop 2024-05-04T03:15:10.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.551+0000 7fa7cc258700 1 -- 172.21.15.1:0/3414234363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7c41045f0 msgr2=0x7fa7c419aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:10.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.551+0000 7fa7cc258700 1 --2- 172.21.15.1:0/3414234363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7c41045f0 0x7fa7c419aec0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0005910 tx=0x7fa7c00045e0 comp rx=0 tx=0).stop 2024-05-04T03:15:10.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.551+0000 7fa7cc258700 1 -- 172.21.15.1:0/3414234363 shutdown_connections 2024-05-04T03:15:10.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.551+0000 7fa7cc258700 1 --2- 172.21.15.1:0/3414234363 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa7b00b2880 0x7fa7b00b4d90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:10.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.551+0000 7fa7cc258700 1 --2- 172.21.15.1:0/3414234363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7c4109870 0x7fa7c419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:10.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.551+0000 7fa7cc258700 1 --2- 172.21.15.1:0/3414234363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7c41045f0 0x7fa7c419aec0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:10.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.551+0000 7fa7cc258700 1 --2- 172.21.15.1:0/3414234363 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa7c4108f20 0x7fa7c419b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:10.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.551+0000 7fa7cc258700 1 -- 172.21.15.1:0/3414234363 >> 172.21.15.1:0/3414234363 conn(0x7fa7c4100040 msgr2=0x7fa7c4108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:10.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.551+0000 7fa7cc258700 1 -- 172.21.15.1:0/3414234363 shutdown_connections 2024-05-04T03:15:10.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:10.551+0000 7fa7cc258700 1 -- 172.21.15.1:0/3414234363 wait complete. 2024-05-04T03:15:10.553 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 5 2024-05-04T03:15:10.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:10 smithi072 ceph-mon[33332]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:10.840 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:10 smithi001 ceph-mon[29731]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:10.840 INFO:journalctl@ceph.mds.d.smithi001.stdout:May 04 03:15:10 smithi001 podman[68253]: 2024-05-04 03:15:10.402556236 +0000 UTC m=+1.363155644 container init efad207677c1b861078dbc90370ca477a7475957600b32031493eb4462f16d19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-d, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-04T03:15:10.841 INFO:journalctl@ceph.mds.d.smithi001.stdout:May 04 03:15:10 smithi001 podman[68253]: 2024-05-04 03:15:10.405457057 +0000 UTC m=+1.366056473 container start efad207677c1b861078dbc90370ca477a7475957600b32031493eb4462f16d19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-05-04T03:15:10.841 INFO:journalctl@ceph.mds.d.smithi001.stdout:May 04 03:15:10 smithi001 ceph-mds[68322]: -- 172.21.15.1:0/3865913517 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555eaffd7a40 con 0x555eb001fa80 2024-05-04T03:15:10.841 INFO:journalctl@ceph.mds.d.smithi001.stdout:May 04 03:15:10 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-d[68296]: starting mds.d at 2024-05-04T03:15:10.841 INFO:journalctl@ceph.mds.d.smithi001.stdout:May 04 03:15:10 smithi001 bash[68253]: efad207677c1b861078dbc90370ca477a7475957600b32031493eb4462f16d19 2024-05-04T03:15:10.841 INFO:journalctl@ceph.mds.d.smithi001.stdout:May 04 03:15:10 smithi001 systemd[1]: Started Ceph mds.d for 4a572aa8-09c3-11ef-bc97-c7b262605968. 2024-05-04T03:15:10.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:10 smithi077 ceph-mon[33531]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:11.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:11 smithi072 ceph-mon[33332]: mds.? [v2:172.21.15.1:6836/2747191047,v1:172.21.15.1:6837/2747191047] up:boot 2024-05-04T03:15:11.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:11 smithi072 ceph-mon[33332]: fsmap 4 up:standby 2024-05-04T03:15:11.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:11 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-04T03:15:11.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:11 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/3414234363' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:15:11.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:11 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:11.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:11 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:11.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:11 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:11.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:11 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:11.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:11 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:11.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:11 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:11.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:11 smithi072 ceph-mon[33332]: Deploying daemon mds.e on smithi072 2024-05-04T03:15:11.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:11 smithi001 ceph-mon[29731]: mds.? [v2:172.21.15.1:6836/2747191047,v1:172.21.15.1:6837/2747191047] up:boot 2024-05-04T03:15:11.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:11 smithi001 ceph-mon[29731]: fsmap 4 up:standby 2024-05-04T03:15:11.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:11 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-04T03:15:11.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:11 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3414234363' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:15:11.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:11 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:11.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:11 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:11.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:11 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:11.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:11 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:11.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:11 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:11.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:11 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:11.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:11 smithi001 ceph-mon[29731]: Deploying daemon mds.e on smithi072 2024-05-04T03:15:11.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:11 smithi077 ceph-mon[33531]: mds.? [v2:172.21.15.1:6836/2747191047,v1:172.21.15.1:6837/2747191047] up:boot 2024-05-04T03:15:11.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:11 smithi077 ceph-mon[33531]: fsmap 4 up:standby 2024-05-04T03:15:11.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:11 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-04T03:15:11.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:11 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/3414234363' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:15:11.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:11 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:11.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:11 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:11.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:11 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:11.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:11 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:11.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:11 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:11.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:11 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:11.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:11 smithi077 ceph-mon[33531]: Deploying daemon mds.e on smithi072 2024-05-04T03:15:12.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:12 smithi001 ceph-mon[29731]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:12.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:12 smithi077 ceph-mon[33531]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:12.957 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:12 smithi072 ceph-mon[33332]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:12.957 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 04 03:15:12 smithi072 systemd[1]: Starting Ceph mds.e for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:15:13.212 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 04 03:15:12 smithi072 podman[62686]: 2024-05-04 03:15:12.85839799 +0000 UTC m=+0.017781749 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:15:13.212 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 04 03:15:12 smithi072 podman[62686]: 2024-05-04 03:15:12.962352147 +0000 UTC m=+0.121735891 container create f59a9e96a9bc80a1aacb27b33e8390798d9bda5ee7fec1f985c9b825f720f870 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-e, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-04T03:15:13.213 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 04 03:15:13 smithi072 podman[62686]: 2024-05-04 03:15:13.175164849 +0000 UTC m=+0.334548603 container init f59a9e96a9bc80a1aacb27b33e8390798d9bda5ee7fec1f985c9b825f720f870 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-e, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-04T03:15:13.213 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 04 03:15:13 smithi072 podman[62686]: 2024-05-04 03:15:13.177871887 +0000 UTC m=+0.337255630 container start f59a9e96a9bc80a1aacb27b33e8390798d9bda5ee7fec1f985c9b825f720f870 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-e, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-04T03:15:13.213 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 04 03:15:13 smithi072 ceph-mds[62705]: -- 172.21.15.72:0/2579922990 <== mon.2 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5601bda25a40 con 0x5601bda6da80 2024-05-04T03:15:13.230 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:15:13.496 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 04 03:15:13 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-e[62701]: starting mds.e at 2024-05-04T03:15:13.496 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 04 03:15:13 smithi072 bash[62686]: f59a9e96a9bc80a1aacb27b33e8390798d9bda5ee7fec1f985c9b825f720f870 2024-05-04T03:15:13.496 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 04 03:15:13 smithi072 systemd[1]: Started Ceph mds.e for 4a572aa8-09c3-11ef-bc97-c7b262605968. 2024-05-04T03:15:13.513 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:15:14.398 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:14 smithi072 ceph-mon[33332]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:14.398 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:14 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:14.398 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:14 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:14.398 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:14 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:14.398 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:14 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:14.399 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:14 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:14.399 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:14 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:14.399 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:14 smithi072 ceph-mon[33332]: Deploying daemon mds.f on smithi077 2024-05-04T03:15:14.399 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:14 smithi072 ceph-mon[33332]: mds.? [v2:172.21.15.72:6834/3930984976,v1:172.21.15.72:6835/3930984976] up:boot 2024-05-04T03:15:14.399 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:14 smithi072 ceph-mon[33332]: fsmap 5 up:standby 2024-05-04T03:15:14.399 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:14 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-04T03:15:14.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:14 smithi001 ceph-mon[29731]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:14.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:14.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:14.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:14.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:14.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:14.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:14.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:14 smithi001 ceph-mon[29731]: Deploying daemon mds.f on smithi077 2024-05-04T03:15:14.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:14 smithi001 ceph-mon[29731]: mds.? [v2:172.21.15.72:6834/3930984976,v1:172.21.15.72:6835/3930984976] up:boot 2024-05-04T03:15:14.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:14 smithi001 ceph-mon[29731]: fsmap 5 up:standby 2024-05-04T03:15:14.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-04T03:15:14.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:14 smithi077 ceph-mon[33531]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:14.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:14 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:14.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:14 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:14.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:14 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:14.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:14 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:14.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:14 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:14.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:14 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:14.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:14 smithi077 ceph-mon[33531]: Deploying daemon mds.f on smithi077 2024-05-04T03:15:14.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:14 smithi077 ceph-mon[33531]: mds.? [v2:172.21.15.72:6834/3930984976,v1:172.21.15.72:6835/3930984976] up:boot 2024-05-04T03:15:14.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:14 smithi077 ceph-mon[33531]: fsmap 5 up:standby 2024-05-04T03:15:14.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:14 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-04T03:15:15.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.126+0000 7f596a297700 1 -- 172.21.15.1:0/1544947821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5964107df0 msgr2=0x7f5964108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:15.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.126+0000 7f596a297700 1 --2- 172.21.15.1:0/1544947821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5964107df0 0x7f5964108200 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f59540099c0 tx=0x7f5954009cd0 comp rx=0 tx=0).stop 2024-05-04T03:15:15.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.126+0000 7f596a297700 1 -- 172.21.15.1:0/1544947821 shutdown_connections 2024-05-04T03:15:15.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.126+0000 7f596a297700 1 --2- 172.21.15.1:0/1544947821 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5964108740 0x7f59641100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:15.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.126+0000 7f596a297700 1 --2- 172.21.15.1:0/1544947821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59641034c0 0x7f59641038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:15.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.126+0000 7f596a297700 1 --2- 172.21.15.1:0/1544947821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5964107df0 0x7f5964108200 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:15.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.126+0000 7f596a297700 1 -- 172.21.15.1:0/1544947821 >> 172.21.15.1:0/1544947821 conn(0x7f59640fedc0 msgr2=0x7f59641011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:15.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.126+0000 7f596a297700 1 -- 172.21.15.1:0/1544947821 shutdown_connections 2024-05-04T03:15:15.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.126+0000 7f596a297700 1 -- 172.21.15.1:0/1544947821 wait complete. 2024-05-04T03:15:15.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.127+0000 7f596a297700 1 Processor -- start 2024-05-04T03:15:15.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.127+0000 7f596a297700 1 -- start start 2024-05-04T03:15:15.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.128+0000 7f596a297700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59641034c0 0x7f5964199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:15.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.128+0000 7f596a297700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5964107df0 0x7f5964199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:15.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.128+0000 7f596a297700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5964108740 0x7f596419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:15.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.128+0000 7f596a297700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f59641133a0 con 0x7f59641034c0 2024-05-04T03:15:15.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.128+0000 7f596a297700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5964113220 con 0x7f5964108740 2024-05-04T03:15:15.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.128+0000 7f596a297700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5964113520 con 0x7f5964107df0 2024-05-04T03:15:15.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.128+0000 7f5963fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59641034c0 0x7f5964199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:15.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.128+0000 7f5963fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59641034c0 0x7f5964199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34602/0 (socket says 172.21.15.1:34602) 2024-05-04T03:15:15.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.128+0000 7f5963fff700 1 -- 172.21.15.1:0/1705968654 learned_addr learned my addr 172.21.15.1:0/1705968654 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:15:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.128+0000 7f5968834700 1 --2- 172.21.15.1:0/1705968654 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5964108740 0x7f596419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.128+0000 7f59637fe700 1 --2- 172.21.15.1:0/1705968654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5964107df0 0x7f5964199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.128+0000 7f5968834700 1 -- 172.21.15.1:0/1705968654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5964107df0 msgr2=0x7f5964199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.128+0000 7f5968834700 1 --2- 172.21.15.1:0/1705968654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5964107df0 0x7f5964199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.128+0000 7f5968834700 1 -- 172.21.15.1:0/1705968654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59641034c0 msgr2=0x7f5964199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.128+0000 7f5968834700 1 --2- 172.21.15.1:0/1705968654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59641034c0 0x7f5964199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.128+0000 7f5968834700 1 -- 172.21.15.1:0/1705968654 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596419edd0 con 0x7f5964108740 2024-05-04T03:15:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.128+0000 7f5968834700 1 --2- 172.21.15.1:0/1705968654 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5964108740 0x7f596419a550 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f595c00efc0 tx=0x7f595c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.129+0000 7f59617fa700 1 -- 172.21.15.1:0/1705968654 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c0092e0 con 0x7f5964108740 2024-05-04T03:15:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.129+0000 7f596a297700 1 -- 172.21.15.1:0/1705968654 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f596419f060 con 0x7f5964108740 2024-05-04T03:15:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.129+0000 7f59617fa700 1 -- 172.21.15.1:0/1705968654 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f595c004bb0 con 0x7f5964108740 2024-05-04T03:15:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.129+0000 7f59617fa700 1 -- 172.21.15.1:0/1705968654 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c004080 con 0x7f5964108740 2024-05-04T03:15:15.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.129+0000 7f596a297700 1 -- 172.21.15.1:0/1705968654 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f596419f570 con 0x7f5964108740 2024-05-04T03:15:15.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.130+0000 7f596a297700 1 -- 172.21.15.1:0/1705968654 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59640512c0 con 0x7f5964108740 2024-05-04T03:15:15.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.131+0000 7f59617fa700 1 -- 172.21.15.1:0/1705968654 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f595c01b030 con 0x7f5964108740 2024-05-04T03:15:15.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.132+0000 7f59617fa700 1 --2- 172.21.15.1:0/1705968654 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f59500b2880 0x7f59500b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:15.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.132+0000 7f5963fff700 1 --2- 172.21.15.1:0/1705968654 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f59500b2880 0x7f59500b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:15.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.132+0000 7f59617fa700 1 -- 172.21.15.1:0/1705968654 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f595c0e2210 con 0x7f5964108740 2024-05-04T03:15:15.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.132+0000 7f5963fff700 1 --2- 172.21.15.1:0/1705968654 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f59500b2880 0x7f59500b4d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f594c005950 tx=0x7f594c004080 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:15.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.135+0000 7f59617fa700 1 -- 172.21.15.1:0/1705968654 <== mon.2 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f595c022020 con 0x7f5964108740 2024-05-04T03:15:15.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.406+0000 7f596a297700 1 -- 172.21.15.1:0/1705968654 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f596410b1d0 con 0x7f5964108740 2024-05-04T03:15:15.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.407+0000 7f59617fa700 1 -- 172.21.15.1:0/1705968654 <== mon.2 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4684 (secure 0 0 0) 0x7f595c010040 con 0x7f5964108740 2024-05-04T03:15:15.409 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:15:15.409 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2186681910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2186681910},{"type":"v1","addr":"172.21.15.1:6835","nonce":2186681910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2747191047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2747191047},{"type":"v1","addr":"172.21.15.1:6837","nonce":2747191047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/3816098761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":3816098761},{"type":"v1","addr":"172.21.15.72:6833","nonce":3816098761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/379026128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":379026128},{"type":"v1","addr":"172.21.15.77:6833","nonce":379026128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3930984976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3930984976},{"type":"v1","addr":"172.21.15.72:6835","nonce":3930984976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-04T03:15:15.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.410+0000 7f596a297700 1 -- 172.21.15.1:0/1705968654 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f59500b2880 msgr2=0x7f59500b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:15.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.410+0000 7f596a297700 1 --2- 172.21.15.1:0/1705968654 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f59500b2880 0x7f59500b4d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f594c005950 tx=0x7f594c004080 comp rx=0 tx=0).stop 2024-05-04T03:15:15.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.410+0000 7f596a297700 1 -- 172.21.15.1:0/1705968654 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5964108740 msgr2=0x7f596419a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:15.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.410+0000 7f596a297700 1 --2- 172.21.15.1:0/1705968654 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5964108740 0x7f596419a550 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f595c00efc0 tx=0x7f595c00c3a0 comp rx=0 tx=0).stop 2024-05-04T03:15:15.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.410+0000 7f596a297700 1 -- 172.21.15.1:0/1705968654 shutdown_connections 2024-05-04T03:15:15.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.410+0000 7f596a297700 1 --2- 172.21.15.1:0/1705968654 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5964108740 0x7f596419a550 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:15.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.410+0000 7f596a297700 1 --2- 172.21.15.1:0/1705968654 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f59500b2880 0x7f59500b4d90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:15.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.410+0000 7f596a297700 1 --2- 172.21.15.1:0/1705968654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f59641034c0 0x7f5964199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:15.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.410+0000 7f596a297700 1 --2- 172.21.15.1:0/1705968654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5964107df0 0x7f5964199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:15.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.410+0000 7f596a297700 1 -- 172.21.15.1:0/1705968654 >> 172.21.15.1:0/1705968654 conn(0x7f59640fedc0 msgr2=0x7f59641096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:15.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.410+0000 7f596a297700 1 -- 172.21.15.1:0/1705968654 shutdown_connections 2024-05-04T03:15:15.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:15.410+0000 7f596a297700 1 -- 172.21.15.1:0/1705968654 wait complete. 2024-05-04T03:15:15.412 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 6 2024-05-04T03:15:15.678 INFO:journalctl@ceph.mds.f.smithi077.stdout:May 04 03:15:15 smithi077 systemd[1]: Starting Ceph mds.f for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:15:15.966 INFO:journalctl@ceph.mds.f.smithi077.stdout:May 04 03:15:15 smithi077 podman[63489]: 2024-05-04 03:15:15.592578741 +0000 UTC m=+0.018209769 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:15:15.966 INFO:journalctl@ceph.mds.f.smithi077.stdout:May 04 03:15:15 smithi077 podman[63489]: 2024-05-04 03:15:15.692696387 +0000 UTC m=+0.118327472 container create f7ab088fc14c4d6ec13e9419e16cc280ef396e76da45868154a0f7ded879d349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-04T03:15:15.966 INFO:journalctl@ceph.mds.f.smithi077.stdout:May 04 03:15:15 smithi077 podman[63489]: 2024-05-04 03:15:15.928687013 +0000 UTC m=+0.354318026 container init f7ab088fc14c4d6ec13e9419e16cc280ef396e76da45868154a0f7ded879d349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-f, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-05-04T03:15:15.966 INFO:journalctl@ceph.mds.f.smithi077.stdout:May 04 03:15:15 smithi077 podman[63489]: 2024-05-04 03:15:15.931276803 +0000 UTC m=+0.356907816 container start f7ab088fc14c4d6ec13e9419e16cc280ef396e76da45868154a0f7ded879d349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-04T03:15:15.966 INFO:journalctl@ceph.mds.f.smithi077.stdout:May 04 03:15:15 smithi077 ceph-mds[63508]: -- 172.21.15.77:0/2276456564 <== mon.1 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e4e0069a40 con 0x55e4e00b1600 2024-05-04T03:15:16.287 INFO:journalctl@ceph.mds.f.smithi077.stdout:May 04 03:15:15 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-f[63504]: starting mds.f at 2024-05-04T03:15:16.287 INFO:journalctl@ceph.mds.f.smithi077.stdout:May 04 03:15:16 smithi077 bash[63489]: f7ab088fc14c4d6ec13e9419e16cc280ef396e76da45868154a0f7ded879d349 2024-05-04T03:15:16.287 INFO:journalctl@ceph.mds.f.smithi077.stdout:May 04 03:15:16 smithi077 systemd[1]: Started Ceph mds.f for 4a572aa8-09c3-11ef-bc97-c7b262605968. 2024-05-04T03:15:16.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:16 smithi001 ceph-mon[29731]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:16.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:16 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/1705968654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:15:16.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:16 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:16.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:16 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:16.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:16 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:16.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:16 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:16.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:16 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:16.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:16 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:16.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:16 smithi077 ceph-mon[33531]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:16.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:16 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/1705968654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:15:16.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:16 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:16.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:16 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:16.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:16 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:16.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:16 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:16.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:16 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:16.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:16 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:16.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:16 smithi072 ceph-mon[33332]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:16.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:16 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/1705968654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:15:16.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:16 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:16.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:16 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:16.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:16 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:16.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:16 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:16.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:16 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:16.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:16 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:17.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:17 smithi001 ceph-mon[29731]: Deploying daemon mds.c on smithi077 2024-05-04T03:15:17.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:17 smithi001 ceph-mon[29731]: mds.? [v2:172.21.15.77:6834/4279951305,v1:172.21.15.77:6835/4279951305] up:boot 2024-05-04T03:15:17.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:17 smithi001 ceph-mon[29731]: fsmap 6 up:standby 2024-05-04T03:15:17.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:17 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-04T03:15:17.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:17 smithi077 ceph-mon[33531]: Deploying daemon mds.c on smithi077 2024-05-04T03:15:17.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:17 smithi077 ceph-mon[33531]: mds.? [v2:172.21.15.77:6834/4279951305,v1:172.21.15.77:6835/4279951305] up:boot 2024-05-04T03:15:17.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:17 smithi077 ceph-mon[33531]: fsmap 6 up:standby 2024-05-04T03:15:17.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:17 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-04T03:15:17.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:17 smithi072 ceph-mon[33332]: Deploying daemon mds.c on smithi077 2024-05-04T03:15:17.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:17 smithi072 ceph-mon[33332]: mds.? [v2:172.21.15.77:6834/4279951305,v1:172.21.15.77:6835/4279951305] up:boot 2024-05-04T03:15:17.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:17 smithi072 ceph-mon[33332]: fsmap 6 up:standby 2024-05-04T03:15:17.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:17 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-04T03:15:18.058 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:15:18.321 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:15:18.342 INFO:journalctl@ceph.mds.c.smithi077.stdout:May 04 03:15:18 smithi077 systemd[1]: Starting Ceph mds.c for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:15:18.596 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:18 smithi077 ceph-mon[33531]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:18.596 INFO:journalctl@ceph.mds.c.smithi077.stdout:May 04 03:15:18 smithi077 podman[63802]: 2024-05-04 03:15:18.37483107 +0000 UTC m=+0.017895522 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:15:18.596 INFO:journalctl@ceph.mds.c.smithi077.stdout:May 04 03:15:18 smithi077 podman[63802]: 2024-05-04 03:15:18.486897534 +0000 UTC m=+0.129961964 container create 8c5f0f199e56162e53c6da217a95e25f14e30c1a31b260d486cc23176c097f6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-04T03:15:18.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:18 smithi001 ceph-mon[29731]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:18.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:18 smithi072 ceph-mon[33332]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:19.178 INFO:journalctl@ceph.mds.c.smithi077.stdout:May 04 03:15:18 smithi077 podman[63802]: 2024-05-04 03:15:18.705974188 +0000 UTC m=+0.349038619 container init 8c5f0f199e56162e53c6da217a95e25f14e30c1a31b260d486cc23176c097f6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-c, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-04T03:15:19.178 INFO:journalctl@ceph.mds.c.smithi077.stdout:May 04 03:15:18 smithi077 podman[63802]: 2024-05-04 03:15:18.708786177 +0000 UTC m=+0.351850611 container start 8c5f0f199e56162e53c6da217a95e25f14e30c1a31b260d486cc23176c097f6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-04T03:15:19.178 INFO:journalctl@ceph.mds.c.smithi077.stdout:May 04 03:15:18 smithi077 ceph-mds[63821]: -- 172.21.15.77:0/2005792472 <== mon.1 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563dd2d51a40 con 0x563dd2d99600 2024-05-04T03:15:19.178 INFO:journalctl@ceph.mds.c.smithi077.stdout:May 04 03:15:18 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-c[63817]: starting mds.c at 2024-05-04T03:15:19.179 INFO:journalctl@ceph.mds.c.smithi077.stdout:May 04 03:15:18 smithi077 bash[63802]: 8c5f0f199e56162e53c6da217a95e25f14e30c1a31b260d486cc23176c097f6f 2024-05-04T03:15:19.179 INFO:journalctl@ceph.mds.c.smithi077.stdout:May 04 03:15:18 smithi077 systemd[1]: Started Ceph mds.c for 4a572aa8-09c3-11ef-bc97-c7b262605968. 2024-05-04T03:15:19.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.895+0000 7f920619a700 1 -- 172.21.15.1:0/3278322811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9200108740 msgr2=0x7f92001100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:19.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.895+0000 7f920619a700 1 --2- 172.21.15.1:0/3278322811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9200108740 0x7f92001100a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f91f80099c0 tx=0x7f91f8009cd0 comp rx=0 tx=0).stop 2024-05-04T03:15:19.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.895+0000 7f920619a700 1 -- 172.21.15.1:0/3278322811 shutdown_connections 2024-05-04T03:15:19.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.895+0000 7f920619a700 1 --2- 172.21.15.1:0/3278322811 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92001034c0 0x7f92001038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:19.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.895+0000 7f920619a700 1 --2- 172.21.15.1:0/3278322811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9200108740 0x7f92001100a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:19.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.895+0000 7f920619a700 1 --2- 172.21.15.1:0/3278322811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9200107df0 0x7f9200108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:19.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.895+0000 7f920619a700 1 -- 172.21.15.1:0/3278322811 >> 172.21.15.1:0/3278322811 conn(0x7f92000fedf0 msgr2=0x7f9200101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:19.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.895+0000 7f920619a700 1 -- 172.21.15.1:0/3278322811 shutdown_connections 2024-05-04T03:15:19.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.895+0000 7f920619a700 1 -- 172.21.15.1:0/3278322811 wait complete. 2024-05-04T03:15:19.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.896+0000 7f920619a700 1 Processor -- start 2024-05-04T03:15:19.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.896+0000 7f920619a700 1 -- start start 2024-05-04T03:15:19.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.896+0000 7f920619a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92001034c0 0x7f920019ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:19.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.896+0000 7f920619a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9200107df0 0x7f920019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:19.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.896+0000 7f920619a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9200108740 0x7f920019f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:19.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.896+0000 7f920619a700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9200113370 con 0x7f9200107df0 2024-05-04T03:15:19.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.896+0000 7f920619a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f92001131f0 con 0x7f92001034c0 2024-05-04T03:15:19.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.896+0000 7f920619a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f92001134f0 con 0x7f9200108740 2024-05-04T03:15:19.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.896+0000 7f91feffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9200107df0 0x7f920019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:19.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.897+0000 7f91feffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9200107df0 0x7f920019b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34616/0 (socket says 172.21.15.1:34616) 2024-05-04T03:15:19.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.897+0000 7f91feffd700 1 -- 172.21.15.1:0/1761236047 learned_addr learned my addr 172.21.15.1:0/1761236047 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:15:19.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.897+0000 7f91ff7fe700 1 --2- 172.21.15.1:0/1761236047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92001034c0 0x7f920019ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:19.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.897+0000 7f91fffff700 1 --2- 172.21.15.1:0/1761236047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9200108740 0x7f920019f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:19.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.897+0000 7f91ff7fe700 1 -- 172.21.15.1:0/1761236047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9200108740 msgr2=0x7f920019f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:19.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.897+0000 7f91ff7fe700 1 --2- 172.21.15.1:0/1761236047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9200108740 0x7f920019f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:19.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.897+0000 7f91ff7fe700 1 -- 172.21.15.1:0/1761236047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9200107df0 msgr2=0x7f920019b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:19.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.897+0000 7f91ff7fe700 1 --2- 172.21.15.1:0/1761236047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9200107df0 0x7f920019b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:19.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.897+0000 7f91ff7fe700 1 -- 172.21.15.1:0/1761236047 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920019fbb0 con 0x7f92001034c0 2024-05-04T03:15:19.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.897+0000 7f91feffd700 1 --2- 172.21.15.1:0/1761236047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9200107df0 0x7f920019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:15:19.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.897+0000 7f91ff7fe700 1 --2- 172.21.15.1:0/1761236047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92001034c0 0x7f920019ad90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f91f000ef10 tx=0x7f91f000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:19.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.897+0000 7f91fcff9700 1 -- 172.21.15.1:0/1761236047 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f000c9c0 con 0x7f92001034c0 2024-05-04T03:15:19.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.897+0000 7f91fcff9700 1 -- 172.21.15.1:0/1761236047 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91f0019070 con 0x7f92001034c0 2024-05-04T03:15:19.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.897+0000 7f920619a700 1 -- 172.21.15.1:0/1761236047 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f920019fe40 con 0x7f92001034c0 2024-05-04T03:15:19.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.897+0000 7f91fcff9700 1 -- 172.21.15.1:0/1761236047 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f0014480 con 0x7f92001034c0 2024-05-04T03:15:19.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.897+0000 7f920619a700 1 -- 172.21.15.1:0/1761236047 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92001a0380 con 0x7f92001034c0 2024-05-04T03:15:19.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.898+0000 7f920619a700 1 -- 172.21.15.1:0/1761236047 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92000512c0 con 0x7f92001034c0 2024-05-04T03:15:19.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.900+0000 7f91fcff9700 1 -- 172.21.15.1:0/1761236047 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f91f0012050 con 0x7f92001034c0 2024-05-04T03:15:19.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.900+0000 7f91fcff9700 1 --2- 172.21.15.1:0/1761236047 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f91ec0b2880 0x7f91ec0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:19.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.900+0000 7f91feffd700 1 --2- 172.21.15.1:0/1761236047 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f91ec0b2880 0x7f91ec0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:19.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.900+0000 7f91fcff9700 1 -- 172.21.15.1:0/1761236047 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f91f00e1da0 con 0x7f92001034c0 2024-05-04T03:15:19.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.901+0000 7f91feffd700 1 --2- 172.21.15.1:0/1761236047 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f91ec0b2880 0x7f91ec0b4d90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f92001027f0 tx=0x7f91e800a400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:19.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:19.904+0000 7f91fcff9700 1 -- 172.21.15.1:0/1761236047 <== mon.2 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f91f0010040 con 0x7f92001034c0 2024-05-04T03:15:20.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:19 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:20.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:19 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:20.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:19 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:20.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:19 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:20.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:19 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:20.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:19 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:20.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:19 smithi001 ceph-mon[29731]: Deploying daemon mds.h on smithi072 2024-05-04T03:15:20.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:19 smithi001 ceph-mon[29731]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:20.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:19 smithi001 ceph-mon[29731]: mds.? [v2:172.21.15.77:6836/2003521417,v1:172.21.15.77:6837/2003521417] up:boot 2024-05-04T03:15:20.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:19 smithi001 ceph-mon[29731]: fsmap 7 up:standby 2024-05-04T03:15:20.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:19 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-04T03:15:20.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:20.164+0000 7f920619a700 1 -- 172.21.15.1:0/1761236047 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92001a78d0 con 0x7f92001034c0 2024-05-04T03:15:20.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:20.165+0000 7f91fcff9700 1 -- 172.21.15.1:0/1761236047 <== mon.2 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6334 (secure 0 0 0) 0x7f91f00a9ec0 con 0x7f92001034c0 2024-05-04T03:15:20.167 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:15:20.167 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2186681910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2186681910},{"type":"v1","addr":"172.21.15.1:6835","nonce":2186681910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2747191047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2747191047},{"type":"v1","addr":"172.21.15.1:6837","nonce":2747191047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/3816098761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":3816098761},{"type":"v1","addr":"172.21.15.72:6833","nonce":3816098761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/379026128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":379026128},{"type":"v1","addr":"172.21.15.77:6833","nonce":379026128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3930984976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3930984976},{"type":"v1","addr":"172.21.15.72:6835","nonce":3930984976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/4279951305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4279951305},{"type":"v1","addr":"172.21.15.77:6835","nonce":4279951305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/2003521417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2003521417},{"type":"v1","addr":"172.21.15.77:6837","nonce":2003521417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-04T03:15:20.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:20.168+0000 7f920619a700 1 -- 172.21.15.1:0/1761236047 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f91ec0b2880 msgr2=0x7f91ec0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:20.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:20.168+0000 7f920619a700 1 --2- 172.21.15.1:0/1761236047 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f91ec0b2880 0x7f91ec0b4d90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f92001027f0 tx=0x7f91e800a400 comp rx=0 tx=0).stop 2024-05-04T03:15:20.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:20.168+0000 7f920619a700 1 -- 172.21.15.1:0/1761236047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92001034c0 msgr2=0x7f920019ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:20.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:20.168+0000 7f920619a700 1 --2- 172.21.15.1:0/1761236047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92001034c0 0x7f920019ad90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f91f000ef10 tx=0x7f91f000c3a0 comp rx=0 tx=0).stop 2024-05-04T03:15:20.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:20.168+0000 7f920619a700 1 -- 172.21.15.1:0/1761236047 shutdown_connections 2024-05-04T03:15:20.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:20.168+0000 7f920619a700 1 --2- 172.21.15.1:0/1761236047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92001034c0 0x7f920019ad90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:20.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:20.168+0000 7f920619a700 1 --2- 172.21.15.1:0/1761236047 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f91ec0b2880 0x7f91ec0b4d90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:20.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:20.168+0000 7f920619a700 1 --2- 172.21.15.1:0/1761236047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9200108740 0x7f920019f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:20.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:20.168+0000 7f920619a700 1 --2- 172.21.15.1:0/1761236047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9200107df0 0x7f920019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:20.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:20.168+0000 7f920619a700 1 -- 172.21.15.1:0/1761236047 >> 172.21.15.1:0/1761236047 conn(0x7f92000fedf0 msgr2=0x7f9200107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:20.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:20.168+0000 7f920619a700 1 -- 172.21.15.1:0/1761236047 shutdown_connections 2024-05-04T03:15:20.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:20.168+0000 7f920619a700 1 -- 172.21.15.1:0/1761236047 wait complete. 2024-05-04T03:15:20.171 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 8 2024-05-04T03:15:20.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:19 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:20.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:19 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:20.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:19 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:20.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:19 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:20.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:19 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:20.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:19 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:20.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:19 smithi077 ceph-mon[33531]: Deploying daemon mds.h on smithi072 2024-05-04T03:15:20.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:19 smithi077 ceph-mon[33531]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:20.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:19 smithi077 ceph-mon[33531]: mds.? [v2:172.21.15.77:6836/2003521417,v1:172.21.15.77:6837/2003521417] up:boot 2024-05-04T03:15:20.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:19 smithi077 ceph-mon[33531]: fsmap 7 up:standby 2024-05-04T03:15:20.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:19 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-04T03:15:20.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:19 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:20.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:19 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:20.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:19 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:20.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:19 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:20.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:19 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:20.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:19 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:20.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:19 smithi072 ceph-mon[33332]: Deploying daemon mds.h on smithi072 2024-05-04T03:15:20.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:19 smithi072 ceph-mon[33332]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:20.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:19 smithi072 ceph-mon[33332]: mds.? [v2:172.21.15.77:6836/2003521417,v1:172.21.15.77:6837/2003521417] up:boot 2024-05-04T03:15:20.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:19 smithi072 ceph-mon[33332]: fsmap 7 up:standby 2024-05-04T03:15:20.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:19 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-04T03:15:21.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:20 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/1761236047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:15:21.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:20 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/1761236047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:15:21.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:20 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/1761236047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:15:21.247 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 04 03:15:21 smithi072 systemd[1]: Starting Ceph mds.h for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:15:21.557 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 04 03:15:21 smithi072 podman[62996]: 2024-05-04 03:15:21.208557117 +0000 UTC m=+0.018221220 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:15:21.557 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 04 03:15:21 smithi072 podman[62996]: 2024-05-04 03:15:21.307727407 +0000 UTC m=+0.117391499 container create 0cc596835e9bdda9a0da128f7a87a740e10713c235308f608bd6e7cb776ef3a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-h, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-04T03:15:21.558 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 04 03:15:21 smithi072 podman[62996]: 2024-05-04 03:15:21.520717355 +0000 UTC m=+0.330381446 container init 0cc596835e9bdda9a0da128f7a87a740e10713c235308f608bd6e7cb776ef3a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-h, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-04T03:15:21.558 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 04 03:15:21 smithi072 podman[62996]: 2024-05-04 03:15:21.523536015 +0000 UTC m=+0.333200111 container start 0cc596835e9bdda9a0da128f7a87a740e10713c235308f608bd6e7cb776ef3a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-04T03:15:21.558 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 04 03:15:21 smithi072 ceph-mds[63016]: -- 172.21.15.72:0/284003610 <== mon.2 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56440dba1a40 con 0x56440dbe9600 2024-05-04T03:15:21.996 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 04 03:15:21 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-h[63012]: starting mds.h at 2024-05-04T03:15:21.996 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 04 03:15:21 smithi072 bash[62996]: 0cc596835e9bdda9a0da128f7a87a740e10713c235308f608bd6e7cb776ef3a9 2024-05-04T03:15:21.996 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 04 03:15:21 smithi072 systemd[1]: Started Ceph mds.h for 4a572aa8-09c3-11ef-bc97-c7b262605968. 2024-05-04T03:15:22.830 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:15:22.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:22 smithi077 ceph-mon[33531]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:22.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:22 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:22.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:22 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:22.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:22 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:22.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:22 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:22.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:22 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:22.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:22 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:22.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:22 smithi077 ceph-mon[33531]: mds.? [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] up:boot 2024-05-04T03:15:22.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:22 smithi077 ceph-mon[33531]: fsmap 8 up:standby 2024-05-04T03:15:22.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:22 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-04T03:15:22.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:22 smithi072 ceph-mon[33332]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:22.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:22 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:22.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:22 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:22.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:22 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:22.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:22 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:22.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:22 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:22.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:22 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:22.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:22 smithi072 ceph-mon[33332]: mds.? [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] up:boot 2024-05-04T03:15:22.998 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:22 smithi072 ceph-mon[33332]: fsmap 8 up:standby 2024-05-04T03:15:22.998 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:22 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-04T03:15:23.092 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:15:23.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:22 smithi001 ceph-mon[29731]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:23.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:22 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:23.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:22 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:23.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:22 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:23.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:22 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:23.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:22 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:23.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:22 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:23.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:22 smithi001 ceph-mon[29731]: mds.? [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] up:boot 2024-05-04T03:15:23.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:22 smithi001 ceph-mon[29731]: fsmap 8 up:standby 2024-05-04T03:15:23.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:22 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-04T03:15:23.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:23 smithi001 ceph-mon[29731]: Deploying daemon mds.i on smithi077 2024-05-04T03:15:23.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:23 smithi077 ceph-mon[33531]: Deploying daemon mds.i on smithi077 2024-05-04T03:15:23.928 INFO:journalctl@ceph.mds.i.smithi077.stdout:May 04 03:15:23 smithi077 systemd[1]: Starting Ceph mds.i for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:15:23.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:23 smithi072 ceph-mon[33332]: Deploying daemon mds.i on smithi077 2024-05-04T03:15:24.221 INFO:journalctl@ceph.mds.i.smithi077.stdout:May 04 03:15:23 smithi077 podman[64119]: 2024-05-04 03:15:23.861231705 +0000 UTC m=+0.018063462 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:15:24.221 INFO:journalctl@ceph.mds.i.smithi077.stdout:May 04 03:15:23 smithi077 podman[64119]: 2024-05-04 03:15:23.976062941 +0000 UTC m=+0.132894687 container create 33378ccac362362ec807e5664c01d9a9f09a3eeab97185ba74f81a6c806fc1cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2) 2024-05-04T03:15:24.221 INFO:journalctl@ceph.mds.i.smithi077.stdout:May 04 03:15:24 smithi077 podman[64119]: 2024-05-04 03:15:24.220202085 +0000 UTC m=+0.377033830 container init 33378ccac362362ec807e5664c01d9a9f09a3eeab97185ba74f81a6c806fc1cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-i, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-04T03:15:24.665 INFO:journalctl@ceph.mds.i.smithi077.stdout:May 04 03:15:24 smithi077 podman[64119]: 2024-05-04 03:15:24.223083215 +0000 UTC m=+0.379914970 container start 33378ccac362362ec807e5664c01d9a9f09a3eeab97185ba74f81a6c806fc1cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-04T03:15:24.665 INFO:journalctl@ceph.mds.i.smithi077.stdout:May 04 03:15:24 smithi077 ceph-mds[64138]: -- 172.21.15.77:0/1712305178 <== mon.1 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b47c019a40 con 0x55b47c061180 2024-05-04T03:15:24.665 INFO:journalctl@ceph.mds.i.smithi077.stdout:May 04 03:15:24 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-i[64134]: starting mds.i at 2024-05-04T03:15:24.666 INFO:journalctl@ceph.mds.i.smithi077.stdout:May 04 03:15:24 smithi077 bash[64119]: 33378ccac362362ec807e5664c01d9a9f09a3eeab97185ba74f81a6c806fc1cd 2024-05-04T03:15:24.666 INFO:journalctl@ceph.mds.i.smithi077.stdout:May 04 03:15:24 smithi077 systemd[1]: Started Ceph mds.i for 4a572aa8-09c3-11ef-bc97-c7b262605968. 2024-05-04T03:15:24.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.743+0000 7efd7b2a2700 1 -- 172.21.15.1:0/568981297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd741045d0 msgr2=0x7efd741049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:24.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.743+0000 7efd7b2a2700 1 --2- 172.21.15.1:0/568981297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd741045d0 0x7efd741049e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efd640099c0 tx=0x7efd64009cd0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.743+0000 7efd7b2a2700 1 -- 172.21.15.1:0/568981297 shutdown_connections 2024-05-04T03:15:24.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.743+0000 7efd7b2a2700 1 --2- 172.21.15.1:0/568981297 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd74109850 0x7efd741111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.743+0000 7efd7b2a2700 1 --2- 172.21.15.1:0/568981297 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd741045d0 0x7efd741049e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.743+0000 7efd7b2a2700 1 --2- 172.21.15.1:0/568981297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd74108f00 0x7efd74109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.743+0000 7efd7b2a2700 1 -- 172.21.15.1:0/568981297 >> 172.21.15.1:0/568981297 conn(0x7efd740ffea0 msgr2=0x7efd741022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:24.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.743+0000 7efd7b2a2700 1 -- 172.21.15.1:0/568981297 shutdown_connections 2024-05-04T03:15:24.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.743+0000 7efd7b2a2700 1 -- 172.21.15.1:0/568981297 wait complete. 2024-05-04T03:15:24.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.744+0000 7efd7b2a2700 1 Processor -- start 2024-05-04T03:15:24.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.744+0000 7efd7b2a2700 1 -- start start 2024-05-04T03:15:24.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.744+0000 7efd7b2a2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd741045d0 0x7efd7419ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:24.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.744+0000 7efd7b2a2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd74108f00 0x7efd7419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:24.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.744+0000 7efd7b2a2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd74109850 0x7efd7419f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:24.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.744+0000 7efd7b2a2700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efd74114480 con 0x7efd741045d0 2024-05-04T03:15:24.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.744+0000 7efd7b2a2700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7efd74114300 con 0x7efd74109850 2024-05-04T03:15:24.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.744+0000 7efd7b2a2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efd74114600 con 0x7efd74108f00 2024-05-04T03:15:24.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.744+0000 7efd7903e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd741045d0 0x7efd7419ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:24.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.745+0000 7efd7903e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd741045d0 0x7efd7419ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39544/0 (socket says 172.21.15.1:39544) 2024-05-04T03:15:24.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.745+0000 7efd7903e700 1 -- 172.21.15.1:0/3718200975 learned_addr learned my addr 172.21.15.1:0/3718200975 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:15:24.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.745+0000 7efd7883d700 1 --2- 172.21.15.1:0/3718200975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd74108f00 0x7efd7419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:24.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.745+0000 7efd7983f700 1 --2- 172.21.15.1:0/3718200975 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd74109850 0x7efd7419f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:24.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.745+0000 7efd7883d700 1 -- 172.21.15.1:0/3718200975 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd74109850 msgr2=0x7efd7419f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:24.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.745+0000 7efd7883d700 1 --2- 172.21.15.1:0/3718200975 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd74109850 0x7efd7419f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.745+0000 7efd7883d700 1 -- 172.21.15.1:0/3718200975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd741045d0 msgr2=0x7efd7419ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:24.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.745+0000 7efd7883d700 1 --2- 172.21.15.1:0/3718200975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd741045d0 0x7efd7419ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:24.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.745+0000 7efd7883d700 1 -- 172.21.15.1:0/3718200975 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7419fc60 con 0x7efd74108f00 2024-05-04T03:15:24.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.745+0000 7efd7983f700 1 --2- 172.21.15.1:0/3718200975 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd74109850 0x7efd7419f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:15:24.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.745+0000 7efd7903e700 1 --2- 172.21.15.1:0/3718200975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd741045d0 0x7efd7419ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:15:24.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.745+0000 7efd7883d700 1 --2- 172.21.15.1:0/3718200975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd74108f00 0x7efd7419b330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efd7000e9c0 tx=0x7efd7000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:24.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.745+0000 7efd6a7fc700 1 -- 172.21.15.1:0/3718200975 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7000cbd0 con 0x7efd74108f00 2024-05-04T03:15:24.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.745+0000 7efd6a7fc700 1 -- 172.21.15.1:0/3718200975 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd7000cd70 con 0x7efd74108f00 2024-05-04T03:15:24.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.745+0000 7efd7b2a2700 1 -- 172.21.15.1:0/3718200975 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd7419ff50 con 0x7efd74108f00 2024-05-04T03:15:24.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.745+0000 7efd6a7fc700 1 -- 172.21.15.1:0/3718200975 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70018680 con 0x7efd74108f00 2024-05-04T03:15:24.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.745+0000 7efd7b2a2700 1 -- 172.21.15.1:0/3718200975 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd741a7870 con 0x7efd74108f00 2024-05-04T03:15:24.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.746+0000 7efd7b2a2700 1 -- 172.21.15.1:0/3718200975 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd740512c0 con 0x7efd74108f00 2024-05-04T03:15:24.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.748+0000 7efd6a7fc700 1 -- 172.21.15.1:0/3718200975 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7efd7004aab0 con 0x7efd74108f00 2024-05-04T03:15:24.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.749+0000 7efd6a7fc700 1 --2- 172.21.15.1:0/3718200975 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7efd600b2880 0x7efd600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:24.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.749+0000 7efd7903e700 1 --2- 172.21.15.1:0/3718200975 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7efd600b2880 0x7efd600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:24.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.749+0000 7efd6a7fc700 1 -- 172.21.15.1:0/3718200975 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7efd700e1fa0 con 0x7efd74108f00 2024-05-04T03:15:24.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.749+0000 7efd7903e700 1 --2- 172.21.15.1:0/3718200975 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7efd600b2880 0x7efd600b4d90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efd64009990 tx=0x7efd6401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:24.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:24.752+0000 7efd6a7fc700 1 -- 172.21.15.1:0/3718200975 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efd700aa0c0 con 0x7efd74108f00 2024-05-04T03:15:24.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:24 smithi001 ceph-mon[29731]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:24.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:24 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:24.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:24 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:24.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:24 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:24.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:24 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:24.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:24 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:24.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:24 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:24.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:24 smithi001 ceph-mon[29731]: Deploying daemon mds.g on smithi001 2024-05-04T03:15:24.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:24 smithi077 ceph-mon[33531]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:24.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:24 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:24.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:24 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:24.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:24 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:24.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:24 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:24.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:24 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:24.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:24 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:24.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:24 smithi077 ceph-mon[33531]: Deploying daemon mds.g on smithi001 2024-05-04T03:15:24.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:24 smithi072 ceph-mon[33332]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:24.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:24 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:24.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:24 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:24.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:24 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:24.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:24 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:24.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:24 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:24.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:24 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:24.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:24 smithi072 ceph-mon[33332]: Deploying daemon mds.g on smithi001 2024-05-04T03:15:25.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:25.041+0000 7efd7b2a2700 1 -- 172.21.15.1:0/3718200975 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd7402e5f0 con 0x7efd74108f00 2024-05-04T03:15:25.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:25.042+0000 7efd6a7fc700 1 -- 172.21.15.1:0/3718200975 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7983 (secure 0 0 0) 0x7efd700b2140 con 0x7efd74108f00 2024-05-04T03:15:25.044 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:15:25.044 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2186681910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2186681910},{"type":"v1","addr":"172.21.15.1:6835","nonce":2186681910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2747191047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2747191047},{"type":"v1","addr":"172.21.15.1:6837","nonce":2747191047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/3816098761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":3816098761},{"type":"v1","addr":"172.21.15.72:6833","nonce":3816098761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/379026128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":379026128},{"type":"v1","addr":"172.21.15.77:6833","nonce":379026128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3930984976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3930984976},{"type":"v1","addr":"172.21.15.72:6835","nonce":3930984976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/4279951305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4279951305},{"type":"v1","addr":"172.21.15.77:6835","nonce":4279951305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/2003521417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2003521417},{"type":"v1","addr":"172.21.15.77:6837","nonce":2003521417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1417614203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1417614203},{"type":"v1","addr":"172.21.15.72:6837","nonce":1417614203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6839/922723189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":922723189},{"type":"v1","addr":"172.21.15.77:6839","nonce":922723189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-04T03:15:25.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:25.045+0000 7efd7b2a2700 1 -- 172.21.15.1:0/3718200975 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7efd600b2880 msgr2=0x7efd600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:25.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:25.045+0000 7efd7b2a2700 1 --2- 172.21.15.1:0/3718200975 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7efd600b2880 0x7efd600b4d90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efd64009990 tx=0x7efd6401a040 comp rx=0 tx=0).stop 2024-05-04T03:15:25.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:25.045+0000 7efd7b2a2700 1 -- 172.21.15.1:0/3718200975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd74108f00 msgr2=0x7efd7419b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:25.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:25.045+0000 7efd7b2a2700 1 --2- 172.21.15.1:0/3718200975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd74108f00 0x7efd7419b330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efd7000e9c0 tx=0x7efd7000ecd0 comp rx=0 tx=0).stop 2024-05-04T03:15:25.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:25.045+0000 7efd7b2a2700 1 -- 172.21.15.1:0/3718200975 shutdown_connections 2024-05-04T03:15:25.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:25.045+0000 7efd7b2a2700 1 --2- 172.21.15.1:0/3718200975 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efd74109850 0x7efd7419f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:25.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:25.045+0000 7efd7b2a2700 1 --2- 172.21.15.1:0/3718200975 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7efd600b2880 0x7efd600b4d90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:25.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:25.045+0000 7efd7b2a2700 1 --2- 172.21.15.1:0/3718200975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd741045d0 0x7efd7419ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:25.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:25.045+0000 7efd7b2a2700 1 --2- 172.21.15.1:0/3718200975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd74108f00 0x7efd7419b330 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:25.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:25.045+0000 7efd7b2a2700 1 -- 172.21.15.1:0/3718200975 >> 172.21.15.1:0/3718200975 conn(0x7efd740ffea0 msgr2=0x7efd7410a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:25.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:25.045+0000 7efd7b2a2700 1 -- 172.21.15.1:0/3718200975 shutdown_connections 2024-05-04T03:15:25.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:25.045+0000 7efd7b2a2700 1 -- 172.21.15.1:0/3718200975 wait complete. 2024-05-04T03:15:25.048 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 10 2024-05-04T03:15:25.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:25 smithi077 ceph-mon[33531]: mds.? [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] up:boot 2024-05-04T03:15:25.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:25 smithi077 ceph-mon[33531]: fsmap 9 up:standby 2024-05-04T03:15:25.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:25 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-04T03:15:25.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:25 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/3718200975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:15:25.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:25 smithi077 ceph-mon[33531]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:25.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:25 smithi072 ceph-mon[33332]: mds.? [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] up:boot 2024-05-04T03:15:25.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:25 smithi072 ceph-mon[33332]: fsmap 9 up:standby 2024-05-04T03:15:25.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:25 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-04T03:15:25.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:25 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/3718200975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:15:25.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:25 smithi072 ceph-mon[33332]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:26.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:25 smithi001 ceph-mon[29731]: mds.? [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] up:boot 2024-05-04T03:15:26.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:25 smithi001 ceph-mon[29731]: fsmap 9 up:standby 2024-05-04T03:15:26.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:25 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-04T03:15:26.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:25 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3718200975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:15:26.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:25 smithi001 ceph-mon[29731]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:27.629 INFO:journalctl@ceph.mds.g.smithi001.stdout:May 04 03:15:27 smithi001 systemd[1]: Started Ceph mds.g for 4a572aa8-09c3-11ef-bc97-c7b262605968. 2024-05-04T03:15:28.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:28 smithi001 ceph-mon[29731]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:28.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:28 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:28.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:28 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:28.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:28 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:28.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:28 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:28.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:28 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:28.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:28 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:28.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:28 smithi001 ceph-mon[29731]: Deploying daemon mds.a on smithi001 2024-05-04T03:15:28.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:28 smithi001 ceph-mon[29731]: mds.? [v2:172.21.15.1:6838/4115120540,v1:172.21.15.1:6839/4115120540] up:boot 2024-05-04T03:15:28.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:28 smithi001 ceph-mon[29731]: fsmap 10 up:standby 2024-05-04T03:15:28.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:28 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-04T03:15:28.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:28 smithi077 ceph-mon[33531]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:28.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:28 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:28.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:28 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:28.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:28 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:28.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:28 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:28.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:28 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:28.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:28 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:28.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:28 smithi077 ceph-mon[33531]: Deploying daemon mds.a on smithi001 2024-05-04T03:15:28.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:28 smithi077 ceph-mon[33531]: mds.? [v2:172.21.15.1:6838/4115120540,v1:172.21.15.1:6839/4115120540] up:boot 2024-05-04T03:15:28.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:28 smithi077 ceph-mon[33531]: fsmap 10 up:standby 2024-05-04T03:15:28.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:28 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-04T03:15:28.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:28 smithi072 ceph-mon[33332]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:28.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:28 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:28.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:28 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:28.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:28 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:28.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:28 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:28.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:28 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:28.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:28 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:28.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:28 smithi072 ceph-mon[33332]: Deploying daemon mds.a on smithi001 2024-05-04T03:15:28.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:28 smithi072 ceph-mon[33332]: mds.? [v2:172.21.15.1:6838/4115120540,v1:172.21.15.1:6839/4115120540] up:boot 2024-05-04T03:15:28.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:28 smithi072 ceph-mon[33332]: fsmap 10 up:standby 2024-05-04T03:15:28.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:28 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-04T03:15:28.811 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:15:29.119 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:15:30.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:30 smithi001 ceph-mon[29731]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:30.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:30 smithi077 ceph-mon[33531]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:30.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:30 smithi072 ceph-mon[33332]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:30.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.888+0000 7fbe90582700 1 -- 172.21.15.1:0/2095620100 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe88107e10 msgr2=0x7fbe88108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:30.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.888+0000 7fbe90582700 1 --2- 172.21.15.1:0/2095620100 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe88107e10 0x7fbe88108220 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbe780099c0 tx=0x7fbe78009cd0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.888+0000 7fbe90582700 1 -- 172.21.15.1:0/2095620100 shutdown_connections 2024-05-04T03:15:30.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.888+0000 7fbe90582700 1 --2- 172.21.15.1:0/2095620100 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbe881034e0 0x7fbe881038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.888+0000 7fbe90582700 1 --2- 172.21.15.1:0/2095620100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe88108760 0x7fbe881100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.888+0000 7fbe90582700 1 --2- 172.21.15.1:0/2095620100 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe88107e10 0x7fbe88108220 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.888+0000 7fbe90582700 1 -- 172.21.15.1:0/2095620100 >> 172.21.15.1:0/2095620100 conn(0x7fbe880fef30 msgr2=0x7fbe88101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:30.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.888+0000 7fbe90582700 1 -- 172.21.15.1:0/2095620100 shutdown_connections 2024-05-04T03:15:30.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.888+0000 7fbe90582700 1 -- 172.21.15.1:0/2095620100 wait complete. 2024-05-04T03:15:30.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.889+0000 7fbe90582700 1 Processor -- start 2024-05-04T03:15:30.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.889+0000 7fbe90582700 1 -- start start 2024-05-04T03:15:30.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.890+0000 7fbe90582700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbe881034e0 0x7fbe88110ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:30.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.890+0000 7fbe90582700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe88107e10 0x7fbe88111000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:30.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.890+0000 7fbe90582700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe88108760 0x7fbe88111540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:30.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.890+0000 7fbe90582700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbe88114530 con 0x7fbe88107e10 2024-05-04T03:15:30.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.890+0000 7fbe90582700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fbe881143b0 con 0x7fbe881034e0 2024-05-04T03:15:30.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.890+0000 7fbe90582700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbe881146b0 con 0x7fbe88108760 2024-05-04T03:15:30.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.890+0000 7fbe8db1d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe88107e10 0x7fbe88111000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:30.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.890+0000 7fbe8db1d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe88107e10 0x7fbe88111000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39586/0 (socket says 172.21.15.1:39586) 2024-05-04T03:15:30.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.890+0000 7fbe8db1d700 1 -- 172.21.15.1:0/1271464485 learned_addr learned my addr 172.21.15.1:0/1271464485 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:15:30.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.890+0000 7fbe8e31e700 1 --2- 172.21.15.1:0/1271464485 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbe881034e0 0x7fbe88110ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:30.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.890+0000 7fbe8eb1f700 1 --2- 172.21.15.1:0/1271464485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe88108760 0x7fbe88111540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:30.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.890+0000 7fbe8db1d700 1 -- 172.21.15.1:0/1271464485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe88108760 msgr2=0x7fbe88111540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:30.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.890+0000 7fbe8db1d700 1 --2- 172.21.15.1:0/1271464485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe88108760 0x7fbe88111540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.890+0000 7fbe8db1d700 1 -- 172.21.15.1:0/1271464485 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbe881034e0 msgr2=0x7fbe88110ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:30.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.890+0000 7fbe8db1d700 1 --2- 172.21.15.1:0/1271464485 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbe881034e0 0x7fbe88110ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:30.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.890+0000 7fbe8db1d700 1 -- 172.21.15.1:0/1271464485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe881a9100 con 0x7fbe88107e10 2024-05-04T03:15:30.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.890+0000 7fbe8db1d700 1 --2- 172.21.15.1:0/1271464485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe88107e10 0x7fbe88111000 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbe78009f50 tx=0x7fbe780036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:30.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.890+0000 7fbe7f7fe700 1 -- 172.21.15.1:0/1271464485 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe78026070 con 0x7fbe88107e10 2024-05-04T03:15:30.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.890+0000 7fbe90582700 1 -- 172.21.15.1:0/1271464485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe881a9390 con 0x7fbe88107e10 2024-05-04T03:15:30.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.890+0000 7fbe7f7fe700 1 -- 172.21.15.1:0/1271464485 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe78004020 con 0x7fbe88107e10 2024-05-04T03:15:30.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.890+0000 7fbe7f7fe700 1 -- 172.21.15.1:0/1271464485 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7800fe30 con 0x7fbe88107e10 2024-05-04T03:15:30.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.891+0000 7fbe90582700 1 -- 172.21.15.1:0/1271464485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe881a97e0 con 0x7fbe88107e10 2024-05-04T03:15:30.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.892+0000 7fbe90582700 1 -- 172.21.15.1:0/1271464485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe880512c0 con 0x7fbe88107e10 2024-05-04T03:15:30.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.893+0000 7fbe7f7fe700 1 -- 172.21.15.1:0/1271464485 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fbe78029070 con 0x7fbe88107e10 2024-05-04T03:15:30.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.894+0000 7fbe7f7fe700 1 --2- 172.21.15.1:0/1271464485 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fbe740b2880 0x7fbe740b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:30.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.894+0000 7fbe8e31e700 1 --2- 172.21.15.1:0/1271464485 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fbe740b2880 0x7fbe740b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:30.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.894+0000 7fbe7f7fe700 1 -- 172.21.15.1:0/1271464485 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fbe780e2890 con 0x7fbe88107e10 2024-05-04T03:15:30.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.895+0000 7fbe8e31e700 1 --2- 172.21.15.1:0/1271464485 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fbe740b2880 0x7fbe740b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbe84005950 tx=0x7fbe84005ed0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:30.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:30.899+0000 7fbe7f7fe700 1 -- 172.21.15.1:0/1271464485 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbe78013070 con 0x7fbe88107e10 2024-05-04T03:15:31.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:31.166+0000 7fbe90582700 1 -- 172.21.15.1:0/1271464485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe8802e690 con 0x7fbe88107e10 2024-05-04T03:15:31.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:31.167+0000 7fbe7f7fe700 1 -- 172.21.15.1:0/1271464485 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8806 (secure 0 0 0) 0x7fbe780aaae0 con 0x7fbe88107e10 2024-05-04T03:15:31.169 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:15:31.169 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2186681910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2186681910},{"type":"v1","addr":"172.21.15.1:6835","nonce":2186681910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2747191047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2747191047},{"type":"v1","addr":"172.21.15.1:6837","nonce":2747191047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4115120540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4115120540},{"type":"v1","addr":"172.21.15.1:6839","nonce":4115120540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/3816098761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":3816098761},{"type":"v1","addr":"172.21.15.72:6833","nonce":3816098761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/379026128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":379026128},{"type":"v1","addr":"172.21.15.77:6833","nonce":379026128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3930984976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3930984976},{"type":"v1","addr":"172.21.15.72:6835","nonce":3930984976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/4279951305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4279951305},{"type":"v1","addr":"172.21.15.77:6835","nonce":4279951305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/2003521417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2003521417},{"type":"v1","addr":"172.21.15.77:6837","nonce":2003521417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1417614203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1417614203},{"type":"v1","addr":"172.21.15.72:6837","nonce":1417614203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6839/922723189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":922723189},{"type":"v1","addr":"172.21.15.77:6839","nonce":922723189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-04T03:15:31.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:31.169+0000 7fbe90582700 1 -- 172.21.15.1:0/1271464485 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fbe740b2880 msgr2=0x7fbe740b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:31.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:31.169+0000 7fbe90582700 1 --2- 172.21.15.1:0/1271464485 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fbe740b2880 0x7fbe740b4d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbe84005950 tx=0x7fbe84005ed0 comp rx=0 tx=0).stop 2024-05-04T03:15:31.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:31.169+0000 7fbe90582700 1 -- 172.21.15.1:0/1271464485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe88107e10 msgr2=0x7fbe88111000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:31.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:31.169+0000 7fbe90582700 1 --2- 172.21.15.1:0/1271464485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe88107e10 0x7fbe88111000 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbe78009f50 tx=0x7fbe780036f0 comp rx=0 tx=0).stop 2024-05-04T03:15:31.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:31.169+0000 7fbe90582700 1 -- 172.21.15.1:0/1271464485 shutdown_connections 2024-05-04T03:15:31.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:31.169+0000 7fbe90582700 1 --2- 172.21.15.1:0/1271464485 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbe881034e0 0x7fbe88110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:31.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:31.169+0000 7fbe90582700 1 --2- 172.21.15.1:0/1271464485 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fbe740b2880 0x7fbe740b4d90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:31.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:31.169+0000 7fbe90582700 1 --2- 172.21.15.1:0/1271464485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe88108760 0x7fbe88111540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:31.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:31.169+0000 7fbe90582700 1 --2- 172.21.15.1:0/1271464485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe88107e10 0x7fbe88111000 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:31.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:31.169+0000 7fbe90582700 1 -- 172.21.15.1:0/1271464485 >> 172.21.15.1:0/1271464485 conn(0x7fbe880fef30 msgr2=0x7fbe88101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:31.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:31.170+0000 7fbe90582700 1 -- 172.21.15.1:0/1271464485 shutdown_connections 2024-05-04T03:15:31.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:31.170+0000 7fbe90582700 1 -- 172.21.15.1:0/1271464485 wait complete. 2024-05-04T03:15:31.172 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 11 2024-05-04T03:15:31.546 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:31 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/1271464485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:15:31.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:31 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/1271464485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:15:31.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:31 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/1271464485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:15:31.879 INFO:journalctl@ceph.mds.a.smithi001.stdout:May 04 03:15:31 smithi001 systemd[1]: Started Ceph mds.a for 4a572aa8-09c3-11ef-bc97-c7b262605968. 2024-05-04T03:15:32.580 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:32 smithi072 ceph-mon[33332]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:32.580 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:32 smithi072 ceph-mon[33332]: mds.? [v2:172.21.15.1:6840/3405709780,v1:172.21.15.1:6841/3405709780] up:boot 2024-05-04T03:15:32.580 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:32 smithi072 ceph-mon[33332]: fsmap 11 up:standby 2024-05-04T03:15:32.580 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:32 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-04T03:15:32.580 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:32 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:32.580 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:32 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:32.580 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:32 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:32.580 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:32 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:32.580 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:32 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:32.581 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:32 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:32.581 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:32 smithi072 ceph-mon[33332]: Deploying daemon mds.b on smithi072 2024-05-04T03:15:32.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:32 smithi077 ceph-mon[33531]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:32.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:32 smithi077 ceph-mon[33531]: mds.? [v2:172.21.15.1:6840/3405709780,v1:172.21.15.1:6841/3405709780] up:boot 2024-05-04T03:15:32.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:32 smithi077 ceph-mon[33531]: fsmap 11 up:standby 2024-05-04T03:15:32.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:32 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-04T03:15:32.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:32 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:32.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:32 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:32.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:32 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:32.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:32 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:32.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:32 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:32.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:32 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:32.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:32 smithi077 ceph-mon[33531]: Deploying daemon mds.b on smithi072 2024-05-04T03:15:32.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:32 smithi001 ceph-mon[29731]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:32.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:32 smithi001 ceph-mon[29731]: mds.? [v2:172.21.15.1:6840/3405709780,v1:172.21.15.1:6841/3405709780] up:boot 2024-05-04T03:15:32.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:32 smithi001 ceph-mon[29731]: fsmap 11 up:standby 2024-05-04T03:15:32.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:32 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-04T03:15:32.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:32 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:32.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:32 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:32.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:32 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:32.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:32 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-04T03:15:32.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:32 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-04T03:15:32.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:32 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:32.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:32 smithi001 ceph-mon[29731]: Deploying daemon mds.b on smithi072 2024-05-04T03:15:33.996 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 04 03:15:33 smithi072 systemd[1]: Starting Ceph mds.b for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:15:34.071 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:15:34.336 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:15:34.371 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 04 03:15:34 smithi072 podman[63303]: 2024-05-04 03:15:34.119745734 +0000 UTC m=+0.106846675 container create 1e23e5da397c3c694217040d2f1c98a9cfd1822899f0085f6eb81d58ac79050a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:15:34.371 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 04 03:15:34 smithi072 podman[63303]: 2024-05-04 03:15:34.031241104 +0000 UTC m=+0.018342039 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:15:34.371 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 04 03:15:34 smithi072 podman[63303]: 2024-05-04 03:15:34.332389885 +0000 UTC m=+0.319490819 container init 1e23e5da397c3c694217040d2f1c98a9cfd1822899f0085f6eb81d58ac79050a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-04T03:15:34.371 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 04 03:15:34 smithi072 podman[63303]: 2024-05-04 03:15:34.334984381 +0000 UTC m=+0.322085324 container start 1e23e5da397c3c694217040d2f1c98a9cfd1822899f0085f6eb81d58ac79050a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:15:34.371 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 04 03:15:34 smithi072 ceph-mds[63322]: -- 172.21.15.72:0/3465376480 <== mon.2 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ef05825a40 con 0x55ef0586d180 2024-05-04T03:15:34.372 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 04 03:15:34 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-b[63318]: starting mds.b at 2024-05-04T03:15:34.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:34 smithi077 ceph-mon[33531]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:34.703 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:34 smithi001 ceph-mon[29731]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:34.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:34 smithi072 ceph-mon[33332]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:34.746 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 04 03:15:34 smithi072 bash[63303]: 1e23e5da397c3c694217040d2f1c98a9cfd1822899f0085f6eb81d58ac79050a 2024-05-04T03:15:34.746 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 04 03:15:34 smithi072 systemd[1]: Started Ceph mds.b for 4a572aa8-09c3-11ef-bc97-c7b262605968. 2024-05-04T03:15:35.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:35 smithi077 ceph-mon[33531]: mds.? [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] up:boot 2024-05-04T03:15:35.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:35 smithi077 ceph-mon[33531]: fsmap 12 up:standby 2024-05-04T03:15:35.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:35 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-04T03:15:35.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:35 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:35.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:35 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:35.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:35 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:35.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:35 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:35.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:35 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:35.677 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:35 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:15:35.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:35 smithi072 ceph-mon[33332]: mds.? [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] up:boot 2024-05-04T03:15:35.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:35 smithi072 ceph-mon[33332]: fsmap 12 up:standby 2024-05-04T03:15:35.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:35 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-04T03:15:35.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:35 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:35.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:35 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:35.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:35 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:35.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:35 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:35.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:35 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:35.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:35 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:15:35.764 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:35 smithi001 ceph-mon[29731]: mds.? [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] up:boot 2024-05-04T03:15:35.764 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:35 smithi001 ceph-mon[29731]: fsmap 12 up:standby 2024-05-04T03:15:35.764 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:35 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-04T03:15:35.764 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:35 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:35.764 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:35 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:35.764 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:35 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:35.764 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:35 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:35.764 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:35 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:35.764 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:35 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:15:36.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.988+0000 7fa5cc4a7700 1 -- 172.21.15.1:0/4091095537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5c4107df0 msgr2=0x7fa5c4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:36.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.988+0000 7fa5cc4a7700 1 --2- 172.21.15.1:0/4091095537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5c4107df0 0x7fa5c4108200 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40099c0 tx=0x7fa5b4009cd0 comp rx=0 tx=0).stop 2024-05-04T03:15:36.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.989+0000 7fa5cc4a7700 1 -- 172.21.15.1:0/4091095537 shutdown_connections 2024-05-04T03:15:36.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.989+0000 7fa5cc4a7700 1 --2- 172.21.15.1:0/4091095537 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5c4108740 0x7fa5c41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:36.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.989+0000 7fa5cc4a7700 1 --2- 172.21.15.1:0/4091095537 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5c41034c0 0x7fa5c41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:36.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.989+0000 7fa5cc4a7700 1 --2- 172.21.15.1:0/4091095537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5c4107df0 0x7fa5c4108200 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:36.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.989+0000 7fa5cc4a7700 1 -- 172.21.15.1:0/4091095537 >> 172.21.15.1:0/4091095537 conn(0x7fa5c40fedc0 msgr2=0x7fa5c41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:36.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.989+0000 7fa5cc4a7700 1 -- 172.21.15.1:0/4091095537 shutdown_connections 2024-05-04T03:15:36.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.989+0000 7fa5cc4a7700 1 -- 172.21.15.1:0/4091095537 wait complete. 2024-05-04T03:15:36.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.990+0000 7fa5cc4a7700 1 Processor -- start 2024-05-04T03:15:36.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.990+0000 7fa5cc4a7700 1 -- start start 2024-05-04T03:15:36.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.990+0000 7fa5cc4a7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5c41034c0 0x7fa5c419f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:36.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.990+0000 7fa5cc4a7700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5c4107df0 0x7fa5c419fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:36.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.990+0000 7fa5cc4a7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5c4108740 0x7fa5c4199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:36.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.990+0000 7fa5cc4a7700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c4113350 con 0x7fa5c4107df0 2024-05-04T03:15:36.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.990+0000 7fa5cc4a7700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c41131d0 con 0x7fa5c41034c0 2024-05-04T03:15:36.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.990+0000 7fa5cc4a7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c41134d0 con 0x7fa5c4108740 2024-05-04T03:15:36.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.990+0000 7fa5c9a42700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5c4107df0 0x7fa5c419fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:36.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.990+0000 7fa5c9a42700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5c4107df0 0x7fa5c419fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42158/0 (socket says 172.21.15.1:42158) 2024-05-04T03:15:36.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.990+0000 7fa5c9a42700 1 -- 172.21.15.1:0/67576262 learned_addr learned my addr 172.21.15.1:0/67576262 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:15:36.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.990+0000 7fa5ca243700 1 --2- 172.21.15.1:0/67576262 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5c41034c0 0x7fa5c419f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:36.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.990+0000 7fa5caa44700 1 --2- 172.21.15.1:0/67576262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5c4108740 0x7fa5c4199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:36.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.990+0000 7fa5c9a42700 1 -- 172.21.15.1:0/67576262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5c4108740 msgr2=0x7fa5c4199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:36.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.990+0000 7fa5c9a42700 1 --2- 172.21.15.1:0/67576262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5c4108740 0x7fa5c4199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:36.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.990+0000 7fa5c9a42700 1 -- 172.21.15.1:0/67576262 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5c41034c0 msgr2=0x7fa5c419f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:36.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.990+0000 7fa5c9a42700 1 --2- 172.21.15.1:0/67576262 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5c41034c0 0x7fa5c419f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:36.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.990+0000 7fa5c9a42700 1 -- 172.21.15.1:0/67576262 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c419a260 con 0x7fa5c4107df0 2024-05-04T03:15:36.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.991+0000 7fa5c9a42700 1 --2- 172.21.15.1:0/67576262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5c4107df0 0x7fa5c419fee0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4005f50 tx=0x7fa5b4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:36.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.991+0000 7fa5bb7fe700 1 -- 172.21.15.1:0/67576262 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4026070 con 0x7fa5c4107df0 2024-05-04T03:15:36.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.991+0000 7fa5bb7fe700 1 -- 172.21.15.1:0/67576262 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5b4003c70 con 0x7fa5c4107df0 2024-05-04T03:15:36.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.991+0000 7fa5cc4a7700 1 -- 172.21.15.1:0/67576262 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c419a4f0 con 0x7fa5c4107df0 2024-05-04T03:15:36.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.991+0000 7fa5bb7fe700 1 -- 172.21.15.1:0/67576262 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b400f9c0 con 0x7fa5c4107df0 2024-05-04T03:15:36.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.991+0000 7fa5cc4a7700 1 -- 172.21.15.1:0/67576262 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c41a6750 con 0x7fa5c4107df0 2024-05-04T03:15:36.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.992+0000 7fa5cc4a7700 1 -- 172.21.15.1:0/67576262 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c4104970 con 0x7fa5c4107df0 2024-05-04T03:15:36.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.994+0000 7fa5bb7fe700 1 -- 172.21.15.1:0/67576262 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fa5b4029070 con 0x7fa5c4107df0 2024-05-04T03:15:36.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.994+0000 7fa5bb7fe700 1 --2- 172.21.15.1:0/67576262 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa5b00b2900 0x7fa5b00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:36.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.994+0000 7fa5ca243700 1 --2- 172.21.15.1:0/67576262 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa5b00b2900 0x7fa5b00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:36.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.994+0000 7fa5bb7fe700 1 -- 172.21.15.1:0/67576262 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7fa5b40e2920 con 0x7fa5c4107df0 2024-05-04T03:15:36.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.995+0000 7fa5ca243700 1 --2- 172.21.15.1:0/67576262 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa5b00b2900 0x7fa5b00b4e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0009c60 tx=0x7fa5c0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:36.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:35.998+0000 7fa5bb7fe700 1 -- 172.21.15.1:0/67576262 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa5b4030080 con 0x7fa5c4107df0 2024-05-04T03:15:36.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:36.280+0000 7fa5cc4a7700 1 -- 172.21.15.1:0/67576262 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5c419afd0 con 0x7fa5c4107df0 2024-05-04T03:15:36.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:36.281+0000 7fa5bb7fe700 1 -- 172.21.15.1:0/67576262 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10455 (secure 0 0 0) 0x7fa5b4013070 con 0x7fa5c4107df0 2024-05-04T03:15:36.286 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:15:36.286 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2186681910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2186681910},{"type":"v1","addr":"172.21.15.1:6835","nonce":2186681910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2747191047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2747191047},{"type":"v1","addr":"172.21.15.1:6837","nonce":2747191047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4115120540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4115120540},{"type":"v1","addr":"172.21.15.1:6839","nonce":4115120540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3405709780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3405709780},{"type":"v1","addr":"172.21.15.1:6841","nonce":3405709780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/3816098761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":3816098761},{"type":"v1","addr":"172.21.15.72:6833","nonce":3816098761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/379026128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":379026128},{"type":"v1","addr":"172.21.15.77:6833","nonce":379026128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/3930984976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3930984976},{"type":"v1","addr":"172.21.15.72:6835","nonce":3930984976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/4279951305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4279951305},{"type":"v1","addr":"172.21.15.77:6835","nonce":4279951305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/2003521417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2003521417},{"type":"v1","addr":"172.21.15.77:6837","nonce":2003521417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/1417614203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1417614203},{"type":"v1","addr":"172.21.15.72:6837","nonce":1417614203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6839/922723189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":922723189},{"type":"v1","addr":"172.21.15.77:6839","nonce":922723189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24494,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6839/2976050650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2976050650},{"type":"v1","addr":"172.21.15.72:6839","nonce":2976050650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-04T03:15:36.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:36.283+0000 7fa5cc4a7700 1 -- 172.21.15.1:0/67576262 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa5b00b2900 msgr2=0x7fa5b00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:36.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:36.283+0000 7fa5cc4a7700 1 --2- 172.21.15.1:0/67576262 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa5b00b2900 0x7fa5b00b4e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0009c60 tx=0x7fa5c0009260 comp rx=0 tx=0).stop 2024-05-04T03:15:36.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:36.283+0000 7fa5cc4a7700 1 -- 172.21.15.1:0/67576262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5c4107df0 msgr2=0x7fa5c419fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:36.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:36.283+0000 7fa5cc4a7700 1 --2- 172.21.15.1:0/67576262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5c4107df0 0x7fa5c419fee0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4005f50 tx=0x7fa5b4004270 comp rx=0 tx=0).stop 2024-05-04T03:15:36.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:36.283+0000 7fa5cc4a7700 1 -- 172.21.15.1:0/67576262 shutdown_connections 2024-05-04T03:15:36.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:36.283+0000 7fa5cc4a7700 1 --2- 172.21.15.1:0/67576262 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5c41034c0 0x7fa5c419f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:36.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:36.283+0000 7fa5cc4a7700 1 --2- 172.21.15.1:0/67576262 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa5b00b2900 0x7fa5b00b4e10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:36.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:36.283+0000 7fa5cc4a7700 1 --2- 172.21.15.1:0/67576262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5c4108740 0x7fa5c4199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:36.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:36.283+0000 7fa5cc4a7700 1 --2- 172.21.15.1:0/67576262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5c4107df0 0x7fa5c419fee0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:36.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:36.283+0000 7fa5cc4a7700 1 -- 172.21.15.1:0/67576262 >> 172.21.15.1:0/67576262 conn(0x7fa5c40fedc0 msgr2=0x7fa5c41096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:36.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:36.283+0000 7fa5cc4a7700 1 -- 172.21.15.1:0/67576262 shutdown_connections 2024-05-04T03:15:36.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:36.283+0000 7fa5cc4a7700 1 -- 172.21.15.1:0/67576262 wait complete. 2024-05-04T03:15:36.287 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 13 2024-05-04T03:15:36.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:36 smithi001 ceph-mon[29731]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:36.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:36 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/67576262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:15:36.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:36.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:36.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:36.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:36 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:36.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:36 smithi077 ceph-mon[33531]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:36.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:36 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/67576262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:15:36.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:36 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:36.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:36 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:36.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:36 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:36.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:36 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:36.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:36 smithi072 ceph-mon[33332]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:36.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:36 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/67576262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:15:36.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:36 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:36.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:36 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:36.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:36 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:36.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:36 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:37.423 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-04T03:15:37.698 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:15:38.815 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:38 smithi001 ceph-mon[29731]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:38.815 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:38 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:38.815 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:38 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:38.815 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:38 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:38.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:38 smithi077 ceph-mon[33531]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:38.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:38 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:38.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:38 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:38.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:38 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:38.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:38 smithi072 ceph-mon[33332]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:38.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:38 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:38.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:38 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:38.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:38 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:39.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.486+0000 7f0ce6f5b700 1 -- 172.21.15.1:0/2029156756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ce0108200 msgr2=0x7f0ce0108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:39.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.486+0000 7f0ce6f5b700 1 --2- 172.21.15.1:0/2029156756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ce0108200 0x7f0ce0108610 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009990 tx=0x7f0cd4009ca0 comp rx=0 tx=0).stop 2024-05-04T03:15:39.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.486+0000 7f0ce6f5b700 1 -- 172.21.15.1:0/2029156756 shutdown_connections 2024-05-04T03:15:39.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.486+0000 7f0ce6f5b700 1 --2- 172.21.15.1:0/2029156756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ce0104240 0x7f0ce0110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:39.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.486+0000 7f0ce6f5b700 1 --2- 172.21.15.1:0/2029156756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ce0108200 0x7f0ce0108610 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:39.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.486+0000 7f0ce6f5b700 1 --2- 172.21.15.1:0/2029156756 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0ce01038f0 0x7f0ce0103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:39.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.486+0000 7f0ce6f5b700 1 -- 172.21.15.1:0/2029156756 >> 172.21.15.1:0/2029156756 conn(0x7f0ce00fe6f0 msgr2=0x7f0ce0100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:39.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.486+0000 7f0ce6f5b700 1 -- 172.21.15.1:0/2029156756 shutdown_connections 2024-05-04T03:15:39.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.486+0000 7f0ce6f5b700 1 -- 172.21.15.1:0/2029156756 wait complete. 2024-05-04T03:15:39.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.487+0000 7f0ce6f5b700 1 Processor -- start 2024-05-04T03:15:39.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.487+0000 7f0ce6f5b700 1 -- start start 2024-05-04T03:15:39.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.487+0000 7f0ce6f5b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0ce01038f0 0x7f0ce0199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:39.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.487+0000 7f0ce6f5b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ce0104240 0x7f0ce019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:39.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.487+0000 7f0ce6f5b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ce0108200 0x7f0ce019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:39.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.487+0000 7f0ce6f5b700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce0113460 con 0x7f0ce0108200 2024-05-04T03:15:39.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.487+0000 7f0ce6f5b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce01132e0 con 0x7f0ce01038f0 2024-05-04T03:15:39.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.487+0000 7f0ce6f5b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce01135e0 con 0x7f0ce0104240 2024-05-04T03:15:39.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.487+0000 7f0ce54f8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ce0108200 0x7f0ce019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:39.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.487+0000 7f0ce54f8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ce0108200 0x7f0ce019e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42172/0 (socket says 172.21.15.1:42172) 2024-05-04T03:15:39.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.487+0000 7f0ce54f8700 1 -- 172.21.15.1:0/145016547 learned_addr learned my addr 172.21.15.1:0/145016547 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:15:39.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.487+0000 7f0cdffff700 1 --2- 172.21.15.1:0/145016547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ce0104240 0x7f0ce019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:39.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.488+0000 7f0ce4cf7700 1 --2- 172.21.15.1:0/145016547 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0ce01038f0 0x7f0ce0199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:39.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.488+0000 7f0ce54f8700 1 -- 172.21.15.1:0/145016547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ce0104240 msgr2=0x7f0ce019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:39.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.488+0000 7f0ce54f8700 1 --2- 172.21.15.1:0/145016547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ce0104240 0x7f0ce019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:39.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.488+0000 7f0ce54f8700 1 -- 172.21.15.1:0/145016547 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0ce01038f0 msgr2=0x7f0ce0199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:39.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.488+0000 7f0ce54f8700 1 --2- 172.21.15.1:0/145016547 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0ce01038f0 0x7f0ce0199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:39.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.488+0000 7f0ce54f8700 1 -- 172.21.15.1:0/145016547 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce019ed50 con 0x7f0ce0108200 2024-05-04T03:15:39.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.488+0000 7f0ce4cf7700 1 --2- 172.21.15.1:0/145016547 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0ce01038f0 0x7f0ce0199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:15:39.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.488+0000 7f0ce54f8700 1 --2- 172.21.15.1:0/145016547 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ce0108200 0x7f0ce019e6b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0cd800b970 tx=0x7f0cd800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:39.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.488+0000 7f0cddffb700 1 -- 172.21.15.1:0/145016547 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8019070 con 0x7f0ce0108200 2024-05-04T03:15:39.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.488+0000 7f0cddffb700 1 -- 172.21.15.1:0/145016547 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cd800cde0 con 0x7f0ce0108200 2024-05-04T03:15:39.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.488+0000 7f0cddffb700 1 -- 172.21.15.1:0/145016547 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd8007370 con 0x7f0ce0108200 2024-05-04T03:15:39.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.488+0000 7f0ce6f5b700 1 -- 172.21.15.1:0/145016547 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce019f040 con 0x7f0ce0108200 2024-05-04T03:15:39.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.488+0000 7f0ce6f5b700 1 -- 172.21.15.1:0/145016547 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce01a6960 con 0x7f0ce0108200 2024-05-04T03:15:39.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.489+0000 7f0ce6f5b700 1 -- 172.21.15.1:0/145016547 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ce00512c0 con 0x7f0ce0108200 2024-05-04T03:15:39.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.491+0000 7f0cddffb700 1 -- 172.21.15.1:0/145016547 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f0cd801c070 con 0x7f0ce0108200 2024-05-04T03:15:39.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.492+0000 7f0cddffb700 1 --2- 172.21.15.1:0/145016547 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0cd00b2880 0x7f0cd00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:39.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.492+0000 7f0ce4cf7700 1 --2- 172.21.15.1:0/145016547 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0cd00b2880 0x7f0cd00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:39.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.492+0000 7f0cddffb700 1 -- 172.21.15.1:0/145016547 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f0cd80e1a70 con 0x7f0ce0108200 2024-05-04T03:15:39.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.492+0000 7f0ce4cf7700 1 --2- 172.21.15.1:0/145016547 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0cd00b2880 0x7f0cd00b4d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4006010 tx=0x7f0cd401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:39.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.495+0000 7f0cddffb700 1 -- 172.21.15.1:0/145016547 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0cd8023070 con 0x7f0ce0108200 2024-05-04T03:15:39.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.729+0000 7f0ce6f5b700 1 -- 172.21.15.1:0/145016547 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f0ce010b7a0 con 0x7f0ce0108200 2024-05-04T03:15:39.731 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:39 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:39.732 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:39 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:15:39.732 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:39 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:39.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.730+0000 7f0cddffb700 1 -- 172.21.15.1:0/145016547 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v74) ==== 84+0+65 (secure 0 0 0) 0x7f0cd80a9b90 con 0x7f0ce0108200 2024-05-04T03:15:39.733 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:15:39.733 INFO:teuthology.orchestra.run.smithi001.stdout:[ 2024-05-04T03:15:39.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:15:39.733 INFO:teuthology.orchestra.run.smithi001.stdout: "poolnum": 1, 2024-05-04T03:15:39.733 INFO:teuthology.orchestra.run.smithi001.stdout: "poolname": ".mgr" 2024-05-04T03:15:39.734 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:15:39.734 INFO:teuthology.orchestra.run.smithi001.stdout:] 2024-05-04T03:15:39.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.732+0000 7f0ce6f5b700 1 -- 172.21.15.1:0/145016547 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0cd00b2880 msgr2=0x7f0cd00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:39.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.732+0000 7f0ce6f5b700 1 --2- 172.21.15.1:0/145016547 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0cd00b2880 0x7f0cd00b4d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4006010 tx=0x7f0cd401a040 comp rx=0 tx=0).stop 2024-05-04T03:15:39.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.732+0000 7f0ce6f5b700 1 -- 172.21.15.1:0/145016547 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ce0108200 msgr2=0x7f0ce019e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:39.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.732+0000 7f0ce6f5b700 1 --2- 172.21.15.1:0/145016547 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ce0108200 0x7f0ce019e6b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0cd800b970 tx=0x7f0cd800bc80 comp rx=0 tx=0).stop 2024-05-04T03:15:39.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.732+0000 7f0ce6f5b700 1 -- 172.21.15.1:0/145016547 shutdown_connections 2024-05-04T03:15:39.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.732+0000 7f0ce6f5b700 1 --2- 172.21.15.1:0/145016547 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0ce01038f0 0x7f0ce0199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:39.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.732+0000 7f0ce6f5b700 1 --2- 172.21.15.1:0/145016547 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0cd00b2880 0x7f0cd00b4d90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:39.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.732+0000 7f0ce6f5b700 1 --2- 172.21.15.1:0/145016547 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ce0108200 0x7f0ce019e6b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:39.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.732+0000 7f0ce6f5b700 1 --2- 172.21.15.1:0/145016547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ce0104240 0x7f0ce019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:39.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.732+0000 7f0ce6f5b700 1 -- 172.21.15.1:0/145016547 >> 172.21.15.1:0/145016547 conn(0x7f0ce00fe6f0 msgr2=0x7f0ce0109690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:39.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.732+0000 7f0ce6f5b700 1 -- 172.21.15.1:0/145016547 shutdown_connections 2024-05-04T03:15:39.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:39.732+0000 7f0ce6f5b700 1 -- 172.21.15.1:0/145016547 wait complete. 2024-05-04T03:15:39.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:39 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:39.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:39 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:15:39.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:39 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:39.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:39 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:39.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:39 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:15:39.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:39 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:40.368 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-04T03:15:40.368 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-04T03:15:40.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:40 smithi072 ceph-mon[33332]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:40.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:40 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:15:40.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:40 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/145016547' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-04T03:15:40.867 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:40 smithi001 ceph-mon[29731]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:40.867 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:40 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:15:40.867 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:40 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/145016547' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-04T03:15:40.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:40 smithi077 ceph-mon[33531]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:40.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:40 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:15:40.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:40 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/145016547' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-04T03:15:41.275 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:15:42.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:42 smithi072 ceph-mon[33332]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:42.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:42 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:42.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:42 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:42.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:42 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:42.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:42 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:42.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:42 smithi001 ceph-mon[29731]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:42.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:42 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:42.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:42 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:42.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:42 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:42.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:42 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:42.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:42 smithi077 ceph-mon[33531]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:42.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:42 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:42.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:42 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:42.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:42 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:42.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:42 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:42.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.915+0000 7f4d8c514700 1 -- 172.21.15.1:0/1166338312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d8410a3e0 msgr2=0x7f4d8410a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:42.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.915+0000 7f4d8c514700 1 --2- 172.21.15.1:0/1166338312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d8410a3e0 0x7f4d8410a7f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4d80009990 tx=0x7f4d80009ca0 comp rx=0 tx=0).stop 2024-05-04T03:15:42.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.916+0000 7f4d8c514700 1 -- 172.21.15.1:0/1166338312 shutdown_connections 2024-05-04T03:15:42.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.916+0000 7f4d8c514700 1 --2- 172.21.15.1:0/1166338312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d84103130 0x7f4d84111120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:42.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.916+0000 7f4d8c514700 1 --2- 172.21.15.1:0/1166338312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d8410a3e0 0x7f4d8410a7f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:42.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.916+0000 7f4d8c514700 1 --2- 172.21.15.1:0/1166338312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d841027e0 0x7f4d84102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:42.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.916+0000 7f4d8c514700 1 -- 172.21.15.1:0/1166338312 >> 172.21.15.1:0/1166338312 conn(0x7f4d840fe6f0 msgr2=0x7f4d84100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:42.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.916+0000 7f4d8c514700 1 -- 172.21.15.1:0/1166338312 shutdown_connections 2024-05-04T03:15:42.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.916+0000 7f4d8c514700 1 -- 172.21.15.1:0/1166338312 wait complete. 2024-05-04T03:15:42.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.917+0000 7f4d8c514700 1 Processor -- start 2024-05-04T03:15:42.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.917+0000 7f4d8c514700 1 -- start start 2024-05-04T03:15:42.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.917+0000 7f4d8c514700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d841027e0 0x7f4d8419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:42.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.917+0000 7f4d8c514700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d84103130 0x7f4d8419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:42.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.917+0000 7f4d8c514700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d8410a3e0 0x7f4d8419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:42.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.917+0000 7f4d8c514700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4d84114480 con 0x7f4d841027e0 2024-05-04T03:15:42.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.917+0000 7f4d8c514700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4d84114300 con 0x7f4d8410a3e0 2024-05-04T03:15:42.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.917+0000 7f4d8c514700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4d84114600 con 0x7f4d84103130 2024-05-04T03:15:42.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.917+0000 7f4d8a2b0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d841027e0 0x7f4d8419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:42.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.917+0000 7f4d8a2b0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d841027e0 0x7f4d8419ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48416/0 (socket says 172.21.15.1:48416) 2024-05-04T03:15:42.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.917+0000 7f4d8a2b0700 1 -- 172.21.15.1:0/3255500495 learned_addr learned my addr 172.21.15.1:0/3255500495 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:15:42.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.917+0000 7f4d89aaf700 1 --2- 172.21.15.1:0/3255500495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d84103130 0x7f4d8419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:42.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.917+0000 7f4d8aab1700 1 --2- 172.21.15.1:0/3255500495 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d8410a3e0 0x7f4d8419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:42.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.918+0000 7f4d8a2b0700 1 -- 172.21.15.1:0/3255500495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d84103130 msgr2=0x7f4d8419b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:42.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.918+0000 7f4d8a2b0700 1 --2- 172.21.15.1:0/3255500495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d84103130 0x7f4d8419b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:42.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.918+0000 7f4d8a2b0700 1 -- 172.21.15.1:0/3255500495 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d8410a3e0 msgr2=0x7f4d8419f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:42.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.918+0000 7f4d8a2b0700 1 --2- 172.21.15.1:0/3255500495 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d8410a3e0 0x7f4d8419f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:42.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.918+0000 7f4d8a2b0700 1 -- 172.21.15.1:0/3255500495 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8419fce0 con 0x7f4d841027e0 2024-05-04T03:15:42.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.918+0000 7f4d8a2b0700 1 --2- 172.21.15.1:0/3255500495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d841027e0 0x7f4d8419ae10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4d80000c00 tx=0x7f4d800047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:42.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.918+0000 7f4d7b7fe700 1 -- 172.21.15.1:0/3255500495 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d80027070 con 0x7f4d841027e0 2024-05-04T03:15:42.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.918+0000 7f4d7b7fe700 1 -- 172.21.15.1:0/3255500495 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d80003f20 con 0x7f4d841027e0 2024-05-04T03:15:42.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.918+0000 7f4d8c514700 1 -- 172.21.15.1:0/3255500495 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d8419ff70 con 0x7f4d841027e0 2024-05-04T03:15:42.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.918+0000 7f4d7b7fe700 1 -- 172.21.15.1:0/3255500495 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d80022ca0 con 0x7f4d841027e0 2024-05-04T03:15:42.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.918+0000 7f4d8c514700 1 -- 172.21.15.1:0/3255500495 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d841a0450 con 0x7f4d841027e0 2024-05-04T03:15:42.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.920+0000 7f4d8c514700 1 -- 172.21.15.1:0/3255500495 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d8410c850 con 0x7f4d841027e0 2024-05-04T03:15:42.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.921+0000 7f4d7b7fe700 1 -- 172.21.15.1:0/3255500495 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f4d8002a070 con 0x7f4d841027e0 2024-05-04T03:15:42.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.922+0000 7f4d7b7fe700 1 --2- 172.21.15.1:0/3255500495 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4d700b2880 0x7f4d700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:42.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.922+0000 7f4d89aaf700 1 --2- 172.21.15.1:0/3255500495 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4d700b2880 0x7f4d700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:42.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.922+0000 7f4d7b7fe700 1 -- 172.21.15.1:0/3255500495 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7127+0+0 (secure 0 0 0) 0x7f4d800e2fc0 con 0x7f4d841027e0 2024-05-04T03:15:42.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.922+0000 7f4d89aaf700 1 --2- 172.21.15.1:0/3255500495 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4d700b2880 0x7f4d700b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4d84100190 tx=0x7f4d7400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:42.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:42.926+0000 7f4d7b7fe700 1 -- 172.21.15.1:0/3255500495 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4d80031080 con 0x7f4d841027e0 2024-05-04T03:15:43.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:43.146+0000 7f4d8c514700 1 -- 172.21.15.1:0/3255500495 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4d8410c230 con 0x7f4d841027e0 2024-05-04T03:15:43.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:43.993+0000 7f4d7b7fe700 1 -- 172.21.15.1:0/3255500495 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v75) ==== 138+0+0 (secure 0 0 0) 0x7f4d800b3070 con 0x7f4d841027e0 2024-05-04T03:15:43.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:43.997+0000 7f4d8c514700 1 -- 172.21.15.1:0/3255500495 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4d700b2880 msgr2=0x7f4d700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:43.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:43.997+0000 7f4d8c514700 1 --2- 172.21.15.1:0/3255500495 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4d700b2880 0x7f4d700b4d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4d84100190 tx=0x7f4d7400b410 comp rx=0 tx=0).stop 2024-05-04T03:15:43.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:43.997+0000 7f4d8c514700 1 -- 172.21.15.1:0/3255500495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d841027e0 msgr2=0x7f4d8419ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:43.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:43.997+0000 7f4d8c514700 1 --2- 172.21.15.1:0/3255500495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d841027e0 0x7f4d8419ae10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4d80000c00 tx=0x7f4d800047b0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:43.997+0000 7f4d8c514700 1 -- 172.21.15.1:0/3255500495 shutdown_connections 2024-05-04T03:15:43.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:43.997+0000 7f4d8c514700 1 --2- 172.21.15.1:0/3255500495 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d8410a3e0 0x7f4d8419f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:43.997+0000 7f4d8c514700 1 --2- 172.21.15.1:0/3255500495 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4d700b2880 0x7f4d700b4d90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:43.997+0000 7f4d8c514700 1 --2- 172.21.15.1:0/3255500495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d841027e0 0x7f4d8419ae10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:43.997+0000 7f4d8c514700 1 --2- 172.21.15.1:0/3255500495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d84103130 0x7f4d8419b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:43.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:43.997+0000 7f4d8c514700 1 -- 172.21.15.1:0/3255500495 >> 172.21.15.1:0/3255500495 conn(0x7f4d840fe6f0 msgr2=0x7f4d84100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:43.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:43.997+0000 7f4d8c514700 1 -- 172.21.15.1:0/3255500495 shutdown_connections 2024-05-04T03:15:43.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:43.997+0000 7f4d8c514700 1 -- 172.21.15.1:0/3255500495 wait complete. 2024-05-04T03:15:44.000 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' created 2024-05-04T03:15:44.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:43 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:44.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:43 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:44.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:43 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:44.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:43 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:15:44.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:43 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:44.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:43 smithi072 ceph-mon[33332]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:44.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:43 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/3255500495' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-04T03:15:44.366 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:43 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:44.366 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:43 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:44.366 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:43 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:44.366 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:43 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:15:44.367 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:43 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:44.367 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:43 smithi001 ceph-mon[29731]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:44.367 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:43 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3255500495' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-04T03:15:44.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:43 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:44.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:43 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:44.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:43 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:15:44.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:43 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:15:44.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:43 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:15:44.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:43 smithi077 ceph-mon[33531]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:44.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:43 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/3255500495' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-04T03:15:44.685 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-04T03:15:44.939 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:15:45.268 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:45 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3255500495' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-04T03:15:45.269 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:45 smithi001 ceph-mon[29731]: osdmap e75: 12 total, 12 up, 12 in 2024-05-04T03:15:45.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:45 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/3255500495' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-04T03:15:45.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:45 smithi077 ceph-mon[33531]: osdmap e75: 12 total, 12 up, 12 in 2024-05-04T03:15:45.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:45 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/3255500495' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-04T03:15:45.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:45 smithi072 ceph-mon[33332]: osdmap e75: 12 total, 12 up, 12 in 2024-05-04T03:15:46.274 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:46 smithi001 ceph-mon[29731]: osdmap e76: 12 total, 12 up, 12 in 2024-05-04T03:15:46.275 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:46 smithi001 ceph-mon[29731]: pgmap v282: 33 pgs: 10 creating+peering, 1 active+clean, 22 unknown; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:46.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:46 smithi077 ceph-mon[33531]: osdmap e76: 12 total, 12 up, 12 in 2024-05-04T03:15:46.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:46 smithi077 ceph-mon[33531]: pgmap v282: 33 pgs: 10 creating+peering, 1 active+clean, 22 unknown; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:46.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:46 smithi072 ceph-mon[33332]: osdmap e76: 12 total, 12 up, 12 in 2024-05-04T03:15:46.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:46 smithi072 ceph-mon[33332]: pgmap v282: 33 pgs: 10 creating+peering, 1 active+clean, 22 unknown; 577 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:46.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.522+0000 7fd73800e700 1 -- 172.21.15.1:0/2809296884 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd730108750 msgr2=0x7fd7301100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:46.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.522+0000 7fd73800e700 1 --2- 172.21.15.1:0/2809296884 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd730108750 0x7fd7301100b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd72800b0d0 tx=0x7fd72800b3e0 comp rx=0 tx=0).stop 2024-05-04T03:15:46.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.522+0000 7fd73800e700 1 -- 172.21.15.1:0/2809296884 shutdown_connections 2024-05-04T03:15:46.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.522+0000 7fd73800e700 1 --2- 172.21.15.1:0/2809296884 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd730108750 0x7fd7301100b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:46.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.522+0000 7fd73800e700 1 --2- 172.21.15.1:0/2809296884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd730103520 0x7fd730103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:46.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.522+0000 7fd73800e700 1 --2- 172.21.15.1:0/2809296884 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd730107e00 0x7fd730108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:46.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.522+0000 7fd73800e700 1 -- 172.21.15.1:0/2809296884 >> 172.21.15.1:0/2809296884 conn(0x7fd7300fef90 msgr2=0x7fd7301013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:46.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.522+0000 7fd73800e700 1 -- 172.21.15.1:0/2809296884 shutdown_connections 2024-05-04T03:15:46.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.522+0000 7fd73800e700 1 -- 172.21.15.1:0/2809296884 wait complete. 2024-05-04T03:15:46.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.523+0000 7fd73800e700 1 Processor -- start 2024-05-04T03:15:46.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.523+0000 7fd73800e700 1 -- start start 2024-05-04T03:15:46.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.524+0000 7fd73800e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd730103520 0x7fd73019d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:46.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.524+0000 7fd73800e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd730107e00 0x7fd73019d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:46.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.524+0000 7fd73800e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd730108750 0x7fd7301a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:46.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.524+0000 7fd73800e700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd7301133d0 con 0x7fd730107e00 2024-05-04T03:15:46.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.524+0000 7fd73800e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd730113250 con 0x7fd730108750 2024-05-04T03:15:46.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.524+0000 7fd73800e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd730113550 con 0x7fd730103520 2024-05-04T03:15:46.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.524+0000 7fd7355a9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd730107e00 0x7fd73019d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:46.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.524+0000 7fd735daa700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd730103520 0x7fd73019d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:46.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.524+0000 7fd7355a9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd730107e00 0x7fd73019d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48426/0 (socket says 172.21.15.1:48426) 2024-05-04T03:15:46.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.524+0000 7fd7355a9700 1 -- 172.21.15.1:0/3431353262 learned_addr learned my addr 172.21.15.1:0/3431353262 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:15:46.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.524+0000 7fd7365ab700 1 --2- 172.21.15.1:0/3431353262 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd730108750 0x7fd7301a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:46.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.524+0000 7fd735daa700 1 -- 172.21.15.1:0/3431353262 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd730108750 msgr2=0x7fd7301a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:46.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.524+0000 7fd735daa700 1 --2- 172.21.15.1:0/3431353262 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd730108750 0x7fd7301a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:46.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.524+0000 7fd735daa700 1 -- 172.21.15.1:0/3431353262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd730107e00 msgr2=0x7fd73019d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:46.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.524+0000 7fd735daa700 1 --2- 172.21.15.1:0/3431353262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd730107e00 0x7fd73019d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:46.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.524+0000 7fd735daa700 1 -- 172.21.15.1:0/3431353262 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7301a1ed0 con 0x7fd730103520 2024-05-04T03:15:46.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.524+0000 7fd7355a9700 1 --2- 172.21.15.1:0/3431353262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd730107e00 0x7fd73019d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:15:46.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.524+0000 7fd735daa700 1 --2- 172.21.15.1:0/3431353262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd730103520 0x7fd73019d000 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd72c00d950 tx=0x7fd72c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:46.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.525+0000 7fd726ffd700 1 -- 172.21.15.1:0/3431353262 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c016070 con 0x7fd730103520 2024-05-04T03:15:46.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.525+0000 7fd73800e700 1 -- 172.21.15.1:0/3431353262 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7301a21c0 con 0x7fd730103520 2024-05-04T03:15:46.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.525+0000 7fd726ffd700 1 -- 172.21.15.1:0/3431353262 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd72c00fd40 con 0x7fd730103520 2024-05-04T03:15:46.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.525+0000 7fd726ffd700 1 -- 172.21.15.1:0/3431353262 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c00e3b0 con 0x7fd730103520 2024-05-04T03:15:46.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.525+0000 7fd73800e700 1 -- 172.21.15.1:0/3431353262 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7301a9ae0 con 0x7fd730103520 2024-05-04T03:15:46.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.526+0000 7fd73800e700 1 -- 172.21.15.1:0/3431353262 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7300512c0 con 0x7fd730103520 2024-05-04T03:15:46.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.528+0000 7fd726ffd700 1 -- 172.21.15.1:0/3431353262 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fd72c019070 con 0x7fd730103520 2024-05-04T03:15:46.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.529+0000 7fd726ffd700 1 --2- 172.21.15.1:0/3431353262 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd71c0b2880 0x7fd71c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:46.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.529+0000 7fd7355a9700 1 --2- 172.21.15.1:0/3431353262 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd71c0b2880 0x7fd71c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:46.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.529+0000 7fd726ffd700 1 -- 172.21.15.1:0/3431353262 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7497+0+0 (secure 0 0 0) 0x7fd72c0e1d10 con 0x7fd730103520 2024-05-04T03:15:46.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.529+0000 7fd7355a9700 1 --2- 172.21.15.1:0/3431353262 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd71c0b2880 0x7fd71c0b4d90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd73006cd50 tx=0x7fd72000a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:46.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.532+0000 7fd726ffd700 1 -- 172.21.15.1:0/3431353262 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd72c0a9d60 con 0x7fd730103520 2024-05-04T03:15:46.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:46.747+0000 7fd73800e700 1 -- 172.21.15.1:0/3431353262 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd73019e390 con 0x7fd730103520 2024-05-04T03:15:47.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:47.023+0000 7fd726ffd700 1 -- 172.21.15.1:0/3431353262 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v78) ==== 170+0+0 (secure 0 0 0) 0x7fd73019e390 con 0x7fd730103520 2024-05-04T03:15:47.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:47.028+0000 7fd73800e700 1 -- 172.21.15.1:0/3431353262 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd71c0b2880 msgr2=0x7fd71c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:47.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:47.028+0000 7fd73800e700 1 --2- 172.21.15.1:0/3431353262 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd71c0b2880 0x7fd71c0b4d90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd73006cd50 tx=0x7fd72000a3b0 comp rx=0 tx=0).stop 2024-05-04T03:15:47.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:47.028+0000 7fd73800e700 1 -- 172.21.15.1:0/3431353262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd730103520 msgr2=0x7fd73019d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:47.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:47.028+0000 7fd73800e700 1 --2- 172.21.15.1:0/3431353262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd730103520 0x7fd73019d000 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd72c00d950 tx=0x7fd72c00dc60 comp rx=0 tx=0).stop 2024-05-04T03:15:47.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:47.029+0000 7fd73800e700 1 -- 172.21.15.1:0/3431353262 shutdown_connections 2024-05-04T03:15:47.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:47.029+0000 7fd73800e700 1 --2- 172.21.15.1:0/3431353262 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd730108750 0x7fd7301a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:47.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:47.029+0000 7fd73800e700 1 --2- 172.21.15.1:0/3431353262 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd71c0b2880 0x7fd71c0b4d90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:47.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:47.029+0000 7fd73800e700 1 --2- 172.21.15.1:0/3431353262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd730103520 0x7fd73019d000 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:47.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:47.029+0000 7fd73800e700 1 --2- 172.21.15.1:0/3431353262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd730107e00 0x7fd73019d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:47.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:47.029+0000 7fd73800e700 1 -- 172.21.15.1:0/3431353262 >> 172.21.15.1:0/3431353262 conn(0x7fd7300fef90 msgr2=0x7fd73010fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:47.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:47.029+0000 7fd73800e700 1 -- 172.21.15.1:0/3431353262 shutdown_connections 2024-05-04T03:15:47.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:47.029+0000 7fd73800e700 1 -- 172.21.15.1:0/3431353262 wait complete. 2024-05-04T03:15:47.032 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' created 2024-05-04T03:15:47.328 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:47 smithi001 ceph-mon[29731]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-04T03:15:47.328 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:47 smithi001 ceph-mon[29731]: osdmap e77: 12 total, 12 up, 12 in 2024-05-04T03:15:47.329 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:47 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3431353262' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-04T03:15:47.329 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:47 smithi001 ceph-mon[29731]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-04T03:15:47.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:47 smithi077 ceph-mon[33531]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-04T03:15:47.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:47 smithi077 ceph-mon[33531]: osdmap e77: 12 total, 12 up, 12 in 2024-05-04T03:15:47.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:47 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/3431353262' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-04T03:15:47.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:47 smithi077 ceph-mon[33531]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-04T03:15:47.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:47 smithi072 ceph-mon[33332]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-04T03:15:47.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:47 smithi072 ceph-mon[33332]: osdmap e77: 12 total, 12 up, 12 in 2024-05-04T03:15:47.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:47 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/3431353262' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-04T03:15:47.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:47 smithi072 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-04T03:15:47.656 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-04T03:15:47.916 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:15:48.282 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:48 smithi001 ceph-mon[29731]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-04T03:15:48.282 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:48 smithi001 ceph-mon[29731]: osdmap e78: 12 total, 12 up, 12 in 2024-05-04T03:15:48.282 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:48 smithi001 ceph-mon[29731]: pgmap v285: 97 pgs: 18 creating+peering, 4 active+clean, 75 unknown; 577 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:48.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:48 smithi077 ceph-mon[33531]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-04T03:15:48.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:48 smithi077 ceph-mon[33531]: osdmap e78: 12 total, 12 up, 12 in 2024-05-04T03:15:48.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:48 smithi077 ceph-mon[33531]: pgmap v285: 97 pgs: 18 creating+peering, 4 active+clean, 75 unknown; 577 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:48.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:48 smithi072 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-04T03:15:48.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:48 smithi072 ceph-mon[33332]: osdmap e78: 12 total, 12 up, 12 in 2024-05-04T03:15:48.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:48 smithi072 ceph-mon[33332]: pgmap v285: 97 pgs: 18 creating+peering, 4 active+clean, 75 unknown; 577 KiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:49.316 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:49 smithi001 ceph-mon[29731]: osdmap e79: 12 total, 12 up, 12 in 2024-05-04T03:15:49.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:49 smithi077 ceph-mon[33531]: osdmap e79: 12 total, 12 up, 12 in 2024-05-04T03:15:49.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:49 smithi072 ceph-mon[33332]: osdmap e79: 12 total, 12 up, 12 in 2024-05-04T03:15:49.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.547+0000 7f4d5380a700 1 -- 172.21.15.1:0/749243319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d4c1034e0 msgr2=0x7f4d4c1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:49.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.547+0000 7f4d5380a700 1 --2- 172.21.15.1:0/749243319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d4c1034e0 0x7f4d4c1038f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4d480099c0 tx=0x7f4d48009cd0 comp rx=0 tx=0).stop 2024-05-04T03:15:49.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.547+0000 7f4d5380a700 1 -- 172.21.15.1:0/749243319 shutdown_connections 2024-05-04T03:15:49.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.547+0000 7f4d5380a700 1 --2- 172.21.15.1:0/749243319 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d4c108760 0x7f4d4c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:49.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.547+0000 7f4d5380a700 1 --2- 172.21.15.1:0/749243319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d4c1034e0 0x7f4d4c1038f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:49.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.547+0000 7f4d5380a700 1 --2- 172.21.15.1:0/749243319 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d4c107e10 0x7f4d4c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:49.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.547+0000 7f4d5380a700 1 -- 172.21.15.1:0/749243319 >> 172.21.15.1:0/749243319 conn(0x7f4d4c0fef30 msgr2=0x7f4d4c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:49.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.547+0000 7f4d5380a700 1 -- 172.21.15.1:0/749243319 shutdown_connections 2024-05-04T03:15:49.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.547+0000 7f4d5380a700 1 -- 172.21.15.1:0/749243319 wait complete. 2024-05-04T03:15:49.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.548+0000 7f4d5380a700 1 Processor -- start 2024-05-04T03:15:49.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.548+0000 7f4d5380a700 1 -- start start 2024-05-04T03:15:49.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.548+0000 7f4d5380a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d4c1034e0 0x7f4d4c199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:49.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.548+0000 7f4d5380a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d4c107e10 0x7f4d4c19a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:49.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.548+0000 7f4d5380a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d4c108760 0x7f4d4c19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:49.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.548+0000 7f4d5380a700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c113400 con 0x7f4d4c1034e0 2024-05-04T03:15:49.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.548+0000 7f4d5380a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c113280 con 0x7f4d4c108760 2024-05-04T03:15:49.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.548+0000 7f4d5380a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c113580 con 0x7f4d4c107e10 2024-05-04T03:15:49.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.548+0000 7f4d515a6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d4c1034e0 0x7f4d4c199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:49.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.548+0000 7f4d515a6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d4c1034e0 0x7f4d4c199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48442/0 (socket says 172.21.15.1:48442) 2024-05-04T03:15:49.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.549+0000 7f4d515a6700 1 -- 172.21.15.1:0/260685677 learned_addr learned my addr 172.21.15.1:0/260685677 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:15:49.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.549+0000 7f4d51da7700 1 --2- 172.21.15.1:0/260685677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d4c108760 0x7f4d4c19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:49.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.549+0000 7f4d50da5700 1 --2- 172.21.15.1:0/260685677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d4c107e10 0x7f4d4c19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:49.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.549+0000 7f4d515a6700 1 -- 172.21.15.1:0/260685677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d4c107e10 msgr2=0x7f4d4c19a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:49.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.549+0000 7f4d515a6700 1 --2- 172.21.15.1:0/260685677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d4c107e10 0x7f4d4c19a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:49.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.549+0000 7f4d515a6700 1 -- 172.21.15.1:0/260685677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d4c108760 msgr2=0x7f4d4c19e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:49.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.549+0000 7f4d515a6700 1 --2- 172.21.15.1:0/260685677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d4c108760 0x7f4d4c19e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:49.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.549+0000 7f4d515a6700 1 -- 172.21.15.1:0/260685677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4c19ec80 con 0x7f4d4c1034e0 2024-05-04T03:15:49.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.549+0000 7f4d50da5700 1 --2- 172.21.15.1:0/260685677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d4c107e10 0x7f4d4c19a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:15:49.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.549+0000 7f4d515a6700 1 --2- 172.21.15.1:0/260685677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d4c1034e0 0x7f4d4c199db0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4d48000c00 tx=0x7f4d48003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:49.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.549+0000 7f4d427fc700 1 -- 172.21.15.1:0/260685677 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d48027070 con 0x7f4d4c1034e0 2024-05-04T03:15:49.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.549+0000 7f4d5380a700 1 -- 172.21.15.1:0/260685677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d4c19ef10 con 0x7f4d4c1034e0 2024-05-04T03:15:49.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.549+0000 7f4d427fc700 1 -- 172.21.15.1:0/260685677 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d48004160 con 0x7f4d4c1034e0 2024-05-04T03:15:49.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.549+0000 7f4d427fc700 1 -- 172.21.15.1:0/260685677 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4800f460 con 0x7f4d4c1034e0 2024-05-04T03:15:49.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.549+0000 7f4d5380a700 1 -- 172.21.15.1:0/260685677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d4c1a6890 con 0x7f4d4c1034e0 2024-05-04T03:15:49.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.550+0000 7f4d5380a700 1 -- 172.21.15.1:0/260685677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d4c104990 con 0x7f4d4c1034e0 2024-05-04T03:15:49.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.552+0000 7f4d427fc700 1 -- 172.21.15.1:0/260685677 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f4d4802a070 con 0x7f4d4c1034e0 2024-05-04T03:15:49.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.553+0000 7f4d427fc700 1 --2- 172.21.15.1:0/260685677 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4d380b2880 0x7f4d380b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:49.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.553+0000 7f4d50da5700 1 --2- 172.21.15.1:0/260685677 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4d380b2880 0x7f4d380b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:49.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.553+0000 7f4d427fc700 1 -- 172.21.15.1:0/260685677 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7879+0+0 (secure 0 0 0) 0x7f4d480e3870 con 0x7f4d4c1034e0 2024-05-04T03:15:49.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.554+0000 7f4d50da5700 1 --2- 172.21.15.1:0/260685677 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4d380b2880 0x7f4d380b4d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c106770 tx=0x7f4d3c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:49.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.557+0000 7f4d427fc700 1 -- 172.21.15.1:0/260685677 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4d48031020 con 0x7f4d4c1034e0 2024-05-04T03:15:49.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:49.822+0000 7f4d5380a700 1 -- 172.21.15.1:0/260685677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4d4c19b110 con 0x7f4d4c1034e0 2024-05-04T03:15:50.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:50.033+0000 7f4d427fc700 1 -- 172.21.15.1:0/260685677 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f4d48013070 con 0x7f4d4c1034e0 2024-05-04T03:15:50.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:50.036+0000 7f4d5380a700 1 -- 172.21.15.1:0/260685677 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4d380b2880 msgr2=0x7f4d380b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:50.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:50.036+0000 7f4d5380a700 1 --2- 172.21.15.1:0/260685677 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4d380b2880 0x7f4d380b4d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c106770 tx=0x7f4d3c008040 comp rx=0 tx=0).stop 2024-05-04T03:15:50.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:50.036+0000 7f4d5380a700 1 -- 172.21.15.1:0/260685677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d4c1034e0 msgr2=0x7f4d4c199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:50.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:50.036+0000 7f4d5380a700 1 --2- 172.21.15.1:0/260685677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d4c1034e0 0x7f4d4c199db0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4d48000c00 tx=0x7f4d48003730 comp rx=0 tx=0).stop 2024-05-04T03:15:50.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:50.037+0000 7f4d5380a700 1 -- 172.21.15.1:0/260685677 shutdown_connections 2024-05-04T03:15:50.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:50.037+0000 7f4d5380a700 1 --2- 172.21.15.1:0/260685677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4d4c108760 0x7f4d4c19e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:50.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:50.037+0000 7f4d5380a700 1 --2- 172.21.15.1:0/260685677 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4d380b2880 0x7f4d380b4d90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:50.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:50.037+0000 7f4d5380a700 1 --2- 172.21.15.1:0/260685677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d4c1034e0 0x7f4d4c199db0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:50.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:50.037+0000 7f4d5380a700 1 --2- 172.21.15.1:0/260685677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d4c107e10 0x7f4d4c19a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:50.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:50.037+0000 7f4d5380a700 1 -- 172.21.15.1:0/260685677 >> 172.21.15.1:0/260685677 conn(0x7f4d4c0fef30 msgr2=0x7f4d4c1071d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:50.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:50.037+0000 7f4d5380a700 1 -- 172.21.15.1:0/260685677 shutdown_connections 2024-05-04T03:15:50.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:50.037+0000 7f4d5380a700 1 -- 172.21.15.1:0/260685677 wait complete. 2024-05-04T03:15:50.039 INFO:teuthology.orchestra.run.smithi001.stderr:new fs with metadata pool 2 and data pool 3 2024-05-04T03:15:50.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:50 smithi001 ceph-mon[29731]: osdmap e80: 12 total, 12 up, 12 in 2024-05-04T03:15:50.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:50 smithi001 ceph-mon[29731]: pgmap v288: 97 pgs: 27 creating+peering, 46 active+clean, 24 unknown; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:50.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:50 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/260685677' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-04T03:15:50.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:50 smithi001 ceph-mon[29731]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-04T03:15:50.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:50 smithi077 ceph-mon[33531]: osdmap e80: 12 total, 12 up, 12 in 2024-05-04T03:15:50.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:50 smithi077 ceph-mon[33531]: pgmap v288: 97 pgs: 27 creating+peering, 46 active+clean, 24 unknown; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:50.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:50 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/260685677' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-04T03:15:50.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:50 smithi077 ceph-mon[33531]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-04T03:15:50.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:50 smithi072 ceph-mon[33332]: osdmap e80: 12 total, 12 up, 12 in 2024-05-04T03:15:50.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:50 smithi072 ceph-mon[33332]: pgmap v288: 97 pgs: 27 creating+peering, 46 active+clean, 24 unknown; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:50.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:50 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/260685677' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-04T03:15:50.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:50 smithi072 ceph-mon[33332]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-04T03:15:50.620 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-04T03:15:50.868 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:15:51.354 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:51 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/260685677' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-04T03:15:51.354 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:51 smithi001 ceph-mon[29731]: osdmap e81: 12 total, 12 up, 12 in 2024-05-04T03:15:51.354 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:51 smithi001 ceph-mon[29731]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-04T03:15:51.354 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:51 smithi001 ceph-mon[29731]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-04T03:15:51.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:51 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/260685677' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-04T03:15:51.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:51 smithi077 ceph-mon[33531]: osdmap e81: 12 total, 12 up, 12 in 2024-05-04T03:15:51.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:51 smithi077 ceph-mon[33531]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-04T03:15:51.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:51 smithi077 ceph-mon[33531]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-04T03:15:51.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:51 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/260685677' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-04T03:15:51.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:51 smithi072 ceph-mon[33332]: osdmap e81: 12 total, 12 up, 12 in 2024-05-04T03:15:51.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:51 smithi072 ceph-mon[33332]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-04T03:15:51.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:51 smithi072 ceph-mon[33332]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-04T03:15:52.170 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:52 smithi001 ceph-mon[29731]: mds.? [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] up:active 2024-05-04T03:15:52.171 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:52 smithi001 ceph-mon[29731]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-04T03:15:52.171 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:52 smithi001 ceph-mon[29731]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-04T03:15:52.171 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:52 smithi001 ceph-mon[29731]: pgmap v290: 97 pgs: 20 creating+peering, 73 active+clean, 4 unknown; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:52.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.381+0000 7f5a3f0a5700 1 -- 172.21.15.1:0/4261178635 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a38108740 msgr2=0x7f5a381100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:52.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.381+0000 7f5a3f0a5700 1 --2- 172.21.15.1:0/4261178635 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a38108740 0x7f5a381100a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5a3000b0d0 tx=0x7f5a3000b3e0 comp rx=0 tx=0).stop 2024-05-04T03:15:52.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.381+0000 7f5a3f0a5700 1 -- 172.21.15.1:0/4261178635 shutdown_connections 2024-05-04T03:15:52.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.381+0000 7f5a3f0a5700 1 --2- 172.21.15.1:0/4261178635 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a38108740 0x7f5a381100a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:52.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.381+0000 7f5a3f0a5700 1 --2- 172.21.15.1:0/4261178635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a381034c0 0x7f5a381038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:52.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.381+0000 7f5a3f0a5700 1 --2- 172.21.15.1:0/4261178635 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a38107df0 0x7f5a38108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:52.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.381+0000 7f5a3f0a5700 1 -- 172.21.15.1:0/4261178635 >> 172.21.15.1:0/4261178635 conn(0x7f5a380fedf0 msgr2=0x7f5a38101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:52.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.381+0000 7f5a3f0a5700 1 -- 172.21.15.1:0/4261178635 shutdown_connections 2024-05-04T03:15:52.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.381+0000 7f5a3f0a5700 1 -- 172.21.15.1:0/4261178635 wait complete. 2024-05-04T03:15:52.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.382+0000 7f5a3f0a5700 1 Processor -- start 2024-05-04T03:15:52.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.382+0000 7f5a3f0a5700 1 -- start start 2024-05-04T03:15:52.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.382+0000 7f5a3f0a5700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a381034c0 0x7f5a3819bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:52.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.382+0000 7f5a3f0a5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a38107df0 0x7f5a3819c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:52.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.382+0000 7f5a3f0a5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a38108740 0x7f5a3819c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:52.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.382+0000 7f5a3f0a5700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5a381133e0 con 0x7f5a381034c0 2024-05-04T03:15:52.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.382+0000 7f5a3f0a5700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5a38113260 con 0x7f5a38108740 2024-05-04T03:15:52.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.382+0000 7f5a3f0a5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5a38113560 con 0x7f5a38107df0 2024-05-04T03:15:52.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.382+0000 7f5a3ce41700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a381034c0 0x7f5a3819bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:52.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.382+0000 7f5a3ce41700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a381034c0 0x7f5a3819bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33448/0 (socket says 172.21.15.1:33448) 2024-05-04T03:15:52.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.382+0000 7f5a3ce41700 1 -- 172.21.15.1:0/2497418107 learned_addr learned my addr 172.21.15.1:0/2497418107 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:15:52.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.382+0000 7f5a37fff700 1 --2- 172.21.15.1:0/2497418107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a38107df0 0x7f5a3819c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:52.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.382+0000 7f5a3d642700 1 --2- 172.21.15.1:0/2497418107 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a38108740 0x7f5a3819c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:52.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.383+0000 7f5a3ce41700 1 -- 172.21.15.1:0/2497418107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a38107df0 msgr2=0x7f5a3819c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:52.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.383+0000 7f5a3ce41700 1 --2- 172.21.15.1:0/2497418107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a38107df0 0x7f5a3819c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:52.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.383+0000 7f5a3ce41700 1 -- 172.21.15.1:0/2497418107 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a38108740 msgr2=0x7f5a3819c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:52.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.383+0000 7f5a3ce41700 1 --2- 172.21.15.1:0/2497418107 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a38108740 0x7f5a3819c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:52.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.383+0000 7f5a3ce41700 1 -- 172.21.15.1:0/2497418107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a381a6ff0 con 0x7f5a381034c0 2024-05-04T03:15:52.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.383+0000 7f5a3ce41700 1 --2- 172.21.15.1:0/2497418107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a381034c0 0x7f5a3819bc80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00d950 tx=0x7f5a2c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:52.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.383+0000 7f5a35ffb700 1 -- 172.21.15.1:0/2497418107 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c016070 con 0x7f5a381034c0 2024-05-04T03:15:52.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.383+0000 7f5a35ffb700 1 -- 172.21.15.1:0/2497418107 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a2c00fd40 con 0x7f5a381034c0 2024-05-04T03:15:52.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.383+0000 7f5a3f0a5700 1 -- 172.21.15.1:0/2497418107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a381a72e0 con 0x7f5a381034c0 2024-05-04T03:15:52.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.383+0000 7f5a35ffb700 1 -- 172.21.15.1:0/2497418107 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c00e3b0 con 0x7f5a381034c0 2024-05-04T03:15:52.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.383+0000 7f5a3f0a5700 1 -- 172.21.15.1:0/2497418107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a381a7820 con 0x7f5a381034c0 2024-05-04T03:15:52.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.384+0000 7f5a3f0a5700 1 -- 172.21.15.1:0/2497418107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a380512c0 con 0x7f5a381034c0 2024-05-04T03:15:52.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.386+0000 7f5a35ffb700 1 -- 172.21.15.1:0/2497418107 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f5a2c019070 con 0x7f5a381034c0 2024-05-04T03:15:52.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.386+0000 7f5a35ffb700 1 --2- 172.21.15.1:0/2497418107 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5a280b2880 0x7f5a280b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:52.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.386+0000 7f5a37fff700 1 --2- 172.21.15.1:0/2497418107 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5a280b2880 0x7f5a280b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:52.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.386+0000 7f5a35ffb700 1 -- 172.21.15.1:0/2497418107 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f5a2c0e1b10 con 0x7f5a381034c0 2024-05-04T03:15:52.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.387+0000 7f5a37fff700 1 --2- 172.21.15.1:0/2497418107 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5a280b2880 0x7f5a280b4d90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5a3806b8a0 tx=0x7f5a24009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:52.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.390+0000 7f5a35ffb700 1 -- 172.21.15.1:0/2497418107 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5a2c0a9980 con 0x7f5a381034c0 2024-05-04T03:15:52.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:52 smithi077 ceph-mon[33531]: mds.? [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] up:active 2024-05-04T03:15:52.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:52 smithi077 ceph-mon[33531]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-04T03:15:52.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:52 smithi077 ceph-mon[33531]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-04T03:15:52.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:52 smithi077 ceph-mon[33531]: pgmap v290: 97 pgs: 20 creating+peering, 73 active+clean, 4 unknown; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:52.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:52 smithi072 ceph-mon[33332]: mds.? [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] up:active 2024-05-04T03:15:52.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:52 smithi072 ceph-mon[33332]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-04T03:15:52.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:52 smithi072 ceph-mon[33332]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-04T03:15:52.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:52 smithi072 ceph-mon[33332]: pgmap v290: 97 pgs: 20 creating+peering, 73 active+clean, 4 unknown; 577 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:15:52.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.606+0000 7f5a3f0a5700 1 -- 172.21.15.1:0/2497418107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a3802e5f0 con 0x7f5a381034c0 2024-05-04T03:15:52.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.607+0000 7f5a35ffb700 1 -- 172.21.15.1:0/2497418107 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22088 (secure 0 0 0) 0x7f5a2c0a9280 con 0x7f5a381034c0 2024-05-04T03:15:52.609 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:15:52.610 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":81,"fsid":"4a572aa8-09c3-11ef-bc97-c7b262605968","created":"2024-05-04T03:07:32.845547+0000","modified":"2024-05-04T03:15:49.823513+0000","last_up_change":"2024-05-04T03:14:44.192571+0000","last_in_change":"2024-05-04T03:14:23.353342+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T03:11:01.139189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T03:15:43.151696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T03:15:46.753539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"100fe035-2454-4c95-90c7-759956ec36a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6803","nonce":3479011484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6805","nonce":3479011484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6809","nonce":3479011484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6807","nonce":3479011484}]},"public_addr":"172.21.15.1:6803/3479011484","cluster_addr":"172.21.15.1:6805/3479011484","heartbeat_back_addr":"172.21.15.1:6809/3479011484","heartbeat_front_addr":"172.21.15.1:6807/3479011484","state":["exists","up"]},{"osd":1,"uuid":"5826ac76-ef6b-4040-8ad1-305db355af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6801","nonce":3725706149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6803","nonce":3725706149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6807","nonce":3725706149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6805","nonce":3725706149}]},"public_addr":"172.21.15.72:6801/3725706149","cluster_addr":"172.21.15.72:6803/3725706149","heartbeat_back_addr":"172.21.15.72:6807/3725706149","heartbeat_front_addr":"172.21.15.72:6805/3725706149","state":["exists","up"]},{"osd":2,"uuid":"923d76e5-b032-4190-9e35-546d27722a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6801","nonce":515038343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6803","nonce":515038343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6807","nonce":515038343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6805","nonce":515038343}]},"public_addr":"172.21.15.77:6801/515038343","cluster_addr":"172.21.15.77:6803/515038343","heartbeat_back_addr":"172.21.15.77:6807/515038343","heartbeat_front_addr":"172.21.15.77:6805/515038343","state":["exists","up"]},{"osd":3,"uuid":"495cc0c5-6a99-481f-ad9e-38f05fb84356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6811","nonce":4050439365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6813","nonce":4050439365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6817","nonce":4050439365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6815","nonce":4050439365}]},"public_addr":"172.21.15.1:6811/4050439365","cluster_addr":"172.21.15.1:6813/4050439365","heartbeat_back_addr":"172.21.15.1:6817/4050439365","heartbeat_front_addr":"172.21.15.1:6815/4050439365","state":["exists","up"]},{"osd":4,"uuid":"b86b8026-79c0-4056-9e67-a61f54b6ab5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6809","nonce":2501732391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6811","nonce":2501732391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6815","nonce":2501732391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6813","nonce":2501732391}]},"public_addr":"172.21.15.72:6809/2501732391","cluster_addr":"172.21.15.72:6811/2501732391","heartbeat_back_addr":"172.21.15.72:6815/2501732391","heartbeat_front_addr":"172.21.15.72:6813/2501732391","state":["exists","up"]},{"osd":5,"uuid":"80975cb9-39e0-46b8-a343-9ff69194a14e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6809","nonce":3582852165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6811","nonce":3582852165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6815","nonce":3582852165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6813","nonce":3582852165}]},"public_addr":"172.21.15.77:6809/3582852165","cluster_addr":"172.21.15.77:6811/3582852165","heartbeat_back_addr":"172.21.15.77:6815/3582852165","heartbeat_front_addr":"172.21.15.77:6813/3582852165","state":["exists","up"]},{"osd":6,"uuid":"aca3fac5-fbb5-42de-bf0e-e4a0db7381e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6819","nonce":1799033649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6821","nonce":1799033649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6825","nonce":1799033649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6823","nonce":1799033649}]},"public_addr":"172.21.15.1:6819/1799033649","cluster_addr":"172.21.15.1:6821/1799033649","heartbeat_back_addr":"172.21.15.1:6825/1799033649","heartbeat_front_addr":"172.21.15.1:6823/1799033649","state":["exists","up"]},{"osd":7,"uuid":"cbff6b8a-c463-4249-ae45-fdc6f99a5489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6817","nonce":3460835447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6819","nonce":3460835447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6823","nonce":3460835447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6821","nonce":3460835447}]},"public_addr":"172.21.15.72:6817/3460835447","cluster_addr":"172.21.15.72:6819/3460835447","heartbeat_back_addr":"172.21.15.72:6823/3460835447","heartbeat_front_addr":"172.21.15.72:6821/3460835447","state":["exists","up"]},{"osd":8,"uuid":"e140bf13-4887-4d68-9373-3d4fc1c4584a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6817","nonce":1124479323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6819","nonce":1124479323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6823","nonce":1124479323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6821","nonce":1124479323}]},"public_addr":"172.21.15.77:6817/1124479323","cluster_addr":"172.21.15.77:6819/1124479323","heartbeat_back_addr":"172.21.15.77:6823/1124479323","heartbeat_front_addr":"172.21.15.77:6821/1124479323","state":["exists","up"]},{"osd":9,"uuid":"f1e8a232-cb11-4b9d-8789-e1637fa5aaa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6827","nonce":2681322272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6829","nonce":2681322272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6833","nonce":2681322272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6831","nonce":2681322272}]},"public_addr":"172.21.15.1:6827/2681322272","cluster_addr":"172.21.15.1:6829/2681322272","heartbeat_back_addr":"172.21.15.1:6833/2681322272","heartbeat_front_addr":"172.21.15.1:6831/2681322272","state":["exists","up"]},{"osd":10,"uuid":"79b570af-3d1e-439d-a328-9e6bc7d67839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6825","nonce":3293703569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6827","nonce":3293703569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6831","nonce":3293703569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6829","nonce":3293703569}]},"public_addr":"172.21.15.72:6825/3293703569","cluster_addr":"172.21.15.72:6827/3293703569","heartbeat_back_addr":"172.21.15.72:6831/3293703569","heartbeat_front_addr":"172.21.15.72:6829/3293703569","state":["exists","up"]},{"osd":11,"uuid":"e35bf83d-9ee4-4745-96c2-1c35cc8eefba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6825","nonce":4155947548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6827","nonce":4155947548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6831","nonce":4155947548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6829","nonce":4155947548}]},"public_addr":"172.21.15.77:6825/4155947548","cluster_addr":"172.21.15.77:6827/4155947548","heartbeat_back_addr":"172.21.15.77:6831/4155947548","heartbeat_front_addr":"172.21.15.77:6829/4155947548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:09.467126+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:33.861139+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:57.741962+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:11:21.746021+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:11:46.323252+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:12:10.640180+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:12:35.750591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:00.306549+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:25.012495+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:51.185229+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:14:16.403850+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:14:41.653316+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/2029897106":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/402488450":"2024-05-05T03:08:33.045884+0000","172.21.15.1:6801/128793132":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/2233526606":"2024-05-05T03:08:06.135783+0000","172.21.15.1:0/2715541855":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/2415697325":"2024-05-05T03:08:06.135783+0000","172.21.15.1:0/2305298106":"2024-05-05T03:08:06.135783+0000","172.21.15.1:6800/128793132":"2024-05-05T03:08:33.045884+0000","172.21.15.1:6800/4200529863":"2024-05-05T03:08:06.135783+0000","172.21.15.1:6801/4200529863":"2024-05-05T03:08:06.135783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T03:15:52.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.610+0000 7f5a3f0a5700 1 -- 172.21.15.1:0/2497418107 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5a280b2880 msgr2=0x7f5a280b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:52.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.610+0000 7f5a3f0a5700 1 --2- 172.21.15.1:0/2497418107 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5a280b2880 0x7f5a280b4d90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5a3806b8a0 tx=0x7f5a24009190 comp rx=0 tx=0).stop 2024-05-04T03:15:52.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.610+0000 7f5a3f0a5700 1 -- 172.21.15.1:0/2497418107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a381034c0 msgr2=0x7f5a3819bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:52.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.610+0000 7f5a3f0a5700 1 --2- 172.21.15.1:0/2497418107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a381034c0 0x7f5a3819bc80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00d950 tx=0x7f5a2c00dc60 comp rx=0 tx=0).stop 2024-05-04T03:15:52.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.610+0000 7f5a3f0a5700 1 -- 172.21.15.1:0/2497418107 shutdown_connections 2024-05-04T03:15:52.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.610+0000 7f5a3f0a5700 1 --2- 172.21.15.1:0/2497418107 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5a38108740 0x7f5a3819c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:52.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.610+0000 7f5a3f0a5700 1 --2- 172.21.15.1:0/2497418107 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5a280b2880 0x7f5a280b4d90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:52.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.610+0000 7f5a3f0a5700 1 --2- 172.21.15.1:0/2497418107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a381034c0 0x7f5a3819bc80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:52.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.610+0000 7f5a3f0a5700 1 --2- 172.21.15.1:0/2497418107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a38107df0 0x7f5a3819c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:52.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.610+0000 7f5a3f0a5700 1 -- 172.21.15.1:0/2497418107 >> 172.21.15.1:0/2497418107 conn(0x7f5a380fedf0 msgr2=0x7f5a38107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:52.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.610+0000 7f5a3f0a5700 1 -- 172.21.15.1:0/2497418107 shutdown_connections 2024-05-04T03:15:52.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:52.610+0000 7f5a3f0a5700 1 -- 172.21.15.1:0/2497418107 wait complete. 2024-05-04T03:15:53.222 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:53 smithi001 ceph-mon[29731]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-04T03:15:53.222 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:53 smithi001 ceph-mon[29731]: Cluster is now healthy 2024-05-04T03:15:53.222 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:53 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2497418107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:15:53.253 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-04T03:15:53.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:53 smithi077 ceph-mon[33531]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-04T03:15:53.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:53 smithi077 ceph-mon[33531]: Cluster is now healthy 2024-05-04T03:15:53.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:53 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/2497418107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:15:53.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:53 smithi072 ceph-mon[33332]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-04T03:15:53.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:53 smithi072 ceph-mon[33332]: Cluster is now healthy 2024-05-04T03:15:53.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:53 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/2497418107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:15:53.505 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:15:54.330 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:54 smithi072 ceph-mon[33332]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2024-05-04T03:15:54.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:54 smithi001 ceph-mon[29731]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2024-05-04T03:15:54.428 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:54 smithi077 ceph-mon[33531]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2024-05-04T03:15:54.746 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 04 03:15:54 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-b[63318]: 2024-05-04T03:15:54.416+0000 7f08f83d3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-04T03:15:55.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.005+0000 7f5715a0d700 1 -- 172.21.15.1:0/3292094988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57101045f0 msgr2=0x7f5710104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:55.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.005+0000 7f5715a0d700 1 --2- 172.21.15.1:0/3292094988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57101045f0 0x7f5710104a00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f57000099c0 tx=0x7f5700009cd0 comp rx=0 tx=0).stop 2024-05-04T03:15:55.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.005+0000 7f5715a0d700 1 -- 172.21.15.1:0/3292094988 shutdown_connections 2024-05-04T03:15:55.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.005+0000 7f5715a0d700 1 --2- 172.21.15.1:0/3292094988 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5710109870 0x7f57101111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:55.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.005+0000 7f5715a0d700 1 --2- 172.21.15.1:0/3292094988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57101045f0 0x7f5710104a00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:55.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.005+0000 7f5715a0d700 1 --2- 172.21.15.1:0/3292094988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5710108f20 0x7f5710109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:55.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.005+0000 7f5715a0d700 1 -- 172.21.15.1:0/3292094988 >> 172.21.15.1:0/3292094988 conn(0x7f5710100040 msgr2=0x7f5710102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:55.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.005+0000 7f5715a0d700 1 -- 172.21.15.1:0/3292094988 shutdown_connections 2024-05-04T03:15:55.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.005+0000 7f5715a0d700 1 -- 172.21.15.1:0/3292094988 wait complete. 2024-05-04T03:15:55.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.006+0000 7f5715a0d700 1 Processor -- start 2024-05-04T03:15:55.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.006+0000 7f5715a0d700 1 -- start start 2024-05-04T03:15:55.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.006+0000 7f5715a0d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57101045f0 0x7f5710075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:55.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.006+0000 7f5715a0d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5710108f20 0x7f5710076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:55.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.006+0000 7f5715a0d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5710109870 0x7f5710076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:55.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.006+0000 7f5715a0d700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5710114540 con 0x7f57101045f0 2024-05-04T03:15:55.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.006+0000 7f5715a0d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f57101143c0 con 0x7f5710108f20 2024-05-04T03:15:55.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.006+0000 7f5715a0d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f57101146c0 con 0x7f5710109870 2024-05-04T03:15:55.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.006+0000 7f570effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57101045f0 0x7f5710075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:55.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.006+0000 7f570effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57101045f0 0x7f5710075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33472/0 (socket says 172.21.15.1:33472) 2024-05-04T03:15:55.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.006+0000 7f570effd700 1 -- 172.21.15.1:0/1798685234 learned_addr learned my addr 172.21.15.1:0/1798685234 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:15:55.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.007+0000 7f570f7fe700 1 --2- 172.21.15.1:0/1798685234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5710109870 0x7f5710076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:55.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.007+0000 7f570e7fc700 1 --2- 172.21.15.1:0/1798685234 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5710108f20 0x7f5710076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:55.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.007+0000 7f570effd700 1 -- 172.21.15.1:0/1798685234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5710109870 msgr2=0x7f5710076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:55.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.007+0000 7f570effd700 1 --2- 172.21.15.1:0/1798685234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5710109870 0x7f5710076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:55.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.007+0000 7f570effd700 1 -- 172.21.15.1:0/1798685234 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5710108f20 msgr2=0x7f5710076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:55.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.007+0000 7f570effd700 1 --2- 172.21.15.1:0/1798685234 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5710108f20 0x7f5710076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:55.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.007+0000 7f570effd700 1 -- 172.21.15.1:0/1798685234 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57101a9230 con 0x7f57101045f0 2024-05-04T03:15:55.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.007+0000 7f570f7fe700 1 --2- 172.21.15.1:0/1798685234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5710109870 0x7f5710076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:15:55.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.007+0000 7f570effd700 1 --2- 172.21.15.1:0/1798685234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57101045f0 0x7f5710075ac0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f570000b580 tx=0x7f57000047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:55.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.007+0000 7f5714a0b700 1 -- 172.21.15.1:0/1798685234 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5700027070 con 0x7f57101045f0 2024-05-04T03:15:55.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.007+0000 7f5715a0d700 1 -- 172.21.15.1:0/1798685234 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57101a94c0 con 0x7f57101045f0 2024-05-04T03:15:55.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.007+0000 7f5714a0b700 1 -- 172.21.15.1:0/1798685234 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5700004020 con 0x7f57101045f0 2024-05-04T03:15:55.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.007+0000 7f5714a0b700 1 -- 172.21.15.1:0/1798685234 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5700022e30 con 0x7f57101045f0 2024-05-04T03:15:55.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.007+0000 7f5715a0d700 1 -- 172.21.15.1:0/1798685234 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57101a9910 con 0x7f57101045f0 2024-05-04T03:15:55.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.008+0000 7f5715a0d700 1 -- 172.21.15.1:0/1798685234 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57100512c0 con 0x7f57101045f0 2024-05-04T03:15:55.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.010+0000 7f5714a0b700 1 -- 172.21.15.1:0/1798685234 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f5700022590 con 0x7f57101045f0 2024-05-04T03:15:55.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.011+0000 7f5714a0b700 1 --2- 172.21.15.1:0/1798685234 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f56f80b2880 0x7f56f80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:55.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.011+0000 7f570e7fc700 1 --2- 172.21.15.1:0/1798685234 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f56f80b2880 0x7f56f80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:55.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.011+0000 7f5714a0b700 1 -- 172.21.15.1:0/1798685234 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f57000e3290 con 0x7f57101045f0 2024-05-04T03:15:55.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.011+0000 7f570e7fc700 1 --2- 172.21.15.1:0/1798685234 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f56f80b2880 0x7f56f80b4d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5704005950 tx=0x7f57040058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:55.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.014+0000 7f5714a0b700 1 -- 172.21.15.1:0/1798685234 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5700013070 con 0x7f57101045f0 2024-05-04T03:15:55.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.235+0000 7f5715a0d700 1 -- 172.21.15.1:0/1798685234 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f571002e7b0 con 0x7f57101045f0 2024-05-04T03:15:55.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.237+0000 7f5714a0b700 1 -- 172.21.15.1:0/1798685234 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22088 (secure 0 0 0) 0x7f57000a6070 con 0x7f57101045f0 2024-05-04T03:15:55.239 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:15:55.239 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":81,"fsid":"4a572aa8-09c3-11ef-bc97-c7b262605968","created":"2024-05-04T03:07:32.845547+0000","modified":"2024-05-04T03:15:49.823513+0000","last_up_change":"2024-05-04T03:14:44.192571+0000","last_in_change":"2024-05-04T03:14:23.353342+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T03:11:01.139189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T03:15:43.151696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T03:15:46.753539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"100fe035-2454-4c95-90c7-759956ec36a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6803","nonce":3479011484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6805","nonce":3479011484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6809","nonce":3479011484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6807","nonce":3479011484}]},"public_addr":"172.21.15.1:6803/3479011484","cluster_addr":"172.21.15.1:6805/3479011484","heartbeat_back_addr":"172.21.15.1:6809/3479011484","heartbeat_front_addr":"172.21.15.1:6807/3479011484","state":["exists","up"]},{"osd":1,"uuid":"5826ac76-ef6b-4040-8ad1-305db355af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6801","nonce":3725706149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6803","nonce":3725706149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6807","nonce":3725706149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6805","nonce":3725706149}]},"public_addr":"172.21.15.72:6801/3725706149","cluster_addr":"172.21.15.72:6803/3725706149","heartbeat_back_addr":"172.21.15.72:6807/3725706149","heartbeat_front_addr":"172.21.15.72:6805/3725706149","state":["exists","up"]},{"osd":2,"uuid":"923d76e5-b032-4190-9e35-546d27722a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6801","nonce":515038343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6803","nonce":515038343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6807","nonce":515038343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6805","nonce":515038343}]},"public_addr":"172.21.15.77:6801/515038343","cluster_addr":"172.21.15.77:6803/515038343","heartbeat_back_addr":"172.21.15.77:6807/515038343","heartbeat_front_addr":"172.21.15.77:6805/515038343","state":["exists","up"]},{"osd":3,"uuid":"495cc0c5-6a99-481f-ad9e-38f05fb84356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6811","nonce":4050439365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6813","nonce":4050439365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6817","nonce":4050439365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6815","nonce":4050439365}]},"public_addr":"172.21.15.1:6811/4050439365","cluster_addr":"172.21.15.1:6813/4050439365","heartbeat_back_addr":"172.21.15.1:6817/4050439365","heartbeat_front_addr":"172.21.15.1:6815/4050439365","state":["exists","up"]},{"osd":4,"uuid":"b86b8026-79c0-4056-9e67-a61f54b6ab5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6809","nonce":2501732391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6811","nonce":2501732391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6815","nonce":2501732391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6813","nonce":2501732391}]},"public_addr":"172.21.15.72:6809/2501732391","cluster_addr":"172.21.15.72:6811/2501732391","heartbeat_back_addr":"172.21.15.72:6815/2501732391","heartbeat_front_addr":"172.21.15.72:6813/2501732391","state":["exists","up"]},{"osd":5,"uuid":"80975cb9-39e0-46b8-a343-9ff69194a14e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6809","nonce":3582852165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6811","nonce":3582852165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6815","nonce":3582852165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6813","nonce":3582852165}]},"public_addr":"172.21.15.77:6809/3582852165","cluster_addr":"172.21.15.77:6811/3582852165","heartbeat_back_addr":"172.21.15.77:6815/3582852165","heartbeat_front_addr":"172.21.15.77:6813/3582852165","state":["exists","up"]},{"osd":6,"uuid":"aca3fac5-fbb5-42de-bf0e-e4a0db7381e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6819","nonce":1799033649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6821","nonce":1799033649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6825","nonce":1799033649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6823","nonce":1799033649}]},"public_addr":"172.21.15.1:6819/1799033649","cluster_addr":"172.21.15.1:6821/1799033649","heartbeat_back_addr":"172.21.15.1:6825/1799033649","heartbeat_front_addr":"172.21.15.1:6823/1799033649","state":["exists","up"]},{"osd":7,"uuid":"cbff6b8a-c463-4249-ae45-fdc6f99a5489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6817","nonce":3460835447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6819","nonce":3460835447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6823","nonce":3460835447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6821","nonce":3460835447}]},"public_addr":"172.21.15.72:6817/3460835447","cluster_addr":"172.21.15.72:6819/3460835447","heartbeat_back_addr":"172.21.15.72:6823/3460835447","heartbeat_front_addr":"172.21.15.72:6821/3460835447","state":["exists","up"]},{"osd":8,"uuid":"e140bf13-4887-4d68-9373-3d4fc1c4584a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6817","nonce":1124479323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6819","nonce":1124479323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6823","nonce":1124479323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6821","nonce":1124479323}]},"public_addr":"172.21.15.77:6817/1124479323","cluster_addr":"172.21.15.77:6819/1124479323","heartbeat_back_addr":"172.21.15.77:6823/1124479323","heartbeat_front_addr":"172.21.15.77:6821/1124479323","state":["exists","up"]},{"osd":9,"uuid":"f1e8a232-cb11-4b9d-8789-e1637fa5aaa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6827","nonce":2681322272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6829","nonce":2681322272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6833","nonce":2681322272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6831","nonce":2681322272}]},"public_addr":"172.21.15.1:6827/2681322272","cluster_addr":"172.21.15.1:6829/2681322272","heartbeat_back_addr":"172.21.15.1:6833/2681322272","heartbeat_front_addr":"172.21.15.1:6831/2681322272","state":["exists","up"]},{"osd":10,"uuid":"79b570af-3d1e-439d-a328-9e6bc7d67839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6825","nonce":3293703569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6827","nonce":3293703569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6831","nonce":3293703569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6829","nonce":3293703569}]},"public_addr":"172.21.15.72:6825/3293703569","cluster_addr":"172.21.15.72:6827/3293703569","heartbeat_back_addr":"172.21.15.72:6831/3293703569","heartbeat_front_addr":"172.21.15.72:6829/3293703569","state":["exists","up"]},{"osd":11,"uuid":"e35bf83d-9ee4-4745-96c2-1c35cc8eefba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6825","nonce":4155947548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6827","nonce":4155947548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6831","nonce":4155947548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6829","nonce":4155947548}]},"public_addr":"172.21.15.77:6825/4155947548","cluster_addr":"172.21.15.77:6827/4155947548","heartbeat_back_addr":"172.21.15.77:6831/4155947548","heartbeat_front_addr":"172.21.15.77:6829/4155947548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:09.467126+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:33.861139+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:57.741962+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:11:21.746021+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:11:46.323252+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:12:10.640180+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:12:35.750591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:00.306549+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:25.012495+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:51.185229+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:14:16.403850+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:14:41.653316+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/2029897106":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/402488450":"2024-05-05T03:08:33.045884+0000","172.21.15.1:6801/128793132":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/2233526606":"2024-05-05T03:08:06.135783+0000","172.21.15.1:0/2715541855":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/2415697325":"2024-05-05T03:08:06.135783+0000","172.21.15.1:0/2305298106":"2024-05-05T03:08:06.135783+0000","172.21.15.1:6800/128793132":"2024-05-05T03:08:33.045884+0000","172.21.15.1:6800/4200529863":"2024-05-05T03:08:06.135783+0000","172.21.15.1:6801/4200529863":"2024-05-05T03:08:06.135783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T03:15:55.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.239+0000 7f5715a0d700 1 -- 172.21.15.1:0/1798685234 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f56f80b2880 msgr2=0x7f56f80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:55.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.239+0000 7f5715a0d700 1 --2- 172.21.15.1:0/1798685234 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f56f80b2880 0x7f56f80b4d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5704005950 tx=0x7f57040058e0 comp rx=0 tx=0).stop 2024-05-04T03:15:55.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.239+0000 7f5715a0d700 1 -- 172.21.15.1:0/1798685234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57101045f0 msgr2=0x7f5710075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:55.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.239+0000 7f5715a0d700 1 --2- 172.21.15.1:0/1798685234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57101045f0 0x7f5710075ac0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f570000b580 tx=0x7f57000047b0 comp rx=0 tx=0).stop 2024-05-04T03:15:55.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.239+0000 7f5715a0d700 1 -- 172.21.15.1:0/1798685234 shutdown_connections 2024-05-04T03:15:55.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.239+0000 7f5715a0d700 1 --2- 172.21.15.1:0/1798685234 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f56f80b2880 0x7f56f80b4d90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:55.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.239+0000 7f5715a0d700 1 --2- 172.21.15.1:0/1798685234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5710109870 0x7f5710076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:55.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.239+0000 7f5715a0d700 1 --2- 172.21.15.1:0/1798685234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57101045f0 0x7f5710075ac0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:55.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.239+0000 7f5715a0d700 1 --2- 172.21.15.1:0/1798685234 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5710108f20 0x7f5710076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:55.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.239+0000 7f5715a0d700 1 -- 172.21.15.1:0/1798685234 >> 172.21.15.1:0/1798685234 conn(0x7f5710100040 msgr2=0x7f571010a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:55.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.239+0000 7f5715a0d700 1 -- 172.21.15.1:0/1798685234 shutdown_connections 2024-05-04T03:15:55.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:55.239+0000 7f5715a0d700 1 -- 172.21.15.1:0/1798685234 wait complete. 2024-05-04T03:15:55.945 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-04T03:15:56.200 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:15:56.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:56 smithi072 ceph-mon[33332]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-04T03:15:56.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:56 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/1798685234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:15:56.507 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:56 smithi001 ceph-mon[29731]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-04T03:15:56.507 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:56 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/1798685234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:15:56.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:56 smithi077 ceph-mon[33531]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-04T03:15:56.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:56 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/1798685234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:15:57.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.560+0000 7f8aaa50d700 1 -- 172.21.15.1:0/3893452436 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8aa41034c0 msgr2=0x7f8aa41038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:57.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.560+0000 7f8aaa50d700 1 --2- 172.21.15.1:0/3893452436 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8aa41034c0 0x7f8aa41038d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8a940099c0 tx=0x7f8a94009cd0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.560+0000 7f8aaa50d700 1 -- 172.21.15.1:0/3893452436 shutdown_connections 2024-05-04T03:15:57.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.560+0000 7f8aaa50d700 1 --2- 172.21.15.1:0/3893452436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8aa4108740 0x7f8aa41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.560+0000 7f8aaa50d700 1 --2- 172.21.15.1:0/3893452436 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8aa41034c0 0x7f8aa41038d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.560+0000 7f8aaa50d700 1 --2- 172.21.15.1:0/3893452436 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8aa4107df0 0x7f8aa4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.560+0000 7f8aaa50d700 1 -- 172.21.15.1:0/3893452436 >> 172.21.15.1:0/3893452436 conn(0x7f8aa40fedf0 msgr2=0x7f8aa4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:57.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.560+0000 7f8aaa50d700 1 -- 172.21.15.1:0/3893452436 shutdown_connections 2024-05-04T03:15:57.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.560+0000 7f8aaa50d700 1 -- 172.21.15.1:0/3893452436 wait complete. 2024-05-04T03:15:57.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.561+0000 7f8aaa50d700 1 Processor -- start 2024-05-04T03:15:57.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.561+0000 7f8aaa50d700 1 -- start start 2024-05-04T03:15:57.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.562+0000 7f8aaa50d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8aa41034c0 0x7f8aa4199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:57.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.562+0000 7f8aaa50d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8aa4107df0 0x7f8aa419a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:57.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.562+0000 7f8aaa50d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8aa4108740 0x7f8aa419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:57.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.562+0000 7f8aaa50d700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa4113390 con 0x7f8aa4108740 2024-05-04T03:15:57.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.562+0000 7f8aaa50d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa4113210 con 0x7f8aa4107df0 2024-05-04T03:15:57.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.562+0000 7f8aaa50d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa4113510 con 0x7f8aa41034c0 2024-05-04T03:15:57.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.562+0000 7f8aa8aaa700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8aa4108740 0x7f8aa419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:57.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.562+0000 7f8aa8aaa700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8aa4108740 0x7f8aa419e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33484/0 (socket says 172.21.15.1:33484) 2024-05-04T03:15:57.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.562+0000 7f8aa8aaa700 1 -- 172.21.15.1:0/4145059502 learned_addr learned my addr 172.21.15.1:0/4145059502 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:15:57.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.562+0000 7f8aa37fe700 1 --2- 172.21.15.1:0/4145059502 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8aa4107df0 0x7f8aa419a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:57.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.562+0000 7f8aa3fff700 1 --2- 172.21.15.1:0/4145059502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8aa41034c0 0x7f8aa4199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:57.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.562+0000 7f8aa8aaa700 1 -- 172.21.15.1:0/4145059502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8aa41034c0 msgr2=0x7f8aa4199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:57.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.562+0000 7f8aa8aaa700 1 --2- 172.21.15.1:0/4145059502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8aa41034c0 0x7f8aa4199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.562+0000 7f8aa8aaa700 1 -- 172.21.15.1:0/4145059502 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8aa4107df0 msgr2=0x7f8aa419a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:57.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.562+0000 7f8aa8aaa700 1 --2- 172.21.15.1:0/4145059502 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8aa4107df0 0x7f8aa419a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:57.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.562+0000 7f8aa8aaa700 1 -- 172.21.15.1:0/4145059502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa419ec90 con 0x7f8aa4108740 2024-05-04T03:15:57.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.563+0000 7f8aa8aaa700 1 --2- 172.21.15.1:0/4145059502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8aa4108740 0x7f8aa419e5f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c007ee0 tx=0x7f8a9c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:57.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.563+0000 7f8aa17fa700 1 -- 172.21.15.1:0/4145059502 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c004bb0 con 0x7f8aa4108740 2024-05-04T03:15:57.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.563+0000 7f8aa17fa700 1 -- 172.21.15.1:0/4145059502 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a9c004d50 con 0x7f8aa4108740 2024-05-04T03:15:57.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.563+0000 7f8aaa50d700 1 -- 172.21.15.1:0/4145059502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa419ef80 con 0x7f8aa4108740 2024-05-04T03:15:57.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.563+0000 7f8aa17fa700 1 -- 172.21.15.1:0/4145059502 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c005920 con 0x7f8aa4108740 2024-05-04T03:15:57.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.563+0000 7f8aaa50d700 1 -- 172.21.15.1:0/4145059502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa41a68a0 con 0x7f8aa4108740 2024-05-04T03:15:57.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.564+0000 7f8aaa50d700 1 -- 172.21.15.1:0/4145059502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aa40512c0 con 0x7f8aa4108740 2024-05-04T03:15:57.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.566+0000 7f8aa17fa700 1 -- 172.21.15.1:0/4145059502 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f8a9c01a020 con 0x7f8aa4108740 2024-05-04T03:15:57.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.566+0000 7f8aa17fa700 1 --2- 172.21.15.1:0/4145059502 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f8a8c0b2900 0x7f8a8c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:15:57.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.566+0000 7f8aa3fff700 1 --2- 172.21.15.1:0/4145059502 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f8a8c0b2900 0x7f8a8c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:15:57.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.566+0000 7f8aa17fa700 1 -- 172.21.15.1:0/4145059502 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f8a9c0e1cb0 con 0x7f8aa4108740 2024-05-04T03:15:57.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.567+0000 7f8aa3fff700 1 --2- 172.21.15.1:0/4145059502 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f8a8c0b2900 0x7f8a8c0b4e10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009990 tx=0x7f8a940036e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:15:57.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.570+0000 7f8aa17fa700 1 -- 172.21.15.1:0/4145059502 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8a9c010070 con 0x7f8aa4108740 2024-05-04T03:15:57.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:57.830+0000 7f8aaa50d700 1 -- 172.21.15.1:0/4145059502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f8aa402e5f0 con 0x7f8aa4108740 2024-05-04T03:15:58.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:58.192+0000 7f8aa17fa700 1 -- 172.21.15.1:0/4145059502 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f8a9c0a9b20 con 0x7f8aa4108740 2024-05-04T03:15:58.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:58.195+0000 7f8aaa50d700 1 -- 172.21.15.1:0/4145059502 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f8a8c0b2900 msgr2=0x7f8a8c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:58.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:58.195+0000 7f8aaa50d700 1 --2- 172.21.15.1:0/4145059502 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f8a8c0b2900 0x7f8a8c0b4e10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009990 tx=0x7f8a940036e0 comp rx=0 tx=0).stop 2024-05-04T03:15:58.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:58.195+0000 7f8aaa50d700 1 -- 172.21.15.1:0/4145059502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8aa4108740 msgr2=0x7f8aa419e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:15:58.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:58.195+0000 7f8aaa50d700 1 --2- 172.21.15.1:0/4145059502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8aa4108740 0x7f8aa419e5f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c007ee0 tx=0x7f8a9c00d3b0 comp rx=0 tx=0).stop 2024-05-04T03:15:58.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:58.195+0000 7f8aaa50d700 1 -- 172.21.15.1:0/4145059502 shutdown_connections 2024-05-04T03:15:58.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:58.195+0000 7f8aaa50d700 1 --2- 172.21.15.1:0/4145059502 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f8a8c0b2900 0x7f8a8c0b4e10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:58.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:58.195+0000 7f8aaa50d700 1 --2- 172.21.15.1:0/4145059502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8aa4108740 0x7f8aa419e5f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:58.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:58.195+0000 7f8aaa50d700 1 --2- 172.21.15.1:0/4145059502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8aa41034c0 0x7f8aa4199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:58.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:58.195+0000 7f8aaa50d700 1 --2- 172.21.15.1:0/4145059502 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8aa4107df0 0x7f8aa419a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:15:58.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:58.195+0000 7f8aaa50d700 1 -- 172.21.15.1:0/4145059502 >> 172.21.15.1:0/4145059502 conn(0x7f8aa40fedf0 msgr2=0x7f8aa41075d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:15:58.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:58.195+0000 7f8aaa50d700 1 -- 172.21.15.1:0/4145059502 shutdown_connections 2024-05-04T03:15:58.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:15:58.195+0000 7f8aaa50d700 1 -- 172.21.15.1:0/4145059502 wait complete. 2024-05-04T03:15:58.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:58 smithi072 ceph-mon[33332]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-04T03:15:58.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:58 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/4145059502' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-04T03:15:58.522 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:58 smithi001 ceph-mon[29731]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-04T03:15:58.523 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:58 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/4145059502' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-04T03:15:58.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:58 smithi077 ceph-mon[33531]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-04T03:15:58.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:58 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/4145059502' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-04T03:15:58.933 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-05-04T03:15:59.086 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-05-04T03:15:59.349 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:15:59.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:59 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/4145059502' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-04T03:15:59.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:15:59 smithi001 ceph-mon[29731]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-04T03:15:59.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:59 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/4145059502' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-04T03:15:59.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:15:59 smithi072 ceph-mon[33332]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-04T03:15:59.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:59 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/4145059502' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-04T03:15:59.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:15:59 smithi077 ceph-mon[33531]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-04T03:16:00.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:00 smithi001 ceph-mon[29731]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-04T03:16:00.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:00 smithi072 ceph-mon[33332]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-04T03:16:00.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:00 smithi077 ceph-mon[33531]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-04T03:16:01.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.034+0000 7f532164b700 1 -- 172.21.15.1:0/3382755813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f531c1034c0 msgr2=0x7f531c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:01.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.034+0000 7f532164b700 1 --2- 172.21.15.1:0/3382755813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f531c1034c0 0x7f531c1038d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f531400b1d0 tx=0x7f531400b4e0 comp rx=0 tx=0).stop 2024-05-04T03:16:01.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.034+0000 7f532164b700 1 -- 172.21.15.1:0/3382755813 shutdown_connections 2024-05-04T03:16:01.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.034+0000 7f532164b700 1 --2- 172.21.15.1:0/3382755813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f531c108740 0x7f531c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:01.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.034+0000 7f532164b700 1 --2- 172.21.15.1:0/3382755813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f531c1034c0 0x7f531c1038d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:01.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.034+0000 7f532164b700 1 --2- 172.21.15.1:0/3382755813 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f531c107df0 0x7f531c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:01.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.034+0000 7f532164b700 1 -- 172.21.15.1:0/3382755813 >> 172.21.15.1:0/3382755813 conn(0x7f531c0fedc0 msgr2=0x7f531c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:01.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.034+0000 7f532164b700 1 -- 172.21.15.1:0/3382755813 shutdown_connections 2024-05-04T03:16:01.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.034+0000 7f532164b700 1 -- 172.21.15.1:0/3382755813 wait complete. 2024-05-04T03:16:01.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.035+0000 7f532164b700 1 Processor -- start 2024-05-04T03:16:01.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.035+0000 7f532164b700 1 -- start start 2024-05-04T03:16:01.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.035+0000 7f532164b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f531c1034c0 0x7f531c19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:01.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.035+0000 7f532164b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f531c107df0 0x7f531c19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:01.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.035+0000 7f532164b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f531c108740 0x7f531c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:01.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.035+0000 7f532164b700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f531c1122d0 con 0x7f531c107df0 2024-05-04T03:16:01.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.035+0000 7f532164b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f531c112150 con 0x7f531c108740 2024-05-04T03:16:01.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.035+0000 7f532164b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f531c112450 con 0x7f531c1034c0 2024-05-04T03:16:01.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.036+0000 7f531a7fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f531c107df0 0x7f531c19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:01.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.036+0000 7f531a7fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f531c107df0 0x7f531c19ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33508/0 (socket says 172.21.15.1:33508) 2024-05-04T03:16:01.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.036+0000 7f531affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f531c1034c0 0x7f531c19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:01.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.036+0000 7f531a7fc700 1 -- 172.21.15.1:0/3602034515 learned_addr learned my addr 172.21.15.1:0/3602034515 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:16:01.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.036+0000 7f531b7fe700 1 --2- 172.21.15.1:0/3602034515 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f531c108740 0x7f531c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:01.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.036+0000 7f531affd700 1 -- 172.21.15.1:0/3602034515 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f531c108740 msgr2=0x7f531c1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:01.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.036+0000 7f531affd700 1 --2- 172.21.15.1:0/3602034515 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f531c108740 0x7f531c1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:01.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.036+0000 7f531affd700 1 -- 172.21.15.1:0/3602034515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f531c107df0 msgr2=0x7f531c19ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:01.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.036+0000 7f531affd700 1 --2- 172.21.15.1:0/3602034515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f531c107df0 0x7f531c19ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:01.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.036+0000 7f531affd700 1 -- 172.21.15.1:0/3602034515 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f531c1990a0 con 0x7f531c1034c0 2024-05-04T03:16:01.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.036+0000 7f531b7fe700 1 --2- 172.21.15.1:0/3602034515 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f531c108740 0x7f531c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:16:01.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.036+0000 7f531affd700 1 --2- 172.21.15.1:0/3602034515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f531c1034c0 0x7f531c19e7d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f531400b1d0 tx=0x7f5314008d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:01.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.036+0000 7f5313fff700 1 -- 172.21.15.1:0/3602034515 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5314003c10 con 0x7f531c1034c0 2024-05-04T03:16:01.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.036+0000 7f532164b700 1 -- 172.21.15.1:0/3602034515 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f531c199330 con 0x7f531c1034c0 2024-05-04T03:16:01.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.037+0000 7f532164b700 1 -- 172.21.15.1:0/3602034515 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f531c1a5660 con 0x7f531c1034c0 2024-05-04T03:16:01.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.037+0000 7f5313fff700 1 -- 172.21.15.1:0/3602034515 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5314003db0 con 0x7f531c1034c0 2024-05-04T03:16:01.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.037+0000 7f5313fff700 1 -- 172.21.15.1:0/3602034515 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53140052a0 con 0x7f531c1034c0 2024-05-04T03:16:01.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.037+0000 7f532164b700 1 -- 172.21.15.1:0/3602034515 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f531c0512c0 con 0x7f531c1034c0 2024-05-04T03:16:01.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.040+0000 7f5313fff700 1 -- 172.21.15.1:0/3602034515 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f5314029060 con 0x7f531c1034c0 2024-05-04T03:16:01.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.040+0000 7f5313fff700 1 --2- 172.21.15.1:0/3602034515 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f53080b2880 0x7f53080b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:01.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.040+0000 7f531a7fc700 1 --2- 172.21.15.1:0/3602034515 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f53080b2880 0x7f53080b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:01.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.040+0000 7f5313fff700 1 -- 172.21.15.1:0/3602034515 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f53140e4f10 con 0x7f531c1034c0 2024-05-04T03:16:01.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.041+0000 7f531a7fc700 1 --2- 172.21.15.1:0/3602034515 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f53080b2880 0x7f53080b4d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f531c102360 tx=0x7f530400a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:01.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.044+0000 7f5313fff700 1 -- 172.21.15.1:0/3602034515 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f531401f070 con 0x7f531c1034c0 2024-05-04T03:16:01.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:01.309+0000 7f532164b700 1 -- 172.21.15.1:0/3602034515 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f531c199f20 con 0x7f531c1034c0 2024-05-04T03:16:02.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:02.218+0000 7f5313fff700 1 -- 172.21.15.1:0/3602034515 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f5314015020 con 0x7f531c1034c0 2024-05-04T03:16:02.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:02.221+0000 7f532164b700 1 -- 172.21.15.1:0/3602034515 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f53080b2880 msgr2=0x7f53080b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:02.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:02.221+0000 7f532164b700 1 --2- 172.21.15.1:0/3602034515 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f53080b2880 0x7f53080b4d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f531c102360 tx=0x7f530400a3b0 comp rx=0 tx=0).stop 2024-05-04T03:16:02.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:02.221+0000 7f532164b700 1 -- 172.21.15.1:0/3602034515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f531c1034c0 msgr2=0x7f531c19e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:02.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:02.221+0000 7f532164b700 1 --2- 172.21.15.1:0/3602034515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f531c1034c0 0x7f531c19e7d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f531400b1d0 tx=0x7f5314008d90 comp rx=0 tx=0).stop 2024-05-04T03:16:02.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:02.221+0000 7f532164b700 1 -- 172.21.15.1:0/3602034515 shutdown_connections 2024-05-04T03:16:02.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:02.221+0000 7f532164b700 1 --2- 172.21.15.1:0/3602034515 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f531c108740 0x7f531c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:02.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:02.221+0000 7f532164b700 1 --2- 172.21.15.1:0/3602034515 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f53080b2880 0x7f53080b4d90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:02.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:02.221+0000 7f532164b700 1 --2- 172.21.15.1:0/3602034515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f531c1034c0 0x7f531c19e7d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:02.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:02.221+0000 7f532164b700 1 --2- 172.21.15.1:0/3602034515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f531c107df0 0x7f531c19ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:02.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:02.221+0000 7f532164b700 1 -- 172.21.15.1:0/3602034515 >> 172.21.15.1:0/3602034515 conn(0x7f531c0fedc0 msgr2=0x7f531c1075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:02.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:02.222+0000 7f532164b700 1 -- 172.21.15.1:0/3602034515 shutdown_connections 2024-05-04T03:16:02.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:02.222+0000 7f532164b700 1 -- 172.21.15.1:0/3602034515 wait complete. 2024-05-04T03:16:02.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:02 smithi072 ceph-mon[33332]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-04T03:16:02.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:02 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/3602034515' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-04T03:16:02.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:02 smithi072 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-04T03:16:02.555 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:02 smithi001 ceph-mon[29731]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-04T03:16:02.555 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:02 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3602034515' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-04T03:16:02.555 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:02 smithi001 ceph-mon[29731]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-04T03:16:02.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:02 smithi077 ceph-mon[33531]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-04T03:16:02.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:02 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/3602034515' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-04T03:16:02.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:02 smithi077 ceph-mon[33531]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-04T03:16:02.954 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-04T03:16:03.209 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:03.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:03 smithi072 ceph-mon[33332]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-04T03:16:03.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:03 smithi072 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-04T03:16:03.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:03 smithi072 ceph-mon[33332]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-04T03:16:03.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:03 smithi072 ceph-mon[33332]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-04T03:16:03.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:03 smithi072 ceph-mon[33332]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-04T03:16:03.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:03 smithi072 ceph-mon[33332]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-04T03:16:03.539 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:03 smithi001 ceph-mon[29731]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-04T03:16:03.540 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:03 smithi001 ceph-mon[29731]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-04T03:16:03.540 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:03 smithi001 ceph-mon[29731]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-04T03:16:03.540 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:03 smithi001 ceph-mon[29731]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-04T03:16:03.540 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:03 smithi001 ceph-mon[29731]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-04T03:16:03.540 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:03 smithi001 ceph-mon[29731]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-04T03:16:03.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:03 smithi077 ceph-mon[33531]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-04T03:16:03.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:03 smithi077 ceph-mon[33531]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-04T03:16:03.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:03 smithi077 ceph-mon[33531]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-04T03:16:03.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:03 smithi077 ceph-mon[33531]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-04T03:16:03.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:03 smithi077 ceph-mon[33531]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-04T03:16:03.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:03 smithi077 ceph-mon[33531]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-04T03:16:04.302 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:04 smithi001 ceph-mon[29731]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-05-04T03:16:04.302 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:04 smithi001 ceph-mon[29731]: mds.? [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] up:active 2024-05-04T03:16:04.302 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:04 smithi001 ceph-mon[29731]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-04T03:16:04.302 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:04 smithi001 ceph-mon[29731]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-04T03:16:04.302 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:04 smithi001 ceph-mon[29731]: Cluster is now healthy 2024-05-04T03:16:04.302 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:04 smithi001 ceph-mon[29731]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-04T03:16:04.302 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:04 smithi001 ceph-mon[29731]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-05-04T03:16:04.302 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:04 smithi001 ceph-mon[29731]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-05-04T03:16:04.496 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 04 03:16:04 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-b[63318]: 2024-05-04T03:16:04.417+0000 7f08f83d3700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-04T03:16:04.496 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 04 03:16:04 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-b[63318]: 2024-05-04T03:16:04.417+0000 7f08f83d3700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-04T03:16:04.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:04 smithi072 ceph-mon[33332]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-05-04T03:16:04.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:04 smithi072 ceph-mon[33332]: mds.? [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] up:active 2024-05-04T03:16:04.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:04 smithi072 ceph-mon[33332]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-04T03:16:04.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:04 smithi072 ceph-mon[33332]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-04T03:16:04.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:04 smithi072 ceph-mon[33332]: Cluster is now healthy 2024-05-04T03:16:04.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:04 smithi072 ceph-mon[33332]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-04T03:16:04.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:04 smithi072 ceph-mon[33332]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-05-04T03:16:04.497 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:04 smithi072 ceph-mon[33332]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-05-04T03:16:04.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:04 smithi077 ceph-mon[33531]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 4 op/s 2024-05-04T03:16:04.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:04 smithi077 ceph-mon[33531]: mds.? [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] up:active 2024-05-04T03:16:04.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:04 smithi077 ceph-mon[33531]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-04T03:16:04.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:04 smithi077 ceph-mon[33531]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-04T03:16:04.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:04 smithi077 ceph-mon[33531]: Cluster is now healthy 2024-05-04T03:16:04.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:04 smithi077 ceph-mon[33531]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-04T03:16:04.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:04 smithi077 ceph-mon[33531]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-05-04T03:16:04.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:04 smithi077 ceph-mon[33531]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-05-04T03:16:04.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.794+0000 7f54c7ab6700 1 -- 172.21.15.1:0/2298587211 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c01054c0 msgr2=0x7f54c01058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:04.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.794+0000 7f54c7ab6700 1 --2- 172.21.15.1:0/2298587211 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c01054c0 0x7f54c01058d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f54bc009990 tx=0x7f54bc009ca0 comp rx=0 tx=0).stop 2024-05-04T03:16:04.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.794+0000 7f54c7ab6700 1 -- 172.21.15.1:0/2298587211 shutdown_connections 2024-05-04T03:16:04.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.794+0000 7f54c7ab6700 1 --2- 172.21.15.1:0/2298587211 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54c010e8f0 0x7f54c0110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:04.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.794+0000 7f54c7ab6700 1 --2- 172.21.15.1:0/2298587211 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c01054c0 0x7f54c01058d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:04.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.794+0000 7f54c7ab6700 1 --2- 172.21.15.1:0/2298587211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54c0105e10 0x7f54c010e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:04.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.794+0000 7f54c7ab6700 1 -- 172.21.15.1:0/2298587211 >> 172.21.15.1:0/2298587211 conn(0x7f54c00fedf0 msgr2=0x7f54c0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:04.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.794+0000 7f54c7ab6700 1 -- 172.21.15.1:0/2298587211 shutdown_connections 2024-05-04T03:16:04.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.794+0000 7f54c7ab6700 1 -- 172.21.15.1:0/2298587211 wait complete. 2024-05-04T03:16:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.795+0000 7f54c7ab6700 1 Processor -- start 2024-05-04T03:16:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.795+0000 7f54c7ab6700 1 -- start start 2024-05-04T03:16:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.796+0000 7f54c7ab6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c01054c0 0x7f54c019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.796+0000 7f54c7ab6700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54c0105e10 0x7f54c019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.796+0000 7f54c7ab6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54c010e8f0 0x7f54c019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.796+0000 7f54c7ab6700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f54c0114440 con 0x7f54c01054c0 2024-05-04T03:16:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.796+0000 7f54c7ab6700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f54c01142c0 con 0x7f54c0105e10 2024-05-04T03:16:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.796+0000 7f54c7ab6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f54c01145c0 con 0x7f54c010e8f0 2024-05-04T03:16:04.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.796+0000 7f54c5852700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c01054c0 0x7f54c019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.796+0000 7f54c5852700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c01054c0 0x7f54c019ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38488/0 (socket says 172.21.15.1:38488) 2024-05-04T03:16:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.796+0000 7f54c5852700 1 -- 172.21.15.1:0/3773367563 learned_addr learned my addr 172.21.15.1:0/3773367563 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:16:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.796+0000 7f54c5051700 1 --2- 172.21.15.1:0/3773367563 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54c0105e10 0x7f54c019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.796+0000 7f54c6053700 1 --2- 172.21.15.1:0/3773367563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54c010e8f0 0x7f54c019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.796+0000 7f54c5051700 1 -- 172.21.15.1:0/3773367563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54c010e8f0 msgr2=0x7f54c019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.796+0000 7f54c5051700 1 --2- 172.21.15.1:0/3773367563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54c010e8f0 0x7f54c019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.796+0000 7f54c5051700 1 -- 172.21.15.1:0/3773367563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c01054c0 msgr2=0x7f54c019ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.796+0000 7f54c5051700 1 --2- 172.21.15.1:0/3773367563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c01054c0 0x7f54c019ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.796+0000 7f54c5051700 1 -- 172.21.15.1:0/3773367563 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c019fce0 con 0x7f54c0105e10 2024-05-04T03:16:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.796+0000 7f54c5852700 1 --2- 172.21.15.1:0/3773367563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c01054c0 0x7f54c019ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:16:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.796+0000 7f54c5051700 1 --2- 172.21.15.1:0/3773367563 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54c0105e10 0x7f54c019b3b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f54b000d950 tx=0x7f54b000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.797+0000 7f54b6ffd700 1 -- 172.21.15.1:0/3773367563 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b0017070 con 0x7f54c0105e10 2024-05-04T03:16:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.797+0000 7f54c7ab6700 1 -- 172.21.15.1:0/3773367563 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c019ffd0 con 0x7f54c0105e10 2024-05-04T03:16:04.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.797+0000 7f54b6ffd700 1 -- 172.21.15.1:0/3773367563 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54b0009de0 con 0x7f54c0105e10 2024-05-04T03:16:04.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.797+0000 7f54b6ffd700 1 -- 172.21.15.1:0/3773367563 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b0010d90 con 0x7f54c0105e10 2024-05-04T03:16:04.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.797+0000 7f54c7ab6700 1 -- 172.21.15.1:0/3773367563 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c01a78f0 con 0x7f54c0105e10 2024-05-04T03:16:04.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.798+0000 7f54c7ab6700 1 -- 172.21.15.1:0/3773367563 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54c00512c0 con 0x7f54c0105e10 2024-05-04T03:16:04.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.799+0000 7f54b6ffd700 1 -- 172.21.15.1:0/3773367563 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f54b001a070 con 0x7f54c0105e10 2024-05-04T03:16:04.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.800+0000 7f54b6ffd700 1 --2- 172.21.15.1:0/3773367563 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f54ac0b2880 0x7f54ac0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:04.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.800+0000 7f54c5852700 1 --2- 172.21.15.1:0/3773367563 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f54ac0b2880 0x7f54ac0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:04.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.800+0000 7f54b6ffd700 1 -- 172.21.15.1:0/3773367563 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f54b00e1af0 con 0x7f54c0105e10 2024-05-04T03:16:04.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.800+0000 7f54c5852700 1 --2- 172.21.15.1:0/3773367563 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f54ac0b2880 0x7f54ac0b4d90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f54bc006010 tx=0x7f54bc00b2a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:04.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:04.803+0000 7f54b6ffd700 1 -- 172.21.15.1:0/3773367563 <== mon.2 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f54b00a98c0 con 0x7f54c0105e10 2024-05-04T03:16:05.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:05.071+0000 7f54c7ab6700 1 -- 172.21.15.1:0/3773367563 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f54c019c170 con 0x7f54c0105e10 2024-05-04T03:16:05.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:05.241+0000 7f54b6ffd700 1 -- 172.21.15.1:0/3773367563 <== mon.2 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f54b00a91c0 con 0x7f54c0105e10 2024-05-04T03:16:05.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:05.245+0000 7f54c7ab6700 1 -- 172.21.15.1:0/3773367563 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f54ac0b2880 msgr2=0x7f54ac0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:05.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:05.245+0000 7f54c7ab6700 1 --2- 172.21.15.1:0/3773367563 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f54ac0b2880 0x7f54ac0b4d90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f54bc006010 tx=0x7f54bc00b2a0 comp rx=0 tx=0).stop 2024-05-04T03:16:05.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:05.245+0000 7f54c7ab6700 1 -- 172.21.15.1:0/3773367563 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54c0105e10 msgr2=0x7f54c019b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:05.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:05.245+0000 7f54c7ab6700 1 --2- 172.21.15.1:0/3773367563 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54c0105e10 0x7f54c019b3b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f54b000d950 tx=0x7f54b000dc60 comp rx=0 tx=0).stop 2024-05-04T03:16:05.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:05.245+0000 7f54c7ab6700 1 -- 172.21.15.1:0/3773367563 shutdown_connections 2024-05-04T03:16:05.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:05.245+0000 7f54c7ab6700 1 --2- 172.21.15.1:0/3773367563 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f54ac0b2880 0x7f54ac0b4d90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:05.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:05.245+0000 7f54c7ab6700 1 --2- 172.21.15.1:0/3773367563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54c010e8f0 0x7f54c019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:05.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:05.245+0000 7f54c7ab6700 1 --2- 172.21.15.1:0/3773367563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c01054c0 0x7f54c019ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:05.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:05.245+0000 7f54c7ab6700 1 --2- 172.21.15.1:0/3773367563 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54c0105e10 0x7f54c019b3b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:05.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:05.245+0000 7f54c7ab6700 1 -- 172.21.15.1:0/3773367563 >> 172.21.15.1:0/3773367563 conn(0x7f54c00fedf0 msgr2=0x7f54c0100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:05.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:05.245+0000 7f54c7ab6700 1 -- 172.21.15.1:0/3773367563 shutdown_connections 2024-05-04T03:16:05.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:05.245+0000 7f54c7ab6700 1 -- 172.21.15.1:0/3773367563 wait complete. 2024-05-04T03:16:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:05 smithi001 ceph-mon[29731]: mds.? [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] up:active 2024-05-04T03:16:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:05 smithi001 ceph-mon[29731]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-04T03:16:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:05 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3773367563' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-04T03:16:05.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:05 smithi001 ceph-mon[29731]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-04T03:16:05.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:05 smithi072 ceph-mon[33332]: mds.? [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] up:active 2024-05-04T03:16:05.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:05 smithi072 ceph-mon[33332]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-04T03:16:05.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:05 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/3773367563' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-04T03:16:05.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:05 smithi072 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-04T03:16:05.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:05 smithi077 ceph-mon[33531]: mds.? [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] up:active 2024-05-04T03:16:05.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:05 smithi077 ceph-mon[33531]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-04T03:16:05.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:05 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/3773367563' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-04T03:16:05.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:05 smithi077 ceph-mon[33531]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-04T03:16:05.899 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-04T03:16:06.151 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:06.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:06 smithi072 ceph-mon[33332]: pgmap v297: 97 pgs: 97 active+clean; 581 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-05-04T03:16:06.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:06 smithi072 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-04T03:16:06.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:06 smithi072 ceph-mon[33332]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-04T03:16:06.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:06 smithi072 ceph-mon[33332]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:16:06.586 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:06 smithi001 ceph-mon[29731]: pgmap v297: 97 pgs: 97 active+clean; 581 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-05-04T03:16:06.586 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:06 smithi001 ceph-mon[29731]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-04T03:16:06.586 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:06 smithi001 ceph-mon[29731]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-04T03:16:06.586 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:06 smithi001 ceph-mon[29731]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:16:06.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:06 smithi077 ceph-mon[33531]: pgmap v297: 97 pgs: 97 active+clean; 581 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-05-04T03:16:06.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:06 smithi077 ceph-mon[33531]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-04T03:16:06.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:06 smithi077 ceph-mon[33531]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-05-04T03:16:06.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:06 smithi077 ceph-mon[33531]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:16:07.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.640+0000 7f5b54013700 1 -- 172.21.15.1:0/1037788745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b4c1034e0 msgr2=0x7f5b4c1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:07.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.640+0000 7f5b54013700 1 --2- 172.21.15.1:0/1037788745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b4c1034e0 0x7f5b4c1038f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5b480099c0 tx=0x7f5b48009cd0 comp rx=0 tx=0).stop 2024-05-04T03:16:07.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.641+0000 7f5b54013700 1 -- 172.21.15.1:0/1037788745 shutdown_connections 2024-05-04T03:16:07.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.641+0000 7f5b54013700 1 --2- 172.21.15.1:0/1037788745 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b4c108760 0x7f5b4c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:07.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.641+0000 7f5b54013700 1 --2- 172.21.15.1:0/1037788745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b4c1034e0 0x7f5b4c1038f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:07.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.641+0000 7f5b54013700 1 --2- 172.21.15.1:0/1037788745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b4c107e10 0x7f5b4c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:07.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.641+0000 7f5b54013700 1 -- 172.21.15.1:0/1037788745 >> 172.21.15.1:0/1037788745 conn(0x7f5b4c0fef50 msgr2=0x7f5b4c101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:07.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.641+0000 7f5b54013700 1 -- 172.21.15.1:0/1037788745 shutdown_connections 2024-05-04T03:16:07.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.641+0000 7f5b54013700 1 -- 172.21.15.1:0/1037788745 wait complete. 2024-05-04T03:16:07.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.641+0000 7f5b54013700 1 Processor -- start 2024-05-04T03:16:07.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.641+0000 7f5b54013700 1 -- start start 2024-05-04T03:16:07.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.642+0000 7f5b54013700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b4c1034e0 0x7f5b4c110aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:07.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.642+0000 7f5b54013700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b4c107e10 0x7f5b4c110fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:07.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.642+0000 7f5b54013700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b4c108760 0x7f5b4c111520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:07.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.642+0000 7f5b54013700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c1144e0 con 0x7f5b4c108760 2024-05-04T03:16:07.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.642+0000 7f5b54013700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c114360 con 0x7f5b4c1034e0 2024-05-04T03:16:07.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.642+0000 7f5b54013700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c114660 con 0x7f5b4c107e10 2024-05-04T03:16:07.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.642+0000 7f5b525b0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b4c108760 0x7f5b4c111520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:07.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.642+0000 7f5b525b0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b4c108760 0x7f5b4c111520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38504/0 (socket says 172.21.15.1:38504) 2024-05-04T03:16:07.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.642+0000 7f5b525b0700 1 -- 172.21.15.1:0/3655069463 learned_addr learned my addr 172.21.15.1:0/3655069463 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:16:07.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.642+0000 7f5b515ae700 1 --2- 172.21.15.1:0/3655069463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b4c107e10 0x7f5b4c110fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:07.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.642+0000 7f5b51daf700 1 --2- 172.21.15.1:0/3655069463 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b4c1034e0 0x7f5b4c110aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:07.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.642+0000 7f5b525b0700 1 -- 172.21.15.1:0/3655069463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b4c107e10 msgr2=0x7f5b4c110fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:07.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.642+0000 7f5b525b0700 1 --2- 172.21.15.1:0/3655069463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b4c107e10 0x7f5b4c110fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:07.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.642+0000 7f5b525b0700 1 -- 172.21.15.1:0/3655069463 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b4c1034e0 msgr2=0x7f5b4c110aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:07.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.642+0000 7f5b525b0700 1 --2- 172.21.15.1:0/3655069463 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b4c1034e0 0x7f5b4c110aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:07.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.642+0000 7f5b525b0700 1 -- 172.21.15.1:0/3655069463 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4c1a9060 con 0x7f5b4c108760 2024-05-04T03:16:07.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.643+0000 7f5b525b0700 1 --2- 172.21.15.1:0/3655069463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b4c108760 0x7f5b4c111520 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5b44007d70 tx=0x7f5b4400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:07.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.643+0000 7f5b42ffd700 1 -- 172.21.15.1:0/3655069463 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44016020 con 0x7f5b4c108760 2024-05-04T03:16:07.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.643+0000 7f5b42ffd700 1 -- 172.21.15.1:0/3655069463 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b4400ac90 con 0x7f5b4c108760 2024-05-04T03:16:07.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.643+0000 7f5b54013700 1 -- 172.21.15.1:0/3655069463 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b4c1a9350 con 0x7f5b4c108760 2024-05-04T03:16:07.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.643+0000 7f5b42ffd700 1 -- 172.21.15.1:0/3655069463 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b440115e0 con 0x7f5b4c108760 2024-05-04T03:16:07.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.643+0000 7f5b54013700 1 -- 172.21.15.1:0/3655069463 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b4c1a9840 con 0x7f5b4c108760 2024-05-04T03:16:07.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.644+0000 7f5b54013700 1 -- 172.21.15.1:0/3655069463 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b4c0512c0 con 0x7f5b4c108760 2024-05-04T03:16:07.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.645+0000 7f5b42ffd700 1 -- 172.21.15.1:0/3655069463 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f5b44019020 con 0x7f5b4c108760 2024-05-04T03:16:07.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.646+0000 7f5b42ffd700 1 --2- 172.21.15.1:0/3655069463 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5b380b2880 0x7f5b380b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:07.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.647+0000 7f5b51daf700 1 --2- 172.21.15.1:0/3655069463 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5b380b2880 0x7f5b380b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:07.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.647+0000 7f5b42ffd700 1 -- 172.21.15.1:0/3655069463 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f5b440e1f90 con 0x7f5b4c108760 2024-05-04T03:16:07.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.647+0000 7f5b51daf700 1 --2- 172.21.15.1:0/3655069463 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5b380b2880 0x7f5b380b4d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009990 tx=0x7f5b48000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:07.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.652+0000 7f5b42ffd700 1 -- 172.21.15.1:0/3655069463 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5b44020020 con 0x7f5b4c108760 2024-05-04T03:16:07.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:07.916+0000 7f5b54013700 1 -- 172.21.15.1:0/3655069463 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f5b4c02e5f0 con 0x7f5b4c108760 2024-05-04T03:16:08.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:08.247+0000 7f5b42ffd700 1 -- 172.21.15.1:0/3655069463 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7f5b440a3e20 con 0x7f5b4c108760 2024-05-04T03:16:08.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:08.250+0000 7f5b54013700 1 -- 172.21.15.1:0/3655069463 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5b380b2880 msgr2=0x7f5b380b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:08.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:08.250+0000 7f5b54013700 1 --2- 172.21.15.1:0/3655069463 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5b380b2880 0x7f5b380b4d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009990 tx=0x7f5b48000bc0 comp rx=0 tx=0).stop 2024-05-04T03:16:08.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:08.250+0000 7f5b54013700 1 -- 172.21.15.1:0/3655069463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b4c108760 msgr2=0x7f5b4c111520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:08.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:08.250+0000 7f5b54013700 1 --2- 172.21.15.1:0/3655069463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b4c108760 0x7f5b4c111520 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5b44007d70 tx=0x7f5b4400a340 comp rx=0 tx=0).stop 2024-05-04T03:16:08.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:08.250+0000 7f5b54013700 1 -- 172.21.15.1:0/3655069463 shutdown_connections 2024-05-04T03:16:08.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:08.250+0000 7f5b54013700 1 --2- 172.21.15.1:0/3655069463 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b4c1034e0 0x7f5b4c110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:08.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:08.250+0000 7f5b54013700 1 --2- 172.21.15.1:0/3655069463 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5b380b2880 0x7f5b380b4d90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:08.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:08.250+0000 7f5b54013700 1 --2- 172.21.15.1:0/3655069463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5b4c108760 0x7f5b4c111520 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:08.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:08.250+0000 7f5b54013700 1 --2- 172.21.15.1:0/3655069463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b4c107e10 0x7f5b4c110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:08.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:08.250+0000 7f5b54013700 1 -- 172.21.15.1:0/3655069463 >> 172.21.15.1:0/3655069463 conn(0x7f5b4c0fef50 msgr2=0x7f5b4c107000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:08.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:08.250+0000 7f5b54013700 1 -- 172.21.15.1:0/3655069463 shutdown_connections 2024-05-04T03:16:08.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:08.250+0000 7f5b54013700 1 -- 172.21.15.1:0/3655069463 wait complete. 2024-05-04T03:16:08.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:08 smithi072 ceph-mon[33332]: pgmap v298: 97 pgs: 97 active+clean; 582 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.0 KiB/s wr, 5 op/s 2024-05-04T03:16:08.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:08 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/3655069463' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-04T03:16:08.570 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:08 smithi001 ceph-mon[29731]: pgmap v298: 97 pgs: 97 active+clean; 582 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.0 KiB/s wr, 5 op/s 2024-05-04T03:16:08.571 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:08 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3655069463' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-04T03:16:08.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:08 smithi077 ceph-mon[33531]: pgmap v298: 97 pgs: 97 active+clean; 582 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.0 KiB/s wr, 5 op/s 2024-05-04T03:16:08.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:08 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/3655069463' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-04T03:16:09.006 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-05-04T03:16:09.006 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-04T03:16:09.262 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:09.508 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:09 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3655069463' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-04T03:16:09.508 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:09 smithi001 ceph-mon[29731]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:16:09.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:09 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/3655069463' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-04T03:16:09.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:09 smithi077 ceph-mon[33531]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:16:09.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:09 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/3655069463' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-04T03:16:09.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:09 smithi072 ceph-mon[33332]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:16:10.520 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:10 smithi001 ceph-mon[29731]: pgmap v299: 97 pgs: 97 active+clean; 582 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-04T03:16:10.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:10 smithi077 ceph-mon[33531]: pgmap v299: 97 pgs: 97 active+clean; 582 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-04T03:16:10.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:10 smithi072 ceph-mon[33332]: pgmap v299: 97 pgs: 97 active+clean; 582 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-04T03:16:11.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.064+0000 7f200636c700 1 -- 172.21.15.1:0/2545201144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2000109860 msgr2=0x7f20001111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:11.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.064+0000 7f200636c700 1 --2- 172.21.15.1:0/2545201144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2000109860 0x7f20001111c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800b0d0 tx=0x7f1ff800b3e0 comp rx=0 tx=0).stop 2024-05-04T03:16:11.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.064+0000 7f200636c700 1 -- 172.21.15.1:0/2545201144 shutdown_connections 2024-05-04T03:16:11.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.064+0000 7f200636c700 1 --2- 172.21.15.1:0/2545201144 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20001045e0 0x7f20001049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:11.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.064+0000 7f200636c700 1 --2- 172.21.15.1:0/2545201144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2000109860 0x7f20001111c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:11.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.064+0000 7f200636c700 1 --2- 172.21.15.1:0/2545201144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2000108f10 0x7f2000109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:11.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.064+0000 7f200636c700 1 -- 172.21.15.1:0/2545201144 >> 172.21.15.1:0/2545201144 conn(0x7f20000ffed0 msgr2=0x7f2000102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:11.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.064+0000 7f200636c700 1 -- 172.21.15.1:0/2545201144 shutdown_connections 2024-05-04T03:16:11.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.064+0000 7f200636c700 1 -- 172.21.15.1:0/2545201144 wait complete. 2024-05-04T03:16:11.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.065+0000 7f200636c700 1 Processor -- start 2024-05-04T03:16:11.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.065+0000 7f200636c700 1 -- start start 2024-05-04T03:16:11.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.066+0000 7f200636c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20001045e0 0x7f2000076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:11.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.066+0000 7f200636c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2000108f10 0x7f2000077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:11.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.066+0000 7f200636c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2000109860 0x7f2000077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:11.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.066+0000 7f200636c700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2000114510 con 0x7f2000108f10 2024-05-04T03:16:11.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.066+0000 7f200636c700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2000114390 con 0x7f2000109860 2024-05-04T03:16:11.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.066+0000 7f200636c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2000114690 con 0x7f20001045e0 2024-05-04T03:16:11.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.066+0000 7f1fff7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2000108f10 0x7f2000077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:11.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.066+0000 7f1fff7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2000108f10 0x7f2000077110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38524/0 (socket says 172.21.15.1:38524) 2024-05-04T03:16:11.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.066+0000 7f1fff7fe700 1 -- 172.21.15.1:0/2521252984 learned_addr learned my addr 172.21.15.1:0/2521252984 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:16:11.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.066+0000 7f2004909700 1 --2- 172.21.15.1:0/2521252984 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2000109860 0x7f2000077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:11.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.066+0000 7f1ffffff700 1 --2- 172.21.15.1:0/2521252984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20001045e0 0x7f2000076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:11.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.066+0000 7f1fff7fe700 1 -- 172.21.15.1:0/2521252984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20001045e0 msgr2=0x7f2000076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:11.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.066+0000 7f1fff7fe700 1 --2- 172.21.15.1:0/2521252984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20001045e0 0x7f2000076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:11.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.066+0000 7f1fff7fe700 1 -- 172.21.15.1:0/2521252984 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2000109860 msgr2=0x7f2000077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:11.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.066+0000 7f1fff7fe700 1 --2- 172.21.15.1:0/2521252984 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2000109860 0x7f2000077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:11.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.066+0000 7f1fff7fe700 1 -- 172.21.15.1:0/2521252984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20001a9160 con 0x7f2000108f10 2024-05-04T03:16:11.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.066+0000 7f1fff7fe700 1 --2- 172.21.15.1:0/2521252984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2000108f10 0x7f2000077110 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400c9e0 tx=0x7f1ff400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:11.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.067+0000 7f1ffd7fa700 1 -- 172.21.15.1:0/2521252984 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4007aa0 con 0x7f2000108f10 2024-05-04T03:16:11.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.067+0000 7f1ffd7fa700 1 -- 172.21.15.1:0/2521252984 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ff4007c40 con 0x7f2000108f10 2024-05-04T03:16:11.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.067+0000 7f200636c700 1 -- 172.21.15.1:0/2521252984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20001a9450 con 0x7f2000108f10 2024-05-04T03:16:11.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.067+0000 7f1ffd7fa700 1 -- 172.21.15.1:0/2521252984 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4018660 con 0x7f2000108f10 2024-05-04T03:16:11.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.067+0000 7f200636c700 1 -- 172.21.15.1:0/2521252984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20001a99e0 con 0x7f2000108f10 2024-05-04T03:16:11.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.068+0000 7f200636c700 1 -- 172.21.15.1:0/2521252984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20000512c0 con 0x7f2000108f10 2024-05-04T03:16:11.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.069+0000 7f1ffd7fa700 1 -- 172.21.15.1:0/2521252984 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f1ff4018800 con 0x7f2000108f10 2024-05-04T03:16:11.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.070+0000 7f1ffd7fa700 1 --2- 172.21.15.1:0/2521252984 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1fe80b2900 0x7f1fe80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:11.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.070+0000 7f1ffffff700 1 --2- 172.21.15.1:0/2521252984 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1fe80b2900 0x7f1fe80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:11.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.070+0000 7f1ffd7fa700 1 -- 172.21.15.1:0/2521252984 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f1ff40e2040 con 0x7f2000108f10 2024-05-04T03:16:11.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.071+0000 7f1ffffff700 1 --2- 172.21.15.1:0/2521252984 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1fe80b2900 0x7f1fe80b4e10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0006fd0 tx=0x7f1ff0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:11.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.075+0000 7f1ffd7fa700 1 -- 172.21.15.1:0/2521252984 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1ff4014030 con 0x7f2000108f10 2024-05-04T03:16:11.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.322+0000 7f200636c700 1 -- 172.21.15.1:0/2521252984 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f2000078100 con 0x7f1fe80b2900 2024-05-04T03:16:11.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.350+0000 7f1ffd7fa700 1 -- 172.21.15.1:0/2521252984 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2000078100 con 0x7f1fe80b2900 2024-05-04T03:16:11.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.352+0000 7f200636c700 1 -- 172.21.15.1:0/2521252984 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1fe80b2900 msgr2=0x7f1fe80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:11.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.352+0000 7f200636c700 1 --2- 172.21.15.1:0/2521252984 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1fe80b2900 0x7f1fe80b4e10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0006fd0 tx=0x7f1ff0008040 comp rx=0 tx=0).stop 2024-05-04T03:16:11.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.352+0000 7f200636c700 1 -- 172.21.15.1:0/2521252984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2000108f10 msgr2=0x7f2000077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:11.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.352+0000 7f200636c700 1 --2- 172.21.15.1:0/2521252984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2000108f10 0x7f2000077110 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400c9e0 tx=0x7f1ff400ccf0 comp rx=0 tx=0).stop 2024-05-04T03:16:11.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.352+0000 7f200636c700 1 -- 172.21.15.1:0/2521252984 shutdown_connections 2024-05-04T03:16:11.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.352+0000 7f200636c700 1 --2- 172.21.15.1:0/2521252984 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2000109860 0x7f2000077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:11.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.352+0000 7f200636c700 1 --2- 172.21.15.1:0/2521252984 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1fe80b2900 0x7f1fe80b4e10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:11.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.352+0000 7f200636c700 1 --2- 172.21.15.1:0/2521252984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f20001045e0 0x7f2000076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:11.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.352+0000 7f200636c700 1 --2- 172.21.15.1:0/2521252984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2000108f10 0x7f2000077110 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:11.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.352+0000 7f200636c700 1 -- 172.21.15.1:0/2521252984 >> 172.21.15.1:0/2521252984 conn(0x7f20000ffed0 msgr2=0x7f2000108620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:11.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.352+0000 7f200636c700 1 -- 172.21.15.1:0/2521252984 shutdown_connections 2024-05-04T03:16:11.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:11.352+0000 7f200636c700 1 -- 172.21.15.1:0/2521252984 wait complete. 2024-05-04T03:16:11.629 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:16:11 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:16:11.327+0000 7f3fca2e8700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-04T03:16:11.629 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:16:11 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:16:11.327+0000 7f3fca2e8700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-04T03:16:11.629 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:16:11 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:16:11.327+0000 7f3fca2e8700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-04T03:16:11.629 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:16:11 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:16:11.327+0000 7f3fca2e8700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-04T03:16:11.629 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:16:11 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:16:11.327+0000 7f3fca2e8700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-04T03:16:11.966 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-04T03:16:12.221 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:12.571 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:12 smithi001 ceph-mon[29731]: pgmap v300: 97 pgs: 97 active+clean; 582 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-04T03:16:12.571 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:12 smithi001 ceph-mon[29731]: from='client.14655 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:16:12.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:12 smithi077 ceph-mon[33531]: pgmap v300: 97 pgs: 97 active+clean; 582 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-04T03:16:12.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:12 smithi077 ceph-mon[33531]: from='client.14655 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:16:12.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:12 smithi072 ceph-mon[33332]: pgmap v300: 97 pgs: 97 active+clean; 582 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-04T03:16:12.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:12 smithi072 ceph-mon[33332]: from='client.14655 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:16:13.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.896+0000 7f50b649b700 1 -- 172.21.15.1:0/4270460323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50b0107df0 msgr2=0x7f50b0108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:13.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.896+0000 7f50b649b700 1 --2- 172.21.15.1:0/4270460323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50b0107df0 0x7f50b0108200 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f50a40099c0 tx=0x7f50a4009cd0 comp rx=0 tx=0).stop 2024-05-04T03:16:13.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.896+0000 7f50b649b700 1 -- 172.21.15.1:0/4270460323 shutdown_connections 2024-05-04T03:16:13.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.896+0000 7f50b649b700 1 --2- 172.21.15.1:0/4270460323 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50b0108740 0x7f50b01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:13.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.896+0000 7f50b649b700 1 --2- 172.21.15.1:0/4270460323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50b01034c0 0x7f50b01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:13.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.896+0000 7f50b649b700 1 --2- 172.21.15.1:0/4270460323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50b0107df0 0x7f50b0108200 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:13.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.896+0000 7f50b649b700 1 -- 172.21.15.1:0/4270460323 >> 172.21.15.1:0/4270460323 conn(0x7f50b00fedf0 msgr2=0x7f50b0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:13.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.896+0000 7f50b649b700 1 -- 172.21.15.1:0/4270460323 shutdown_connections 2024-05-04T03:16:13.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.896+0000 7f50b649b700 1 -- 172.21.15.1:0/4270460323 wait complete. 2024-05-04T03:16:13.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.897+0000 7f50b649b700 1 Processor -- start 2024-05-04T03:16:13.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.897+0000 7f50b649b700 1 -- start start 2024-05-04T03:16:13.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.897+0000 7f50b649b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50b01034c0 0x7f50b019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:13.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.897+0000 7f50b649b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50b0107df0 0x7f50b019b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:13.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.897+0000 7f50b649b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50b0108740 0x7f50b019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:13.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.897+0000 7f50b649b700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f50b0113350 con 0x7f50b0108740 2024-05-04T03:16:13.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.897+0000 7f50b649b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f50b01131d0 con 0x7f50b01034c0 2024-05-04T03:16:13.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.897+0000 7f50b649b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f50b01134d0 con 0x7f50b0107df0 2024-05-04T03:16:13.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.897+0000 7f50b4a38700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50b0108740 0x7f50b019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:13.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.897+0000 7f50b4a38700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50b0108740 0x7f50b019f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55216/0 (socket says 172.21.15.1:55216) 2024-05-04T03:16:13.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.897+0000 7f50b4a38700 1 -- 172.21.15.1:0/2607596628 learned_addr learned my addr 172.21.15.1:0/2607596628 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:16:13.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.897+0000 7f50affff700 1 --2- 172.21.15.1:0/2607596628 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50b01034c0 0x7f50b019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:13.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.897+0000 7f50af7fe700 1 --2- 172.21.15.1:0/2607596628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50b0107df0 0x7f50b019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:13.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.898+0000 7f50b4a38700 1 -- 172.21.15.1:0/2607596628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50b0107df0 msgr2=0x7f50b019b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:13.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.898+0000 7f50b4a38700 1 --2- 172.21.15.1:0/2607596628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50b0107df0 0x7f50b019b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:13.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.898+0000 7f50b4a38700 1 -- 172.21.15.1:0/2607596628 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50b01034c0 msgr2=0x7f50b019ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:13.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.898+0000 7f50b4a38700 1 --2- 172.21.15.1:0/2607596628 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50b01034c0 0x7f50b019ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:13.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.898+0000 7f50b4a38700 1 -- 172.21.15.1:0/2607596628 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b019fc90 con 0x7f50b0108740 2024-05-04T03:16:13.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.898+0000 7f50b4a38700 1 --2- 172.21.15.1:0/2607596628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50b0108740 0x7f50b019f6a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f50a8007d70 tx=0x7f50a800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:13.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.898+0000 7f50ad7fa700 1 -- 172.21.15.1:0/2607596628 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a8016020 con 0x7f50b0108740 2024-05-04T03:16:13.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.898+0000 7f50b649b700 1 -- 172.21.15.1:0/2607596628 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b019ff80 con 0x7f50b0108740 2024-05-04T03:16:13.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.898+0000 7f50ad7fa700 1 -- 172.21.15.1:0/2607596628 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50a800ac90 con 0x7f50b0108740 2024-05-04T03:16:13.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.898+0000 7f50ad7fa700 1 -- 172.21.15.1:0/2607596628 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a80115e0 con 0x7f50b0108740 2024-05-04T03:16:13.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.898+0000 7f50b649b700 1 -- 172.21.15.1:0/2607596628 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b01a0490 con 0x7f50b0108740 2024-05-04T03:16:13.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.899+0000 7f50b649b700 1 -- 172.21.15.1:0/2607596628 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50b0104970 con 0x7f50b0108740 2024-05-04T03:16:13.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.901+0000 7f50ad7fa700 1 -- 172.21.15.1:0/2607596628 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f50a8019020 con 0x7f50b0108740 2024-05-04T03:16:13.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.901+0000 7f50ad7fa700 1 --2- 172.21.15.1:0/2607596628 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f50980b2910 0x7f50980b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:13.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.901+0000 7f50affff700 1 --2- 172.21.15.1:0/2607596628 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f50980b2910 0x7f50980b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:13.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.901+0000 7f50ad7fa700 1 -- 172.21.15.1:0/2607596628 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f50a80e24e0 con 0x7f50b0108740 2024-05-04T03:16:13.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.902+0000 7f50affff700 1 --2- 172.21.15.1:0/2607596628 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f50980b2910 0x7f50980b4e20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f50a0005950 tx=0x7f50a000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:13.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:13.905+0000 7f50ad7fa700 1 -- 172.21.15.1:0/2607596628 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f50a8020020 con 0x7f50b0108740 2024-05-04T03:16:14.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:14.151+0000 7f50b649b700 1 -- 172.21.15.1:0/2607596628 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f50b0063c00 con 0x7f50980b2910 2024-05-04T03:16:14.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:14.177+0000 7f50ad7fa700 1 -- 172.21.15.1:0/2607596628 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f50b0063c00 con 0x7f50980b2910 2024-05-04T03:16:14.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:14.180+0000 7f50b649b700 1 -- 172.21.15.1:0/2607596628 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f50980b2910 msgr2=0x7f50980b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:14.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:14.180+0000 7f50b649b700 1 --2- 172.21.15.1:0/2607596628 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f50980b2910 0x7f50980b4e20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f50a0005950 tx=0x7f50a000b410 comp rx=0 tx=0).stop 2024-05-04T03:16:14.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:14.180+0000 7f50b649b700 1 -- 172.21.15.1:0/2607596628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50b0108740 msgr2=0x7f50b019f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:14.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:14.180+0000 7f50b649b700 1 --2- 172.21.15.1:0/2607596628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50b0108740 0x7f50b019f6a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f50a8007d70 tx=0x7f50a800a340 comp rx=0 tx=0).stop 2024-05-04T03:16:14.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:14.180+0000 7f50b649b700 1 -- 172.21.15.1:0/2607596628 shutdown_connections 2024-05-04T03:16:14.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:14.180+0000 7f50b649b700 1 --2- 172.21.15.1:0/2607596628 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f50b01034c0 0x7f50b019ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:14.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:14.180+0000 7f50b649b700 1 --2- 172.21.15.1:0/2607596628 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f50980b2910 0x7f50980b4e20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:14.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:14.180+0000 7f50b649b700 1 --2- 172.21.15.1:0/2607596628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50b0108740 0x7f50b019f6a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:14.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:14.180+0000 7f50b649b700 1 --2- 172.21.15.1:0/2607596628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50b0107df0 0x7f50b019b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:14.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:14.180+0000 7f50b649b700 1 -- 172.21.15.1:0/2607596628 >> 172.21.15.1:0/2607596628 conn(0x7f50b00fedf0 msgr2=0x7f50b0075dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:14.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:14.180+0000 7f50b649b700 1 -- 172.21.15.1:0/2607596628 shutdown_connections 2024-05-04T03:16:14.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:14.180+0000 7f50b649b700 1 -- 172.21.15.1:0/2607596628 wait complete. 2024-05-04T03:16:14.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:14 smithi001 ceph-mon[29731]: pgmap v301: 97 pgs: 97 active+clean; 588 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-05-04T03:16:14.498 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:14 smithi001 ceph-mon[29731]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-04T03:16:14.549 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:14 smithi072 ceph-mon[33332]: pgmap v301: 97 pgs: 97 active+clean; 588 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-05-04T03:16:14.549 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:14 smithi072 ceph-mon[33332]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-04T03:16:14.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:14 smithi077 ceph-mon[33531]: pgmap v301: 97 pgs: 97 active+clean; 588 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-05-04T03:16:14.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:14 smithi077 ceph-mon[33531]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-04T03:16:14.836 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-04T03:16:15.086 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:15.517 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:15 smithi001 ceph-mon[29731]: from='client.14667 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:16:15.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:15 smithi077 ceph-mon[33531]: from='client.14667 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:16:15.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:15 smithi072 ceph-mon[33332]: from='client.14667 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:16:16.359 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:16 smithi001 ceph-mon[29731]: pgmap v302: 97 pgs: 97 active+clean; 588 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-05-04T03:16:16.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.583+0000 7f8275494700 1 -- 172.21.15.1:0/2870569842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8270109890 msgr2=0x7f82701111f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:16.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.583+0000 7f8275494700 1 --2- 172.21.15.1:0/2870569842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8270109890 0x7f82701111f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f826800b0d0 tx=0x7f826800b3e0 comp rx=0 tx=0).stop 2024-05-04T03:16:16.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.583+0000 7f8275494700 1 -- 172.21.15.1:0/2870569842 shutdown_connections 2024-05-04T03:16:16.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.583+0000 7f8275494700 1 --2- 172.21.15.1:0/2870569842 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8270104610 0x7f8270104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:16.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.583+0000 7f8275494700 1 --2- 172.21.15.1:0/2870569842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8270109890 0x7f82701111f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:16.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.583+0000 7f8275494700 1 --2- 172.21.15.1:0/2870569842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8270108f40 0x7f8270109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:16.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.583+0000 7f8275494700 1 -- 172.21.15.1:0/2870569842 >> 172.21.15.1:0/2870569842 conn(0x7f827007af70 msgr2=0x7f827007b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:16.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.583+0000 7f8275494700 1 -- 172.21.15.1:0/2870569842 shutdown_connections 2024-05-04T03:16:16.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.583+0000 7f8275494700 1 -- 172.21.15.1:0/2870569842 wait complete. 2024-05-04T03:16:16.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.584+0000 7f8275494700 1 Processor -- start 2024-05-04T03:16:16.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.584+0000 7f8275494700 1 -- start start 2024-05-04T03:16:16.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.585+0000 7f8275494700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8270104610 0x7f827019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:16.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.585+0000 7f8275494700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8270108f40 0x7f827019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:16.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.585+0000 7f8275494700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8270109890 0x7f827019f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:16.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.585+0000 7f8275494700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f82701144c0 con 0x7f8270109890 2024-05-04T03:16:16.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.585+0000 7f8275494700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8270114340 con 0x7f8270104610 2024-05-04T03:16:16.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.585+0000 7f8275494700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8270114640 con 0x7f8270108f40 2024-05-04T03:16:16.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.585+0000 7f826f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8270109890 0x7f827019f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:16.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.585+0000 7f826e7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8270108f40 0x7f827019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:16.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.585+0000 7f826e7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8270108f40 0x7f827019b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.1:59708/0 (socket says 172.21.15.1:59708) 2024-05-04T03:16:16.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.585+0000 7f826f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8270109890 0x7f827019f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55244/0 (socket says 172.21.15.1:55244) 2024-05-04T03:16:16.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.585+0000 7f826e7fc700 1 -- 172.21.15.1:0/1748317314 learned_addr learned my addr 172.21.15.1:0/1748317314 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:16:16.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.585+0000 7f826effd700 1 --2- 172.21.15.1:0/1748317314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8270104610 0x7f827019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:16.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.585+0000 7f826e7fc700 1 -- 172.21.15.1:0/1748317314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8270104610 msgr2=0x7f827019ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:16.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.585+0000 7f826e7fc700 1 --2- 172.21.15.1:0/1748317314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8270104610 0x7f827019ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:16.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.585+0000 7f826e7fc700 1 -- 172.21.15.1:0/1748317314 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8270109890 msgr2=0x7f827019f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:16.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.585+0000 7f826e7fc700 1 --2- 172.21.15.1:0/1748317314 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8270109890 0x7f827019f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:16.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.585+0000 7f826e7fc700 1 -- 172.21.15.1:0/1748317314 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827019fcb0 con 0x7f8270108f40 2024-05-04T03:16:16.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.585+0000 7f826f7fe700 1 --2- 172.21.15.1:0/1748317314 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8270109890 0x7f827019f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:16:16.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.585+0000 7f826e7fc700 1 --2- 172.21.15.1:0/1748317314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8270108f40 0x7f827019b3d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f826400e970 tx=0x7f826400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:16.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.585+0000 7f8257fff700 1 -- 172.21.15.1:0/1748317314 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8264004a30 con 0x7f8270108f40 2024-05-04T03:16:16.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.586+0000 7f8257fff700 1 -- 172.21.15.1:0/1748317314 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f826400cc90 con 0x7f8270108f40 2024-05-04T03:16:16.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.586+0000 7f8275494700 1 -- 172.21.15.1:0/1748317314 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f827019ffa0 con 0x7f8270108f40 2024-05-04T03:16:16.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.586+0000 7f8257fff700 1 -- 172.21.15.1:0/1748317314 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8264005610 con 0x7f8270108f40 2024-05-04T03:16:16.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.586+0000 7f8275494700 1 -- 172.21.15.1:0/1748317314 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82701a78c0 con 0x7f8270108f40 2024-05-04T03:16:16.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.586+0000 7f8275494700 1 -- 172.21.15.1:0/1748317314 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82700512c0 con 0x7f8270108f40 2024-05-04T03:16:16.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.588+0000 7f8257fff700 1 -- 172.21.15.1:0/1748317314 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f826400c7e0 con 0x7f8270108f40 2024-05-04T03:16:16.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.589+0000 7f8257fff700 1 --2- 172.21.15.1:0/1748317314 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f82580b2910 0x7f82580b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:16.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.589+0000 7f826effd700 1 --2- 172.21.15.1:0/1748317314 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f82580b2910 0x7f82580b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:16.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.589+0000 7f8257fff700 1 -- 172.21.15.1:0/1748317314 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f82640e2440 con 0x7f8270108f40 2024-05-04T03:16:16.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.589+0000 7f826effd700 1 --2- 172.21.15.1:0/1748317314 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f82580b2910 0x7f82580b4e20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8260007be0 tx=0x7f82600073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:16.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.592+0000 7f8257fff700 1 -- 172.21.15.1:0/1748317314 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f82640aa0b0 con 0x7f8270108f40 2024-05-04T03:16:16.676 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:16 smithi077 ceph-mon[33531]: pgmap v302: 97 pgs: 97 active+clean; 588 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-05-04T03:16:16.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:16 smithi072 ceph-mon[33332]: pgmap v302: 97 pgs: 97 active+clean; 588 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-05-04T03:16:16.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.840+0000 7f8275494700 1 -- 172.21.15.1:0/1748317314 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f827019c140 con 0x7f82580b2910 2024-05-04T03:16:16.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.846+0000 7f8257fff700 1 -- 172.21.15.1:0/1748317314 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f827019c140 con 0x7f82580b2910 2024-05-04T03:16:16.847 INFO:teuthology.orchestra.run.smithi001.stdout:/volumes/qa/sv_0/48e28e33-5c65-4820-8f93-da64ac44444f 2024-05-04T03:16:16.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.848+0000 7f8275494700 1 -- 172.21.15.1:0/1748317314 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f82580b2910 msgr2=0x7f82580b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:16.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.848+0000 7f8275494700 1 --2- 172.21.15.1:0/1748317314 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f82580b2910 0x7f82580b4e20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8260007be0 tx=0x7f82600073d0 comp rx=0 tx=0).stop 2024-05-04T03:16:16.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.848+0000 7f8275494700 1 -- 172.21.15.1:0/1748317314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8270108f40 msgr2=0x7f827019b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:16.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.848+0000 7f8275494700 1 --2- 172.21.15.1:0/1748317314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8270108f40 0x7f827019b3d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f826400e970 tx=0x7f826400ec80 comp rx=0 tx=0).stop 2024-05-04T03:16:16.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.848+0000 7f8275494700 1 -- 172.21.15.1:0/1748317314 shutdown_connections 2024-05-04T03:16:16.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.848+0000 7f8275494700 1 --2- 172.21.15.1:0/1748317314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8270104610 0x7f827019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:16.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.848+0000 7f8275494700 1 --2- 172.21.15.1:0/1748317314 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f82580b2910 0x7f82580b4e20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:16.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.848+0000 7f8275494700 1 --2- 172.21.15.1:0/1748317314 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8270109890 0x7f827019f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:16.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.848+0000 7f8275494700 1 --2- 172.21.15.1:0/1748317314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8270108f40 0x7f827019b3d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:16.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.848+0000 7f8275494700 1 -- 172.21.15.1:0/1748317314 >> 172.21.15.1:0/1748317314 conn(0x7f827007af70 msgr2=0x7f82701018e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:16.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.848+0000 7f8275494700 1 -- 172.21.15.1:0/1748317314 shutdown_connections 2024-05-04T03:16:16.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:16.848+0000 7f8275494700 1 -- 172.21.15.1:0/1748317314 wait complete. 2024-05-04T03:16:17.582 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-04T03:16:17.833 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:18.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:18 smithi001 ceph-mon[29731]: from='client.24568 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:16:18.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:18 smithi001 ceph-mon[29731]: pgmap v303: 97 pgs: 97 active+clean; 588 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1023 B/s wr, 8 op/s 2024-05-04T03:16:18.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:18 smithi077 ceph-mon[33531]: from='client.24568 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:16:18.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:18 smithi077 ceph-mon[33531]: pgmap v303: 97 pgs: 97 active+clean; 588 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1023 B/s wr, 8 op/s 2024-05-04T03:16:18.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:18 smithi072 ceph-mon[33332]: from='client.24568 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:16:18.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:18 smithi072 ceph-mon[33332]: pgmap v303: 97 pgs: 97 active+clean; 588 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1023 B/s wr, 8 op/s 2024-05-04T03:16:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.447+0000 7f01de5ca700 1 -- 172.21.15.1:0/3164488184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01d8107df0 msgr2=0x7f01d8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.447+0000 7f01de5ca700 1 --2- 172.21.15.1:0/3164488184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01d8107df0 0x7f01d8108200 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f01cc0099c0 tx=0x7f01cc009cd0 comp rx=0 tx=0).stop 2024-05-04T03:16:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.447+0000 7f01de5ca700 1 -- 172.21.15.1:0/3164488184 shutdown_connections 2024-05-04T03:16:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.447+0000 7f01de5ca700 1 --2- 172.21.15.1:0/3164488184 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01d8108740 0x7f01d81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.447+0000 7f01de5ca700 1 --2- 172.21.15.1:0/3164488184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01d81034c0 0x7f01d81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.447+0000 7f01de5ca700 1 --2- 172.21.15.1:0/3164488184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01d8107df0 0x7f01d8108200 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.447+0000 7f01de5ca700 1 -- 172.21.15.1:0/3164488184 >> 172.21.15.1:0/3164488184 conn(0x7f01d80fedf0 msgr2=0x7f01d8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.447+0000 7f01de5ca700 1 -- 172.21.15.1:0/3164488184 shutdown_connections 2024-05-04T03:16:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.447+0000 7f01de5ca700 1 -- 172.21.15.1:0/3164488184 wait complete. 2024-05-04T03:16:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.448+0000 7f01de5ca700 1 Processor -- start 2024-05-04T03:16:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.448+0000 7f01de5ca700 1 -- start start 2024-05-04T03:16:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.448+0000 7f01de5ca700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01d81034c0 0x7f01d819bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.448+0000 7f01de5ca700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01d8107df0 0x7f01d819c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.448+0000 7f01de5ca700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01d8108740 0x7f01d819c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.448+0000 7f01de5ca700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f01d8113350 con 0x7f01d81034c0 2024-05-04T03:16:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.448+0000 7f01de5ca700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f01d81131d0 con 0x7f01d8107df0 2024-05-04T03:16:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.448+0000 7f01de5ca700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f01d81134d0 con 0x7f01d8108740 2024-05-04T03:16:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.448+0000 7f01d7fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01d81034c0 0x7f01d819bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.448+0000 7f01d7fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01d81034c0 0x7f01d819bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55252/0 (socket says 172.21.15.1:55252) 2024-05-04T03:16:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.448+0000 7f01d7fff700 1 -- 172.21.15.1:0/1164015700 learned_addr learned my addr 172.21.15.1:0/1164015700 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:16:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.449+0000 7f01d77fe700 1 --2- 172.21.15.1:0/1164015700 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01d8107df0 0x7f01d819c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.449+0000 7f01dcb67700 1 --2- 172.21.15.1:0/1164015700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01d8108740 0x7f01d819c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.449+0000 7f01d7fff700 1 -- 172.21.15.1:0/1164015700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01d8108740 msgr2=0x7f01d819c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.449+0000 7f01d7fff700 1 --2- 172.21.15.1:0/1164015700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01d8108740 0x7f01d819c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.449+0000 7f01d7fff700 1 -- 172.21.15.1:0/1164015700 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01d8107df0 msgr2=0x7f01d819c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.449+0000 7f01d7fff700 1 --2- 172.21.15.1:0/1164015700 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01d8107df0 0x7f01d819c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.449+0000 7f01d7fff700 1 -- 172.21.15.1:0/1164015700 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d81a7000 con 0x7f01d81034c0 2024-05-04T03:16:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.449+0000 7f01d7fff700 1 --2- 172.21.15.1:0/1164015700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01d81034c0 0x7f01d819bcb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f01c800d950 tx=0x7f01c800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.449+0000 7f01d57fa700 1 -- 172.21.15.1:0/1164015700 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8016070 con 0x7f01d81034c0 2024-05-04T03:16:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.449+0000 7f01d57fa700 1 -- 172.21.15.1:0/1164015700 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01c8009df0 con 0x7f01d81034c0 2024-05-04T03:16:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.449+0000 7f01de5ca700 1 -- 172.21.15.1:0/1164015700 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d81a72f0 con 0x7f01d81034c0 2024-05-04T03:16:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.449+0000 7f01d57fa700 1 -- 172.21.15.1:0/1164015700 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c800f3c0 con 0x7f01d81034c0 2024-05-04T03:16:19.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.449+0000 7f01de5ca700 1 -- 172.21.15.1:0/1164015700 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d81a7800 con 0x7f01d81034c0 2024-05-04T03:16:19.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.451+0000 7f01de5ca700 1 -- 172.21.15.1:0/1164015700 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01d80512c0 con 0x7f01d81034c0 2024-05-04T03:16:19.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.452+0000 7f01d57fa700 1 -- 172.21.15.1:0/1164015700 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f01c8019070 con 0x7f01d81034c0 2024-05-04T03:16:19.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.453+0000 7f01d57fa700 1 --2- 172.21.15.1:0/1164015700 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f01c00b2910 0x7f01c00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:19.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.453+0000 7f01d77fe700 1 --2- 172.21.15.1:0/1164015700 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f01c00b2910 0x7f01c00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:19.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.453+0000 7f01d57fa700 1 -- 172.21.15.1:0/1164015700 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f01c80e1e10 con 0x7f01d81034c0 2024-05-04T03:16:19.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.453+0000 7f01d77fe700 1 --2- 172.21.15.1:0/1164015700 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f01c00b2910 0x7f01c00b4e20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f01cc006010 tx=0x7f01cc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:19.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.458+0000 7f01d57fa700 1 -- 172.21.15.1:0/1164015700 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f01c80a9b80 con 0x7f01d81034c0 2024-05-04T03:16:19.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.705+0000 7f01de5ca700 1 -- 172.21.15.1:0/1164015700 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f01d8063c00 con 0x7f01c00b2910 2024-05-04T03:16:19.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.731+0000 7f01d57fa700 1 -- 172.21.15.1:0/1164015700 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f01d8063c00 con 0x7f01c00b2910 2024-05-04T03:16:19.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.734+0000 7f01de5ca700 1 -- 172.21.15.1:0/1164015700 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f01c00b2910 msgr2=0x7f01c00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:19.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.734+0000 7f01de5ca700 1 --2- 172.21.15.1:0/1164015700 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f01c00b2910 0x7f01c00b4e20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f01cc006010 tx=0x7f01cc000bc0 comp rx=0 tx=0).stop 2024-05-04T03:16:19.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.734+0000 7f01de5ca700 1 -- 172.21.15.1:0/1164015700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01d81034c0 msgr2=0x7f01d819bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:19.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.734+0000 7f01de5ca700 1 --2- 172.21.15.1:0/1164015700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01d81034c0 0x7f01d819bcb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f01c800d950 tx=0x7f01c800dc60 comp rx=0 tx=0).stop 2024-05-04T03:16:19.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.734+0000 7f01de5ca700 1 -- 172.21.15.1:0/1164015700 shutdown_connections 2024-05-04T03:16:19.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.734+0000 7f01de5ca700 1 --2- 172.21.15.1:0/1164015700 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f01c00b2910 0x7f01c00b4e20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:19.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.734+0000 7f01de5ca700 1 --2- 172.21.15.1:0/1164015700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01d8108740 0x7f01d819c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:19.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.734+0000 7f01de5ca700 1 --2- 172.21.15.1:0/1164015700 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01d81034c0 0x7f01d819bcb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:19.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.734+0000 7f01de5ca700 1 --2- 172.21.15.1:0/1164015700 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01d8107df0 0x7f01d819c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:19.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.734+0000 7f01de5ca700 1 -- 172.21.15.1:0/1164015700 >> 172.21.15.1:0/1164015700 conn(0x7f01d80fedf0 msgr2=0x7f01d8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:19.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.734+0000 7f01de5ca700 1 -- 172.21.15.1:0/1164015700 shutdown_connections 2024-05-04T03:16:19.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:19.734+0000 7f01de5ca700 1 -- 172.21.15.1:0/1164015700 wait complete. 2024-05-04T03:16:20.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:20 smithi001 ceph-mon[29731]: pgmap v304: 97 pgs: 97 active+clean; 607 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-05-04T03:16:20.486 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-04T03:16:20.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:20 smithi077 ceph-mon[33531]: pgmap v304: 97 pgs: 97 active+clean; 607 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-05-04T03:16:20.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:20 smithi072 ceph-mon[33332]: pgmap v304: 97 pgs: 97 active+clean; 607 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-05-04T03:16:20.749 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:21.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:21 smithi001 ceph-mon[29731]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:16:21.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:21 smithi077 ceph-mon[33531]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:16:21.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:21 smithi072 ceph-mon[33332]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:16:22.298 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:22 smithi001 ceph-mon[29731]: pgmap v305: 97 pgs: 97 active+clean; 607 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-05-04T03:16:22.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.295+0000 7f085f3c7700 1 -- 172.21.15.1:0/1618299980 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0858108f10 msgr2=0x7f0858109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:22.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.295+0000 7f085f3c7700 1 --2- 172.21.15.1:0/1618299980 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0858108f10 0x7f0858109320 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f08480099c0 tx=0x7f0848009cd0 comp rx=0 tx=0).stop 2024-05-04T03:16:22.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.296+0000 7f085f3c7700 1 -- 172.21.15.1:0/1618299980 shutdown_connections 2024-05-04T03:16:22.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.296+0000 7f085f3c7700 1 --2- 172.21.15.1:0/1618299980 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0858109860 0x7f08581111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:22.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.296+0000 7f085f3c7700 1 --2- 172.21.15.1:0/1618299980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08581045e0 0x7f08581049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:22.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.296+0000 7f085f3c7700 1 --2- 172.21.15.1:0/1618299980 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0858108f10 0x7f0858109320 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:22.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.296+0000 7f085f3c7700 1 -- 172.21.15.1:0/1618299980 >> 172.21.15.1:0/1618299980 conn(0x7f08580ffe70 msgr2=0x7f08581022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:22.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.296+0000 7f085f3c7700 1 -- 172.21.15.1:0/1618299980 shutdown_connections 2024-05-04T03:16:22.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.296+0000 7f085f3c7700 1 -- 172.21.15.1:0/1618299980 wait complete. 2024-05-04T03:16:22.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.297+0000 7f085f3c7700 1 Processor -- start 2024-05-04T03:16:22.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.297+0000 7f085f3c7700 1 -- start start 2024-05-04T03:16:22.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.297+0000 7f085f3c7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08581045e0 0x7f085819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:22.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.297+0000 7f085f3c7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0858108f10 0x7f085819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:22.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.297+0000 7f085f3c7700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0858109860 0x7f085819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:22.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.297+0000 7f085f3c7700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0858114470 con 0x7f0858109860 2024-05-04T03:16:22.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.297+0000 7f085f3c7700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f08581142f0 con 0x7f0858108f10 2024-05-04T03:16:22.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.297+0000 7f085f3c7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f08581145f0 con 0x7f08581045e0 2024-05-04T03:16:22.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.297+0000 7f085d163700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08581045e0 0x7f085819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:22.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.297+0000 7f085d964700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0858109860 0x7f085819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:22.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.297+0000 7f085d163700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08581045e0 0x7f085819ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.1:34922/0 (socket says 172.21.15.1:34922) 2024-05-04T03:16:22.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.297+0000 7f085d163700 1 -- 172.21.15.1:0/992214754 learned_addr learned my addr 172.21.15.1:0/992214754 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:16:22.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.297+0000 7f085c962700 1 --2- 172.21.15.1:0/992214754 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0858108f10 0x7f085819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:22.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.297+0000 7f085d163700 1 -- 172.21.15.1:0/992214754 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0858108f10 msgr2=0x7f085819b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:22.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.297+0000 7f085d163700 1 --2- 172.21.15.1:0/992214754 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0858108f10 0x7f085819b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:22.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.297+0000 7f085d163700 1 -- 172.21.15.1:0/992214754 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0858109860 msgr2=0x7f085819f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:22.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.297+0000 7f085d163700 1 --2- 172.21.15.1:0/992214754 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0858109860 0x7f085819f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:22.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.298+0000 7f085d163700 1 -- 172.21.15.1:0/992214754 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085819fd10 con 0x7f08581045e0 2024-05-04T03:16:22.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.298+0000 7f085d964700 1 --2- 172.21.15.1:0/992214754 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0858109860 0x7f085819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:16:22.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.298+0000 7f085c962700 1 --2- 172.21.15.1:0/992214754 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0858108f10 0x7f085819b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:16:22.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.298+0000 7f085d163700 1 --2- 172.21.15.1:0/992214754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08581045e0 0x7f085819ae40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f085400e9a0 tx=0x7f085400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:22.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.298+0000 7f084e7fc700 1 -- 172.21.15.1:0/992214754 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085400ca10 con 0x7f08581045e0 2024-05-04T03:16:22.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.298+0000 7f085f3c7700 1 -- 172.21.15.1:0/992214754 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08581a0000 con 0x7f08581045e0 2024-05-04T03:16:22.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.298+0000 7f084e7fc700 1 -- 172.21.15.1:0/992214754 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f085400cbb0 con 0x7f08581045e0 2024-05-04T03:16:22.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.298+0000 7f084e7fc700 1 -- 172.21.15.1:0/992214754 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0854018680 con 0x7f08581045e0 2024-05-04T03:16:22.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.298+0000 7f085f3c7700 1 -- 172.21.15.1:0/992214754 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08581a7920 con 0x7f08581045e0 2024-05-04T03:16:22.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.299+0000 7f085f3c7700 1 -- 172.21.15.1:0/992214754 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08580512c0 con 0x7f08581045e0 2024-05-04T03:16:22.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.301+0000 7f084e7fc700 1 -- 172.21.15.1:0/992214754 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f085404aae0 con 0x7f08581045e0 2024-05-04T03:16:22.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.301+0000 7f084e7fc700 1 --2- 172.21.15.1:0/992214754 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f08440b2910 0x7f08440b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:22.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.301+0000 7f085c962700 1 --2- 172.21.15.1:0/992214754 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f08440b2910 0x7f08440b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:22.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.301+0000 7f084e7fc700 1 -- 172.21.15.1:0/992214754 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f08540e2550 con 0x7f08581045e0 2024-05-04T03:16:22.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.302+0000 7f085c962700 1 --2- 172.21.15.1:0/992214754 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f08440b2910 0x7f08440b4e20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0848000c00 tx=0x7f084801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:22.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.305+0000 7f084e7fc700 1 -- 172.21.15.1:0/992214754 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f08540aa1c0 con 0x7f08581045e0 2024-05-04T03:16:22.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.552+0000 7f085f3c7700 1 -- 172.21.15.1:0/992214754 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f0858063c00 con 0x7f08440b2910 2024-05-04T03:16:22.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.558+0000 7f084e7fc700 1 -- 172.21.15.1:0/992214754 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f0858063c00 con 0x7f08440b2910 2024-05-04T03:16:22.559 INFO:teuthology.orchestra.run.smithi001.stdout:/volumes/qa/sv_1/fca5699f-7058-441f-8333-bcf3e9a0f362 2024-05-04T03:16:22.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.560+0000 7f085f3c7700 1 -- 172.21.15.1:0/992214754 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f08440b2910 msgr2=0x7f08440b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:22.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.560+0000 7f085f3c7700 1 --2- 172.21.15.1:0/992214754 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f08440b2910 0x7f08440b4e20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0848000c00 tx=0x7f084801a040 comp rx=0 tx=0).stop 2024-05-04T03:16:22.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.560+0000 7f085f3c7700 1 -- 172.21.15.1:0/992214754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08581045e0 msgr2=0x7f085819ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:22.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.560+0000 7f085f3c7700 1 --2- 172.21.15.1:0/992214754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08581045e0 0x7f085819ae40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f085400e9a0 tx=0x7f085400ecb0 comp rx=0 tx=0).stop 2024-05-04T03:16:22.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.560+0000 7f085f3c7700 1 -- 172.21.15.1:0/992214754 shutdown_connections 2024-05-04T03:16:22.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.560+0000 7f085f3c7700 1 --2- 172.21.15.1:0/992214754 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f08440b2910 0x7f08440b4e20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:22.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.560+0000 7f085f3c7700 1 --2- 172.21.15.1:0/992214754 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0858109860 0x7f085819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:22.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.560+0000 7f085f3c7700 1 --2- 172.21.15.1:0/992214754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08581045e0 0x7f085819ae40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:22.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.560+0000 7f085f3c7700 1 --2- 172.21.15.1:0/992214754 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0858108f10 0x7f085819b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:22.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.560+0000 7f085f3c7700 1 -- 172.21.15.1:0/992214754 >> 172.21.15.1:0/992214754 conn(0x7f08580ffe70 msgr2=0x7f0858107160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:22.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.560+0000 7f085f3c7700 1 -- 172.21.15.1:0/992214754 shutdown_connections 2024-05-04T03:16:22.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:22.560+0000 7f085f3c7700 1 -- 172.21.15.1:0/992214754 wait complete. 2024-05-04T03:16:22.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:22 smithi077 ceph-mon[33531]: pgmap v305: 97 pgs: 97 active+clean; 607 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-05-04T03:16:22.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:22 smithi072 ceph-mon[33332]: pgmap v305: 97 pgs: 97 active+clean; 607 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-05-04T03:16:23.347 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:16:23.600 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:23.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:23 smithi001 ceph-mon[29731]: from='client.24580 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:16:23.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:23 smithi077 ceph-mon[33531]: from='client.24580 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:16:23.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:23 smithi072 ceph-mon[33332]: from='client.24580 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:16:24.579 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:24 smithi001 ceph-mon[29731]: pgmap v306: 97 pgs: 97 active+clean; 624 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-05-04T03:16:24.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:24 smithi077 ceph-mon[33531]: pgmap v306: 97 pgs: 97 active+clean; 624 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-05-04T03:16:24.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:24 smithi072 ceph-mon[33332]: pgmap v306: 97 pgs: 97 active+clean; 624 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-05-04T03:16:25.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.129+0000 7f537cb5d700 1 -- 172.21.15.1:0/398604094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5378103b30 msgr2=0x7f5378103f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:25.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.129+0000 7f537cb5d700 1 --2- 172.21.15.1:0/398604094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5378103b30 0x7f5378103f40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f53680099c0 tx=0x7f5368009cd0 comp rx=0 tx=0).stop 2024-05-04T03:16:25.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.129+0000 7f537cb5d700 1 -- 172.21.15.1:0/398604094 shutdown_connections 2024-05-04T03:16:25.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.129+0000 7f537cb5d700 1 --2- 172.21.15.1:0/398604094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53781087a0 0x7f5378110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:25.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.129+0000 7f537cb5d700 1 --2- 172.21.15.1:0/398604094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5378103b30 0x7f5378103f40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:25.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.129+0000 7f537cb5d700 1 --2- 172.21.15.1:0/398604094 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5378107e50 0x7f5378108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:25.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.129+0000 7f537cb5d700 1 -- 172.21.15.1:0/398604094 >> 172.21.15.1:0/398604094 conn(0x7f53780fe500 msgr2=0x7f5378100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:25.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.129+0000 7f537cb5d700 1 -- 172.21.15.1:0/398604094 shutdown_connections 2024-05-04T03:16:25.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.129+0000 7f537cb5d700 1 -- 172.21.15.1:0/398604094 wait complete. 2024-05-04T03:16:25.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.130+0000 7f537cb5d700 1 Processor -- start 2024-05-04T03:16:25.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.130+0000 7f537cb5d700 1 -- start start 2024-05-04T03:16:25.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.130+0000 7f537cb5d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5378103b30 0x7f537819ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:25.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.130+0000 7f537cb5d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5378107e50 0x7f537819b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:25.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.130+0000 7f537cb5d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53781087a0 0x7f537819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:25.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.130+0000 7f537cb5d700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5378114560 con 0x7f5378103b30 2024-05-04T03:16:25.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.130+0000 7f537cb5d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f53781143e0 con 0x7f53781087a0 2024-05-04T03:16:25.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.130+0000 7f537cb5d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f53781146e0 con 0x7f5378107e50 2024-05-04T03:16:25.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.131+0000 7f537659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5378103b30 0x7f537819ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:25.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.131+0000 7f537659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5378103b30 0x7f537819ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60504/0 (socket says 172.21.15.1:60504) 2024-05-04T03:16:25.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.131+0000 7f537659c700 1 -- 172.21.15.1:0/2173624842 learned_addr learned my addr 172.21.15.1:0/2173624842 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:16:25.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.131+0000 7f5376d9d700 1 --2- 172.21.15.1:0/2173624842 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53781087a0 0x7f537819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:25.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.131+0000 7f5375d9b700 1 --2- 172.21.15.1:0/2173624842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5378107e50 0x7f537819b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:25.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.131+0000 7f537659c700 1 -- 172.21.15.1:0/2173624842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5378107e50 msgr2=0x7f537819b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:25.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.131+0000 7f537659c700 1 --2- 172.21.15.1:0/2173624842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5378107e50 0x7f537819b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:25.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.131+0000 7f537659c700 1 -- 172.21.15.1:0/2173624842 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53781087a0 msgr2=0x7f537819f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:25.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.131+0000 7f537659c700 1 --2- 172.21.15.1:0/2173624842 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53781087a0 0x7f537819f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:25.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.131+0000 7f537659c700 1 -- 172.21.15.1:0/2173624842 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537819fd40 con 0x7f5378103b30 2024-05-04T03:16:25.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.131+0000 7f5376d9d700 1 --2- 172.21.15.1:0/2173624842 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53781087a0 0x7f537819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:16:25.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.131+0000 7f5375d9b700 1 --2- 172.21.15.1:0/2173624842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5378107e50 0x7f537819b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:16:25.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.131+0000 7f537659c700 1 --2- 172.21.15.1:0/2173624842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5378103b30 0x7f537819ae70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f536800b580 tx=0x7f53680047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:25.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.131+0000 7f53677fe700 1 -- 172.21.15.1:0/2173624842 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368027070 con 0x7f5378103b30 2024-05-04T03:16:25.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.131+0000 7f53677fe700 1 -- 172.21.15.1:0/2173624842 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5368004020 con 0x7f5378103b30 2024-05-04T03:16:25.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.131+0000 7f53677fe700 1 -- 172.21.15.1:0/2173624842 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368022e30 con 0x7f5378103b30 2024-05-04T03:16:25.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.131+0000 7f537cb5d700 1 -- 172.21.15.1:0/2173624842 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537819ffd0 con 0x7f5378103b30 2024-05-04T03:16:25.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.131+0000 7f537cb5d700 1 -- 172.21.15.1:0/2173624842 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53781a0460 con 0x7f5378103b30 2024-05-04T03:16:25.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.132+0000 7f537cb5d700 1 -- 172.21.15.1:0/2173624842 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f537810b7c0 con 0x7f5378103b30 2024-05-04T03:16:25.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.134+0000 7f53677fe700 1 -- 172.21.15.1:0/2173624842 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f5368022590 con 0x7f5378103b30 2024-05-04T03:16:25.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.135+0000 7f53677fe700 1 --2- 172.21.15.1:0/2173624842 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f53600b2910 0x7f53600b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:25.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.135+0000 7f5375d9b700 1 --2- 172.21.15.1:0/2173624842 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f53600b2910 0x7f53600b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:25.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.135+0000 7f53677fe700 1 -- 172.21.15.1:0/2173624842 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f53680e3300 con 0x7f5378103b30 2024-05-04T03:16:25.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.135+0000 7f5375d9b700 1 --2- 172.21.15.1:0/2173624842 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f53600b2910 0x7f53600b4e20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f536c005fd0 tx=0x7f536c005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:25.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.139+0000 7f53677fe700 1 -- 172.21.15.1:0/2173624842 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5368030080 con 0x7f5378103b30 2024-05-04T03:16:25.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.401+0000 7f537cb5d700 1 -- 172.21.15.1:0/2173624842 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53780512c0 con 0x7f5378103b30 2024-05-04T03:16:25.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.401+0000 7f53677fe700 1 -- 172.21.15.1:0/2173624842 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11844 (secure 0 0 0) 0x7f5368013070 con 0x7f5378103b30 2024-05-04T03:16:25.403 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:16:25.404 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2186681910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2186681910},{"type":"v1","addr":"172.21.15.1:6835","nonce":2186681910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2747191047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2747191047},{"type":"v1","addr":"172.21.15.1:6837","nonce":2747191047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4115120540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4115120540},{"type":"v1","addr":"172.21.15.1:6839","nonce":4115120540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3405709780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3405709780},{"type":"v1","addr":"172.21.15.1:6841","nonce":3405709780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/3816098761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":3816098761},{"type":"v1","addr":"172.21.15.72:6833","nonce":3816098761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/379026128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":379026128},{"type":"v1","addr":"172.21.15.77:6833","nonce":379026128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:15:49.823471+0000","modified":"2024-05-04T03:16:07.918400+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24472,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/3930984976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3930984976},{"type":"v1","addr":"172.21.15.72:6835","nonce":3930984976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/4279951305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4279951305},{"type":"v1","addr":"172.21.15.77:6835","nonce":4279951305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2003521417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2003521417},{"type":"v1","addr":"172.21.15.77:6837","nonce":2003521417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/1417614203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1417614203},{"type":"v1","addr":"172.21.15.72:6837","nonce":1417614203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.77:6839/922723189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":922723189},{"type":"v1","addr":"172.21.15.77:6839","nonce":922723189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2976050650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2976050650},{"type":"v1","addr":"172.21.15.72:6839","nonce":2976050650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24472,24494,24467]},"id":1}]} 2024-05-04T03:16:25.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.404+0000 7f537cb5d700 1 -- 172.21.15.1:0/2173624842 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f53600b2910 msgr2=0x7f53600b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:25.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.404+0000 7f537cb5d700 1 --2- 172.21.15.1:0/2173624842 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f53600b2910 0x7f53600b4e20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f536c005fd0 tx=0x7f536c005dc0 comp rx=0 tx=0).stop 2024-05-04T03:16:25.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.404+0000 7f537cb5d700 1 -- 172.21.15.1:0/2173624842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5378103b30 msgr2=0x7f537819ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:25.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.404+0000 7f537cb5d700 1 --2- 172.21.15.1:0/2173624842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5378103b30 0x7f537819ae70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f536800b580 tx=0x7f53680047b0 comp rx=0 tx=0).stop 2024-05-04T03:16:25.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.404+0000 7f537cb5d700 1 -- 172.21.15.1:0/2173624842 shutdown_connections 2024-05-04T03:16:25.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.404+0000 7f537cb5d700 1 --2- 172.21.15.1:0/2173624842 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53781087a0 0x7f537819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:25.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.404+0000 7f537cb5d700 1 --2- 172.21.15.1:0/2173624842 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f53600b2910 0x7f53600b4e20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:25.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.404+0000 7f537cb5d700 1 --2- 172.21.15.1:0/2173624842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5378103b30 0x7f537819ae70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:25.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.404+0000 7f537cb5d700 1 --2- 172.21.15.1:0/2173624842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5378107e50 0x7f537819b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:25.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.404+0000 7f537cb5d700 1 -- 172.21.15.1:0/2173624842 >> 172.21.15.1:0/2173624842 conn(0x7f53780fe500 msgr2=0x7f5378100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:25.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.404+0000 7f537cb5d700 1 -- 172.21.15.1:0/2173624842 shutdown_connections 2024-05-04T03:16:25.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:25.404+0000 7f537cb5d700 1 -- 172.21.15.1:0/2173624842 wait complete. 2024-05-04T03:16:25.407 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-05-04T03:16:25.963 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-04T03:16:26.212 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:26.622 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:26 smithi001 ceph-mon[29731]: pgmap v307: 97 pgs: 97 active+clean; 625 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-05-04T03:16:26.622 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:26 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2173624842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:16:26.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:26 smithi077 ceph-mon[33531]: pgmap v307: 97 pgs: 97 active+clean; 625 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-05-04T03:16:26.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:26 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/2173624842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:16:26.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:26 smithi072 ceph-mon[33332]: pgmap v307: 97 pgs: 97 active+clean; 625 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-05-04T03:16:26.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:26 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/2173624842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:16:27.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.725+0000 7f76d9290700 1 -- 172.21.15.1:0/3070988416 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76d41045e0 msgr2=0x7f76d41049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:27.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.725+0000 7f76d9290700 1 --2- 172.21.15.1:0/3070988416 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76d41045e0 0x7f76d41049f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f76cc00b0d0 tx=0x7f76cc00b3e0 comp rx=0 tx=0).stop 2024-05-04T03:16:27.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.725+0000 7f76d9290700 1 -- 172.21.15.1:0/3070988416 shutdown_connections 2024-05-04T03:16:27.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.725+0000 7f76d9290700 1 --2- 172.21.15.1:0/3070988416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76d4109860 0x7f76d41111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:27.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.725+0000 7f76d9290700 1 --2- 172.21.15.1:0/3070988416 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76d41045e0 0x7f76d41049f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:27.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.725+0000 7f76d9290700 1 --2- 172.21.15.1:0/3070988416 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f76d4108f10 0x7f76d4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:27.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.725+0000 7f76d9290700 1 -- 172.21.15.1:0/3070988416 >> 172.21.15.1:0/3070988416 conn(0x7f76d40ffed0 msgr2=0x7f76d4102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:27.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.725+0000 7f76d9290700 1 -- 172.21.15.1:0/3070988416 shutdown_connections 2024-05-04T03:16:27.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.725+0000 7f76d9290700 1 -- 172.21.15.1:0/3070988416 wait complete. 2024-05-04T03:16:27.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.726+0000 7f76d9290700 1 Processor -- start 2024-05-04T03:16:27.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.726+0000 7f76d9290700 1 -- start start 2024-05-04T03:16:27.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.726+0000 7f76d9290700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f76d41045e0 0x7f76d419ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:27.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.726+0000 7f76d9290700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76d4108f10 0x7f76d419b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:27.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.726+0000 7f76d9290700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76d4109860 0x7f76d419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:27.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.726+0000 7f76d9290700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f76d4114500 con 0x7f76d4109860 2024-05-04T03:16:27.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.726+0000 7f76d9290700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f76d4114380 con 0x7f76d41045e0 2024-05-04T03:16:27.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.726+0000 7f76d9290700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f76d4114680 con 0x7f76d4108f10 2024-05-04T03:16:27.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.726+0000 7f76d37fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76d4109860 0x7f76d419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:27.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.726+0000 7f76d37fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76d4109860 0x7f76d419f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60526/0 (socket says 172.21.15.1:60526) 2024-05-04T03:16:27.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.726+0000 7f76d37fe700 1 -- 172.21.15.1:0/2565845471 learned_addr learned my addr 172.21.15.1:0/2565845471 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:16:27.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.727+0000 7f76d27fc700 1 --2- 172.21.15.1:0/2565845471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76d4108f10 0x7f76d419b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:27.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.727+0000 7f76d2ffd700 1 --2- 172.21.15.1:0/2565845471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f76d41045e0 0x7f76d419ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:27.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.727+0000 7f76d37fe700 1 -- 172.21.15.1:0/2565845471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76d4108f10 msgr2=0x7f76d419b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:27.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.727+0000 7f76d37fe700 1 --2- 172.21.15.1:0/2565845471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76d4108f10 0x7f76d419b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:27.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.727+0000 7f76d37fe700 1 -- 172.21.15.1:0/2565845471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f76d41045e0 msgr2=0x7f76d419ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:27.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.727+0000 7f76d37fe700 1 --2- 172.21.15.1:0/2565845471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f76d41045e0 0x7f76d419ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:27.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.727+0000 7f76d37fe700 1 -- 172.21.15.1:0/2565845471 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d419fd60 con 0x7f76d4109860 2024-05-04T03:16:27.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.727+0000 7f76d2ffd700 1 --2- 172.21.15.1:0/2565845471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f76d41045e0 0x7f76d419ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:16:27.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.727+0000 7f76d37fe700 1 --2- 172.21.15.1:0/2565845471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76d4109860 0x7f76d419f6c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f76c400c970 tx=0x7f76c400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:27.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.727+0000 7f76cbfff700 1 -- 172.21.15.1:0/2565845471 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c400ce60 con 0x7f76d4109860 2024-05-04T03:16:27.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.727+0000 7f76cbfff700 1 -- 172.21.15.1:0/2565845471 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76c4004830 con 0x7f76d4109860 2024-05-04T03:16:27.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.727+0000 7f76cbfff700 1 -- 172.21.15.1:0/2565845471 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c40054c0 con 0x7f76d4109860 2024-05-04T03:16:27.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.727+0000 7f76d9290700 1 -- 172.21.15.1:0/2565845471 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d41a0050 con 0x7f76d4109860 2024-05-04T03:16:27.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.727+0000 7f76d9290700 1 -- 172.21.15.1:0/2565845471 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d41a7970 con 0x7f76d4109860 2024-05-04T03:16:27.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.728+0000 7f76d9290700 1 -- 172.21.15.1:0/2565845471 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76d40512c0 con 0x7f76d4109860 2024-05-04T03:16:27.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.730+0000 7f76cbfff700 1 -- 172.21.15.1:0/2565845471 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f76c40049e0 con 0x7f76d4109860 2024-05-04T03:16:27.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.730+0000 7f76cbfff700 1 --2- 172.21.15.1:0/2565845471 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f76c00b2990 0x7f76c00b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:27.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.731+0000 7f76d2ffd700 1 --2- 172.21.15.1:0/2565845471 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f76c00b2990 0x7f76c00b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:27.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.731+0000 7f76cbfff700 1 -- 172.21.15.1:0/2565845471 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f76c40e2090 con 0x7f76d4109860 2024-05-04T03:16:27.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.731+0000 7f76d2ffd700 1 --2- 172.21.15.1:0/2565845471 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f76c00b2990 0x7f76c00b4ea0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f76cc00d010 tx=0x7f76cc00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:27.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.734+0000 7f76cbfff700 1 -- 172.21.15.1:0/2565845471 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f76c4014030 con 0x7f76d4109860 2024-05-04T03:16:27.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.956+0000 7f76d9290700 1 -- 172.21.15.1:0/2565845471 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76d402e5f0 con 0x7f76d4109860 2024-05-04T03:16:27.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.958+0000 7f76cbfff700 1 -- 172.21.15.1:0/2565845471 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22088 (secure 0 0 0) 0x7f76c40a9e80 con 0x7f76d4109860 2024-05-04T03:16:27.959 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:16:27.960 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":81,"fsid":"4a572aa8-09c3-11ef-bc97-c7b262605968","created":"2024-05-04T03:07:32.845547+0000","modified":"2024-05-04T03:15:49.823513+0000","last_up_change":"2024-05-04T03:14:44.192571+0000","last_in_change":"2024-05-04T03:14:23.353342+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T03:11:01.139189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T03:15:43.151696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T03:15:46.753539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"100fe035-2454-4c95-90c7-759956ec36a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6803","nonce":3479011484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6805","nonce":3479011484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6809","nonce":3479011484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6807","nonce":3479011484}]},"public_addr":"172.21.15.1:6803/3479011484","cluster_addr":"172.21.15.1:6805/3479011484","heartbeat_back_addr":"172.21.15.1:6809/3479011484","heartbeat_front_addr":"172.21.15.1:6807/3479011484","state":["exists","up"]},{"osd":1,"uuid":"5826ac76-ef6b-4040-8ad1-305db355af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6801","nonce":3725706149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6803","nonce":3725706149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6807","nonce":3725706149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6805","nonce":3725706149}]},"public_addr":"172.21.15.72:6801/3725706149","cluster_addr":"172.21.15.72:6803/3725706149","heartbeat_back_addr":"172.21.15.72:6807/3725706149","heartbeat_front_addr":"172.21.15.72:6805/3725706149","state":["exists","up"]},{"osd":2,"uuid":"923d76e5-b032-4190-9e35-546d27722a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6801","nonce":515038343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6803","nonce":515038343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6807","nonce":515038343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6805","nonce":515038343}]},"public_addr":"172.21.15.77:6801/515038343","cluster_addr":"172.21.15.77:6803/515038343","heartbeat_back_addr":"172.21.15.77:6807/515038343","heartbeat_front_addr":"172.21.15.77:6805/515038343","state":["exists","up"]},{"osd":3,"uuid":"495cc0c5-6a99-481f-ad9e-38f05fb84356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6811","nonce":4050439365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6813","nonce":4050439365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6817","nonce":4050439365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6815","nonce":4050439365}]},"public_addr":"172.21.15.1:6811/4050439365","cluster_addr":"172.21.15.1:6813/4050439365","heartbeat_back_addr":"172.21.15.1:6817/4050439365","heartbeat_front_addr":"172.21.15.1:6815/4050439365","state":["exists","up"]},{"osd":4,"uuid":"b86b8026-79c0-4056-9e67-a61f54b6ab5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6809","nonce":2501732391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6811","nonce":2501732391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6815","nonce":2501732391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6813","nonce":2501732391}]},"public_addr":"172.21.15.72:6809/2501732391","cluster_addr":"172.21.15.72:6811/2501732391","heartbeat_back_addr":"172.21.15.72:6815/2501732391","heartbeat_front_addr":"172.21.15.72:6813/2501732391","state":["exists","up"]},{"osd":5,"uuid":"80975cb9-39e0-46b8-a343-9ff69194a14e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6809","nonce":3582852165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6811","nonce":3582852165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6815","nonce":3582852165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6813","nonce":3582852165}]},"public_addr":"172.21.15.77:6809/3582852165","cluster_addr":"172.21.15.77:6811/3582852165","heartbeat_back_addr":"172.21.15.77:6815/3582852165","heartbeat_front_addr":"172.21.15.77:6813/3582852165","state":["exists","up"]},{"osd":6,"uuid":"aca3fac5-fbb5-42de-bf0e-e4a0db7381e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6819","nonce":1799033649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6821","nonce":1799033649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6825","nonce":1799033649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6823","nonce":1799033649}]},"public_addr":"172.21.15.1:6819/1799033649","cluster_addr":"172.21.15.1:6821/1799033649","heartbeat_back_addr":"172.21.15.1:6825/1799033649","heartbeat_front_addr":"172.21.15.1:6823/1799033649","state":["exists","up"]},{"osd":7,"uuid":"cbff6b8a-c463-4249-ae45-fdc6f99a5489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6817","nonce":3460835447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6819","nonce":3460835447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6823","nonce":3460835447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6821","nonce":3460835447}]},"public_addr":"172.21.15.72:6817/3460835447","cluster_addr":"172.21.15.72:6819/3460835447","heartbeat_back_addr":"172.21.15.72:6823/3460835447","heartbeat_front_addr":"172.21.15.72:6821/3460835447","state":["exists","up"]},{"osd":8,"uuid":"e140bf13-4887-4d68-9373-3d4fc1c4584a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6817","nonce":1124479323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6819","nonce":1124479323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6823","nonce":1124479323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6821","nonce":1124479323}]},"public_addr":"172.21.15.77:6817/1124479323","cluster_addr":"172.21.15.77:6819/1124479323","heartbeat_back_addr":"172.21.15.77:6823/1124479323","heartbeat_front_addr":"172.21.15.77:6821/1124479323","state":["exists","up"]},{"osd":9,"uuid":"f1e8a232-cb11-4b9d-8789-e1637fa5aaa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6827","nonce":2681322272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6829","nonce":2681322272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6833","nonce":2681322272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6831","nonce":2681322272}]},"public_addr":"172.21.15.1:6827/2681322272","cluster_addr":"172.21.15.1:6829/2681322272","heartbeat_back_addr":"172.21.15.1:6833/2681322272","heartbeat_front_addr":"172.21.15.1:6831/2681322272","state":["exists","up"]},{"osd":10,"uuid":"79b570af-3d1e-439d-a328-9e6bc7d67839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6825","nonce":3293703569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6827","nonce":3293703569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6831","nonce":3293703569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6829","nonce":3293703569}]},"public_addr":"172.21.15.72:6825/3293703569","cluster_addr":"172.21.15.72:6827/3293703569","heartbeat_back_addr":"172.21.15.72:6831/3293703569","heartbeat_front_addr":"172.21.15.72:6829/3293703569","state":["exists","up"]},{"osd":11,"uuid":"e35bf83d-9ee4-4745-96c2-1c35cc8eefba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6825","nonce":4155947548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6827","nonce":4155947548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6831","nonce":4155947548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6829","nonce":4155947548}]},"public_addr":"172.21.15.77:6825/4155947548","cluster_addr":"172.21.15.77:6827/4155947548","heartbeat_back_addr":"172.21.15.77:6831/4155947548","heartbeat_front_addr":"172.21.15.77:6829/4155947548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:09.467126+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:33.861139+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:57.741962+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:11:21.746021+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:11:46.323252+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:12:10.640180+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:12:35.750591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:00.306549+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:25.012495+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:51.185229+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:14:16.403850+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:14:41.653316+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/2029897106":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/402488450":"2024-05-05T03:08:33.045884+0000","172.21.15.1:6801/128793132":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/2233526606":"2024-05-05T03:08:06.135783+0000","172.21.15.1:0/2715541855":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/2415697325":"2024-05-05T03:08:06.135783+0000","172.21.15.1:0/2305298106":"2024-05-05T03:08:06.135783+0000","172.21.15.1:6800/128793132":"2024-05-05T03:08:33.045884+0000","172.21.15.1:6800/4200529863":"2024-05-05T03:08:06.135783+0000","172.21.15.1:6801/4200529863":"2024-05-05T03:08:06.135783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T03:16:27.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.960+0000 7f76d9290700 1 -- 172.21.15.1:0/2565845471 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f76c00b2990 msgr2=0x7f76c00b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:27.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.960+0000 7f76d9290700 1 --2- 172.21.15.1:0/2565845471 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f76c00b2990 0x7f76c00b4ea0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f76cc00d010 tx=0x7f76cc00b040 comp rx=0 tx=0).stop 2024-05-04T03:16:27.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.960+0000 7f76d9290700 1 -- 172.21.15.1:0/2565845471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76d4109860 msgr2=0x7f76d419f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:27.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.960+0000 7f76d9290700 1 --2- 172.21.15.1:0/2565845471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76d4109860 0x7f76d419f6c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f76c400c970 tx=0x7f76c400cc80 comp rx=0 tx=0).stop 2024-05-04T03:16:27.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.960+0000 7f76d9290700 1 -- 172.21.15.1:0/2565845471 shutdown_connections 2024-05-04T03:16:27.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.960+0000 7f76d9290700 1 --2- 172.21.15.1:0/2565845471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f76d41045e0 0x7f76d419ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:27.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.960+0000 7f76d9290700 1 --2- 172.21.15.1:0/2565845471 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f76c00b2990 0x7f76c00b4ea0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:27.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.960+0000 7f76d9290700 1 --2- 172.21.15.1:0/2565845471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76d4109860 0x7f76d419f6c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:27.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.960+0000 7f76d9290700 1 --2- 172.21.15.1:0/2565845471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76d4108f10 0x7f76d419b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:27.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.960+0000 7f76d9290700 1 -- 172.21.15.1:0/2565845471 >> 172.21.15.1:0/2565845471 conn(0x7f76d40ffed0 msgr2=0x7f76d410a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:27.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.960+0000 7f76d9290700 1 -- 172.21.15.1:0/2565845471 shutdown_connections 2024-05-04T03:16:27.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:27.960+0000 7f76d9290700 1 -- 172.21.15.1:0/2565845471 wait complete. 2024-05-04T03:16:28.607 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:28 smithi001 ceph-mon[29731]: pgmap v308: 97 pgs: 97 active+clean; 625 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 6 op/s 2024-05-04T03:16:28.607 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:28 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2565845471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:16:28.638 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-04T03:16:28.638 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-04T03:16:28.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:28 smithi077 ceph-mon[33531]: pgmap v308: 97 pgs: 97 active+clean; 625 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 6 op/s 2024-05-04T03:16:28.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:28 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/2565845471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:16:28.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:28 smithi072 ceph-mon[33332]: pgmap v308: 97 pgs: 97 active+clean; 625 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 6 op/s 2024-05-04T03:16:28.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:28 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/2565845471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:16:28.888 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:30.382 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:30 smithi001 ceph-mon[29731]: pgmap v309: 97 pgs: 97 active+clean; 634 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-05-04T03:16:30.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.405+0000 7fe5b5013700 1 -- 172.21.15.1:0/759622209 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5b0105e40 msgr2=0x7fe5b010e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:30.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.405+0000 7fe5b5013700 1 --2- 172.21.15.1:0/759622209 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5b0105e40 0x7fe5b010e390 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4009990 tx=0x7fe5a4009ca0 comp rx=0 tx=0).stop 2024-05-04T03:16:30.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.405+0000 7fe5b5013700 1 -- 172.21.15.1:0/759622209 shutdown_connections 2024-05-04T03:16:30.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.405+0000 7fe5b5013700 1 --2- 172.21.15.1:0/759622209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5b01054f0 0x7fe5b0105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:30.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.405+0000 7fe5b5013700 1 --2- 172.21.15.1:0/759622209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5b010e920 0x7fe5b0110e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:30.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.406+0000 7fe5b5013700 1 --2- 172.21.15.1:0/759622209 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5b0105e40 0x7fe5b010e390 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:30.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.406+0000 7fe5b5013700 1 -- 172.21.15.1:0/759622209 >> 172.21.15.1:0/759622209 conn(0x7fe5b0078e00 msgr2=0x7fe5b0079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:30.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.406+0000 7fe5b5013700 1 -- 172.21.15.1:0/759622209 shutdown_connections 2024-05-04T03:16:30.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.406+0000 7fe5b5013700 1 -- 172.21.15.1:0/759622209 wait complete. 2024-05-04T03:16:30.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.406+0000 7fe5b5013700 1 Processor -- start 2024-05-04T03:16:30.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.407+0000 7fe5b5013700 1 -- start start 2024-05-04T03:16:30.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.407+0000 7fe5b5013700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5b01054f0 0x7fe5b019af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:30.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.407+0000 7fe5b5013700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5b0105e40 0x7fe5b019b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:30.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.407+0000 7fe5b5013700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5b010e920 0x7fe5b019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:30.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.407+0000 7fe5b5013700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b01144d0 con 0x7fe5b01054f0 2024-05-04T03:16:30.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.407+0000 7fe5b5013700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b0114350 con 0x7fe5b010e920 2024-05-04T03:16:30.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.407+0000 7fe5b5013700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b0114650 con 0x7fe5b0105e40 2024-05-04T03:16:30.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.407+0000 7fe5aed9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5b01054f0 0x7fe5b019af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:30.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.407+0000 7fe5aed9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5b01054f0 0x7fe5b019af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60552/0 (socket says 172.21.15.1:60552) 2024-05-04T03:16:30.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.407+0000 7fe5aed9d700 1 -- 172.21.15.1:0/429327637 learned_addr learned my addr 172.21.15.1:0/429327637 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:16:30.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.407+0000 7fe5ae59c700 1 --2- 172.21.15.1:0/429327637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5b0105e40 0x7fe5b019b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:30.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.407+0000 7fe5af59e700 1 --2- 172.21.15.1:0/429327637 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5b010e920 0x7fe5b019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:30.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.407+0000 7fe5aed9d700 1 -- 172.21.15.1:0/429327637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5b0105e40 msgr2=0x7fe5b019b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:30.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.407+0000 7fe5aed9d700 1 --2- 172.21.15.1:0/429327637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5b0105e40 0x7fe5b019b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:30.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.407+0000 7fe5aed9d700 1 -- 172.21.15.1:0/429327637 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5b010e920 msgr2=0x7fe5b019f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:30.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.407+0000 7fe5aed9d700 1 --2- 172.21.15.1:0/429327637 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5b010e920 0x7fe5b019f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:30.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.407+0000 7fe5aed9d700 1 -- 172.21.15.1:0/429327637 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b019fdd0 con 0x7fe5b01054f0 2024-05-04T03:16:30.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.407+0000 7fe5af59e700 1 --2- 172.21.15.1:0/429327637 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5b010e920 0x7fe5b019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:16:30.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.408+0000 7fe5aed9d700 1 --2- 172.21.15.1:0/429327637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5b01054f0 0x7fe5b019af00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000d950 tx=0x7fe5a000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:30.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.408+0000 7fe597fff700 1 -- 172.21.15.1:0/429327637 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0016070 con 0x7fe5b01054f0 2024-05-04T03:16:30.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.408+0000 7fe5b5013700 1 -- 172.21.15.1:0/429327637 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5b01a00c0 con 0x7fe5b01054f0 2024-05-04T03:16:30.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.408+0000 7fe597fff700 1 -- 172.21.15.1:0/429327637 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5a0010460 con 0x7fe5b01054f0 2024-05-04T03:16:30.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.408+0000 7fe597fff700 1 -- 172.21.15.1:0/429327637 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a000f620 con 0x7fe5b01054f0 2024-05-04T03:16:30.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.408+0000 7fe5b5013700 1 -- 172.21.15.1:0/429327637 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5b01a79e0 con 0x7fe5b01054f0 2024-05-04T03:16:30.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.409+0000 7fe5b5013700 1 -- 172.21.15.1:0/429327637 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5b00512c0 con 0x7fe5b01054f0 2024-05-04T03:16:30.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.410+0000 7fe597fff700 1 -- 172.21.15.1:0/429327637 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fe5a0019070 con 0x7fe5b01054f0 2024-05-04T03:16:30.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.411+0000 7fe597fff700 1 --2- 172.21.15.1:0/429327637 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe5980b2910 0x7fe5980b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:30.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.411+0000 7fe5ae59c700 1 --2- 172.21.15.1:0/429327637 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe5980b2910 0x7fe5980b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:30.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.411+0000 7fe597fff700 1 -- 172.21.15.1:0/429327637 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fe5a00e1fc0 con 0x7fe5b01054f0 2024-05-04T03:16:30.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.412+0000 7fe5ae59c700 1 --2- 172.21.15.1:0/429327637 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe5980b2910 0x7fe5980b4e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4003800 tx=0x7fe5a401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:30.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.416+0000 7fe597fff700 1 -- 172.21.15.1:0/429327637 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe5a00a9c30 con 0x7fe5b01054f0 2024-05-04T03:16:30.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.630+0000 7fe5b5013700 1 -- 172.21.15.1:0/429327637 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe5b019c260 con 0x7fe5980b2910 2024-05-04T03:16:30.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.638+0000 7fe597fff700 1 -- 172.21.15.1:0/429327637 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258695 (secure 0 0 0) 0x7fe5b019c260 con 0x7fe5980b2910 2024-05-04T03:16:30.644 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:16:30.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.641+0000 7fe5b5013700 1 -- 172.21.15.1:0/429327637 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe5980b2910 msgr2=0x7fe5980b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:30.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.641+0000 7fe5b5013700 1 --2- 172.21.15.1:0/429327637 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe5980b2910 0x7fe5980b4e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4003800 tx=0x7fe5a401a040 comp rx=0 tx=0).stop 2024-05-04T03:16:30.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.641+0000 7fe5b5013700 1 -- 172.21.15.1:0/429327637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5b01054f0 msgr2=0x7fe5b019af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:30.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.641+0000 7fe5b5013700 1 --2- 172.21.15.1:0/429327637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5b01054f0 0x7fe5b019af00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000d950 tx=0x7fe5a000dd10 comp rx=0 tx=0).stop 2024-05-04T03:16:30.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.641+0000 7fe5b5013700 1 -- 172.21.15.1:0/429327637 shutdown_connections 2024-05-04T03:16:30.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.641+0000 7fe5b5013700 1 --2- 172.21.15.1:0/429327637 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5b010e920 0x7fe5b019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:30.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.641+0000 7fe5b5013700 1 --2- 172.21.15.1:0/429327637 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe5980b2910 0x7fe5980b4e20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:30.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.641+0000 7fe5b5013700 1 --2- 172.21.15.1:0/429327637 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5b01054f0 0x7fe5b019af00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:30.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.641+0000 7fe5b5013700 1 --2- 172.21.15.1:0/429327637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe5b0105e40 0x7fe5b019b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:30.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.641+0000 7fe5b5013700 1 -- 172.21.15.1:0/429327637 >> 172.21.15.1:0/429327637 conn(0x7fe5b0078e00 msgr2=0x7fe5b01011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:30.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.641+0000 7fe5b5013700 1 -- 172.21.15.1:0/429327637 shutdown_connections 2024-05-04T03:16:30.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:30.641+0000 7fe5b5013700 1 -- 172.21.15.1:0/429327637 wait complete. 2024-05-04T03:16:30.647 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-04T03:16:30.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:30 smithi077 ceph-mon[33531]: pgmap v309: 97 pgs: 97 active+clean; 634 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-05-04T03:16:30.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:30 smithi072 ceph-mon[33332]: pgmap v309: 97 pgs: 97 active+clean; 634 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-05-04T03:16:31.288 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":309,"stamp":"2024-05-04T03:16:29.146773+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":648975,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":238,"num_read_kb":208,"num_write":281,"num_write_kb":2253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":407296,"kb_used_data":12384,"kb_used_omap":18,"kb_used_meta":394797,"kb_avail":1124534528,"statfs":{"total":1151940427776,"available":1151523356672,"internally_reserved":0,"allocated":12681216,"data_stored":7023981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":404272441},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46394,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":89,"num_read_kb":92,"num_write":26,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003408"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.038080+0000","last_change":"2024-05-04T03:15:48.049361+0000","last_active":"2024-05-04T03:15:50.038080+0000","last_peered":"2024-05-04T03:15:50.038080+0000","last_clean":"2024-05-04T03:15:50.038080+0000","last_became_active":"2024-05-04T03:15:48.048876+0000","last_became_peered":"2024-05-04T03:15:48.048876+0000","last_unstale":"2024-05-04T03:15:50.038080+0000","last_undegraded":"2024-05-04T03:15:50.038080+0000","last_fullsized":"2024-05-04T03:15:50.038080+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:50:52.568516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.050624+0000","last_change":"2024-05-04T03:15:48.055554+0000","last_active":"2024-05-04T03:15:50.050624+0000","last_peered":"2024-05-04T03:15:50.050624+0000","last_clean":"2024-05-04T03:15:50.050624+0000","last_became_active":"2024-05-04T03:15:48.055155+0000","last_became_peered":"2024-05-04T03:15:48.055155+0000","last_unstale":"2024-05-04T03:15:50.050624+0000","last_undegraded":"2024-05-04T03:15:50.050624+0000","last_fullsized":"2024-05-04T03:15:50.050624+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:38:07.824031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.050185+0000","last_change":"2024-05-04T03:15:48.047974+0000","last_active":"2024-05-04T03:15:50.050185+0000","last_peered":"2024-05-04T03:15:50.050185+0000","last_clean":"2024-05-04T03:15:50.050185+0000","last_became_active":"2024-05-04T03:15:48.044299+0000","last_became_peered":"2024-05-04T03:15:48.044299+0000","last_unstale":"2024-05-04T03:15:50.050185+0000","last_undegraded":"2024-05-04T03:15:50.050185+0000","last_fullsized":"2024-05-04T03:15:50.050185+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:49:08.014987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.054833+0000","last_change":"2024-05-04T03:15:48.050899+0000","last_active":"2024-05-04T03:15:50.054833+0000","last_peered":"2024-05-04T03:15:50.054833+0000","last_clean":"2024-05-04T03:15:50.054833+0000","last_became_active":"2024-05-04T03:15:48.048812+0000","last_became_peered":"2024-05-04T03:15:48.048812+0000","last_unstale":"2024-05-04T03:15:50.054833+0000","last_undegraded":"2024-05-04T03:15:50.054833+0000","last_fullsized":"2024-05-04T03:15:50.054833+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:21:01.829349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.046558+0000","last_change":"2024-05-04T03:15:48.044403+0000","last_active":"2024-05-04T03:15:50.046558+0000","last_peered":"2024-05-04T03:15:50.046558+0000","last_clean":"2024-05-04T03:15:50.046558+0000","last_became_active":"2024-05-04T03:15:48.044021+0000","last_became_peered":"2024-05-04T03:15:48.044021+0000","last_unstale":"2024-05-04T03:15:50.046558+0000","last_undegraded":"2024-05-04T03:15:50.046558+0000","last_fullsized":"2024-05-04T03:15:50.046558+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:50:54.762070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.040245+0000","last_change":"2024-05-04T03:15:48.053233+0000","last_active":"2024-05-04T03:15:50.040245+0000","last_peered":"2024-05-04T03:15:50.040245+0000","last_clean":"2024-05-04T03:15:50.040245+0000","last_became_active":"2024-05-04T03:15:48.052809+0000","last_became_peered":"2024-05-04T03:15:48.052809+0000","last_unstale":"2024-05-04T03:15:50.040245+0000","last_undegraded":"2024-05-04T03:15:50.040245+0000","last_fullsized":"2024-05-04T03:15:50.040245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:27:20.082694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.051600+0000","last_change":"2024-05-04T03:15:48.045108+0000","last_active":"2024-05-04T03:15:50.051600+0000","last_peered":"2024-05-04T03:15:50.051600+0000","last_clean":"2024-05-04T03:15:50.051600+0000","last_became_active":"2024-05-04T03:15:48.044719+0000","last_became_peered":"2024-05-04T03:15:48.044719+0000","last_unstale":"2024-05-04T03:15:50.051600+0000","last_undegraded":"2024-05-04T03:15:50.051600+0000","last_fullsized":"2024-05-04T03:15:50.051600+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:55:02.586294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.046745+0000","last_change":"2024-05-04T03:15:48.043559+0000","last_active":"2024-05-04T03:15:50.046745+0000","last_peered":"2024-05-04T03:15:50.046745+0000","last_clean":"2024-05-04T03:15:50.046745+0000","last_became_active":"2024-05-04T03:15:48.043138+0000","last_became_peered":"2024-05-04T03:15:48.043138+0000","last_unstale":"2024-05-04T03:15:50.046745+0000","last_undegraded":"2024-05-04T03:15:50.046745+0000","last_fullsized":"2024-05-04T03:15:50.046745+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:05:25.411916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.048825+0000","last_change":"2024-05-04T03:15:48.039632+0000","last_active":"2024-05-04T03:15:50.048825+0000","last_peered":"2024-05-04T03:15:50.048825+0000","last_clean":"2024-05-04T03:15:50.048825+0000","last_became_active":"2024-05-04T03:15:48.039231+0000","last_became_peered":"2024-05-04T03:15:48.039231+0000","last_unstale":"2024-05-04T03:15:50.048825+0000","last_undegraded":"2024-05-04T03:15:50.048825+0000","last_fullsized":"2024-05-04T03:15:50.048825+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:33:12.853172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.040669+0000","last_change":"2024-05-04T03:15:48.055138+0000","last_active":"2024-05-04T03:15:50.040669+0000","last_peered":"2024-05-04T03:15:50.040669+0000","last_clean":"2024-05-04T03:15:50.040669+0000","last_became_active":"2024-05-04T03:15:48.054791+0000","last_became_peered":"2024-05-04T03:15:48.054791+0000","last_unstale":"2024-05-04T03:15:50.040669+0000","last_undegraded":"2024-05-04T03:15:50.040669+0000","last_fullsized":"2024-05-04T03:15:50.040669+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:33:36.822549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.052925+0000","last_change":"2024-05-04T03:15:48.050125+0000","last_active":"2024-05-04T03:15:50.052925+0000","last_peered":"2024-05-04T03:15:50.052925+0000","last_clean":"2024-05-04T03:15:50.052925+0000","last_became_active":"2024-05-04T03:15:48.049390+0000","last_became_peered":"2024-05-04T03:15:48.049390+0000","last_unstale":"2024-05-04T03:15:50.052925+0000","last_undegraded":"2024-05-04T03:15:50.052925+0000","last_fullsized":"2024-05-04T03:15:50.052925+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:27:05.740383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.054736+0000","last_change":"2024-05-04T03:15:48.050700+0000","last_active":"2024-05-04T03:15:50.054736+0000","last_peered":"2024-05-04T03:15:50.054736+0000","last_clean":"2024-05-04T03:15:50.054736+0000","last_became_active":"2024-05-04T03:15:48.049288+0000","last_became_peered":"2024-05-04T03:15:48.049288+0000","last_unstale":"2024-05-04T03:15:50.054736+0000","last_undegraded":"2024-05-04T03:15:50.054736+0000","last_fullsized":"2024-05-04T03:15:50.054736+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:40:07.072006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.037513+0000","last_change":"2024-05-04T03:15:48.037959+0000","last_active":"2024-05-04T03:15:50.037513+0000","last_peered":"2024-05-04T03:15:50.037513+0000","last_clean":"2024-05-04T03:15:50.037513+0000","last_became_active":"2024-05-04T03:15:48.037562+0000","last_became_peered":"2024-05-04T03:15:48.037562+0000","last_unstale":"2024-05-04T03:15:50.037513+0000","last_undegraded":"2024-05-04T03:15:50.037513+0000","last_fullsized":"2024-05-04T03:15:50.037513+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:38:17.971197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.050187+0000","last_change":"2024-05-04T03:15:48.047258+0000","last_active":"2024-05-04T03:15:50.050187+0000","last_peered":"2024-05-04T03:15:50.050187+0000","last_clean":"2024-05-04T03:15:50.050187+0000","last_became_active":"2024-05-04T03:15:48.046897+0000","last_became_peered":"2024-05-04T03:15:48.046897+0000","last_unstale":"2024-05-04T03:15:50.050187+0000","last_undegraded":"2024-05-04T03:15:50.050187+0000","last_fullsized":"2024-05-04T03:15:50.050187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:14:39.217475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.049849+0000","last_change":"2024-05-04T03:15:48.051240+0000","last_active":"2024-05-04T03:15:50.049849+0000","last_peered":"2024-05-04T03:15:50.049849+0000","last_clean":"2024-05-04T03:15:50.049849+0000","last_became_active":"2024-05-04T03:15:48.050081+0000","last_became_peered":"2024-05-04T03:15:48.050081+0000","last_unstale":"2024-05-04T03:15:50.049849+0000","last_undegraded":"2024-05-04T03:15:50.049849+0000","last_fullsized":"2024-05-04T03:15:50.049849+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:39:06.773121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.048192+0000","last_change":"2024-05-04T03:15:48.050936+0000","last_active":"2024-05-04T03:15:50.048192+0000","last_peered":"2024-05-04T03:15:50.048192+0000","last_clean":"2024-05-04T03:15:50.048192+0000","last_became_active":"2024-05-04T03:15:48.050567+0000","last_became_peered":"2024-05-04T03:15:48.050567+0000","last_unstale":"2024-05-04T03:15:50.048192+0000","last_undegraded":"2024-05-04T03:15:50.048192+0000","last_fullsized":"2024-05-04T03:15:50.048192+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:17:43.469889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.038200+0000","last_change":"2024-05-04T03:15:48.047200+0000","last_active":"2024-05-04T03:15:50.038200+0000","last_peered":"2024-05-04T03:15:50.038200+0000","last_clean":"2024-05-04T03:15:50.038200+0000","last_became_active":"2024-05-04T03:15:48.046782+0000","last_became_peered":"2024-05-04T03:15:48.046782+0000","last_unstale":"2024-05-04T03:15:50.038200+0000","last_undegraded":"2024-05-04T03:15:50.038200+0000","last_fullsized":"2024-05-04T03:15:50.038200+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:32:56.773916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.049919+0000","last_change":"2024-05-04T03:15:48.042114+0000","last_active":"2024-05-04T03:15:50.049919+0000","last_peered":"2024-05-04T03:15:50.049919+0000","last_clean":"2024-05-04T03:15:50.049919+0000","last_became_active":"2024-05-04T03:15:48.040921+0000","last_became_peered":"2024-05-04T03:15:48.040921+0000","last_unstale":"2024-05-04T03:15:50.049919+0000","last_undegraded":"2024-05-04T03:15:50.049919+0000","last_fullsized":"2024-05-04T03:15:50.049919+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:16:20.130996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.050660+0000","last_change":"2024-05-04T03:15:48.051060+0000","last_active":"2024-05-04T03:15:50.050660+0000","last_peered":"2024-05-04T03:15:50.050660+0000","last_clean":"2024-05-04T03:15:50.050660+0000","last_became_active":"2024-05-04T03:15:48.050642+0000","last_became_peered":"2024-05-04T03:15:48.050642+0000","last_unstale":"2024-05-04T03:15:50.050660+0000","last_undegraded":"2024-05-04T03:15:50.050660+0000","last_fullsized":"2024-05-04T03:15:50.050660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:23:31.193072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:03.245104+0000","last_change":"2024-05-04T03:15:45.013477+0000","last_active":"2024-05-04T03:16:03.245104+0000","last_peered":"2024-05-04T03:16:03.245104+0000","last_clean":"2024-05-04T03:16:03.245104+0000","last_became_active":"2024-05-04T03:15:45.013161+0000","last_became_peered":"2024-05-04T03:15:45.013161+0000","last_unstale":"2024-05-04T03:16:03.245104+0000","last_undegraded":"2024-05-04T03:16:03.245104+0000","last_fullsized":"2024-05-04T03:16:03.245104+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:08:14.087090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.047134+0000","last_change":"2024-05-04T03:15:48.045810+0000","last_active":"2024-05-04T03:15:50.047134+0000","last_peered":"2024-05-04T03:15:50.047134+0000","last_clean":"2024-05-04T03:15:50.047134+0000","last_became_active":"2024-05-04T03:15:48.045449+0000","last_became_peered":"2024-05-04T03:15:48.045449+0000","last_unstale":"2024-05-04T03:15:50.047134+0000","last_undegraded":"2024-05-04T03:15:50.047134+0000","last_fullsized":"2024-05-04T03:15:50.047134+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:48:05.084208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.052967+0000","last_change":"2024-05-04T03:15:45.204196+0000","last_active":"2024-05-04T03:15:50.052967+0000","last_peered":"2024-05-04T03:15:50.052967+0000","last_clean":"2024-05-04T03:15:50.052967+0000","last_became_active":"2024-05-04T03:15:45.203821+0000","last_became_peered":"2024-05-04T03:15:45.203821+0000","last_unstale":"2024-05-04T03:15:50.052967+0000","last_undegraded":"2024-05-04T03:15:50.052967+0000","last_fullsized":"2024-05-04T03:15:50.052967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:17:05.074450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.050972+0000","last_change":"2024-05-04T03:15:48.045131+0000","last_active":"2024-05-04T03:15:50.050972+0000","last_peered":"2024-05-04T03:15:50.050972+0000","last_clean":"2024-05-04T03:15:50.050972+0000","last_became_active":"2024-05-04T03:15:48.044793+0000","last_became_peered":"2024-05-04T03:15:48.044793+0000","last_unstale":"2024-05-04T03:15:50.050972+0000","last_undegraded":"2024-05-04T03:15:50.050972+0000","last_fullsized":"2024-05-04T03:15:50.050972+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:03:37.769324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:03.242913+0000","last_change":"2024-05-04T03:15:45.013644+0000","last_active":"2024-05-04T03:16:03.242913+0000","last_peered":"2024-05-04T03:16:03.242913+0000","last_clean":"2024-05-04T03:16:03.242913+0000","last_became_active":"2024-05-04T03:15:45.013282+0000","last_became_peered":"2024-05-04T03:15:45.013282+0000","last_unstale":"2024-05-04T03:16:03.242913+0000","last_undegraded":"2024-05-04T03:16:03.242913+0000","last_fullsized":"2024-05-04T03:16:03.242913+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:21:21.204433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.048582+0000","last_change":"2024-05-04T03:15:48.043046+0000","last_active":"2024-05-04T03:15:50.048582+0000","last_peered":"2024-05-04T03:15:50.048582+0000","last_clean":"2024-05-04T03:15:50.048582+0000","last_became_active":"2024-05-04T03:15:48.042621+0000","last_became_peered":"2024-05-04T03:15:48.042621+0000","last_unstale":"2024-05-04T03:15:50.048582+0000","last_undegraded":"2024-05-04T03:15:50.048582+0000","last_fullsized":"2024-05-04T03:15:50.048582+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:03:58.916423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.051927+0000","last_change":"2024-05-04T03:15:45.010618+0000","last_active":"2024-05-04T03:15:50.051927+0000","last_peered":"2024-05-04T03:15:50.051927+0000","last_clean":"2024-05-04T03:15:50.051927+0000","last_became_active":"2024-05-04T03:15:45.009239+0000","last_became_peered":"2024-05-04T03:15:45.009239+0000","last_unstale":"2024-05-04T03:15:50.051927+0000","last_undegraded":"2024-05-04T03:15:50.051927+0000","last_fullsized":"2024-05-04T03:15:50.051927+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:49:46.777571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.051942+0000","last_change":"2024-05-04T03:15:48.049334+0000","last_active":"2024-05-04T03:15:50.051942+0000","last_peered":"2024-05-04T03:15:50.051942+0000","last_clean":"2024-05-04T03:15:50.051942+0000","last_became_active":"2024-05-04T03:15:48.048984+0000","last_became_peered":"2024-05-04T03:15:48.048984+0000","last_unstale":"2024-05-04T03:15:50.051942+0000","last_undegraded":"2024-05-04T03:15:50.051942+0000","last_fullsized":"2024-05-04T03:15:50.051942+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:28:25.366106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:23.281582+0000","last_change":"2024-05-04T03:15:45.015277+0000","last_active":"2024-05-04T03:16:23.281582+0000","last_peered":"2024-05-04T03:16:23.281582+0000","last_clean":"2024-05-04T03:16:23.281582+0000","last_became_active":"2024-05-04T03:15:45.014542+0000","last_became_peered":"2024-05-04T03:15:45.014542+0000","last_unstale":"2024-05-04T03:16:23.281582+0000","last_undegraded":"2024-05-04T03:16:23.281582+0000","last_fullsized":"2024-05-04T03:16:23.281582+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:22:15.594866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.046648+0000","last_change":"2024-05-04T03:15:48.045029+0000","last_active":"2024-05-04T03:15:50.046648+0000","last_peered":"2024-05-04T03:15:50.046648+0000","last_clean":"2024-05-04T03:15:50.046648+0000","last_became_active":"2024-05-04T03:15:48.041262+0000","last_became_peered":"2024-05-04T03:15:48.041262+0000","last_unstale":"2024-05-04T03:15:50.046648+0000","last_undegraded":"2024-05-04T03:15:50.046648+0000","last_fullsized":"2024-05-04T03:15:50.046648+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:45:56.882957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:03.242881+0000","last_change":"2024-05-04T03:15:45.007530+0000","last_active":"2024-05-04T03:16:03.242881+0000","last_peered":"2024-05-04T03:16:03.242881+0000","last_clean":"2024-05-04T03:16:03.242881+0000","last_became_active":"2024-05-04T03:15:45.007054+0000","last_became_peered":"2024-05-04T03:15:45.007054+0000","last_unstale":"2024-05-04T03:16:03.242881+0000","last_undegraded":"2024-05-04T03:16:03.242881+0000","last_fullsized":"2024-05-04T03:16:03.242881+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:09:13.845679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.051231+0000","last_change":"2024-05-04T03:15:48.050956+0000","last_active":"2024-05-04T03:15:50.051231+0000","last_peered":"2024-05-04T03:15:50.051231+0000","last_clean":"2024-05-04T03:15:50.051231+0000","last_became_active":"2024-05-04T03:15:48.050597+0000","last_became_peered":"2024-05-04T03:15:48.050597+0000","last_unstale":"2024-05-04T03:15:50.051231+0000","last_undegraded":"2024-05-04T03:15:50.051231+0000","last_fullsized":"2024-05-04T03:15:50.051231+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:27:34.127889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:03.245334+0000","last_change":"2024-05-04T03:15:45.007027+0000","last_active":"2024-05-04T03:16:03.245334+0000","last_peered":"2024-05-04T03:16:03.245334+0000","last_clean":"2024-05-04T03:16:03.245334+0000","last_became_active":"2024-05-04T03:15:45.006659+0000","last_became_peered":"2024-05-04T03:15:45.006659+0000","last_unstale":"2024-05-04T03:16:03.245334+0000","last_undegraded":"2024-05-04T03:16:03.245334+0000","last_fullsized":"2024-05-04T03:16:03.245334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:06:11.827093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.046952+0000","last_change":"2024-05-04T03:15:48.046196+0000","last_active":"2024-05-04T03:15:50.046952+0000","last_peered":"2024-05-04T03:15:50.046952+0000","last_clean":"2024-05-04T03:15:50.046952+0000","last_became_active":"2024-05-04T03:15:48.044316+0000","last_became_peered":"2024-05-04T03:15:48.044316+0000","last_unstale":"2024-05-04T03:15:50.046952+0000","last_undegraded":"2024-05-04T03:15:50.046952+0000","last_fullsized":"2024-05-04T03:15:50.046952+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:04:39.301705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.050536+0000","last_change":"2024-05-04T03:15:45.015360+0000","last_active":"2024-05-04T03:15:50.050536+0000","last_peered":"2024-05-04T03:15:50.050536+0000","last_clean":"2024-05-04T03:15:50.050536+0000","last_became_active":"2024-05-04T03:15:45.014696+0000","last_became_peered":"2024-05-04T03:15:45.014696+0000","last_unstale":"2024-05-04T03:15:50.050536+0000","last_undegraded":"2024-05-04T03:15:50.050536+0000","last_fullsized":"2024-05-04T03:15:50.050536+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:17:28.838651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.049568+0000","last_change":"2024-05-04T03:15:48.039823+0000","last_active":"2024-05-04T03:15:50.049568+0000","last_peered":"2024-05-04T03:15:50.049568+0000","last_clean":"2024-05-04T03:15:50.049568+0000","last_became_active":"2024-05-04T03:15:48.039447+0000","last_became_peered":"2024-05-04T03:15:48.039447+0000","last_unstale":"2024-05-04T03:15:50.049568+0000","last_undegraded":"2024-05-04T03:15:50.049568+0000","last_fullsized":"2024-05-04T03:15:50.049568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:45:55.136946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":48,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:26.285696+0000","last_change":"2024-05-04T03:15:45.004662+0000","last_active":"2024-05-04T03:16:26.285696+0000","last_peered":"2024-05-04T03:16:26.285696+0000","last_clean":"2024-05-04T03:16:26.285696+0000","last_became_active":"2024-05-04T03:15:45.004265+0000","last_became_peered":"2024-05-04T03:15:45.004265+0000","last_unstale":"2024-05-04T03:16:26.285696+0000","last_undegraded":"2024-05-04T03:16:26.285696+0000","last_fullsized":"2024-05-04T03:16:26.285696+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:14:03.065346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.048754+0000","last_change":"2024-05-04T03:15:48.042093+0000","last_active":"2024-05-04T03:15:50.048754+0000","last_peered":"2024-05-04T03:15:50.048754+0000","last_clean":"2024-05-04T03:15:50.048754+0000","last_became_active":"2024-05-04T03:15:48.041646+0000","last_became_peered":"2024-05-04T03:15:48.041646+0000","last_unstale":"2024-05-04T03:15:50.048754+0000","last_undegraded":"2024-05-04T03:15:50.048754+0000","last_fullsized":"2024-05-04T03:15:50.048754+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:06:32.083493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:02.230658+0000","last_change":"2024-05-04T03:15:45.016731+0000","last_active":"2024-05-04T03:16:02.230658+0000","last_peered":"2024-05-04T03:16:02.230658+0000","last_clean":"2024-05-04T03:16:02.230658+0000","last_became_active":"2024-05-04T03:15:45.016313+0000","last_became_peered":"2024-05-04T03:15:45.016313+0000","last_unstale":"2024-05-04T03:16:02.230658+0000","last_undegraded":"2024-05-04T03:16:02.230658+0000","last_fullsized":"2024-05-04T03:16:02.230658+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:42:40.830986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.050873+0000","last_change":"2024-05-04T03:15:48.053649+0000","last_active":"2024-05-04T03:15:50.050873+0000","last_peered":"2024-05-04T03:15:50.050873+0000","last_clean":"2024-05-04T03:15:50.050873+0000","last_became_active":"2024-05-04T03:15:48.052407+0000","last_became_peered":"2024-05-04T03:15:48.052407+0000","last_unstale":"2024-05-04T03:15:50.050873+0000","last_undegraded":"2024-05-04T03:15:50.050873+0000","last_fullsized":"2024-05-04T03:15:50.050873+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:37:28.359272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:02.231042+0000","last_change":"2024-05-04T03:15:45.205180+0000","last_active":"2024-05-04T03:16:02.231042+0000","last_peered":"2024-05-04T03:16:02.231042+0000","last_clean":"2024-05-04T03:16:02.231042+0000","last_became_active":"2024-05-04T03:15:45.204828+0000","last_became_peered":"2024-05-04T03:15:45.204828+0000","last_unstale":"2024-05-04T03:16:02.231042+0000","last_undegraded":"2024-05-04T03:16:02.231042+0000","last_fullsized":"2024-05-04T03:16:02.231042+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:27:34.063540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.048566+0000","last_change":"2024-05-04T03:15:48.047154+0000","last_active":"2024-05-04T03:15:50.048566+0000","last_peered":"2024-05-04T03:15:50.048566+0000","last_clean":"2024-05-04T03:15:50.048566+0000","last_became_active":"2024-05-04T03:15:48.046794+0000","last_became_peered":"2024-05-04T03:15:48.046794+0000","last_unstale":"2024-05-04T03:15:50.048566+0000","last_undegraded":"2024-05-04T03:15:50.048566+0000","last_fullsized":"2024-05-04T03:15:50.048566+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:21:42.723252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'25","reported_seq":100,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:27.287741+0000","last_change":"2024-05-04T03:15:45.008930+0000","last_active":"2024-05-04T03:16:27.287741+0000","last_peered":"2024-05-04T03:16:27.287741+0000","last_clean":"2024-05-04T03:16:27.287741+0000","last_became_active":"2024-05-04T03:15:45.008459+0000","last_became_peered":"2024-05-04T03:15:45.008459+0000","last_unstale":"2024-05-04T03:16:27.287741+0000","last_undegraded":"2024-05-04T03:16:27.287741+0000","last_fullsized":"2024-05-04T03:16:27.287741+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:07:52.609156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54372,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":53,"num_read_kb":79,"num_write":26,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.046965+0000","last_change":"2024-05-04T03:15:48.051848+0000","last_active":"2024-05-04T03:15:50.046965+0000","last_peered":"2024-05-04T03:15:50.046965+0000","last_clean":"2024-05-04T03:15:50.046965+0000","last_became_active":"2024-05-04T03:15:48.051171+0000","last_became_peered":"2024-05-04T03:15:48.051171+0000","last_unstale":"2024-05-04T03:15:50.046965+0000","last_undegraded":"2024-05-04T03:15:50.046965+0000","last_fullsized":"2024-05-04T03:15:50.046965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:20:32.722845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:03.243780+0000","last_change":"2024-05-04T03:15:45.007970+0000","last_active":"2024-05-04T03:16:03.243780+0000","last_peered":"2024-05-04T03:16:03.243780+0000","last_clean":"2024-05-04T03:16:03.243780+0000","last_became_active":"2024-05-04T03:15:45.007605+0000","last_became_peered":"2024-05-04T03:15:45.007605+0000","last_unstale":"2024-05-04T03:16:03.243780+0000","last_undegraded":"2024-05-04T03:16:03.243780+0000","last_fullsized":"2024-05-04T03:16:03.243780+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:52:08.466858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.036909+0000","last_change":"2024-05-04T03:15:48.050647+0000","last_active":"2024-05-04T03:15:50.036909+0000","last_peered":"2024-05-04T03:15:50.036909+0000","last_clean":"2024-05-04T03:15:50.036909+0000","last_became_active":"2024-05-04T03:15:48.050296+0000","last_became_peered":"2024-05-04T03:15:48.050296+0000","last_unstale":"2024-05-04T03:15:50.036909+0000","last_undegraded":"2024-05-04T03:15:50.036909+0000","last_fullsized":"2024-05-04T03:15:50.036909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:14:23.902094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:02.233134+0000","last_change":"2024-05-04T03:15:45.014936+0000","last_active":"2024-05-04T03:16:02.233134+0000","last_peered":"2024-05-04T03:16:02.233134+0000","last_clean":"2024-05-04T03:16:02.233134+0000","last_became_active":"2024-05-04T03:15:45.014474+0000","last_became_peered":"2024-05-04T03:15:45.014474+0000","last_unstale":"2024-05-04T03:16:02.233134+0000","last_undegraded":"2024-05-04T03:16:02.233134+0000","last_fullsized":"2024-05-04T03:16:02.233134+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:26:37.972523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.052133+0000","last_change":"2024-05-04T03:15:48.047390+0000","last_active":"2024-05-04T03:15:50.052133+0000","last_peered":"2024-05-04T03:15:50.052133+0000","last_clean":"2024-05-04T03:15:50.052133+0000","last_became_active":"2024-05-04T03:15:48.046747+0000","last_became_peered":"2024-05-04T03:15:48.046747+0000","last_unstale":"2024-05-04T03:15:50.052133+0000","last_undegraded":"2024-05-04T03:15:50.052133+0000","last_fullsized":"2024-05-04T03:15:50.052133+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:30:14.628532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'4","reported_seq":53,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:27.287379+0000","last_change":"2024-05-04T03:15:45.013177+0000","last_active":"2024-05-04T03:16:27.287379+0000","last_peered":"2024-05-04T03:16:27.287379+0000","last_clean":"2024-05-04T03:16:27.287379+0000","last_became_active":"2024-05-04T03:15:45.012764+0000","last_became_peered":"2024-05-04T03:15:45.012764+0000","last_unstale":"2024-05-04T03:16:27.287379+0000","last_undegraded":"2024-05-04T03:16:27.287379+0000","last_fullsized":"2024-05-04T03:16:27.287379+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:07:43.578443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.051645+0000","last_change":"2024-05-04T03:15:48.046453+0000","last_active":"2024-05-04T03:15:50.051645+0000","last_peered":"2024-05-04T03:15:50.051645+0000","last_clean":"2024-05-04T03:15:50.051645+0000","last_became_active":"2024-05-04T03:15:48.045990+0000","last_became_peered":"2024-05-04T03:15:48.045990+0000","last_unstale":"2024-05-04T03:15:50.051645+0000","last_undegraded":"2024-05-04T03:15:50.051645+0000","last_fullsized":"2024-05-04T03:15:50.051645+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:54:57.285122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:03.245575+0000","last_change":"2024-05-04T03:15:45.205539+0000","last_active":"2024-05-04T03:16:03.245575+0000","last_peered":"2024-05-04T03:16:03.245575+0000","last_clean":"2024-05-04T03:16:03.245575+0000","last_became_active":"2024-05-04T03:15:45.205174+0000","last_became_peered":"2024-05-04T03:15:45.205174+0000","last_unstale":"2024-05-04T03:16:03.245575+0000","last_undegraded":"2024-05-04T03:16:03.245575+0000","last_fullsized":"2024-05-04T03:16:03.245575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:03:53.398368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.053186+0000","last_change":"2024-05-04T03:15:48.051824+0000","last_active":"2024-05-04T03:15:50.053186+0000","last_peered":"2024-05-04T03:15:50.053186+0000","last_clean":"2024-05-04T03:15:50.053186+0000","last_became_active":"2024-05-04T03:15:48.051483+0000","last_became_peered":"2024-05-04T03:15:48.051483+0000","last_unstale":"2024-05-04T03:15:50.053186+0000","last_undegraded":"2024-05-04T03:15:50.053186+0000","last_fullsized":"2024-05-04T03:15:50.053186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:57:56.644486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:04.238402+0000","last_change":"2024-05-04T03:15:45.011062+0000","last_active":"2024-05-04T03:16:04.238402+0000","last_peered":"2024-05-04T03:16:04.238402+0000","last_clean":"2024-05-04T03:16:04.238402+0000","last_became_active":"2024-05-04T03:15:45.010730+0000","last_became_peered":"2024-05-04T03:15:45.010730+0000","last_unstale":"2024-05-04T03:16:04.238402+0000","last_undegraded":"2024-05-04T03:16:04.238402+0000","last_fullsized":"2024-05-04T03:16:04.238402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:45:55.979067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.038002+0000","last_change":"2024-05-04T03:15:48.045965+0000","last_active":"2024-05-04T03:15:50.038002+0000","last_peered":"2024-05-04T03:15:50.038002+0000","last_clean":"2024-05-04T03:15:50.038002+0000","last_became_active":"2024-05-04T03:15:48.045615+0000","last_became_peered":"2024-05-04T03:15:48.045615+0000","last_unstale":"2024-05-04T03:15:50.038002+0000","last_undegraded":"2024-05-04T03:15:50.038002+0000","last_fullsized":"2024-05-04T03:15:50.038002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:14:23.497484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.049777+0000","last_change":"2024-05-04T03:15:45.204223+0000","last_active":"2024-05-04T03:15:50.049777+0000","last_peered":"2024-05-04T03:15:50.049777+0000","last_clean":"2024-05-04T03:15:50.049777+0000","last_became_active":"2024-05-04T03:15:45.203774+0000","last_became_peered":"2024-05-04T03:15:45.203774+0000","last_unstale":"2024-05-04T03:15:50.049777+0000","last_undegraded":"2024-05-04T03:15:50.049777+0000","last_fullsized":"2024-05-04T03:15:50.049777+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:02:34.456055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.040518+0000","last_change":"2024-05-04T03:15:48.045574+0000","last_active":"2024-05-04T03:15:50.040518+0000","last_peered":"2024-05-04T03:15:50.040518+0000","last_clean":"2024-05-04T03:15:50.040518+0000","last_became_active":"2024-05-04T03:15:48.045237+0000","last_became_peered":"2024-05-04T03:15:48.045237+0000","last_unstale":"2024-05-04T03:15:50.040518+0000","last_undegraded":"2024-05-04T03:15:50.040518+0000","last_fullsized":"2024-05-04T03:15:50.040518+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:53:46.346199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.051933+0000","last_change":"2024-05-04T03:15:45.013895+0000","last_active":"2024-05-04T03:15:50.051933+0000","last_peered":"2024-05-04T03:15:50.051933+0000","last_clean":"2024-05-04T03:15:50.051933+0000","last_became_active":"2024-05-04T03:15:45.013306+0000","last_became_peered":"2024-05-04T03:15:45.013306+0000","last_unstale":"2024-05-04T03:15:50.051933+0000","last_undegraded":"2024-05-04T03:15:50.051933+0000","last_fullsized":"2024-05-04T03:15:50.051933+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:39:18.616432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.040373+0000","last_change":"2024-05-04T03:15:48.054643+0000","last_active":"2024-05-04T03:15:50.040373+0000","last_peered":"2024-05-04T03:15:50.040373+0000","last_clean":"2024-05-04T03:15:50.040373+0000","last_became_active":"2024-05-04T03:15:48.054171+0000","last_became_peered":"2024-05-04T03:15:48.054171+0000","last_unstale":"2024-05-04T03:15:50.040373+0000","last_undegraded":"2024-05-04T03:15:50.040373+0000","last_fullsized":"2024-05-04T03:15:50.040373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:49:47.732848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.050820+0000","last_change":"2024-05-04T03:15:48.050122+0000","last_active":"2024-05-04T03:15:50.050820+0000","last_peered":"2024-05-04T03:15:50.050820+0000","last_clean":"2024-05-04T03:15:50.050820+0000","last_became_active":"2024-05-04T03:15:48.049794+0000","last_became_peered":"2024-05-04T03:15:48.049794+0000","last_unstale":"2024-05-04T03:15:50.050820+0000","last_undegraded":"2024-05-04T03:15:50.050820+0000","last_fullsized":"2024-05-04T03:15:50.050820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:29:11.066068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:26.284961+0000","last_change":"2024-05-04T03:15:45.205143+0000","last_active":"2024-05-04T03:16:26.284961+0000","last_peered":"2024-05-04T03:16:26.284961+0000","last_clean":"2024-05-04T03:16:26.284961+0000","last_became_active":"2024-05-04T03:15:45.204727+0000","last_became_peered":"2024-05-04T03:15:45.204727+0000","last_unstale":"2024-05-04T03:16:26.284961+0000","last_undegraded":"2024-05-04T03:16:26.284961+0000","last_fullsized":"2024-05-04T03:16:26.284961+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:15:05.799556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.047004+0000","last_change":"2024-05-04T03:15:48.042914+0000","last_active":"2024-05-04T03:15:50.047004+0000","last_peered":"2024-05-04T03:15:50.047004+0000","last_clean":"2024-05-04T03:15:50.047004+0000","last_became_active":"2024-05-04T03:15:48.041936+0000","last_became_peered":"2024-05-04T03:15:48.041936+0000","last_unstale":"2024-05-04T03:15:50.047004+0000","last_undegraded":"2024-05-04T03:15:50.047004+0000","last_fullsized":"2024-05-04T03:15:50.047004+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:11:48.807602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:05.249261+0000","last_change":"2024-05-04T03:15:45.011045+0000","last_active":"2024-05-04T03:16:05.249261+0000","last_peered":"2024-05-04T03:16:05.249261+0000","last_clean":"2024-05-04T03:16:05.249261+0000","last_became_active":"2024-05-04T03:15:45.010608+0000","last_became_peered":"2024-05-04T03:15:45.010608+0000","last_unstale":"2024-05-04T03:16:05.249261+0000","last_undegraded":"2024-05-04T03:16:05.249261+0000","last_fullsized":"2024-05-04T03:16:05.249261+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:00:36.243588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":97,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.051809+0000","last_change":"2024-05-04T03:14:19.895070+0000","last_active":"2024-05-04T03:15:50.051809+0000","last_peered":"2024-05-04T03:15:50.051809+0000","last_clean":"2024-05-04T03:15:50.051809+0000","last_became_active":"2024-05-04T03:14:19.889987+0000","last_became_peered":"2024-05-04T03:14:19.889987+0000","last_unstale":"2024-05-04T03:15:50.051809+0000","last_undegraded":"2024-05-04T03:15:50.051809+0000","last_fullsized":"2024-05-04T03:15:50.051809+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:11:01.712629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:11:01.712629+0000","last_clean_scrub_stamp":"2024-05-04T03:11:01.712629+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:03:02.769719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.036881+0000","last_change":"2024-05-04T03:15:48.044928+0000","last_active":"2024-05-04T03:15:50.036881+0000","last_peered":"2024-05-04T03:15:50.036881+0000","last_clean":"2024-05-04T03:15:50.036881+0000","last_became_active":"2024-05-04T03:15:48.044532+0000","last_became_peered":"2024-05-04T03:15:48.044532+0000","last_unstale":"2024-05-04T03:15:50.036881+0000","last_undegraded":"2024-05-04T03:15:50.036881+0000","last_fullsized":"2024-05-04T03:15:50.036881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:27:00.825692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:03.244129+0000","last_change":"2024-05-04T03:15:45.014469+0000","last_active":"2024-05-04T03:16:03.244129+0000","last_peered":"2024-05-04T03:16:03.244129+0000","last_clean":"2024-05-04T03:16:03.244129+0000","last_became_active":"2024-05-04T03:15:45.013763+0000","last_became_peered":"2024-05-04T03:15:45.013763+0000","last_unstale":"2024-05-04T03:16:03.244129+0000","last_undegraded":"2024-05-04T03:16:03.244129+0000","last_fullsized":"2024-05-04T03:16:03.244129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:19:36.379930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:02.231310+0000","last_change":"2024-05-04T03:15:45.204536+0000","last_active":"2024-05-04T03:16:02.231310+0000","last_peered":"2024-05-04T03:16:02.231310+0000","last_clean":"2024-05-04T03:16:02.231310+0000","last_became_active":"2024-05-04T03:15:45.204072+0000","last_became_peered":"2024-05-04T03:15:45.204072+0000","last_unstale":"2024-05-04T03:16:02.231310+0000","last_undegraded":"2024-05-04T03:16:02.231310+0000","last_fullsized":"2024-05-04T03:16:02.231310+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:49:12.726220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.037172+0000","last_change":"2024-05-04T03:15:48.044982+0000","last_active":"2024-05-04T03:15:50.037172+0000","last_peered":"2024-05-04T03:15:50.037172+0000","last_clean":"2024-05-04T03:15:50.037172+0000","last_became_active":"2024-05-04T03:15:48.043996+0000","last_became_peered":"2024-05-04T03:15:48.043996+0000","last_unstale":"2024-05-04T03:15:50.037172+0000","last_undegraded":"2024-05-04T03:15:50.037172+0000","last_fullsized":"2024-05-04T03:15:50.037172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:09:39.770023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'9","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:25.047196+0000","last_change":"2024-05-04T03:15:45.008153+0000","last_active":"2024-05-04T03:16:25.047196+0000","last_peered":"2024-05-04T03:16:25.047196+0000","last_clean":"2024-05-04T03:16:25.047196+0000","last_became_active":"2024-05-04T03:15:45.007698+0000","last_became_peered":"2024-05-04T03:15:45.007698+0000","last_unstale":"2024-05-04T03:16:25.047196+0000","last_undegraded":"2024-05-04T03:16:25.047196+0000","last_fullsized":"2024-05-04T03:16:25.047196+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:56:52.322049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.047367+0000","last_change":"2024-05-04T03:15:48.055767+0000","last_active":"2024-05-04T03:15:50.047367+0000","last_peered":"2024-05-04T03:15:50.047367+0000","last_clean":"2024-05-04T03:15:50.047367+0000","last_became_active":"2024-05-04T03:15:48.055252+0000","last_became_peered":"2024-05-04T03:15:48.055252+0000","last_unstale":"2024-05-04T03:15:50.047367+0000","last_undegraded":"2024-05-04T03:15:50.047367+0000","last_fullsized":"2024-05-04T03:15:50.047367+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:28:11.736803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:02.230206+0000","last_change":"2024-05-04T03:15:45.012838+0000","last_active":"2024-05-04T03:16:02.230206+0000","last_peered":"2024-05-04T03:16:02.230206+0000","last_clean":"2024-05-04T03:16:02.230206+0000","last_became_active":"2024-05-04T03:15:45.012429+0000","last_became_peered":"2024-05-04T03:15:45.012429+0000","last_unstale":"2024-05-04T03:16:02.230206+0000","last_undegraded":"2024-05-04T03:16:02.230206+0000","last_fullsized":"2024-05-04T03:16:02.230206+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:22:22.602551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.050642+0000","last_change":"2024-05-04T03:15:48.056993+0000","last_active":"2024-05-04T03:15:50.050642+0000","last_peered":"2024-05-04T03:15:50.050642+0000","last_clean":"2024-05-04T03:15:50.050642+0000","last_became_active":"2024-05-04T03:15:48.055938+0000","last_became_peered":"2024-05-04T03:15:48.055938+0000","last_unstale":"2024-05-04T03:15:50.050642+0000","last_undegraded":"2024-05-04T03:15:50.050642+0000","last_fullsized":"2024-05-04T03:15:50.050642+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:23:59.521692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:02.232885+0000","last_change":"2024-05-04T03:15:45.204652+0000","last_active":"2024-05-04T03:16:02.232885+0000","last_peered":"2024-05-04T03:16:02.232885+0000","last_clean":"2024-05-04T03:16:02.232885+0000","last_became_active":"2024-05-04T03:15:45.204266+0000","last_became_peered":"2024-05-04T03:15:45.204266+0000","last_unstale":"2024-05-04T03:16:02.232885+0000","last_undegraded":"2024-05-04T03:16:02.232885+0000","last_fullsized":"2024-05-04T03:16:02.232885+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:06:38.230692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.046466+0000","last_change":"2024-05-04T03:15:48.043614+0000","last_active":"2024-05-04T03:15:50.046466+0000","last_peered":"2024-05-04T03:15:50.046466+0000","last_clean":"2024-05-04T03:15:50.046466+0000","last_became_active":"2024-05-04T03:15:48.042390+0000","last_became_peered":"2024-05-04T03:15:48.042390+0000","last_unstale":"2024-05-04T03:15:50.046466+0000","last_undegraded":"2024-05-04T03:15:50.046466+0000","last_fullsized":"2024-05-04T03:15:50.046466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:56:22.791368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:02.232827+0000","last_change":"2024-05-04T03:15:45.005570+0000","last_active":"2024-05-04T03:16:02.232827+0000","last_peered":"2024-05-04T03:16:02.232827+0000","last_clean":"2024-05-04T03:16:02.232827+0000","last_became_active":"2024-05-04T03:15:45.005105+0000","last_became_peered":"2024-05-04T03:15:45.005105+0000","last_unstale":"2024-05-04T03:16:02.232827+0000","last_undegraded":"2024-05-04T03:16:02.232827+0000","last_fullsized":"2024-05-04T03:16:02.232827+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:40:17.275618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.037743+0000","last_change":"2024-05-04T03:15:48.045081+0000","last_active":"2024-05-04T03:15:50.037743+0000","last_peered":"2024-05-04T03:15:50.037743+0000","last_clean":"2024-05-04T03:15:50.037743+0000","last_became_active":"2024-05-04T03:15:48.044745+0000","last_became_peered":"2024-05-04T03:15:48.044745+0000","last_unstale":"2024-05-04T03:15:50.037743+0000","last_undegraded":"2024-05-04T03:15:50.037743+0000","last_fullsized":"2024-05-04T03:15:50.037743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:40:53.486222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.048056+0000","last_change":"2024-05-04T03:15:45.008926+0000","last_active":"2024-05-04T03:15:50.048056+0000","last_peered":"2024-05-04T03:15:50.048056+0000","last_clean":"2024-05-04T03:15:50.048056+0000","last_became_active":"2024-05-04T03:15:45.008412+0000","last_became_peered":"2024-05-04T03:15:45.008412+0000","last_unstale":"2024-05-04T03:15:50.048056+0000","last_undegraded":"2024-05-04T03:15:50.048056+0000","last_fullsized":"2024-05-04T03:15:50.048056+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:34:33.107866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.046816+0000","last_change":"2024-05-04T03:15:48.054580+0000","last_active":"2024-05-04T03:15:50.046816+0000","last_peered":"2024-05-04T03:15:50.046816+0000","last_clean":"2024-05-04T03:15:50.046816+0000","last_became_active":"2024-05-04T03:15:48.054242+0000","last_became_peered":"2024-05-04T03:15:48.054242+0000","last_unstale":"2024-05-04T03:15:50.046816+0000","last_undegraded":"2024-05-04T03:15:50.046816+0000","last_fullsized":"2024-05-04T03:15:50.046816+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:16:19.426571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:02.232185+0000","last_change":"2024-05-04T03:15:45.015504+0000","last_active":"2024-05-04T03:16:02.232185+0000","last_peered":"2024-05-04T03:16:02.232185+0000","last_clean":"2024-05-04T03:16:02.232185+0000","last_became_active":"2024-05-04T03:15:45.015153+0000","last_became_peered":"2024-05-04T03:15:45.015153+0000","last_unstale":"2024-05-04T03:16:02.232185+0000","last_undegraded":"2024-05-04T03:16:02.232185+0000","last_fullsized":"2024-05-04T03:16:02.232185+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:25:17.293674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.051912+0000","last_change":"2024-05-04T03:15:48.045284+0000","last_active":"2024-05-04T03:15:50.051912+0000","last_peered":"2024-05-04T03:15:50.051912+0000","last_clean":"2024-05-04T03:15:50.051912+0000","last_became_active":"2024-05-04T03:15:48.044673+0000","last_became_peered":"2024-05-04T03:15:48.044673+0000","last_unstale":"2024-05-04T03:15:50.051912+0000","last_undegraded":"2024-05-04T03:15:50.051912+0000","last_fullsized":"2024-05-04T03:15:50.051912+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:51:52.389748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:03.245074+0000","last_change":"2024-05-04T03:15:45.205648+0000","last_active":"2024-05-04T03:16:03.245074+0000","last_peered":"2024-05-04T03:16:03.245074+0000","last_clean":"2024-05-04T03:16:03.245074+0000","last_became_active":"2024-05-04T03:15:45.204169+0000","last_became_peered":"2024-05-04T03:15:45.204169+0000","last_unstale":"2024-05-04T03:16:03.245074+0000","last_undegraded":"2024-05-04T03:16:03.245074+0000","last_fullsized":"2024-05-04T03:16:03.245074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:20:48.859276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.052862+0000","last_change":"2024-05-04T03:15:48.049808+0000","last_active":"2024-05-04T03:15:50.052862+0000","last_peered":"2024-05-04T03:15:50.052862+0000","last_clean":"2024-05-04T03:15:50.052862+0000","last_became_active":"2024-05-04T03:15:48.048998+0000","last_became_peered":"2024-05-04T03:15:48.048998+0000","last_unstale":"2024-05-04T03:15:50.052862+0000","last_undegraded":"2024-05-04T03:15:50.052862+0000","last_fullsized":"2024-05-04T03:15:50.052862+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:54:46.786406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.056975+0000","last_change":"2024-05-04T03:15:45.008874+0000","last_active":"2024-05-04T03:15:50.056975+0000","last_peered":"2024-05-04T03:15:50.056975+0000","last_clean":"2024-05-04T03:15:50.056975+0000","last_became_active":"2024-05-04T03:15:45.008530+0000","last_became_peered":"2024-05-04T03:15:45.008530+0000","last_unstale":"2024-05-04T03:15:50.056975+0000","last_undegraded":"2024-05-04T03:15:50.056975+0000","last_fullsized":"2024-05-04T03:15:50.056975+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:31:41.697386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.052480+0000","last_change":"2024-05-04T03:15:48.046042+0000","last_active":"2024-05-04T03:15:50.052480+0000","last_peered":"2024-05-04T03:15:50.052480+0000","last_clean":"2024-05-04T03:15:50.052480+0000","last_became_active":"2024-05-04T03:15:48.045387+0000","last_became_peered":"2024-05-04T03:15:48.045387+0000","last_unstale":"2024-05-04T03:15:50.052480+0000","last_undegraded":"2024-05-04T03:15:50.052480+0000","last_fullsized":"2024-05-04T03:15:50.052480+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:43:50.782623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.046516+0000","last_change":"2024-05-04T03:15:45.009126+0000","last_active":"2024-05-04T03:15:50.046516+0000","last_peered":"2024-05-04T03:15:50.046516+0000","last_clean":"2024-05-04T03:15:50.046516+0000","last_became_active":"2024-05-04T03:15:45.008802+0000","last_became_peered":"2024-05-04T03:15:45.008802+0000","last_unstale":"2024-05-04T03:15:50.046516+0000","last_undegraded":"2024-05-04T03:15:50.046516+0000","last_fullsized":"2024-05-04T03:15:50.046516+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:25:30.867870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.052793+0000","last_change":"2024-05-04T03:15:48.050478+0000","last_active":"2024-05-04T03:15:50.052793+0000","last_peered":"2024-05-04T03:15:50.052793+0000","last_clean":"2024-05-04T03:15:50.052793+0000","last_became_active":"2024-05-04T03:15:48.045756+0000","last_became_peered":"2024-05-04T03:15:48.045756+0000","last_unstale":"2024-05-04T03:15:50.052793+0000","last_undegraded":"2024-05-04T03:15:50.052793+0000","last_fullsized":"2024-05-04T03:15:50.052793+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:05:11.098381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.037858+0000","last_change":"2024-05-04T03:15:48.042778+0000","last_active":"2024-05-04T03:15:50.037858+0000","last_peered":"2024-05-04T03:15:50.037858+0000","last_clean":"2024-05-04T03:15:50.037858+0000","last_became_active":"2024-05-04T03:15:48.042349+0000","last_became_peered":"2024-05-04T03:15:48.042349+0000","last_unstale":"2024-05-04T03:15:50.037858+0000","last_undegraded":"2024-05-04T03:15:50.037858+0000","last_fullsized":"2024-05-04T03:15:50.037858+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:40:12.260026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.051646+0000","last_change":"2024-05-04T03:15:48.048544+0000","last_active":"2024-05-04T03:15:50.051646+0000","last_peered":"2024-05-04T03:15:50.051646+0000","last_clean":"2024-05-04T03:15:50.051646+0000","last_became_active":"2024-05-04T03:15:48.047873+0000","last_became_peered":"2024-05-04T03:15:48.047873+0000","last_unstale":"2024-05-04T03:15:50.051646+0000","last_undegraded":"2024-05-04T03:15:50.051646+0000","last_fullsized":"2024-05-04T03:15:50.051646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:48:02.290401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.048680+0000","last_change":"2024-05-04T03:15:48.053467+0000","last_active":"2024-05-04T03:15:50.048680+0000","last_peered":"2024-05-04T03:15:50.048680+0000","last_clean":"2024-05-04T03:15:50.048680+0000","last_became_active":"2024-05-04T03:15:48.050796+0000","last_became_peered":"2024-05-04T03:15:48.050796+0000","last_unstale":"2024-05-04T03:15:50.048680+0000","last_undegraded":"2024-05-04T03:15:50.048680+0000","last_fullsized":"2024-05-04T03:15:50.048680+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:11:15.009467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.050345+0000","last_change":"2024-05-04T03:15:48.054616+0000","last_active":"2024-05-04T03:15:50.050345+0000","last_peered":"2024-05-04T03:15:50.050345+0000","last_clean":"2024-05-04T03:15:50.050345+0000","last_became_active":"2024-05-04T03:15:48.054142+0000","last_became_peered":"2024-05-04T03:15:48.054142+0000","last_unstale":"2024-05-04T03:15:50.050345+0000","last_undegraded":"2024-05-04T03:15:50.050345+0000","last_fullsized":"2024-05-04T03:15:50.050345+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:18:24.067666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.037371+0000","last_change":"2024-05-04T03:15:48.046969+0000","last_active":"2024-05-04T03:15:50.037371+0000","last_peered":"2024-05-04T03:15:50.037371+0000","last_clean":"2024-05-04T03:15:50.037371+0000","last_became_active":"2024-05-04T03:15:48.046637+0000","last_became_peered":"2024-05-04T03:15:48.046637+0000","last_unstale":"2024-05-04T03:15:50.037371+0000","last_undegraded":"2024-05-04T03:15:50.037371+0000","last_fullsized":"2024-05-04T03:15:50.037371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:03:01.763037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.037937+0000","last_change":"2024-05-04T03:15:48.041753+0000","last_active":"2024-05-04T03:15:50.037937+0000","last_peered":"2024-05-04T03:15:50.037937+0000","last_clean":"2024-05-04T03:15:50.037937+0000","last_became_active":"2024-05-04T03:15:48.041236+0000","last_became_peered":"2024-05-04T03:15:48.041236+0000","last_unstale":"2024-05-04T03:15:50.037937+0000","last_undegraded":"2024-05-04T03:15:50.037937+0000","last_fullsized":"2024-05-04T03:15:50.037937+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:41:02.928257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.051064+0000","last_change":"2024-05-04T03:15:48.044192+0000","last_active":"2024-05-04T03:15:50.051064+0000","last_peered":"2024-05-04T03:15:50.051064+0000","last_clean":"2024-05-04T03:15:50.051064+0000","last_became_active":"2024-05-04T03:15:48.043849+0000","last_became_peered":"2024-05-04T03:15:48.043849+0000","last_unstale":"2024-05-04T03:15:50.051064+0000","last_undegraded":"2024-05-04T03:15:50.051064+0000","last_fullsized":"2024-05-04T03:15:50.051064+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:55:22.000788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.047162+0000","last_change":"2024-05-04T03:15:48.045675+0000","last_active":"2024-05-04T03:15:50.047162+0000","last_peered":"2024-05-04T03:15:50.047162+0000","last_clean":"2024-05-04T03:15:50.047162+0000","last_became_active":"2024-05-04T03:15:48.045321+0000","last_became_peered":"2024-05-04T03:15:48.045321+0000","last_unstale":"2024-05-04T03:15:50.047162+0000","last_undegraded":"2024-05-04T03:15:50.047162+0000","last_fullsized":"2024-05-04T03:15:50.047162+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:12:06.060729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.050456+0000","last_change":"2024-05-04T03:15:48.040872+0000","last_active":"2024-05-04T03:15:50.050456+0000","last_peered":"2024-05-04T03:15:50.050456+0000","last_clean":"2024-05-04T03:15:50.050456+0000","last_became_active":"2024-05-04T03:15:48.040485+0000","last_became_peered":"2024-05-04T03:15:48.040485+0000","last_unstale":"2024-05-04T03:15:50.050456+0000","last_undegraded":"2024-05-04T03:15:50.050456+0000","last_fullsized":"2024-05-04T03:15:50.050456+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:58:01.789270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.046917+0000","last_change":"2024-05-04T03:15:48.049080+0000","last_active":"2024-05-04T03:15:50.046917+0000","last_peered":"2024-05-04T03:15:50.046917+0000","last_clean":"2024-05-04T03:15:50.046917+0000","last_became_active":"2024-05-04T03:15:48.044624+0000","last_became_peered":"2024-05-04T03:15:48.044624+0000","last_unstale":"2024-05-04T03:15:50.046917+0000","last_undegraded":"2024-05-04T03:15:50.046917+0000","last_fullsized":"2024-05-04T03:15:50.046917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:47:08.647401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.048482+0000","last_change":"2024-05-04T03:15:48.049394+0000","last_active":"2024-05-04T03:15:50.048482+0000","last_peered":"2024-05-04T03:15:50.048482+0000","last_clean":"2024-05-04T03:15:50.048482+0000","last_became_active":"2024-05-04T03:15:48.048980+0000","last_became_peered":"2024-05-04T03:15:48.048980+0000","last_unstale":"2024-05-04T03:15:50.048482+0000","last_undegraded":"2024-05-04T03:15:50.048482+0000","last_fullsized":"2024-05-04T03:15:50.048482+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:43:24.914665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.037305+0000","last_change":"2024-05-04T03:15:48.051672+0000","last_active":"2024-05-04T03:15:50.037305+0000","last_peered":"2024-05-04T03:15:50.037305+0000","last_clean":"2024-05-04T03:15:50.037305+0000","last_became_active":"2024-05-04T03:15:48.051314+0000","last_became_peered":"2024-05-04T03:15:48.051314+0000","last_unstale":"2024-05-04T03:15:50.037305+0000","last_undegraded":"2024-05-04T03:15:50.037305+0000","last_fullsized":"2024-05-04T03:15:50.037305+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:19:22.808102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.037727+0000","last_change":"2024-05-04T03:15:48.053045+0000","last_active":"2024-05-04T03:15:50.037727+0000","last_peered":"2024-05-04T03:15:50.037727+0000","last_clean":"2024-05-04T03:15:50.037727+0000","last_became_active":"2024-05-04T03:15:48.052659+0000","last_became_peered":"2024-05-04T03:15:48.052659+0000","last_unstale":"2024-05-04T03:15:50.037727+0000","last_undegraded":"2024-05-04T03:15:50.037727+0000","last_fullsized":"2024-05-04T03:15:50.037727+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:53:25.624540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58373,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":162,"num_read_kb":144,"num_write":156,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":175119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612631,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":876544,"data_stored":423670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808860,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1568768,"data_stored":1069914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972385,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":423906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168616,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":423826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397548,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32628,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712524,"statfs":{"total":95995035648,"available":95961624576,"internally_reserved":0,"allocated":1486848,"data_stored":1015756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593777,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":880640,"data_stored":423295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":937984,"data_stored":477460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40376,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704776,"statfs":{"total":95995035648,"available":95953690624,"internally_reserved":0,"allocated":901120,"data_stored":424566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":962560,"data_stored":478810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182462,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40372,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704780,"statfs":{"total":95995035648,"available":95953694720,"internally_reserved":0,"allocated":897024,"data_stored":423606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986235,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":424123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790006,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36800,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708352,"statfs":{"total":95995035648,"available":95957352448,"internally_reserved":0,"allocated":1499136,"data_stored":1015049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":54372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":55722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-04T03:16:31.293 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-04T03:16:31.541 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:31.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:31 smithi001 ceph-mon[29731]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:16:31.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:31 smithi077 ceph-mon[33531]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:16:31.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:31 smithi072 ceph-mon[33332]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:16:32.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:32 smithi001 ceph-mon[29731]: pgmap v310: 97 pgs: 97 active+clean; 634 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-05-04T03:16:32.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:32 smithi077 ceph-mon[33531]: pgmap v310: 97 pgs: 97 active+clean; 634 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-05-04T03:16:32.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:32 smithi072 ceph-mon[33332]: pgmap v310: 97 pgs: 97 active+clean; 634 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-05-04T03:16:32.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.960+0000 7f2a34e80700 1 -- 172.21.15.1:0/1342942769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a30107df0 msgr2=0x7f2a30108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:32.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.960+0000 7f2a34e80700 1 --2- 172.21.15.1:0/1342942769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a30107df0 0x7f2a30108200 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2a240099c0 tx=0x7f2a24009cd0 comp rx=0 tx=0).stop 2024-05-04T03:16:32.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.960+0000 7f2a34e80700 1 -- 172.21.15.1:0/1342942769 shutdown_connections 2024-05-04T03:16:32.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.960+0000 7f2a34e80700 1 --2- 172.21.15.1:0/1342942769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a301034c0 0x7f2a301038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:32.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.960+0000 7f2a34e80700 1 --2- 172.21.15.1:0/1342942769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a30108740 0x7f2a301100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:32.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.960+0000 7f2a34e80700 1 --2- 172.21.15.1:0/1342942769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a30107df0 0x7f2a30108200 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:32.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.960+0000 7f2a34e80700 1 -- 172.21.15.1:0/1342942769 >> 172.21.15.1:0/1342942769 conn(0x7f2a300fedf0 msgr2=0x7f2a30101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:32.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.961+0000 7f2a34e80700 1 -- 172.21.15.1:0/1342942769 shutdown_connections 2024-05-04T03:16:32.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.961+0000 7f2a34e80700 1 -- 172.21.15.1:0/1342942769 wait complete. 2024-05-04T03:16:32.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.961+0000 7f2a34e80700 1 Processor -- start 2024-05-04T03:16:32.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.961+0000 7f2a34e80700 1 -- start start 2024-05-04T03:16:32.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.962+0000 7f2a34e80700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a301034c0 0x7f2a3019bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:32.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.962+0000 7f2a34e80700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a30107df0 0x7f2a3019c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:32.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.962+0000 7f2a34e80700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a30108740 0x7f2a301a6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.962+0000 7f2a34e80700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2a30113350 con 0x7f2a30107df0 2024-05-04T03:16:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.962+0000 7f2a34e80700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2a301131d0 con 0x7f2a301034c0 2024-05-04T03:16:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.962+0000 7f2a34e80700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2a301134d0 con 0x7f2a30108740 2024-05-04T03:16:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.962+0000 7f2a2dd9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a30107df0 0x7f2a3019c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.962+0000 7f2a2dd9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a30107df0 0x7f2a3019c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56206/0 (socket says 172.21.15.1:56206) 2024-05-04T03:16:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.962+0000 7f2a2dd9b700 1 -- 172.21.15.1:0/2894972328 learned_addr learned my addr 172.21.15.1:0/2894972328 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:16:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.962+0000 7f2a2e59c700 1 --2- 172.21.15.1:0/2894972328 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a301034c0 0x7f2a3019bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.962+0000 7f2a2ed9d700 1 --2- 172.21.15.1:0/2894972328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a30108740 0x7f2a301a6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:32.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.962+0000 7f2a2dd9b700 1 -- 172.21.15.1:0/2894972328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a30108740 msgr2=0x7f2a301a6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:32.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.962+0000 7f2a2dd9b700 1 --2- 172.21.15.1:0/2894972328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a30108740 0x7f2a301a6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:32.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.962+0000 7f2a2dd9b700 1 -- 172.21.15.1:0/2894972328 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a301034c0 msgr2=0x7f2a3019bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:32.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.962+0000 7f2a2dd9b700 1 --2- 172.21.15.1:0/2894972328 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a301034c0 0x7f2a3019bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:32.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.962+0000 7f2a2dd9b700 1 -- 172.21.15.1:0/2894972328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a3019c9d0 con 0x7f2a30107df0 2024-05-04T03:16:32.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.962+0000 7f2a2dd9b700 1 --2- 172.21.15.1:0/2894972328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a30107df0 0x7f2a3019c1b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2a240098e0 tx=0x7f2a240057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:32.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.962+0000 7f2a1f7fe700 1 -- 172.21.15.1:0/2894972328 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a24027070 con 0x7f2a30107df0 2024-05-04T03:16:32.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.963+0000 7f2a1f7fe700 1 -- 172.21.15.1:0/2894972328 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a24003f20 con 0x7f2a30107df0 2024-05-04T03:16:32.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.963+0000 7f2a34e80700 1 -- 172.21.15.1:0/2894972328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a301a7230 con 0x7f2a30107df0 2024-05-04T03:16:32.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.963+0000 7f2a1f7fe700 1 -- 172.21.15.1:0/2894972328 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a24022d70 con 0x7f2a30107df0 2024-05-04T03:16:32.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.963+0000 7f2a34e80700 1 -- 172.21.15.1:0/2894972328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a301a7680 con 0x7f2a30107df0 2024-05-04T03:16:32.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.963+0000 7f2a34e80700 1 -- 172.21.15.1:0/2894972328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a300512c0 con 0x7f2a30107df0 2024-05-04T03:16:32.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.965+0000 7f2a1f7fe700 1 -- 172.21.15.1:0/2894972328 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f2a2402a070 con 0x7f2a30107df0 2024-05-04T03:16:32.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.966+0000 7f2a1f7fe700 1 --2- 172.21.15.1:0/2894972328 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f2a180b2910 0x7f2a180b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:32.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.966+0000 7f2a2e59c700 1 --2- 172.21.15.1:0/2894972328 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f2a180b2910 0x7f2a180b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:32.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.966+0000 7f2a1f7fe700 1 -- 172.21.15.1:0/2894972328 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f2a240e3810 con 0x7f2a30107df0 2024-05-04T03:16:32.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.966+0000 7f2a2e59c700 1 --2- 172.21.15.1:0/2894972328 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f2a180b2910 0x7f2a180b4e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2a20005950 tx=0x7f2a2000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:32.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:32.970+0000 7f2a1f7fe700 1 -- 172.21.15.1:0/2894972328 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2a24013070 con 0x7f2a30107df0 2024-05-04T03:16:33.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:33.182+0000 7f2a34e80700 1 -- 172.21.15.1:0/2894972328 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2a30063c00 con 0x7f2a180b2910 2024-05-04T03:16:33.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:33.190+0000 7f2a1f7fe700 1 -- 172.21.15.1:0/2894972328 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258699 (secure 0 0 0) 0x7f2a30063c00 con 0x7f2a180b2910 2024-05-04T03:16:33.195 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:16:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:33.193+0000 7f2a34e80700 1 -- 172.21.15.1:0/2894972328 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f2a180b2910 msgr2=0x7f2a180b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:33.193+0000 7f2a34e80700 1 --2- 172.21.15.1:0/2894972328 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f2a180b2910 0x7f2a180b4e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2a20005950 tx=0x7f2a2000a300 comp rx=0 tx=0).stop 2024-05-04T03:16:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:33.193+0000 7f2a34e80700 1 -- 172.21.15.1:0/2894972328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a30107df0 msgr2=0x7f2a3019c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:33.193+0000 7f2a34e80700 1 --2- 172.21.15.1:0/2894972328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a30107df0 0x7f2a3019c1b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2a240098e0 tx=0x7f2a240057c0 comp rx=0 tx=0).stop 2024-05-04T03:16:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:33.193+0000 7f2a34e80700 1 -- 172.21.15.1:0/2894972328 shutdown_connections 2024-05-04T03:16:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:33.193+0000 7f2a34e80700 1 --2- 172.21.15.1:0/2894972328 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2a301034c0 0x7f2a3019bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:33.193+0000 7f2a34e80700 1 --2- 172.21.15.1:0/2894972328 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f2a180b2910 0x7f2a180b4e20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:33.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:33.193+0000 7f2a34e80700 1 --2- 172.21.15.1:0/2894972328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2a30108740 0x7f2a301a6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:33.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:33.193+0000 7f2a34e80700 1 --2- 172.21.15.1:0/2894972328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a30107df0 0x7f2a3019c1b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:33.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:33.193+0000 7f2a34e80700 1 -- 172.21.15.1:0/2894972328 >> 172.21.15.1:0/2894972328 conn(0x7f2a300fedf0 msgr2=0x7f2a301096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:33.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:33.193+0000 7f2a34e80700 1 -- 172.21.15.1:0/2894972328 shutdown_connections 2024-05-04T03:16:33.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:33.193+0000 7f2a34e80700 1 -- 172.21.15.1:0/2894972328 wait complete. 2024-05-04T03:16:33.198 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-04T03:16:33.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:33 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-05-04T03:16:33.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:33 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-05-04T03:16:33.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:33 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-04T03:16:33.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:33 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-05-04T03:16:33.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:33 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-05-04T03:16:33.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:33 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-05-04T03:16:33.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:33 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-05-04T03:16:33.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:33 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-04T03:16:33.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:33 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-05-04T03:16:33.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:33 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-05-04T03:16:33.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:33 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-05-04T03:16:33.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:33 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-05-04T03:16:33.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:33 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-04T03:16:33.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:33 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-05-04T03:16:33.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:33 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-05-04T03:16:33.780 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-05-04T03:16:33.147953+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657171,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":259,"num_read_kb":227,"num_write":288,"num_write_kb":2265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":407320,"kb_used_data":12408,"kb_used_omap":18,"kb_used_meta":394797,"kb_avail":1124534504,"statfs":{"total":1151940427776,"available":1151523332096,"internally_reserved":0,"allocated":12705792,"data_stored":7048569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":404272441},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":35740,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":78,"num_read_kb":77,"num_write":25,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003410"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.038080+0000","last_change":"2024-05-04T03:15:48.049361+0000","last_active":"2024-05-04T03:15:50.038080+0000","last_peered":"2024-05-04T03:15:50.038080+0000","last_clean":"2024-05-04T03:15:50.038080+0000","last_became_active":"2024-05-04T03:15:48.048876+0000","last_became_peered":"2024-05-04T03:15:48.048876+0000","last_unstale":"2024-05-04T03:15:50.038080+0000","last_undegraded":"2024-05-04T03:15:50.038080+0000","last_fullsized":"2024-05-04T03:15:50.038080+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:50:52.568516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.050624+0000","last_change":"2024-05-04T03:15:48.055554+0000","last_active":"2024-05-04T03:15:50.050624+0000","last_peered":"2024-05-04T03:15:50.050624+0000","last_clean":"2024-05-04T03:15:50.050624+0000","last_became_active":"2024-05-04T03:15:48.055155+0000","last_became_peered":"2024-05-04T03:15:48.055155+0000","last_unstale":"2024-05-04T03:15:50.050624+0000","last_undegraded":"2024-05-04T03:15:50.050624+0000","last_fullsized":"2024-05-04T03:15:50.050624+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:38:07.824031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.050185+0000","last_change":"2024-05-04T03:15:48.047974+0000","last_active":"2024-05-04T03:15:50.050185+0000","last_peered":"2024-05-04T03:15:50.050185+0000","last_clean":"2024-05-04T03:15:50.050185+0000","last_became_active":"2024-05-04T03:15:48.044299+0000","last_became_peered":"2024-05-04T03:15:48.044299+0000","last_unstale":"2024-05-04T03:15:50.050185+0000","last_undegraded":"2024-05-04T03:15:50.050185+0000","last_fullsized":"2024-05-04T03:15:50.050185+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:49:08.014987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.054833+0000","last_change":"2024-05-04T03:15:48.050899+0000","last_active":"2024-05-04T03:15:50.054833+0000","last_peered":"2024-05-04T03:15:50.054833+0000","last_clean":"2024-05-04T03:15:50.054833+0000","last_became_active":"2024-05-04T03:15:48.048812+0000","last_became_peered":"2024-05-04T03:15:48.048812+0000","last_unstale":"2024-05-04T03:15:50.054833+0000","last_undegraded":"2024-05-04T03:15:50.054833+0000","last_fullsized":"2024-05-04T03:15:50.054833+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:21:01.829349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.046558+0000","last_change":"2024-05-04T03:15:48.044403+0000","last_active":"2024-05-04T03:15:50.046558+0000","last_peered":"2024-05-04T03:15:50.046558+0000","last_clean":"2024-05-04T03:15:50.046558+0000","last_became_active":"2024-05-04T03:15:48.044021+0000","last_became_peered":"2024-05-04T03:15:48.044021+0000","last_unstale":"2024-05-04T03:15:50.046558+0000","last_undegraded":"2024-05-04T03:15:50.046558+0000","last_fullsized":"2024-05-04T03:15:50.046558+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:50:54.762070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.040245+0000","last_change":"2024-05-04T03:15:48.053233+0000","last_active":"2024-05-04T03:15:50.040245+0000","last_peered":"2024-05-04T03:15:50.040245+0000","last_clean":"2024-05-04T03:15:50.040245+0000","last_became_active":"2024-05-04T03:15:48.052809+0000","last_became_peered":"2024-05-04T03:15:48.052809+0000","last_unstale":"2024-05-04T03:15:50.040245+0000","last_undegraded":"2024-05-04T03:15:50.040245+0000","last_fullsized":"2024-05-04T03:15:50.040245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:27:20.082694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.051600+0000","last_change":"2024-05-04T03:15:48.045108+0000","last_active":"2024-05-04T03:15:50.051600+0000","last_peered":"2024-05-04T03:15:50.051600+0000","last_clean":"2024-05-04T03:15:50.051600+0000","last_became_active":"2024-05-04T03:15:48.044719+0000","last_became_peered":"2024-05-04T03:15:48.044719+0000","last_unstale":"2024-05-04T03:15:50.051600+0000","last_undegraded":"2024-05-04T03:15:50.051600+0000","last_fullsized":"2024-05-04T03:15:50.051600+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:55:02.586294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.046745+0000","last_change":"2024-05-04T03:15:48.043559+0000","last_active":"2024-05-04T03:15:50.046745+0000","last_peered":"2024-05-04T03:15:50.046745+0000","last_clean":"2024-05-04T03:15:50.046745+0000","last_became_active":"2024-05-04T03:15:48.043138+0000","last_became_peered":"2024-05-04T03:15:48.043138+0000","last_unstale":"2024-05-04T03:15:50.046745+0000","last_undegraded":"2024-05-04T03:15:50.046745+0000","last_fullsized":"2024-05-04T03:15:50.046745+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:05:25.411916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.048825+0000","last_change":"2024-05-04T03:15:48.039632+0000","last_active":"2024-05-04T03:15:50.048825+0000","last_peered":"2024-05-04T03:15:50.048825+0000","last_clean":"2024-05-04T03:15:50.048825+0000","last_became_active":"2024-05-04T03:15:48.039231+0000","last_became_peered":"2024-05-04T03:15:48.039231+0000","last_unstale":"2024-05-04T03:15:50.048825+0000","last_undegraded":"2024-05-04T03:15:50.048825+0000","last_fullsized":"2024-05-04T03:15:50.048825+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:33:12.853172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.040669+0000","last_change":"2024-05-04T03:15:48.055138+0000","last_active":"2024-05-04T03:15:50.040669+0000","last_peered":"2024-05-04T03:15:50.040669+0000","last_clean":"2024-05-04T03:15:50.040669+0000","last_became_active":"2024-05-04T03:15:48.054791+0000","last_became_peered":"2024-05-04T03:15:48.054791+0000","last_unstale":"2024-05-04T03:15:50.040669+0000","last_undegraded":"2024-05-04T03:15:50.040669+0000","last_fullsized":"2024-05-04T03:15:50.040669+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:33:36.822549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.052925+0000","last_change":"2024-05-04T03:15:48.050125+0000","last_active":"2024-05-04T03:15:50.052925+0000","last_peered":"2024-05-04T03:15:50.052925+0000","last_clean":"2024-05-04T03:15:50.052925+0000","last_became_active":"2024-05-04T03:15:48.049390+0000","last_became_peered":"2024-05-04T03:15:48.049390+0000","last_unstale":"2024-05-04T03:15:50.052925+0000","last_undegraded":"2024-05-04T03:15:50.052925+0000","last_fullsized":"2024-05-04T03:15:50.052925+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:27:05.740383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.054736+0000","last_change":"2024-05-04T03:15:48.050700+0000","last_active":"2024-05-04T03:15:50.054736+0000","last_peered":"2024-05-04T03:15:50.054736+0000","last_clean":"2024-05-04T03:15:50.054736+0000","last_became_active":"2024-05-04T03:15:48.049288+0000","last_became_peered":"2024-05-04T03:15:48.049288+0000","last_unstale":"2024-05-04T03:15:50.054736+0000","last_undegraded":"2024-05-04T03:15:50.054736+0000","last_fullsized":"2024-05-04T03:15:50.054736+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:40:07.072006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.037513+0000","last_change":"2024-05-04T03:15:48.037959+0000","last_active":"2024-05-04T03:15:50.037513+0000","last_peered":"2024-05-04T03:15:50.037513+0000","last_clean":"2024-05-04T03:15:50.037513+0000","last_became_active":"2024-05-04T03:15:48.037562+0000","last_became_peered":"2024-05-04T03:15:48.037562+0000","last_unstale":"2024-05-04T03:15:50.037513+0000","last_undegraded":"2024-05-04T03:15:50.037513+0000","last_fullsized":"2024-05-04T03:15:50.037513+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:38:17.971197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.050187+0000","last_change":"2024-05-04T03:15:48.047258+0000","last_active":"2024-05-04T03:15:50.050187+0000","last_peered":"2024-05-04T03:15:50.050187+0000","last_clean":"2024-05-04T03:15:50.050187+0000","last_became_active":"2024-05-04T03:15:48.046897+0000","last_became_peered":"2024-05-04T03:15:48.046897+0000","last_unstale":"2024-05-04T03:15:50.050187+0000","last_undegraded":"2024-05-04T03:15:50.050187+0000","last_fullsized":"2024-05-04T03:15:50.050187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:14:39.217475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.049849+0000","last_change":"2024-05-04T03:15:48.051240+0000","last_active":"2024-05-04T03:15:50.049849+0000","last_peered":"2024-05-04T03:15:50.049849+0000","last_clean":"2024-05-04T03:15:50.049849+0000","last_became_active":"2024-05-04T03:15:48.050081+0000","last_became_peered":"2024-05-04T03:15:48.050081+0000","last_unstale":"2024-05-04T03:15:50.049849+0000","last_undegraded":"2024-05-04T03:15:50.049849+0000","last_fullsized":"2024-05-04T03:15:50.049849+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:39:06.773121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.048192+0000","last_change":"2024-05-04T03:15:48.050936+0000","last_active":"2024-05-04T03:15:50.048192+0000","last_peered":"2024-05-04T03:15:50.048192+0000","last_clean":"2024-05-04T03:15:50.048192+0000","last_became_active":"2024-05-04T03:15:48.050567+0000","last_became_peered":"2024-05-04T03:15:48.050567+0000","last_unstale":"2024-05-04T03:15:50.048192+0000","last_undegraded":"2024-05-04T03:15:50.048192+0000","last_fullsized":"2024-05-04T03:15:50.048192+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:17:43.469889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.038200+0000","last_change":"2024-05-04T03:15:48.047200+0000","last_active":"2024-05-04T03:15:50.038200+0000","last_peered":"2024-05-04T03:15:50.038200+0000","last_clean":"2024-05-04T03:15:50.038200+0000","last_became_active":"2024-05-04T03:15:48.046782+0000","last_became_peered":"2024-05-04T03:15:48.046782+0000","last_unstale":"2024-05-04T03:15:50.038200+0000","last_undegraded":"2024-05-04T03:15:50.038200+0000","last_fullsized":"2024-05-04T03:15:50.038200+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:32:56.773916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.049919+0000","last_change":"2024-05-04T03:15:48.042114+0000","last_active":"2024-05-04T03:15:50.049919+0000","last_peered":"2024-05-04T03:15:50.049919+0000","last_clean":"2024-05-04T03:15:50.049919+0000","last_became_active":"2024-05-04T03:15:48.040921+0000","last_became_peered":"2024-05-04T03:15:48.040921+0000","last_unstale":"2024-05-04T03:15:50.049919+0000","last_undegraded":"2024-05-04T03:15:50.049919+0000","last_fullsized":"2024-05-04T03:15:50.049919+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:16:20.130996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.050660+0000","last_change":"2024-05-04T03:15:48.051060+0000","last_active":"2024-05-04T03:15:50.050660+0000","last_peered":"2024-05-04T03:15:50.050660+0000","last_clean":"2024-05-04T03:15:50.050660+0000","last_became_active":"2024-05-04T03:15:48.050642+0000","last_became_peered":"2024-05-04T03:15:48.050642+0000","last_unstale":"2024-05-04T03:15:50.050660+0000","last_undegraded":"2024-05-04T03:15:50.050660+0000","last_fullsized":"2024-05-04T03:15:50.050660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:23:31.193072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:03.245104+0000","last_change":"2024-05-04T03:15:45.013477+0000","last_active":"2024-05-04T03:16:03.245104+0000","last_peered":"2024-05-04T03:16:03.245104+0000","last_clean":"2024-05-04T03:16:03.245104+0000","last_became_active":"2024-05-04T03:15:45.013161+0000","last_became_peered":"2024-05-04T03:15:45.013161+0000","last_unstale":"2024-05-04T03:16:03.245104+0000","last_undegraded":"2024-05-04T03:16:03.245104+0000","last_fullsized":"2024-05-04T03:16:03.245104+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:08:14.087090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.047134+0000","last_change":"2024-05-04T03:15:48.045810+0000","last_active":"2024-05-04T03:15:50.047134+0000","last_peered":"2024-05-04T03:15:50.047134+0000","last_clean":"2024-05-04T03:15:50.047134+0000","last_became_active":"2024-05-04T03:15:48.045449+0000","last_became_peered":"2024-05-04T03:15:48.045449+0000","last_unstale":"2024-05-04T03:15:50.047134+0000","last_undegraded":"2024-05-04T03:15:50.047134+0000","last_fullsized":"2024-05-04T03:15:50.047134+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:48:05.084208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.052967+0000","last_change":"2024-05-04T03:15:45.204196+0000","last_active":"2024-05-04T03:15:50.052967+0000","last_peered":"2024-05-04T03:15:50.052967+0000","last_clean":"2024-05-04T03:15:50.052967+0000","last_became_active":"2024-05-04T03:15:45.203821+0000","last_became_peered":"2024-05-04T03:15:45.203821+0000","last_unstale":"2024-05-04T03:15:50.052967+0000","last_undegraded":"2024-05-04T03:15:50.052967+0000","last_fullsized":"2024-05-04T03:15:50.052967+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:17:05.074450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.050972+0000","last_change":"2024-05-04T03:15:48.045131+0000","last_active":"2024-05-04T03:15:50.050972+0000","last_peered":"2024-05-04T03:15:50.050972+0000","last_clean":"2024-05-04T03:15:50.050972+0000","last_became_active":"2024-05-04T03:15:48.044793+0000","last_became_peered":"2024-05-04T03:15:48.044793+0000","last_unstale":"2024-05-04T03:15:50.050972+0000","last_undegraded":"2024-05-04T03:15:50.050972+0000","last_fullsized":"2024-05-04T03:15:50.050972+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:03:37.769324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:03.242913+0000","last_change":"2024-05-04T03:15:45.013644+0000","last_active":"2024-05-04T03:16:03.242913+0000","last_peered":"2024-05-04T03:16:03.242913+0000","last_clean":"2024-05-04T03:16:03.242913+0000","last_became_active":"2024-05-04T03:15:45.013282+0000","last_became_peered":"2024-05-04T03:15:45.013282+0000","last_unstale":"2024-05-04T03:16:03.242913+0000","last_undegraded":"2024-05-04T03:16:03.242913+0000","last_fullsized":"2024-05-04T03:16:03.242913+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:21:21.204433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.048582+0000","last_change":"2024-05-04T03:15:48.043046+0000","last_active":"2024-05-04T03:15:50.048582+0000","last_peered":"2024-05-04T03:15:50.048582+0000","last_clean":"2024-05-04T03:15:50.048582+0000","last_became_active":"2024-05-04T03:15:48.042621+0000","last_became_peered":"2024-05-04T03:15:48.042621+0000","last_unstale":"2024-05-04T03:15:50.048582+0000","last_undegraded":"2024-05-04T03:15:50.048582+0000","last_fullsized":"2024-05-04T03:15:50.048582+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:03:58.916423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.051927+0000","last_change":"2024-05-04T03:15:45.010618+0000","last_active":"2024-05-04T03:15:50.051927+0000","last_peered":"2024-05-04T03:15:50.051927+0000","last_clean":"2024-05-04T03:15:50.051927+0000","last_became_active":"2024-05-04T03:15:45.009239+0000","last_became_peered":"2024-05-04T03:15:45.009239+0000","last_unstale":"2024-05-04T03:15:50.051927+0000","last_undegraded":"2024-05-04T03:15:50.051927+0000","last_fullsized":"2024-05-04T03:15:50.051927+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:49:46.777571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.051942+0000","last_change":"2024-05-04T03:15:48.049334+0000","last_active":"2024-05-04T03:15:50.051942+0000","last_peered":"2024-05-04T03:15:50.051942+0000","last_clean":"2024-05-04T03:15:50.051942+0000","last_became_active":"2024-05-04T03:15:48.048984+0000","last_became_peered":"2024-05-04T03:15:48.048984+0000","last_unstale":"2024-05-04T03:15:50.051942+0000","last_undegraded":"2024-05-04T03:15:50.051942+0000","last_fullsized":"2024-05-04T03:15:50.051942+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:28:25.366106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":54,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:28.289572+0000","last_change":"2024-05-04T03:15:45.015277+0000","last_active":"2024-05-04T03:16:28.289572+0000","last_peered":"2024-05-04T03:16:28.289572+0000","last_clean":"2024-05-04T03:16:28.289572+0000","last_became_active":"2024-05-04T03:15:45.014542+0000","last_became_peered":"2024-05-04T03:15:45.014542+0000","last_unstale":"2024-05-04T03:16:28.289572+0000","last_undegraded":"2024-05-04T03:16:28.289572+0000","last_fullsized":"2024-05-04T03:16:28.289572+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:22:15.594866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.046648+0000","last_change":"2024-05-04T03:15:48.045029+0000","last_active":"2024-05-04T03:15:50.046648+0000","last_peered":"2024-05-04T03:15:50.046648+0000","last_clean":"2024-05-04T03:15:50.046648+0000","last_became_active":"2024-05-04T03:15:48.041262+0000","last_became_peered":"2024-05-04T03:15:48.041262+0000","last_unstale":"2024-05-04T03:15:50.046648+0000","last_undegraded":"2024-05-04T03:15:50.046648+0000","last_fullsized":"2024-05-04T03:15:50.046648+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:45:56.882957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:03.242881+0000","last_change":"2024-05-04T03:15:45.007530+0000","last_active":"2024-05-04T03:16:03.242881+0000","last_peered":"2024-05-04T03:16:03.242881+0000","last_clean":"2024-05-04T03:16:03.242881+0000","last_became_active":"2024-05-04T03:15:45.007054+0000","last_became_peered":"2024-05-04T03:15:45.007054+0000","last_unstale":"2024-05-04T03:16:03.242881+0000","last_undegraded":"2024-05-04T03:16:03.242881+0000","last_fullsized":"2024-05-04T03:16:03.242881+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:09:13.845679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.051231+0000","last_change":"2024-05-04T03:15:48.050956+0000","last_active":"2024-05-04T03:15:50.051231+0000","last_peered":"2024-05-04T03:15:50.051231+0000","last_clean":"2024-05-04T03:15:50.051231+0000","last_became_active":"2024-05-04T03:15:48.050597+0000","last_became_peered":"2024-05-04T03:15:48.050597+0000","last_unstale":"2024-05-04T03:15:50.051231+0000","last_undegraded":"2024-05-04T03:15:50.051231+0000","last_fullsized":"2024-05-04T03:15:50.051231+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:27:34.127889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:03.245334+0000","last_change":"2024-05-04T03:15:45.007027+0000","last_active":"2024-05-04T03:16:03.245334+0000","last_peered":"2024-05-04T03:16:03.245334+0000","last_clean":"2024-05-04T03:16:03.245334+0000","last_became_active":"2024-05-04T03:15:45.006659+0000","last_became_peered":"2024-05-04T03:15:45.006659+0000","last_unstale":"2024-05-04T03:16:03.245334+0000","last_undegraded":"2024-05-04T03:16:03.245334+0000","last_fullsized":"2024-05-04T03:16:03.245334+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:06:11.827093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.046952+0000","last_change":"2024-05-04T03:15:48.046196+0000","last_active":"2024-05-04T03:15:50.046952+0000","last_peered":"2024-05-04T03:15:50.046952+0000","last_clean":"2024-05-04T03:15:50.046952+0000","last_became_active":"2024-05-04T03:15:48.044316+0000","last_became_peered":"2024-05-04T03:15:48.044316+0000","last_unstale":"2024-05-04T03:15:50.046952+0000","last_undegraded":"2024-05-04T03:15:50.046952+0000","last_fullsized":"2024-05-04T03:15:50.046952+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:04:39.301705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.050536+0000","last_change":"2024-05-04T03:15:45.015360+0000","last_active":"2024-05-04T03:15:50.050536+0000","last_peered":"2024-05-04T03:15:50.050536+0000","last_clean":"2024-05-04T03:15:50.050536+0000","last_became_active":"2024-05-04T03:15:45.014696+0000","last_became_peered":"2024-05-04T03:15:45.014696+0000","last_unstale":"2024-05-04T03:15:50.050536+0000","last_undegraded":"2024-05-04T03:15:50.050536+0000","last_fullsized":"2024-05-04T03:15:50.050536+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:17:28.838651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.049568+0000","last_change":"2024-05-04T03:15:48.039823+0000","last_active":"2024-05-04T03:15:50.049568+0000","last_peered":"2024-05-04T03:15:50.049568+0000","last_clean":"2024-05-04T03:15:50.049568+0000","last_became_active":"2024-05-04T03:15:48.039447+0000","last_became_peered":"2024-05-04T03:15:48.039447+0000","last_unstale":"2024-05-04T03:15:50.049568+0000","last_undegraded":"2024-05-04T03:15:50.049568+0000","last_fullsized":"2024-05-04T03:15:50.049568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:45:55.136946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":53,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:31.293815+0000","last_change":"2024-05-04T03:15:45.004662+0000","last_active":"2024-05-04T03:16:31.293815+0000","last_peered":"2024-05-04T03:16:31.293815+0000","last_clean":"2024-05-04T03:16:31.293815+0000","last_became_active":"2024-05-04T03:15:45.004265+0000","last_became_peered":"2024-05-04T03:15:45.004265+0000","last_unstale":"2024-05-04T03:16:31.293815+0000","last_undegraded":"2024-05-04T03:16:31.293815+0000","last_fullsized":"2024-05-04T03:16:31.293815+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:14:03.065346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":28,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.048754+0000","last_change":"2024-05-04T03:15:48.042093+0000","last_active":"2024-05-04T03:15:50.048754+0000","last_peered":"2024-05-04T03:15:50.048754+0000","last_clean":"2024-05-04T03:15:50.048754+0000","last_became_active":"2024-05-04T03:15:48.041646+0000","last_became_peered":"2024-05-04T03:15:48.041646+0000","last_unstale":"2024-05-04T03:15:50.048754+0000","last_undegraded":"2024-05-04T03:15:50.048754+0000","last_fullsized":"2024-05-04T03:15:50.048754+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:06:32.083493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:02.230658+0000","last_change":"2024-05-04T03:15:45.016731+0000","last_active":"2024-05-04T03:16:02.230658+0000","last_peered":"2024-05-04T03:16:02.230658+0000","last_clean":"2024-05-04T03:16:02.230658+0000","last_became_active":"2024-05-04T03:15:45.016313+0000","last_became_peered":"2024-05-04T03:15:45.016313+0000","last_unstale":"2024-05-04T03:16:02.230658+0000","last_undegraded":"2024-05-04T03:16:02.230658+0000","last_fullsized":"2024-05-04T03:16:02.230658+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:42:40.830986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.050873+0000","last_change":"2024-05-04T03:15:48.053649+0000","last_active":"2024-05-04T03:15:50.050873+0000","last_peered":"2024-05-04T03:15:50.050873+0000","last_clean":"2024-05-04T03:15:50.050873+0000","last_became_active":"2024-05-04T03:15:48.052407+0000","last_became_peered":"2024-05-04T03:15:48.052407+0000","last_unstale":"2024-05-04T03:15:50.050873+0000","last_undegraded":"2024-05-04T03:15:50.050873+0000","last_fullsized":"2024-05-04T03:15:50.050873+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:37:28.359272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:02.231042+0000","last_change":"2024-05-04T03:15:45.205180+0000","last_active":"2024-05-04T03:16:02.231042+0000","last_peered":"2024-05-04T03:16:02.231042+0000","last_clean":"2024-05-04T03:16:02.231042+0000","last_became_active":"2024-05-04T03:15:45.204828+0000","last_became_peered":"2024-05-04T03:15:45.204828+0000","last_unstale":"2024-05-04T03:16:02.231042+0000","last_undegraded":"2024-05-04T03:16:02.231042+0000","last_fullsized":"2024-05-04T03:16:02.231042+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:27:34.063540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.048566+0000","last_change":"2024-05-04T03:15:48.047154+0000","last_active":"2024-05-04T03:15:50.048566+0000","last_peered":"2024-05-04T03:15:50.048566+0000","last_clean":"2024-05-04T03:15:50.048566+0000","last_became_active":"2024-05-04T03:15:48.046794+0000","last_became_peered":"2024-05-04T03:15:48.046794+0000","last_unstale":"2024-05-04T03:15:50.048566+0000","last_undegraded":"2024-05-04T03:15:50.048566+0000","last_fullsized":"2024-05-04T03:15:50.048566+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:21:42.723252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'30","reported_seq":116,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:32.295704+0000","last_change":"2024-05-04T03:15:45.008930+0000","last_active":"2024-05-04T03:16:32.295704+0000","last_peered":"2024-05-04T03:16:32.295704+0000","last_clean":"2024-05-04T03:16:32.295704+0000","last_became_active":"2024-05-04T03:15:45.008459+0000","last_became_peered":"2024-05-04T03:15:45.008459+0000","last_unstale":"2024-05-04T03:16:32.295704+0000","last_undegraded":"2024-05-04T03:16:32.295704+0000","last_fullsized":"2024-05-04T03:16:32.295704+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:07:52.609156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62568,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":64,"num_read_kb":93,"num_write":31,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.046965+0000","last_change":"2024-05-04T03:15:48.051848+0000","last_active":"2024-05-04T03:15:50.046965+0000","last_peered":"2024-05-04T03:15:50.046965+0000","last_clean":"2024-05-04T03:15:50.046965+0000","last_became_active":"2024-05-04T03:15:48.051171+0000","last_became_peered":"2024-05-04T03:15:48.051171+0000","last_unstale":"2024-05-04T03:15:50.046965+0000","last_undegraded":"2024-05-04T03:15:50.046965+0000","last_fullsized":"2024-05-04T03:15:50.046965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:20:32.722845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:03.243780+0000","last_change":"2024-05-04T03:15:45.007970+0000","last_active":"2024-05-04T03:16:03.243780+0000","last_peered":"2024-05-04T03:16:03.243780+0000","last_clean":"2024-05-04T03:16:03.243780+0000","last_became_active":"2024-05-04T03:15:45.007605+0000","last_became_peered":"2024-05-04T03:15:45.007605+0000","last_unstale":"2024-05-04T03:16:03.243780+0000","last_undegraded":"2024-05-04T03:16:03.243780+0000","last_fullsized":"2024-05-04T03:16:03.243780+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:52:08.466858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.036909+0000","last_change":"2024-05-04T03:15:48.050647+0000","last_active":"2024-05-04T03:15:50.036909+0000","last_peered":"2024-05-04T03:15:50.036909+0000","last_clean":"2024-05-04T03:15:50.036909+0000","last_became_active":"2024-05-04T03:15:48.050296+0000","last_became_peered":"2024-05-04T03:15:48.050296+0000","last_unstale":"2024-05-04T03:15:50.036909+0000","last_undegraded":"2024-05-04T03:15:50.036909+0000","last_fullsized":"2024-05-04T03:15:50.036909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:14:23.902094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:02.233134+0000","last_change":"2024-05-04T03:15:45.014936+0000","last_active":"2024-05-04T03:16:02.233134+0000","last_peered":"2024-05-04T03:16:02.233134+0000","last_clean":"2024-05-04T03:16:02.233134+0000","last_became_active":"2024-05-04T03:15:45.014474+0000","last_became_peered":"2024-05-04T03:15:45.014474+0000","last_unstale":"2024-05-04T03:16:02.233134+0000","last_undegraded":"2024-05-04T03:16:02.233134+0000","last_fullsized":"2024-05-04T03:16:02.233134+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:26:37.972523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.052133+0000","last_change":"2024-05-04T03:15:48.047390+0000","last_active":"2024-05-04T03:15:50.052133+0000","last_peered":"2024-05-04T03:15:50.052133+0000","last_clean":"2024-05-04T03:15:50.052133+0000","last_became_active":"2024-05-04T03:15:48.046747+0000","last_became_peered":"2024-05-04T03:15:48.046747+0000","last_unstale":"2024-05-04T03:15:50.052133+0000","last_undegraded":"2024-05-04T03:15:50.052133+0000","last_fullsized":"2024-05-04T03:15:50.052133+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:30:14.628532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'4","reported_seq":53,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:27.287379+0000","last_change":"2024-05-04T03:15:45.013177+0000","last_active":"2024-05-04T03:16:27.287379+0000","last_peered":"2024-05-04T03:16:27.287379+0000","last_clean":"2024-05-04T03:16:27.287379+0000","last_became_active":"2024-05-04T03:15:45.012764+0000","last_became_peered":"2024-05-04T03:15:45.012764+0000","last_unstale":"2024-05-04T03:16:27.287379+0000","last_undegraded":"2024-05-04T03:16:27.287379+0000","last_fullsized":"2024-05-04T03:16:27.287379+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:07:43.578443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.051645+0000","last_change":"2024-05-04T03:15:48.046453+0000","last_active":"2024-05-04T03:15:50.051645+0000","last_peered":"2024-05-04T03:15:50.051645+0000","last_clean":"2024-05-04T03:15:50.051645+0000","last_became_active":"2024-05-04T03:15:48.045990+0000","last_became_peered":"2024-05-04T03:15:48.045990+0000","last_unstale":"2024-05-04T03:15:50.051645+0000","last_undegraded":"2024-05-04T03:15:50.051645+0000","last_fullsized":"2024-05-04T03:15:50.051645+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:54:57.285122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:03.245575+0000","last_change":"2024-05-04T03:15:45.205539+0000","last_active":"2024-05-04T03:16:03.245575+0000","last_peered":"2024-05-04T03:16:03.245575+0000","last_clean":"2024-05-04T03:16:03.245575+0000","last_became_active":"2024-05-04T03:15:45.205174+0000","last_became_peered":"2024-05-04T03:15:45.205174+0000","last_unstale":"2024-05-04T03:16:03.245575+0000","last_undegraded":"2024-05-04T03:16:03.245575+0000","last_fullsized":"2024-05-04T03:16:03.245575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:03:53.398368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.053186+0000","last_change":"2024-05-04T03:15:48.051824+0000","last_active":"2024-05-04T03:15:50.053186+0000","last_peered":"2024-05-04T03:15:50.053186+0000","last_clean":"2024-05-04T03:15:50.053186+0000","last_became_active":"2024-05-04T03:15:48.051483+0000","last_became_peered":"2024-05-04T03:15:48.051483+0000","last_unstale":"2024-05-04T03:15:50.053186+0000","last_undegraded":"2024-05-04T03:15:50.053186+0000","last_fullsized":"2024-05-04T03:15:50.053186+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:57:56.644486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:04.238402+0000","last_change":"2024-05-04T03:15:45.011062+0000","last_active":"2024-05-04T03:16:04.238402+0000","last_peered":"2024-05-04T03:16:04.238402+0000","last_clean":"2024-05-04T03:16:04.238402+0000","last_became_active":"2024-05-04T03:15:45.010730+0000","last_became_peered":"2024-05-04T03:15:45.010730+0000","last_unstale":"2024-05-04T03:16:04.238402+0000","last_undegraded":"2024-05-04T03:16:04.238402+0000","last_fullsized":"2024-05-04T03:16:04.238402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:45:55.979067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.038002+0000","last_change":"2024-05-04T03:15:48.045965+0000","last_active":"2024-05-04T03:15:50.038002+0000","last_peered":"2024-05-04T03:15:50.038002+0000","last_clean":"2024-05-04T03:15:50.038002+0000","last_became_active":"2024-05-04T03:15:48.045615+0000","last_became_peered":"2024-05-04T03:15:48.045615+0000","last_unstale":"2024-05-04T03:15:50.038002+0000","last_undegraded":"2024-05-04T03:15:50.038002+0000","last_fullsized":"2024-05-04T03:15:50.038002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:14:23.497484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.049777+0000","last_change":"2024-05-04T03:15:45.204223+0000","last_active":"2024-05-04T03:15:50.049777+0000","last_peered":"2024-05-04T03:15:50.049777+0000","last_clean":"2024-05-04T03:15:50.049777+0000","last_became_active":"2024-05-04T03:15:45.203774+0000","last_became_peered":"2024-05-04T03:15:45.203774+0000","last_unstale":"2024-05-04T03:15:50.049777+0000","last_undegraded":"2024-05-04T03:15:50.049777+0000","last_fullsized":"2024-05-04T03:15:50.049777+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:02:34.456055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.040518+0000","last_change":"2024-05-04T03:15:48.045574+0000","last_active":"2024-05-04T03:15:50.040518+0000","last_peered":"2024-05-04T03:15:50.040518+0000","last_clean":"2024-05-04T03:15:50.040518+0000","last_became_active":"2024-05-04T03:15:48.045237+0000","last_became_peered":"2024-05-04T03:15:48.045237+0000","last_unstale":"2024-05-04T03:15:50.040518+0000","last_undegraded":"2024-05-04T03:15:50.040518+0000","last_fullsized":"2024-05-04T03:15:50.040518+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:53:46.346199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.051933+0000","last_change":"2024-05-04T03:15:45.013895+0000","last_active":"2024-05-04T03:15:50.051933+0000","last_peered":"2024-05-04T03:15:50.051933+0000","last_clean":"2024-05-04T03:15:50.051933+0000","last_became_active":"2024-05-04T03:15:45.013306+0000","last_became_peered":"2024-05-04T03:15:45.013306+0000","last_unstale":"2024-05-04T03:15:50.051933+0000","last_undegraded":"2024-05-04T03:15:50.051933+0000","last_fullsized":"2024-05-04T03:15:50.051933+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:39:18.616432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.040373+0000","last_change":"2024-05-04T03:15:48.054643+0000","last_active":"2024-05-04T03:15:50.040373+0000","last_peered":"2024-05-04T03:15:50.040373+0000","last_clean":"2024-05-04T03:15:50.040373+0000","last_became_active":"2024-05-04T03:15:48.054171+0000","last_became_peered":"2024-05-04T03:15:48.054171+0000","last_unstale":"2024-05-04T03:15:50.040373+0000","last_undegraded":"2024-05-04T03:15:50.040373+0000","last_fullsized":"2024-05-04T03:15:50.040373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:49:47.732848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.050820+0000","last_change":"2024-05-04T03:15:48.050122+0000","last_active":"2024-05-04T03:15:50.050820+0000","last_peered":"2024-05-04T03:15:50.050820+0000","last_clean":"2024-05-04T03:15:50.050820+0000","last_became_active":"2024-05-04T03:15:48.049794+0000","last_became_peered":"2024-05-04T03:15:48.049794+0000","last_unstale":"2024-05-04T03:15:50.050820+0000","last_undegraded":"2024-05-04T03:15:50.050820+0000","last_fullsized":"2024-05-04T03:15:50.050820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:29:11.066068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:26.284961+0000","last_change":"2024-05-04T03:15:45.205143+0000","last_active":"2024-05-04T03:16:26.284961+0000","last_peered":"2024-05-04T03:16:26.284961+0000","last_clean":"2024-05-04T03:16:26.284961+0000","last_became_active":"2024-05-04T03:15:45.204727+0000","last_became_peered":"2024-05-04T03:15:45.204727+0000","last_unstale":"2024-05-04T03:16:26.284961+0000","last_undegraded":"2024-05-04T03:16:26.284961+0000","last_fullsized":"2024-05-04T03:16:26.284961+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:15:05.799556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.047004+0000","last_change":"2024-05-04T03:15:48.042914+0000","last_active":"2024-05-04T03:15:50.047004+0000","last_peered":"2024-05-04T03:15:50.047004+0000","last_clean":"2024-05-04T03:15:50.047004+0000","last_became_active":"2024-05-04T03:15:48.041936+0000","last_became_peered":"2024-05-04T03:15:48.041936+0000","last_unstale":"2024-05-04T03:15:50.047004+0000","last_undegraded":"2024-05-04T03:15:50.047004+0000","last_fullsized":"2024-05-04T03:15:50.047004+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:11:48.807602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:05.249261+0000","last_change":"2024-05-04T03:15:45.011045+0000","last_active":"2024-05-04T03:16:05.249261+0000","last_peered":"2024-05-04T03:16:05.249261+0000","last_clean":"2024-05-04T03:16:05.249261+0000","last_became_active":"2024-05-04T03:15:45.010608+0000","last_became_peered":"2024-05-04T03:15:45.010608+0000","last_unstale":"2024-05-04T03:16:05.249261+0000","last_undegraded":"2024-05-04T03:16:05.249261+0000","last_fullsized":"2024-05-04T03:16:05.249261+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:00:36.243588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":97,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.051809+0000","last_change":"2024-05-04T03:14:19.895070+0000","last_active":"2024-05-04T03:15:50.051809+0000","last_peered":"2024-05-04T03:15:50.051809+0000","last_clean":"2024-05-04T03:15:50.051809+0000","last_became_active":"2024-05-04T03:14:19.889987+0000","last_became_peered":"2024-05-04T03:14:19.889987+0000","last_unstale":"2024-05-04T03:15:50.051809+0000","last_undegraded":"2024-05-04T03:15:50.051809+0000","last_fullsized":"2024-05-04T03:15:50.051809+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:11:01.712629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:11:01.712629+0000","last_clean_scrub_stamp":"2024-05-04T03:11:01.712629+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:03:02.769719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.036881+0000","last_change":"2024-05-04T03:15:48.044928+0000","last_active":"2024-05-04T03:15:50.036881+0000","last_peered":"2024-05-04T03:15:50.036881+0000","last_clean":"2024-05-04T03:15:50.036881+0000","last_became_active":"2024-05-04T03:15:48.044532+0000","last_became_peered":"2024-05-04T03:15:48.044532+0000","last_unstale":"2024-05-04T03:15:50.036881+0000","last_undegraded":"2024-05-04T03:15:50.036881+0000","last_fullsized":"2024-05-04T03:15:50.036881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:27:00.825692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:03.244129+0000","last_change":"2024-05-04T03:15:45.014469+0000","last_active":"2024-05-04T03:16:03.244129+0000","last_peered":"2024-05-04T03:16:03.244129+0000","last_clean":"2024-05-04T03:16:03.244129+0000","last_became_active":"2024-05-04T03:15:45.013763+0000","last_became_peered":"2024-05-04T03:15:45.013763+0000","last_unstale":"2024-05-04T03:16:03.244129+0000","last_undegraded":"2024-05-04T03:16:03.244129+0000","last_fullsized":"2024-05-04T03:16:03.244129+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:19:36.379930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:02.231310+0000","last_change":"2024-05-04T03:15:45.204536+0000","last_active":"2024-05-04T03:16:02.231310+0000","last_peered":"2024-05-04T03:16:02.231310+0000","last_clean":"2024-05-04T03:16:02.231310+0000","last_became_active":"2024-05-04T03:15:45.204072+0000","last_became_peered":"2024-05-04T03:15:45.204072+0000","last_unstale":"2024-05-04T03:16:02.231310+0000","last_undegraded":"2024-05-04T03:16:02.231310+0000","last_fullsized":"2024-05-04T03:16:02.231310+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:49:12.726220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.037172+0000","last_change":"2024-05-04T03:15:48.044982+0000","last_active":"2024-05-04T03:15:50.037172+0000","last_peered":"2024-05-04T03:15:50.037172+0000","last_clean":"2024-05-04T03:15:50.037172+0000","last_became_active":"2024-05-04T03:15:48.043996+0000","last_became_peered":"2024-05-04T03:15:48.043996+0000","last_unstale":"2024-05-04T03:15:50.037172+0000","last_undegraded":"2024-05-04T03:15:50.037172+0000","last_fullsized":"2024-05-04T03:15:50.037172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:09:39.770023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'10","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:26.047352+0000","last_change":"2024-05-04T03:15:45.008153+0000","last_active":"2024-05-04T03:16:26.047352+0000","last_peered":"2024-05-04T03:16:26.047352+0000","last_clean":"2024-05-04T03:16:26.047352+0000","last_became_active":"2024-05-04T03:15:45.007698+0000","last_became_peered":"2024-05-04T03:15:45.007698+0000","last_unstale":"2024-05-04T03:16:26.047352+0000","last_undegraded":"2024-05-04T03:16:26.047352+0000","last_fullsized":"2024-05-04T03:16:26.047352+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:56:52.322049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.047367+0000","last_change":"2024-05-04T03:15:48.055767+0000","last_active":"2024-05-04T03:15:50.047367+0000","last_peered":"2024-05-04T03:15:50.047367+0000","last_clean":"2024-05-04T03:15:50.047367+0000","last_became_active":"2024-05-04T03:15:48.055252+0000","last_became_peered":"2024-05-04T03:15:48.055252+0000","last_unstale":"2024-05-04T03:15:50.047367+0000","last_undegraded":"2024-05-04T03:15:50.047367+0000","last_fullsized":"2024-05-04T03:15:50.047367+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:28:11.736803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:02.230206+0000","last_change":"2024-05-04T03:15:45.012838+0000","last_active":"2024-05-04T03:16:02.230206+0000","last_peered":"2024-05-04T03:16:02.230206+0000","last_clean":"2024-05-04T03:16:02.230206+0000","last_became_active":"2024-05-04T03:15:45.012429+0000","last_became_peered":"2024-05-04T03:15:45.012429+0000","last_unstale":"2024-05-04T03:16:02.230206+0000","last_undegraded":"2024-05-04T03:16:02.230206+0000","last_fullsized":"2024-05-04T03:16:02.230206+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:22:22.602551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.050642+0000","last_change":"2024-05-04T03:15:48.056993+0000","last_active":"2024-05-04T03:15:50.050642+0000","last_peered":"2024-05-04T03:15:50.050642+0000","last_clean":"2024-05-04T03:15:50.050642+0000","last_became_active":"2024-05-04T03:15:48.055938+0000","last_became_peered":"2024-05-04T03:15:48.055938+0000","last_unstale":"2024-05-04T03:15:50.050642+0000","last_undegraded":"2024-05-04T03:15:50.050642+0000","last_fullsized":"2024-05-04T03:15:50.050642+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:23:59.521692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:02.232885+0000","last_change":"2024-05-04T03:15:45.204652+0000","last_active":"2024-05-04T03:16:02.232885+0000","last_peered":"2024-05-04T03:16:02.232885+0000","last_clean":"2024-05-04T03:16:02.232885+0000","last_became_active":"2024-05-04T03:15:45.204266+0000","last_became_peered":"2024-05-04T03:15:45.204266+0000","last_unstale":"2024-05-04T03:16:02.232885+0000","last_undegraded":"2024-05-04T03:16:02.232885+0000","last_fullsized":"2024-05-04T03:16:02.232885+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:06:38.230692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.046466+0000","last_change":"2024-05-04T03:15:48.043614+0000","last_active":"2024-05-04T03:15:50.046466+0000","last_peered":"2024-05-04T03:15:50.046466+0000","last_clean":"2024-05-04T03:15:50.046466+0000","last_became_active":"2024-05-04T03:15:48.042390+0000","last_became_peered":"2024-05-04T03:15:48.042390+0000","last_unstale":"2024-05-04T03:15:50.046466+0000","last_undegraded":"2024-05-04T03:15:50.046466+0000","last_fullsized":"2024-05-04T03:15:50.046466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:56:22.791368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:02.232827+0000","last_change":"2024-05-04T03:15:45.005570+0000","last_active":"2024-05-04T03:16:02.232827+0000","last_peered":"2024-05-04T03:16:02.232827+0000","last_clean":"2024-05-04T03:16:02.232827+0000","last_became_active":"2024-05-04T03:15:45.005105+0000","last_became_peered":"2024-05-04T03:15:45.005105+0000","last_unstale":"2024-05-04T03:16:02.232827+0000","last_undegraded":"2024-05-04T03:16:02.232827+0000","last_fullsized":"2024-05-04T03:16:02.232827+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:40:17.275618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.037743+0000","last_change":"2024-05-04T03:15:48.045081+0000","last_active":"2024-05-04T03:15:50.037743+0000","last_peered":"2024-05-04T03:15:50.037743+0000","last_clean":"2024-05-04T03:15:50.037743+0000","last_became_active":"2024-05-04T03:15:48.044745+0000","last_became_peered":"2024-05-04T03:15:48.044745+0000","last_unstale":"2024-05-04T03:15:50.037743+0000","last_undegraded":"2024-05-04T03:15:50.037743+0000","last_fullsized":"2024-05-04T03:15:50.037743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:40:53.486222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.048056+0000","last_change":"2024-05-04T03:15:45.008926+0000","last_active":"2024-05-04T03:15:50.048056+0000","last_peered":"2024-05-04T03:15:50.048056+0000","last_clean":"2024-05-04T03:15:50.048056+0000","last_became_active":"2024-05-04T03:15:45.008412+0000","last_became_peered":"2024-05-04T03:15:45.008412+0000","last_unstale":"2024-05-04T03:15:50.048056+0000","last_undegraded":"2024-05-04T03:15:50.048056+0000","last_fullsized":"2024-05-04T03:15:50.048056+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:34:33.107866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.046816+0000","last_change":"2024-05-04T03:15:48.054580+0000","last_active":"2024-05-04T03:15:50.046816+0000","last_peered":"2024-05-04T03:15:50.046816+0000","last_clean":"2024-05-04T03:15:50.046816+0000","last_became_active":"2024-05-04T03:15:48.054242+0000","last_became_peered":"2024-05-04T03:15:48.054242+0000","last_unstale":"2024-05-04T03:15:50.046816+0000","last_undegraded":"2024-05-04T03:15:50.046816+0000","last_fullsized":"2024-05-04T03:15:50.046816+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:16:19.426571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:02.232185+0000","last_change":"2024-05-04T03:15:45.015504+0000","last_active":"2024-05-04T03:16:02.232185+0000","last_peered":"2024-05-04T03:16:02.232185+0000","last_clean":"2024-05-04T03:16:02.232185+0000","last_became_active":"2024-05-04T03:15:45.015153+0000","last_became_peered":"2024-05-04T03:15:45.015153+0000","last_unstale":"2024-05-04T03:16:02.232185+0000","last_undegraded":"2024-05-04T03:16:02.232185+0000","last_fullsized":"2024-05-04T03:16:02.232185+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:25:17.293674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.051912+0000","last_change":"2024-05-04T03:15:48.045284+0000","last_active":"2024-05-04T03:15:50.051912+0000","last_peered":"2024-05-04T03:15:50.051912+0000","last_clean":"2024-05-04T03:15:50.051912+0000","last_became_active":"2024-05-04T03:15:48.044673+0000","last_became_peered":"2024-05-04T03:15:48.044673+0000","last_unstale":"2024-05-04T03:15:50.051912+0000","last_undegraded":"2024-05-04T03:15:50.051912+0000","last_fullsized":"2024-05-04T03:15:50.051912+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:51:52.389748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:16:03.245074+0000","last_change":"2024-05-04T03:15:45.205648+0000","last_active":"2024-05-04T03:16:03.245074+0000","last_peered":"2024-05-04T03:16:03.245074+0000","last_clean":"2024-05-04T03:16:03.245074+0000","last_became_active":"2024-05-04T03:15:45.204169+0000","last_became_peered":"2024-05-04T03:15:45.204169+0000","last_unstale":"2024-05-04T03:16:03.245074+0000","last_undegraded":"2024-05-04T03:16:03.245074+0000","last_fullsized":"2024-05-04T03:16:03.245074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:20:48.859276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.052862+0000","last_change":"2024-05-04T03:15:48.049808+0000","last_active":"2024-05-04T03:15:50.052862+0000","last_peered":"2024-05-04T03:15:50.052862+0000","last_clean":"2024-05-04T03:15:50.052862+0000","last_became_active":"2024-05-04T03:15:48.048998+0000","last_became_peered":"2024-05-04T03:15:48.048998+0000","last_unstale":"2024-05-04T03:15:50.052862+0000","last_undegraded":"2024-05-04T03:15:50.052862+0000","last_fullsized":"2024-05-04T03:15:50.052862+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:54:46.786406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.056975+0000","last_change":"2024-05-04T03:15:45.008874+0000","last_active":"2024-05-04T03:15:50.056975+0000","last_peered":"2024-05-04T03:15:50.056975+0000","last_clean":"2024-05-04T03:15:50.056975+0000","last_became_active":"2024-05-04T03:15:45.008530+0000","last_became_peered":"2024-05-04T03:15:45.008530+0000","last_unstale":"2024-05-04T03:15:50.056975+0000","last_undegraded":"2024-05-04T03:15:50.056975+0000","last_fullsized":"2024-05-04T03:15:50.056975+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:31:41.697386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.052480+0000","last_change":"2024-05-04T03:15:48.046042+0000","last_active":"2024-05-04T03:15:50.052480+0000","last_peered":"2024-05-04T03:15:50.052480+0000","last_clean":"2024-05-04T03:15:50.052480+0000","last_became_active":"2024-05-04T03:15:48.045387+0000","last_became_peered":"2024-05-04T03:15:48.045387+0000","last_unstale":"2024-05-04T03:15:50.052480+0000","last_undegraded":"2024-05-04T03:15:50.052480+0000","last_fullsized":"2024-05-04T03:15:50.052480+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:43:50.782623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.046516+0000","last_change":"2024-05-04T03:15:45.009126+0000","last_active":"2024-05-04T03:15:50.046516+0000","last_peered":"2024-05-04T03:15:50.046516+0000","last_clean":"2024-05-04T03:15:50.046516+0000","last_became_active":"2024-05-04T03:15:45.008802+0000","last_became_peered":"2024-05-04T03:15:45.008802+0000","last_unstale":"2024-05-04T03:15:50.046516+0000","last_undegraded":"2024-05-04T03:15:50.046516+0000","last_fullsized":"2024-05-04T03:15:50.046516+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:25:30.867870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.052793+0000","last_change":"2024-05-04T03:15:48.050478+0000","last_active":"2024-05-04T03:15:50.052793+0000","last_peered":"2024-05-04T03:15:50.052793+0000","last_clean":"2024-05-04T03:15:50.052793+0000","last_became_active":"2024-05-04T03:15:48.045756+0000","last_became_peered":"2024-05-04T03:15:48.045756+0000","last_unstale":"2024-05-04T03:15:50.052793+0000","last_undegraded":"2024-05-04T03:15:50.052793+0000","last_fullsized":"2024-05-04T03:15:50.052793+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:05:11.098381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.037858+0000","last_change":"2024-05-04T03:15:48.042778+0000","last_active":"2024-05-04T03:15:50.037858+0000","last_peered":"2024-05-04T03:15:50.037858+0000","last_clean":"2024-05-04T03:15:50.037858+0000","last_became_active":"2024-05-04T03:15:48.042349+0000","last_became_peered":"2024-05-04T03:15:48.042349+0000","last_unstale":"2024-05-04T03:15:50.037858+0000","last_undegraded":"2024-05-04T03:15:50.037858+0000","last_fullsized":"2024-05-04T03:15:50.037858+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:40:12.260026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.051646+0000","last_change":"2024-05-04T03:15:48.048544+0000","last_active":"2024-05-04T03:15:50.051646+0000","last_peered":"2024-05-04T03:15:50.051646+0000","last_clean":"2024-05-04T03:15:50.051646+0000","last_became_active":"2024-05-04T03:15:48.047873+0000","last_became_peered":"2024-05-04T03:15:48.047873+0000","last_unstale":"2024-05-04T03:15:50.051646+0000","last_undegraded":"2024-05-04T03:15:50.051646+0000","last_fullsized":"2024-05-04T03:15:50.051646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:48:02.290401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.048680+0000","last_change":"2024-05-04T03:15:48.053467+0000","last_active":"2024-05-04T03:15:50.048680+0000","last_peered":"2024-05-04T03:15:50.048680+0000","last_clean":"2024-05-04T03:15:50.048680+0000","last_became_active":"2024-05-04T03:15:48.050796+0000","last_became_peered":"2024-05-04T03:15:48.050796+0000","last_unstale":"2024-05-04T03:15:50.048680+0000","last_undegraded":"2024-05-04T03:15:50.048680+0000","last_fullsized":"2024-05-04T03:15:50.048680+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:11:15.009467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.050345+0000","last_change":"2024-05-04T03:15:48.054616+0000","last_active":"2024-05-04T03:15:50.050345+0000","last_peered":"2024-05-04T03:15:50.050345+0000","last_clean":"2024-05-04T03:15:50.050345+0000","last_became_active":"2024-05-04T03:15:48.054142+0000","last_became_peered":"2024-05-04T03:15:48.054142+0000","last_unstale":"2024-05-04T03:15:50.050345+0000","last_undegraded":"2024-05-04T03:15:50.050345+0000","last_fullsized":"2024-05-04T03:15:50.050345+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:18:24.067666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.037371+0000","last_change":"2024-05-04T03:15:48.046969+0000","last_active":"2024-05-04T03:15:50.037371+0000","last_peered":"2024-05-04T03:15:50.037371+0000","last_clean":"2024-05-04T03:15:50.037371+0000","last_became_active":"2024-05-04T03:15:48.046637+0000","last_became_peered":"2024-05-04T03:15:48.046637+0000","last_unstale":"2024-05-04T03:15:50.037371+0000","last_undegraded":"2024-05-04T03:15:50.037371+0000","last_fullsized":"2024-05-04T03:15:50.037371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:03:01.763037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.037937+0000","last_change":"2024-05-04T03:15:48.041753+0000","last_active":"2024-05-04T03:15:50.037937+0000","last_peered":"2024-05-04T03:15:50.037937+0000","last_clean":"2024-05-04T03:15:50.037937+0000","last_became_active":"2024-05-04T03:15:48.041236+0000","last_became_peered":"2024-05-04T03:15:48.041236+0000","last_unstale":"2024-05-04T03:15:50.037937+0000","last_undegraded":"2024-05-04T03:15:50.037937+0000","last_fullsized":"2024-05-04T03:15:50.037937+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:41:02.928257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.051064+0000","last_change":"2024-05-04T03:15:48.044192+0000","last_active":"2024-05-04T03:15:50.051064+0000","last_peered":"2024-05-04T03:15:50.051064+0000","last_clean":"2024-05-04T03:15:50.051064+0000","last_became_active":"2024-05-04T03:15:48.043849+0000","last_became_peered":"2024-05-04T03:15:48.043849+0000","last_unstale":"2024-05-04T03:15:50.051064+0000","last_undegraded":"2024-05-04T03:15:50.051064+0000","last_fullsized":"2024-05-04T03:15:50.051064+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:55:22.000788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.047162+0000","last_change":"2024-05-04T03:15:48.045675+0000","last_active":"2024-05-04T03:15:50.047162+0000","last_peered":"2024-05-04T03:15:50.047162+0000","last_clean":"2024-05-04T03:15:50.047162+0000","last_became_active":"2024-05-04T03:15:48.045321+0000","last_became_peered":"2024-05-04T03:15:48.045321+0000","last_unstale":"2024-05-04T03:15:50.047162+0000","last_undegraded":"2024-05-04T03:15:50.047162+0000","last_fullsized":"2024-05-04T03:15:50.047162+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:12:06.060729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.050456+0000","last_change":"2024-05-04T03:15:48.040872+0000","last_active":"2024-05-04T03:15:50.050456+0000","last_peered":"2024-05-04T03:15:50.050456+0000","last_clean":"2024-05-04T03:15:50.050456+0000","last_became_active":"2024-05-04T03:15:48.040485+0000","last_became_peered":"2024-05-04T03:15:48.040485+0000","last_unstale":"2024-05-04T03:15:50.050456+0000","last_undegraded":"2024-05-04T03:15:50.050456+0000","last_fullsized":"2024-05-04T03:15:50.050456+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:58:01.789270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.046917+0000","last_change":"2024-05-04T03:15:48.049080+0000","last_active":"2024-05-04T03:15:50.046917+0000","last_peered":"2024-05-04T03:15:50.046917+0000","last_clean":"2024-05-04T03:15:50.046917+0000","last_became_active":"2024-05-04T03:15:48.044624+0000","last_became_peered":"2024-05-04T03:15:48.044624+0000","last_unstale":"2024-05-04T03:15:50.046917+0000","last_undegraded":"2024-05-04T03:15:50.046917+0000","last_fullsized":"2024-05-04T03:15:50.046917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:47:08.647401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.048482+0000","last_change":"2024-05-04T03:15:48.049394+0000","last_active":"2024-05-04T03:15:50.048482+0000","last_peered":"2024-05-04T03:15:50.048482+0000","last_clean":"2024-05-04T03:15:50.048482+0000","last_became_active":"2024-05-04T03:15:48.048980+0000","last_became_peered":"2024-05-04T03:15:48.048980+0000","last_unstale":"2024-05-04T03:15:50.048482+0000","last_undegraded":"2024-05-04T03:15:50.048482+0000","last_fullsized":"2024-05-04T03:15:50.048482+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:43:24.914665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.037305+0000","last_change":"2024-05-04T03:15:48.051672+0000","last_active":"2024-05-04T03:15:50.037305+0000","last_peered":"2024-05-04T03:15:50.037305+0000","last_clean":"2024-05-04T03:15:50.037305+0000","last_became_active":"2024-05-04T03:15:48.051314+0000","last_became_peered":"2024-05-04T03:15:48.051314+0000","last_unstale":"2024-05-04T03:15:50.037305+0000","last_undegraded":"2024-05-04T03:15:50.037305+0000","last_fullsized":"2024-05-04T03:15:50.037305+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:19:22.808102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-04T03:15:50.037727+0000","last_change":"2024-05-04T03:15:48.053045+0000","last_active":"2024-05-04T03:15:50.037727+0000","last_peered":"2024-05-04T03:15:50.037727+0000","last_clean":"2024-05-04T03:15:50.037727+0000","last_became_active":"2024-05-04T03:15:48.052659+0000","last_became_peered":"2024-05-04T03:15:48.052659+0000","last_unstale":"2024-05-04T03:15:50.037727+0000","last_undegraded":"2024-05-04T03:15:50.037727+0000","last_fullsized":"2024-05-04T03:15:50.037727+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:53:25.624540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66569,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":183,"num_read_kb":163,"num_write":163,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":199707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612632,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":876544,"data_stored":423670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808861,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1576960,"data_stored":1078110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972386,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":892928,"data_stored":423906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168616,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":892928,"data_stored":423826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397548,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32628,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712524,"statfs":{"total":95995035648,"available":95961624576,"internally_reserved":0,"allocated":1486848,"data_stored":1015756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593777,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":880640,"data_stored":423295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":946176,"data_stored":485656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40376,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704776,"statfs":{"total":95995035648,"available":95953690624,"internally_reserved":0,"allocated":901120,"data_stored":424566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":970752,"data_stored":487006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182463,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40372,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704780,"statfs":{"total":95995035648,"available":95953694720,"internally_reserved":0,"allocated":897024,"data_stored":423606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986235,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":424123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790007,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36800,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708352,"statfs":{"total":95995035648,"available":95957352448,"internally_reserved":0,"allocated":1499136,"data_stored":1015049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-04T03:16:33.785 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-04T03:16:33.786 INFO:tasks.cephadm:Setting up client nodes... 2024-05-04T03:16:33.786 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-04T03:16:34.032 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:34.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:34 smithi001 ceph-mon[29731]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-05-04T03:16:34.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:34 smithi001 ceph-mon[29731]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:16:34.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:34 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-05-04T03:16:34.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:34 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-05-04T03:16:34.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:34 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-04T03:16:34.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:34 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-05-04T03:16:34.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:34 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-05-04T03:16:34.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:34 smithi001 ceph-mon[29731]: osdmap e82: 12 total, 12 up, 12 in 2024-05-04T03:16:34.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:34 smithi077 ceph-mon[33531]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-05-04T03:16:34.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:34 smithi077 ceph-mon[33531]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:16:34.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:34 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-05-04T03:16:34.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:34 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-05-04T03:16:34.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:34 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-04T03:16:34.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:34 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-05-04T03:16:34.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:34 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-05-04T03:16:34.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:34 smithi077 ceph-mon[33531]: osdmap e82: 12 total, 12 up, 12 in 2024-05-04T03:16:34.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:34 smithi072 ceph-mon[33332]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.7 KiB/s wr, 8 op/s 2024-05-04T03:16:34.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:34 smithi072 ceph-mon[33332]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:16:34.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:34 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-05-04T03:16:34.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:34 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-05-04T03:16:34.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:34 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-04T03:16:34.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:34 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-05-04T03:16:34.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:34 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-05-04T03:16:34.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:34 smithi072 ceph-mon[33332]: osdmap e82: 12 total, 12 up, 12 in 2024-05-04T03:16:35.520 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:35 smithi001 ceph-mon[29731]: osdmap e83: 12 total, 12 up, 12 in 2024-05-04T03:16:35.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.647+0000 7f1c8bc7e700 1 -- 172.21.15.1:0/3268945451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c84104380 msgr2=0x7f1c84104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:35.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.647+0000 7f1c8bc7e700 1 --2- 172.21.15.1:0/3268945451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c84104380 0x7f1c84104790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1c74009990 tx=0x7f1c74009ca0 comp rx=0 tx=0).stop 2024-05-04T03:16:35.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.648+0000 7f1c8bc7e700 1 -- 172.21.15.1:0/3268945451 shutdown_connections 2024-05-04T03:16:35.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.648+0000 7f1c8bc7e700 1 --2- 172.21.15.1:0/3268945451 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c84103250 0x7f1c84103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:35.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.648+0000 7f1c8bc7e700 1 --2- 172.21.15.1:0/3268945451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c84104d20 0x7f1c8410b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:35.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.648+0000 7f1c8bc7e700 1 --2- 172.21.15.1:0/3268945451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c84104380 0x7f1c84104790 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:35.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.648+0000 7f1c8bc7e700 1 -- 172.21.15.1:0/3268945451 >> 172.21.15.1:0/3268945451 conn(0x7f1c840fe870 msgr2=0x7f1c84100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:35.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.648+0000 7f1c8bc7e700 1 -- 172.21.15.1:0/3268945451 shutdown_connections 2024-05-04T03:16:35.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.648+0000 7f1c8bc7e700 1 -- 172.21.15.1:0/3268945451 wait complete. 2024-05-04T03:16:35.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.649+0000 7f1c8bc7e700 1 Processor -- start 2024-05-04T03:16:35.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.649+0000 7f1c8bc7e700 1 -- start start 2024-05-04T03:16:35.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.649+0000 7f1c8bc7e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c84103250 0x7f1c84076450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:35.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.649+0000 7f1c8bc7e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c84104380 0x7f1c8419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:35.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.649+0000 7f1c8bc7e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c84104d20 0x7f1c841a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:35.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.649+0000 7f1c8bc7e700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8410ed10 con 0x7f1c84104380 2024-05-04T03:16:35.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.649+0000 7f1c8bc7e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8410eb90 con 0x7f1c84103250 2024-05-04T03:16:35.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.649+0000 7f1c8bc7e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8410ee90 con 0x7f1c84104d20 2024-05-04T03:16:35.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.649+0000 7f1c89219700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c84104380 0x7f1c8419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:35.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.649+0000 7f1c89219700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c84104380 0x7f1c8419f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56222/0 (socket says 172.21.15.1:56222) 2024-05-04T03:16:35.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.649+0000 7f1c89219700 1 -- 172.21.15.1:0/3697121309 learned_addr learned my addr 172.21.15.1:0/3697121309 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:16:35.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.650+0000 7f1c89a1a700 1 --2- 172.21.15.1:0/3697121309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c84103250 0x7f1c84076450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:35.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.650+0000 7f1c8a21b700 1 --2- 172.21.15.1:0/3697121309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c84104d20 0x7f1c841a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:35.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.650+0000 7f1c89219700 1 -- 172.21.15.1:0/3697121309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c84104d20 msgr2=0x7f1c841a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:35.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.650+0000 7f1c89219700 1 --2- 172.21.15.1:0/3697121309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c84104d20 0x7f1c841a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:35.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.650+0000 7f1c89219700 1 -- 172.21.15.1:0/3697121309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c84103250 msgr2=0x7f1c84076450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:35.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.650+0000 7f1c89219700 1 --2- 172.21.15.1:0/3697121309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c84103250 0x7f1c84076450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:35.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.650+0000 7f1c89219700 1 -- 172.21.15.1:0/3697121309 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c841a24b0 con 0x7f1c84104380 2024-05-04T03:16:35.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.650+0000 7f1c89219700 1 --2- 172.21.15.1:0/3697121309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c84104380 0x7f1c8419f710 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1c740098e0 tx=0x7f1c740057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:35.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.650+0000 7f1c7affd700 1 -- 172.21.15.1:0/3697121309 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c74026070 con 0x7f1c84104380 2024-05-04T03:16:35.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.650+0000 7f1c7affd700 1 -- 172.21.15.1:0/3697121309 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c74021410 con 0x7f1c84104380 2024-05-04T03:16:35.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.650+0000 7f1c7affd700 1 -- 172.21.15.1:0/3697121309 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c74017650 con 0x7f1c84104380 2024-05-04T03:16:35.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.650+0000 7f1c8bc7e700 1 -- 172.21.15.1:0/3697121309 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c841a2740 con 0x7f1c84104380 2024-05-04T03:16:35.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.650+0000 7f1c8bc7e700 1 -- 172.21.15.1:0/3697121309 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c841a2b40 con 0x7f1c84104380 2024-05-04T03:16:35.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.651+0000 7f1c8bc7e700 1 -- 172.21.15.1:0/3697121309 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c840512c0 con 0x7f1c84104380 2024-05-04T03:16:35.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.653+0000 7f1c7affd700 1 -- 172.21.15.1:0/3697121309 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f1c74029070 con 0x7f1c84104380 2024-05-04T03:16:35.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.654+0000 7f1c7affd700 1 --2- 172.21.15.1:0/3697121309 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1c700b2990 0x7f1c700b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:35.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.654+0000 7f1c89a1a700 1 --2- 172.21.15.1:0/3697121309 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1c700b2990 0x7f1c700b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:35.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.654+0000 7f1c7affd700 1 -- 172.21.15.1:0/3697121309 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f1c740e35e0 con 0x7f1c84104380 2024-05-04T03:16:35.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.654+0000 7f1c89a1a700 1 --2- 172.21.15.1:0/3697121309 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1c700b2990 0x7f1c700b4ea0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1c84104280 tx=0x7f1c80009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:35.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.657+0000 7f1c7affd700 1 -- 172.21.15.1:0/3697121309 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1c74013070 con 0x7f1c84104380 2024-05-04T03:16:35.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:35 smithi077 ceph-mon[33531]: osdmap e83: 12 total, 12 up, 12 in 2024-05-04T03:16:35.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:35 smithi072 ceph-mon[33332]: osdmap e83: 12 total, 12 up, 12 in 2024-05-04T03:16:35.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.958+0000 7f1c8bc7e700 1 -- 172.21.15.1:0/3697121309 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f1c8402e760 con 0x7f1c84104380 2024-05-04T03:16:35.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.960+0000 7f1c7affd700 1 -- 172.21.15.1:0/3697121309 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f1c8402e760 con 0x7f1c84104380 2024-05-04T03:16:35.962 INFO:teuthology.orchestra.run.smithi001.stdout:[client.0] 2024-05-04T03:16:35.962 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQCTqDVmxqIwORAAWj3UK9Ch4yPQJGTP+299AA== 2024-05-04T03:16:35.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.962+0000 7f1c8bc7e700 1 -- 172.21.15.1:0/3697121309 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1c700b2990 msgr2=0x7f1c700b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:35.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.963+0000 7f1c8bc7e700 1 --2- 172.21.15.1:0/3697121309 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1c700b2990 0x7f1c700b4ea0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1c84104280 tx=0x7f1c80009260 comp rx=0 tx=0).stop 2024-05-04T03:16:35.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.963+0000 7f1c8bc7e700 1 -- 172.21.15.1:0/3697121309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c84104380 msgr2=0x7f1c8419f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:35.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.963+0000 7f1c8bc7e700 1 --2- 172.21.15.1:0/3697121309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c84104380 0x7f1c8419f710 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1c740098e0 tx=0x7f1c740057a0 comp rx=0 tx=0).stop 2024-05-04T03:16:35.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.963+0000 7f1c8bc7e700 1 -- 172.21.15.1:0/3697121309 shutdown_connections 2024-05-04T03:16:35.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.963+0000 7f1c8bc7e700 1 --2- 172.21.15.1:0/3697121309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c84103250 0x7f1c84076450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:35.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.963+0000 7f1c8bc7e700 1 --2- 172.21.15.1:0/3697121309 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1c700b2990 0x7f1c700b4ea0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:35.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.963+0000 7f1c8bc7e700 1 --2- 172.21.15.1:0/3697121309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c84104d20 0x7f1c841a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:35.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.963+0000 7f1c8bc7e700 1 --2- 172.21.15.1:0/3697121309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c84104380 0x7f1c8419f710 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:35.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.963+0000 7f1c8bc7e700 1 -- 172.21.15.1:0/3697121309 >> 172.21.15.1:0/3697121309 conn(0x7f1c840fe870 msgr2=0x7f1c841075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:35.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.963+0000 7f1c8bc7e700 1 -- 172.21.15.1:0/3697121309 shutdown_connections 2024-05-04T03:16:35.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:35.963+0000 7f1c8bc7e700 1 -- 172.21.15.1:0/3697121309 wait complete. 2024-05-04T03:16:36.607 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:36 smithi001 ceph-mon[29731]: pgmap v314: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-05-04T03:16:36.607 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:36 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3697121309' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-04T03:16:36.607 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:36 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3697121309' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-04T03:16:36.636 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-04T03:16:36.636 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-04T03:16:36.636 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-04T03:16:36.673 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-04T03:16:36.673 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-04T03:16:36.674 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph mgr dump --format=json 2024-05-04T03:16:36.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:36 smithi077 ceph-mon[33531]: pgmap v314: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-05-04T03:16:36.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:36 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/3697121309' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-04T03:16:36.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:36 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/3697121309' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-04T03:16:36.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:36 smithi072 ceph-mon[33332]: pgmap v314: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-05-04T03:16:36.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:36 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/3697121309' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-04T03:16:36.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:36 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/3697121309' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-04T03:16:36.961 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:38.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.548+0000 7f84f53e6700 1 -- 172.21.15.1:0/821092824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84f0108f20 msgr2=0x7f84f0109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:38.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.548+0000 7f84f53e6700 1 --2- 172.21.15.1:0/821092824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84f0108f20 0x7f84f0109330 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f84d80099c0 tx=0x7f84d8009cd0 comp rx=0 tx=0).stop 2024-05-04T03:16:38.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.548+0000 7f84f53e6700 1 -- 172.21.15.1:0/821092824 shutdown_connections 2024-05-04T03:16:38.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.548+0000 7f84f53e6700 1 --2- 172.21.15.1:0/821092824 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84f01045f0 0x7f84f0104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:38.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.548+0000 7f84f53e6700 1 --2- 172.21.15.1:0/821092824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84f0109870 0x7f84f01111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:38.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.548+0000 7f84f53e6700 1 --2- 172.21.15.1:0/821092824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84f0108f20 0x7f84f0109330 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:38.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.548+0000 7f84f53e6700 1 -- 172.21.15.1:0/821092824 >> 172.21.15.1:0/821092824 conn(0x7f84f0100040 msgr2=0x7f84f0102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:38.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.548+0000 7f84f53e6700 1 -- 172.21.15.1:0/821092824 shutdown_connections 2024-05-04T03:16:38.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.548+0000 7f84f53e6700 1 -- 172.21.15.1:0/821092824 wait complete. 2024-05-04T03:16:38.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.549+0000 7f84f53e6700 1 Processor -- start 2024-05-04T03:16:38.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.549+0000 7f84f53e6700 1 -- start start 2024-05-04T03:16:38.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.549+0000 7f84f53e6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84f01045f0 0x7f84f019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:38.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.549+0000 7f84f53e6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84f0108f20 0x7f84f019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:38.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.549+0000 7f84f53e6700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84f0109870 0x7f84f019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:38.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.549+0000 7f84f53e6700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f84f0114480 con 0x7f84f01045f0 2024-05-04T03:16:38.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.549+0000 7f84f53e6700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f84f0114300 con 0x7f84f0109870 2024-05-04T03:16:38.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.549+0000 7f84f53e6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f84f0114600 con 0x7f84f0108f20 2024-05-04T03:16:38.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.550+0000 7f84eeffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84f01045f0 0x7f84f019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:38.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.550+0000 7f84eeffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84f01045f0 0x7f84f019ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56234/0 (socket says 172.21.15.1:56234) 2024-05-04T03:16:38.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.550+0000 7f84eeffd700 1 -- 172.21.15.1:0/2785109727 learned_addr learned my addr 172.21.15.1:0/2785109727 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:16:38.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.550+0000 7f84e7fff700 1 --2- 172.21.15.1:0/2785109727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84f0108f20 0x7f84f019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:38.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.550+0000 7f84ef7fe700 1 --2- 172.21.15.1:0/2785109727 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84f0109870 0x7f84f019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:38.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.550+0000 7f84eeffd700 1 -- 172.21.15.1:0/2785109727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84f0108f20 msgr2=0x7f84f019b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:38.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.550+0000 7f84eeffd700 1 --2- 172.21.15.1:0/2785109727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84f0108f20 0x7f84f019b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:38.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.550+0000 7f84eeffd700 1 -- 172.21.15.1:0/2785109727 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84f0109870 msgr2=0x7f84f019f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:38.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.550+0000 7f84eeffd700 1 --2- 172.21.15.1:0/2785109727 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84f0109870 0x7f84f019f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:38.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.550+0000 7f84eeffd700 1 -- 172.21.15.1:0/2785109727 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f019fc50 con 0x7f84f01045f0 2024-05-04T03:16:38.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.550+0000 7f84eeffd700 1 --2- 172.21.15.1:0/2785109727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84f01045f0 0x7f84f019ae30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f84e000c9b0 tx=0x7f84e000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:38.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.550+0000 7f84ecff9700 1 -- 172.21.15.1:0/2785109727 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e0007b00 con 0x7f84f01045f0 2024-05-04T03:16:38.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.550+0000 7f84ecff9700 1 -- 172.21.15.1:0/2785109727 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84e000ce50 con 0x7f84f01045f0 2024-05-04T03:16:38.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.550+0000 7f84f53e6700 1 -- 172.21.15.1:0/2785109727 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f019ff40 con 0x7f84f01045f0 2024-05-04T03:16:38.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.550+0000 7f84ecff9700 1 -- 172.21.15.1:0/2785109727 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e0018660 con 0x7f84f01045f0 2024-05-04T03:16:38.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.550+0000 7f84f53e6700 1 -- 172.21.15.1:0/2785109727 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f01a0450 con 0x7f84f01045f0 2024-05-04T03:16:38.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.551+0000 7f84f53e6700 1 -- 172.21.15.1:0/2785109727 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84f00512c0 con 0x7f84f01045f0 2024-05-04T03:16:38.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.553+0000 7f84ecff9700 1 -- 172.21.15.1:0/2785109727 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f84e000f450 con 0x7f84f01045f0 2024-05-04T03:16:38.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.554+0000 7f84ecff9700 1 --2- 172.21.15.1:0/2785109727 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f84dc0b2910 0x7f84dc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:38.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.554+0000 7f84e7fff700 1 --2- 172.21.15.1:0/2785109727 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f84dc0b2910 0x7f84dc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:38.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.554+0000 7f84ecff9700 1 -- 172.21.15.1:0/2785109727 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f84e00e2480 con 0x7f84f01045f0 2024-05-04T03:16:38.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.554+0000 7f84e7fff700 1 --2- 172.21.15.1:0/2785109727 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f84dc0b2910 0x7f84dc0b4e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f84d8006010 tx=0x7f84d8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:38.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.557+0000 7f84ecff9700 1 -- 172.21.15.1:0/2785109727 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f84e0014030 con 0x7f84f01045f0 2024-05-04T03:16:38.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:38 smithi001 ceph-mon[29731]: pgmap v315: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-04T03:16:38.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:38 smithi077 ceph-mon[33531]: pgmap v315: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-04T03:16:38.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:38 smithi072 ceph-mon[33332]: pgmap v315: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-04T03:16:38.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.834+0000 7f84f53e6700 1 -- 172.21.15.1:0/2785109727 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f84f002e5f0 con 0x7f84f01045f0 2024-05-04T03:16:38.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.840+0000 7f84ecff9700 1 -- 172.21.15.1:0/2785109727 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288632 (secure 0 0 0) 0x7f84e00aa1f0 con 0x7f84f01045f0 2024-05-04T03:16:38.846 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:16:38.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.843+0000 7f84f53e6700 1 -- 172.21.15.1:0/2785109727 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f84dc0b2910 msgr2=0x7f84dc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:38.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.843+0000 7f84f53e6700 1 --2- 172.21.15.1:0/2785109727 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f84dc0b2910 0x7f84dc0b4e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f84d8006010 tx=0x7f84d8000bc0 comp rx=0 tx=0).stop 2024-05-04T03:16:38.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.843+0000 7f84f53e6700 1 -- 172.21.15.1:0/2785109727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84f01045f0 msgr2=0x7f84f019ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:38.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.843+0000 7f84f53e6700 1 --2- 172.21.15.1:0/2785109727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84f01045f0 0x7f84f019ae30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f84e000c9b0 tx=0x7f84e000ccc0 comp rx=0 tx=0).stop 2024-05-04T03:16:38.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.844+0000 7f84f53e6700 1 -- 172.21.15.1:0/2785109727 shutdown_connections 2024-05-04T03:16:38.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.844+0000 7f84f53e6700 1 --2- 172.21.15.1:0/2785109727 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84f0109870 0x7f84f019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:38.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.844+0000 7f84f53e6700 1 --2- 172.21.15.1:0/2785109727 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f84dc0b2910 0x7f84dc0b4e20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:38.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.844+0000 7f84f53e6700 1 --2- 172.21.15.1:0/2785109727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84f01045f0 0x7f84f019ae30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:38.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.844+0000 7f84f53e6700 1 --2- 172.21.15.1:0/2785109727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84f0108f20 0x7f84f019b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:38.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.844+0000 7f84f53e6700 1 -- 172.21.15.1:0/2785109727 >> 172.21.15.1:0/2785109727 conn(0x7f84f0100040 msgr2=0x7f84f010a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:38.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.844+0000 7f84f53e6700 1 -- 172.21.15.1:0/2785109727 shutdown_connections 2024-05-04T03:16:38.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:38.844+0000 7f84f53e6700 1 -- 172.21.15.1:0/2785109727 wait complete. 2024-05-04T03:16:39.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:39 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2785109727' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-04T03:16:39.500 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":402647351},{"type":"v1","addr":"172.21.15.1:6801","nonce":402647351}]},"active_addr":"172.21.15.1:6801/402647351","active_change":"2024-05-04T03:08:33.046021+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24112,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.1:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":2426286568}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":82390108}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":1105801409}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":629779919}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":2876690767}]}]} 2024-05-04T03:16:39.511 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-04T03:16:39.511 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-04T03:16:39.511 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-04T03:16:39.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:39 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/2785109727' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-04T03:16:39.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:39 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/2785109727' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-04T03:16:39.762 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:40.568 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:40 smithi001 ceph-mon[29731]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.5 KiB/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-04T03:16:40.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:40 smithi077 ceph-mon[33531]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.5 KiB/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-04T03:16:40.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:40 smithi072 ceph-mon[33332]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.5 KiB/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-04T03:16:41.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.227+0000 7faf148c5700 1 -- 172.21.15.1:0/1503187017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf101034c0 msgr2=0x7faf101038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:41.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.227+0000 7faf148c5700 1 --2- 172.21.15.1:0/1503187017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf101034c0 0x7faf101038d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7faf000099c0 tx=0x7faf00009cd0 comp rx=0 tx=0).stop 2024-05-04T03:16:41.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.227+0000 7faf148c5700 1 -- 172.21.15.1:0/1503187017 shutdown_connections 2024-05-04T03:16:41.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.227+0000 7faf148c5700 1 --2- 172.21.15.1:0/1503187017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf10108740 0x7faf101100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:41.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.227+0000 7faf148c5700 1 --2- 172.21.15.1:0/1503187017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf101034c0 0x7faf101038d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:41.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.227+0000 7faf148c5700 1 --2- 172.21.15.1:0/1503187017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf10107df0 0x7faf10108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:41.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.227+0000 7faf148c5700 1 -- 172.21.15.1:0/1503187017 >> 172.21.15.1:0/1503187017 conn(0x7faf100fedc0 msgr2=0x7faf101011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:41.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.227+0000 7faf148c5700 1 -- 172.21.15.1:0/1503187017 shutdown_connections 2024-05-04T03:16:41.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.227+0000 7faf148c5700 1 -- 172.21.15.1:0/1503187017 wait complete. 2024-05-04T03:16:41.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.228+0000 7faf148c5700 1 Processor -- start 2024-05-04T03:16:41.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.228+0000 7faf148c5700 1 -- start start 2024-05-04T03:16:41.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.229+0000 7faf148c5700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf101034c0 0x7faf10199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:41.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.229+0000 7faf148c5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf10107df0 0x7faf1019a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:41.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.229+0000 7faf148c5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf10108740 0x7faf1019e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:41.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.229+0000 7faf148c5700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7faf101133f0 con 0x7faf101034c0 2024-05-04T03:16:41.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.229+0000 7faf148c5700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7faf10113270 con 0x7faf10108740 2024-05-04T03:16:41.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.229+0000 7faf148c5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faf10113570 con 0x7faf10107df0 2024-05-04T03:16:41.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.229+0000 7faf0e59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf101034c0 0x7faf10199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:41.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.229+0000 7faf0e59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf101034c0 0x7faf10199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56246/0 (socket says 172.21.15.1:56246) 2024-05-04T03:16:41.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.229+0000 7faf0e59c700 1 -- 172.21.15.1:0/2121751097 learned_addr learned my addr 172.21.15.1:0/2121751097 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:16:41.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.229+0000 7faf0ed9d700 1 --2- 172.21.15.1:0/2121751097 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf10108740 0x7faf1019e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:41.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.229+0000 7faf0dd9b700 1 --2- 172.21.15.1:0/2121751097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf10107df0 0x7faf1019a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:41.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.229+0000 7faf0e59c700 1 -- 172.21.15.1:0/2121751097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf10107df0 msgr2=0x7faf1019a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:41.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.229+0000 7faf0e59c700 1 --2- 172.21.15.1:0/2121751097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf10107df0 0x7faf1019a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:41.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.229+0000 7faf0e59c700 1 -- 172.21.15.1:0/2121751097 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf10108740 msgr2=0x7faf1019e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:41.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.229+0000 7faf0e59c700 1 --2- 172.21.15.1:0/2121751097 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf10108740 0x7faf1019e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:41.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.229+0000 7faf0e59c700 1 -- 172.21.15.1:0/2121751097 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1019eb40 con 0x7faf101034c0 2024-05-04T03:16:41.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.229+0000 7faf0e59c700 1 --2- 172.21.15.1:0/2121751097 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf101034c0 0x7faf10199d20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7faf000098e0 tx=0x7faf000057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:41.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.230+0000 7faeff7fe700 1 -- 172.21.15.1:0/2121751097 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf00027070 con 0x7faf101034c0 2024-05-04T03:16:41.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.230+0000 7faeff7fe700 1 -- 172.21.15.1:0/2121751097 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf00004020 con 0x7faf101034c0 2024-05-04T03:16:41.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.230+0000 7faf148c5700 1 -- 172.21.15.1:0/2121751097 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf1019edd0 con 0x7faf101034c0 2024-05-04T03:16:41.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.230+0000 7faeff7fe700 1 -- 172.21.15.1:0/2121751097 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf00022e10 con 0x7faf101034c0 2024-05-04T03:16:41.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.230+0000 7faf148c5700 1 -- 172.21.15.1:0/2121751097 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf1019f2b0 con 0x7faf101034c0 2024-05-04T03:16:41.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.231+0000 7faf148c5700 1 -- 172.21.15.1:0/2121751097 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf100512c0 con 0x7faf101034c0 2024-05-04T03:16:41.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.232+0000 7faeff7fe700 1 -- 172.21.15.1:0/2121751097 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7faf0002a070 con 0x7faf101034c0 2024-05-04T03:16:41.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.233+0000 7faeff7fe700 1 --2- 172.21.15.1:0/2121751097 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7faef80b2910 0x7faef80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:41.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.233+0000 7faf0dd9b700 1 --2- 172.21.15.1:0/2121751097 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7faef80b2910 0x7faef80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:41.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.233+0000 7faeff7fe700 1 -- 172.21.15.1:0/2121751097 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7faf00005030 con 0x7faf101034c0 2024-05-04T03:16:41.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.233+0000 7faf0dd9b700 1 --2- 172.21.15.1:0/2121751097 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7faef80b2910 0x7faef80b4e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7faf04005ea0 tx=0x7faf04005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:41.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.237+0000 7faeff7fe700 1 -- 172.21.15.1:0/2121751097 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faf00013070 con 0x7faf101034c0 2024-05-04T03:16:41.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.455+0000 7faf148c5700 1 -- 172.21.15.1:0/2121751097 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faf1002e5f0 con 0x7faf101034c0 2024-05-04T03:16:41.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.457+0000 7faeff7fe700 1 -- 172.21.15.1:0/2121751097 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22380 (secure 0 0 0) 0x7faf000ab1e0 con 0x7faf101034c0 2024-05-04T03:16:41.460 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:16:41.460 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":83,"fsid":"4a572aa8-09c3-11ef-bc97-c7b262605968","created":"2024-05-04T03:07:32.845547+0000","modified":"2024-05-04T03:16:34.321580+0000","last_up_change":"2024-05-04T03:14:44.192571+0000","last_in_change":"2024-05-04T03:14:23.353342+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T03:11:01.139189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T03:15:43.151696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T03:15:46.753539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"100fe035-2454-4c95-90c7-759956ec36a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6803","nonce":3479011484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6805","nonce":3479011484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6809","nonce":3479011484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6807","nonce":3479011484}]},"public_addr":"172.21.15.1:6803/3479011484","cluster_addr":"172.21.15.1:6805/3479011484","heartbeat_back_addr":"172.21.15.1:6809/3479011484","heartbeat_front_addr":"172.21.15.1:6807/3479011484","state":["exists","up"]},{"osd":1,"uuid":"5826ac76-ef6b-4040-8ad1-305db355af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6801","nonce":3725706149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6803","nonce":3725706149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6807","nonce":3725706149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6805","nonce":3725706149}]},"public_addr":"172.21.15.72:6801/3725706149","cluster_addr":"172.21.15.72:6803/3725706149","heartbeat_back_addr":"172.21.15.72:6807/3725706149","heartbeat_front_addr":"172.21.15.72:6805/3725706149","state":["exists","up"]},{"osd":2,"uuid":"923d76e5-b032-4190-9e35-546d27722a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6801","nonce":515038343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6803","nonce":515038343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6807","nonce":515038343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6805","nonce":515038343}]},"public_addr":"172.21.15.77:6801/515038343","cluster_addr":"172.21.15.77:6803/515038343","heartbeat_back_addr":"172.21.15.77:6807/515038343","heartbeat_front_addr":"172.21.15.77:6805/515038343","state":["exists","up"]},{"osd":3,"uuid":"495cc0c5-6a99-481f-ad9e-38f05fb84356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6811","nonce":4050439365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6813","nonce":4050439365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6817","nonce":4050439365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6815","nonce":4050439365}]},"public_addr":"172.21.15.1:6811/4050439365","cluster_addr":"172.21.15.1:6813/4050439365","heartbeat_back_addr":"172.21.15.1:6817/4050439365","heartbeat_front_addr":"172.21.15.1:6815/4050439365","state":["exists","up"]},{"osd":4,"uuid":"b86b8026-79c0-4056-9e67-a61f54b6ab5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6809","nonce":2501732391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6811","nonce":2501732391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6815","nonce":2501732391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6813","nonce":2501732391}]},"public_addr":"172.21.15.72:6809/2501732391","cluster_addr":"172.21.15.72:6811/2501732391","heartbeat_back_addr":"172.21.15.72:6815/2501732391","heartbeat_front_addr":"172.21.15.72:6813/2501732391","state":["exists","up"]},{"osd":5,"uuid":"80975cb9-39e0-46b8-a343-9ff69194a14e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6809","nonce":3582852165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6811","nonce":3582852165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6815","nonce":3582852165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6813","nonce":3582852165}]},"public_addr":"172.21.15.77:6809/3582852165","cluster_addr":"172.21.15.77:6811/3582852165","heartbeat_back_addr":"172.21.15.77:6815/3582852165","heartbeat_front_addr":"172.21.15.77:6813/3582852165","state":["exists","up"]},{"osd":6,"uuid":"aca3fac5-fbb5-42de-bf0e-e4a0db7381e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6819","nonce":1799033649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6821","nonce":1799033649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6825","nonce":1799033649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6823","nonce":1799033649}]},"public_addr":"172.21.15.1:6819/1799033649","cluster_addr":"172.21.15.1:6821/1799033649","heartbeat_back_addr":"172.21.15.1:6825/1799033649","heartbeat_front_addr":"172.21.15.1:6823/1799033649","state":["exists","up"]},{"osd":7,"uuid":"cbff6b8a-c463-4249-ae45-fdc6f99a5489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6817","nonce":3460835447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6819","nonce":3460835447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6823","nonce":3460835447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6821","nonce":3460835447}]},"public_addr":"172.21.15.72:6817/3460835447","cluster_addr":"172.21.15.72:6819/3460835447","heartbeat_back_addr":"172.21.15.72:6823/3460835447","heartbeat_front_addr":"172.21.15.72:6821/3460835447","state":["exists","up"]},{"osd":8,"uuid":"e140bf13-4887-4d68-9373-3d4fc1c4584a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6817","nonce":1124479323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6819","nonce":1124479323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6823","nonce":1124479323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6821","nonce":1124479323}]},"public_addr":"172.21.15.77:6817/1124479323","cluster_addr":"172.21.15.77:6819/1124479323","heartbeat_back_addr":"172.21.15.77:6823/1124479323","heartbeat_front_addr":"172.21.15.77:6821/1124479323","state":["exists","up"]},{"osd":9,"uuid":"f1e8a232-cb11-4b9d-8789-e1637fa5aaa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6827","nonce":2681322272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6829","nonce":2681322272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6833","nonce":2681322272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6831","nonce":2681322272}]},"public_addr":"172.21.15.1:6827/2681322272","cluster_addr":"172.21.15.1:6829/2681322272","heartbeat_back_addr":"172.21.15.1:6833/2681322272","heartbeat_front_addr":"172.21.15.1:6831/2681322272","state":["exists","up"]},{"osd":10,"uuid":"79b570af-3d1e-439d-a328-9e6bc7d67839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6825","nonce":3293703569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6827","nonce":3293703569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6831","nonce":3293703569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6829","nonce":3293703569}]},"public_addr":"172.21.15.72:6825/3293703569","cluster_addr":"172.21.15.72:6827/3293703569","heartbeat_back_addr":"172.21.15.72:6831/3293703569","heartbeat_front_addr":"172.21.15.72:6829/3293703569","state":["exists","up"]},{"osd":11,"uuid":"e35bf83d-9ee4-4745-96c2-1c35cc8eefba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6825","nonce":4155947548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6827","nonce":4155947548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6831","nonce":4155947548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6829","nonce":4155947548}]},"public_addr":"172.21.15.77:6825/4155947548","cluster_addr":"172.21.15.77:6827/4155947548","heartbeat_back_addr":"172.21.15.77:6831/4155947548","heartbeat_front_addr":"172.21.15.77:6829/4155947548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:09.467126+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:33.861139+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:57.741962+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:11:21.746021+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:11:46.323252+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:12:10.640180+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:12:35.750591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:00.306549+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:25.012495+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:51.185229+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:14:16.403850+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:14:41.653316+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/2029897106":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/402488450":"2024-05-05T03:08:33.045884+0000","172.21.15.1:6801/128793132":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/2233526606":"2024-05-05T03:08:06.135783+0000","172.21.15.1:0/2715541855":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/2415697325":"2024-05-05T03:08:06.135783+0000","172.21.15.1:0/2305298106":"2024-05-05T03:08:06.135783+0000","172.21.15.1:6800/128793132":"2024-05-05T03:08:33.045884+0000","172.21.15.1:6800/4200529863":"2024-05-05T03:08:06.135783+0000","172.21.15.1:6801/4200529863":"2024-05-05T03:08:06.135783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T03:16:41.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.459+0000 7faf148c5700 1 -- 172.21.15.1:0/2121751097 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7faef80b2910 msgr2=0x7faef80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:41.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.459+0000 7faf148c5700 1 --2- 172.21.15.1:0/2121751097 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7faef80b2910 0x7faef80b4e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7faf04005ea0 tx=0x7faf04005e30 comp rx=0 tx=0).stop 2024-05-04T03:16:41.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.459+0000 7faf148c5700 1 -- 172.21.15.1:0/2121751097 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf101034c0 msgr2=0x7faf10199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:41.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.459+0000 7faf148c5700 1 --2- 172.21.15.1:0/2121751097 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf101034c0 0x7faf10199d20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7faf000098e0 tx=0x7faf000057c0 comp rx=0 tx=0).stop 2024-05-04T03:16:41.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.459+0000 7faf148c5700 1 -- 172.21.15.1:0/2121751097 shutdown_connections 2024-05-04T03:16:41.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.459+0000 7faf148c5700 1 --2- 172.21.15.1:0/2121751097 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf10108740 0x7faf1019e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:41.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.459+0000 7faf148c5700 1 --2- 172.21.15.1:0/2121751097 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7faef80b2910 0x7faef80b4e20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:41.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.459+0000 7faf148c5700 1 --2- 172.21.15.1:0/2121751097 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf101034c0 0x7faf10199d20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:41.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.459+0000 7faf148c5700 1 --2- 172.21.15.1:0/2121751097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf10107df0 0x7faf1019a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:41.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.459+0000 7faf148c5700 1 -- 172.21.15.1:0/2121751097 >> 172.21.15.1:0/2121751097 conn(0x7faf100fedc0 msgr2=0x7faf101096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:41.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.459+0000 7faf148c5700 1 -- 172.21.15.1:0/2121751097 shutdown_connections 2024-05-04T03:16:41.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:41.460+0000 7faf148c5700 1 -- 172.21.15.1:0/2121751097 wait complete. 2024-05-04T03:16:42.028 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-04T03:16:42.028 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-04T03:16:42.276 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:42.610 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:42 smithi001 ceph-mon[29731]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-04T03:16:42.610 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:42 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2121751097' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:16:42.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:42 smithi077 ceph-mon[33531]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-04T03:16:42.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:42 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/2121751097' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:16:42.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:42 smithi072 ceph-mon[33332]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-04T03:16:42.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:42 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/2121751097' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:16:43.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:43 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:16:43.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:43 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:16:43.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:43 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:16:43.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.898+0000 7f51c9ed6700 1 -- 172.21.15.1:0/645524791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51c41034c0 msgr2=0x7f51c41038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:43.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.898+0000 7f51c9ed6700 1 --2- 172.21.15.1:0/645524791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51c41034c0 0x7f51c41038d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f51b400b0d0 tx=0x7f51b400b3e0 comp rx=0 tx=0).stop 2024-05-04T03:16:43.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.898+0000 7f51c9ed6700 1 -- 172.21.15.1:0/645524791 shutdown_connections 2024-05-04T03:16:43.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.898+0000 7f51c9ed6700 1 --2- 172.21.15.1:0/645524791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51c4108740 0x7f51c41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:43.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.898+0000 7f51c9ed6700 1 --2- 172.21.15.1:0/645524791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51c41034c0 0x7f51c41038d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:43.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.898+0000 7f51c9ed6700 1 --2- 172.21.15.1:0/645524791 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51c4107df0 0x7f51c4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:43.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.898+0000 7f51c9ed6700 1 -- 172.21.15.1:0/645524791 >> 172.21.15.1:0/645524791 conn(0x7f51c40fedc0 msgr2=0x7f51c41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:43.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.898+0000 7f51c9ed6700 1 -- 172.21.15.1:0/645524791 shutdown_connections 2024-05-04T03:16:43.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.898+0000 7f51c9ed6700 1 -- 172.21.15.1:0/645524791 wait complete. 2024-05-04T03:16:43.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.899+0000 7f51c9ed6700 1 Processor -- start 2024-05-04T03:16:43.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.899+0000 7f51c9ed6700 1 -- start start 2024-05-04T03:16:43.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.900+0000 7f51c9ed6700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51c41034c0 0x7f51c419f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:43.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.900+0000 7f51c9ed6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51c4107df0 0x7f51c419ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:43.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.900+0000 7f51c9ed6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51c4108740 0x7f51c4199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:43.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.900+0000 7f51c9ed6700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f51c4113370 con 0x7f51c4107df0 2024-05-04T03:16:43.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.900+0000 7f51c9ed6700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f51c41131f0 con 0x7f51c41034c0 2024-05-04T03:16:43.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.900+0000 7f51c9ed6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f51c41134f0 con 0x7f51c4108740 2024-05-04T03:16:43.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.900+0000 7f51c37fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51c41034c0 0x7f51c419f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:43.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.900+0000 7f51c2ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51c4107df0 0x7f51c419ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:43.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.900+0000 7f51c37fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51c41034c0 0x7f51c419f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:52734/0 (socket says 172.21.15.1:52734) 2024-05-04T03:16:43.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.900+0000 7f51c37fe700 1 -- 172.21.15.1:0/2431774554 learned_addr learned my addr 172.21.15.1:0/2431774554 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:16:43.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.900+0000 7f51c3fff700 1 --2- 172.21.15.1:0/2431774554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51c4108740 0x7f51c4199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:43.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.900+0000 7f51c37fe700 1 -- 172.21.15.1:0/2431774554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51c4108740 msgr2=0x7f51c4199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:43.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.900+0000 7f51c37fe700 1 --2- 172.21.15.1:0/2431774554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51c4108740 0x7f51c4199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:43.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.900+0000 7f51c37fe700 1 -- 172.21.15.1:0/2431774554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51c4107df0 msgr2=0x7f51c419ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:43.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.900+0000 7f51c37fe700 1 --2- 172.21.15.1:0/2431774554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51c4107df0 0x7f51c419ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:43.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.900+0000 7f51c37fe700 1 -- 172.21.15.1:0/2431774554 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c419a280 con 0x7f51c41034c0 2024-05-04T03:16:43.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.900+0000 7f51c2ffd700 1 --2- 172.21.15.1:0/2431774554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51c4107df0 0x7f51c419ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:16:43.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.901+0000 7f51c37fe700 1 --2- 172.21.15.1:0/2431774554 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51c41034c0 0x7f51c419f9c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f51b4007a20 tx=0x7f51b4007a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:43.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.901+0000 7f51c0ff9700 1 -- 172.21.15.1:0/2431774554 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b401f070 con 0x7f51c41034c0 2024-05-04T03:16:43.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.901+0000 7f51c0ff9700 1 -- 172.21.15.1:0/2431774554 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51b4004c50 con 0x7f51c41034c0 2024-05-04T03:16:43.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.901+0000 7f51c0ff9700 1 -- 172.21.15.1:0/2431774554 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b401b690 con 0x7f51c41034c0 2024-05-04T03:16:43.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.901+0000 7f51c9ed6700 1 -- 172.21.15.1:0/2431774554 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51c419a510 con 0x7f51c41034c0 2024-05-04T03:16:43.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.901+0000 7f51c9ed6700 1 -- 172.21.15.1:0/2431774554 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51c419a960 con 0x7f51c41034c0 2024-05-04T03:16:43.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.902+0000 7f51c9ed6700 1 -- 172.21.15.1:0/2431774554 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51c4104970 con 0x7f51c41034c0 2024-05-04T03:16:43.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.904+0000 7f51c0ff9700 1 -- 172.21.15.1:0/2431774554 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f51b4022070 con 0x7f51c41034c0 2024-05-04T03:16:43.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.905+0000 7f51c0ff9700 1 --2- 172.21.15.1:0/2431774554 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f51ac0b2990 0x7f51ac0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:43.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.906+0000 7f51c2ffd700 1 --2- 172.21.15.1:0/2431774554 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f51ac0b2990 0x7f51ac0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:43.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.906+0000 7f51c0ff9700 1 -- 172.21.15.1:0/2431774554 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f51b400f020 con 0x7f51c41034c0 2024-05-04T03:16:43.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.906+0000 7f51c2ffd700 1 --2- 172.21.15.1:0/2431774554 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f51ac0b2990 0x7f51ac0b4ea0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f51c419b280 tx=0x7f51b8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:43.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:43.910+0000 7f51c0ff9700 1 -- 172.21.15.1:0/2431774554 <== mon.2 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f51b4029020 con 0x7f51c41034c0 2024-05-04T03:16:44.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:44.143+0000 7f51c9ed6700 1 -- 172.21.15.1:0/2431774554 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51c410b210 con 0x7f51c41034c0 2024-05-04T03:16:44.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:44.145+0000 7f51c0ff9700 1 -- 172.21.15.1:0/2431774554 <== mon.2 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22380 (secure 0 0 0) 0x7f51b40ab040 con 0x7f51c41034c0 2024-05-04T03:16:44.147 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:16:44.147 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":83,"fsid":"4a572aa8-09c3-11ef-bc97-c7b262605968","created":"2024-05-04T03:07:32.845547+0000","modified":"2024-05-04T03:16:34.321580+0000","last_up_change":"2024-05-04T03:14:44.192571+0000","last_in_change":"2024-05-04T03:14:23.353342+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T03:11:01.139189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T03:15:43.151696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T03:15:46.753539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"100fe035-2454-4c95-90c7-759956ec36a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6803","nonce":3479011484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6805","nonce":3479011484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6809","nonce":3479011484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6807","nonce":3479011484}]},"public_addr":"172.21.15.1:6803/3479011484","cluster_addr":"172.21.15.1:6805/3479011484","heartbeat_back_addr":"172.21.15.1:6809/3479011484","heartbeat_front_addr":"172.21.15.1:6807/3479011484","state":["exists","up"]},{"osd":1,"uuid":"5826ac76-ef6b-4040-8ad1-305db355af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6801","nonce":3725706149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6803","nonce":3725706149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6807","nonce":3725706149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6805","nonce":3725706149}]},"public_addr":"172.21.15.72:6801/3725706149","cluster_addr":"172.21.15.72:6803/3725706149","heartbeat_back_addr":"172.21.15.72:6807/3725706149","heartbeat_front_addr":"172.21.15.72:6805/3725706149","state":["exists","up"]},{"osd":2,"uuid":"923d76e5-b032-4190-9e35-546d27722a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6801","nonce":515038343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6803","nonce":515038343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6807","nonce":515038343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6805","nonce":515038343}]},"public_addr":"172.21.15.77:6801/515038343","cluster_addr":"172.21.15.77:6803/515038343","heartbeat_back_addr":"172.21.15.77:6807/515038343","heartbeat_front_addr":"172.21.15.77:6805/515038343","state":["exists","up"]},{"osd":3,"uuid":"495cc0c5-6a99-481f-ad9e-38f05fb84356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6811","nonce":4050439365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6813","nonce":4050439365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6817","nonce":4050439365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6815","nonce":4050439365}]},"public_addr":"172.21.15.1:6811/4050439365","cluster_addr":"172.21.15.1:6813/4050439365","heartbeat_back_addr":"172.21.15.1:6817/4050439365","heartbeat_front_addr":"172.21.15.1:6815/4050439365","state":["exists","up"]},{"osd":4,"uuid":"b86b8026-79c0-4056-9e67-a61f54b6ab5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6809","nonce":2501732391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6811","nonce":2501732391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6815","nonce":2501732391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6813","nonce":2501732391}]},"public_addr":"172.21.15.72:6809/2501732391","cluster_addr":"172.21.15.72:6811/2501732391","heartbeat_back_addr":"172.21.15.72:6815/2501732391","heartbeat_front_addr":"172.21.15.72:6813/2501732391","state":["exists","up"]},{"osd":5,"uuid":"80975cb9-39e0-46b8-a343-9ff69194a14e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6809","nonce":3582852165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6811","nonce":3582852165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6815","nonce":3582852165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6813","nonce":3582852165}]},"public_addr":"172.21.15.77:6809/3582852165","cluster_addr":"172.21.15.77:6811/3582852165","heartbeat_back_addr":"172.21.15.77:6815/3582852165","heartbeat_front_addr":"172.21.15.77:6813/3582852165","state":["exists","up"]},{"osd":6,"uuid":"aca3fac5-fbb5-42de-bf0e-e4a0db7381e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6819","nonce":1799033649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6821","nonce":1799033649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6825","nonce":1799033649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6823","nonce":1799033649}]},"public_addr":"172.21.15.1:6819/1799033649","cluster_addr":"172.21.15.1:6821/1799033649","heartbeat_back_addr":"172.21.15.1:6825/1799033649","heartbeat_front_addr":"172.21.15.1:6823/1799033649","state":["exists","up"]},{"osd":7,"uuid":"cbff6b8a-c463-4249-ae45-fdc6f99a5489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6817","nonce":3460835447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6819","nonce":3460835447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6823","nonce":3460835447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6821","nonce":3460835447}]},"public_addr":"172.21.15.72:6817/3460835447","cluster_addr":"172.21.15.72:6819/3460835447","heartbeat_back_addr":"172.21.15.72:6823/3460835447","heartbeat_front_addr":"172.21.15.72:6821/3460835447","state":["exists","up"]},{"osd":8,"uuid":"e140bf13-4887-4d68-9373-3d4fc1c4584a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6817","nonce":1124479323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6819","nonce":1124479323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6823","nonce":1124479323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6821","nonce":1124479323}]},"public_addr":"172.21.15.77:6817/1124479323","cluster_addr":"172.21.15.77:6819/1124479323","heartbeat_back_addr":"172.21.15.77:6823/1124479323","heartbeat_front_addr":"172.21.15.77:6821/1124479323","state":["exists","up"]},{"osd":9,"uuid":"f1e8a232-cb11-4b9d-8789-e1637fa5aaa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6827","nonce":2681322272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6829","nonce":2681322272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6833","nonce":2681322272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6831","nonce":2681322272}]},"public_addr":"172.21.15.1:6827/2681322272","cluster_addr":"172.21.15.1:6829/2681322272","heartbeat_back_addr":"172.21.15.1:6833/2681322272","heartbeat_front_addr":"172.21.15.1:6831/2681322272","state":["exists","up"]},{"osd":10,"uuid":"79b570af-3d1e-439d-a328-9e6bc7d67839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6825","nonce":3293703569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6827","nonce":3293703569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6831","nonce":3293703569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6829","nonce":3293703569}]},"public_addr":"172.21.15.72:6825/3293703569","cluster_addr":"172.21.15.72:6827/3293703569","heartbeat_back_addr":"172.21.15.72:6831/3293703569","heartbeat_front_addr":"172.21.15.72:6829/3293703569","state":["exists","up"]},{"osd":11,"uuid":"e35bf83d-9ee4-4745-96c2-1c35cc8eefba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6825","nonce":4155947548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6827","nonce":4155947548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6831","nonce":4155947548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6829","nonce":4155947548}]},"public_addr":"172.21.15.77:6825/4155947548","cluster_addr":"172.21.15.77:6827/4155947548","heartbeat_back_addr":"172.21.15.77:6831/4155947548","heartbeat_front_addr":"172.21.15.77:6829/4155947548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:09.467126+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:33.861139+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:57.741962+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:11:21.746021+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:11:46.323252+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:12:10.640180+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:12:35.750591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:00.306549+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:25.012495+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:51.185229+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:14:16.403850+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:14:41.653316+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/2029897106":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/402488450":"2024-05-05T03:08:33.045884+0000","172.21.15.1:6801/128793132":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/2233526606":"2024-05-05T03:08:06.135783+0000","172.21.15.1:0/2715541855":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/2415697325":"2024-05-05T03:08:06.135783+0000","172.21.15.1:0/2305298106":"2024-05-05T03:08:06.135783+0000","172.21.15.1:6800/128793132":"2024-05-05T03:08:33.045884+0000","172.21.15.1:6800/4200529863":"2024-05-05T03:08:06.135783+0000","172.21.15.1:6801/4200529863":"2024-05-05T03:08:06.135783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T03:16:44.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:44.147+0000 7f51c9ed6700 1 -- 172.21.15.1:0/2431774554 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f51ac0b2990 msgr2=0x7f51ac0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:44.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:44.147+0000 7f51c9ed6700 1 --2- 172.21.15.1:0/2431774554 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f51ac0b2990 0x7f51ac0b4ea0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f51c419b280 tx=0x7f51b8009260 comp rx=0 tx=0).stop 2024-05-04T03:16:44.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:44.147+0000 7f51c9ed6700 1 -- 172.21.15.1:0/2431774554 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51c41034c0 msgr2=0x7f51c419f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:44.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:44.147+0000 7f51c9ed6700 1 --2- 172.21.15.1:0/2431774554 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51c41034c0 0x7f51c419f9c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f51b4007a20 tx=0x7f51b4007a50 comp rx=0 tx=0).stop 2024-05-04T03:16:44.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:44.147+0000 7f51c9ed6700 1 -- 172.21.15.1:0/2431774554 shutdown_connections 2024-05-04T03:16:44.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:44.147+0000 7f51c9ed6700 1 --2- 172.21.15.1:0/2431774554 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f51c41034c0 0x7f51c419f9c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:44.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:44.147+0000 7f51c9ed6700 1 --2- 172.21.15.1:0/2431774554 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f51ac0b2990 0x7f51ac0b4ea0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:44.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:44.147+0000 7f51c9ed6700 1 --2- 172.21.15.1:0/2431774554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51c4108740 0x7f51c4199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:44.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:44.147+0000 7f51c9ed6700 1 --2- 172.21.15.1:0/2431774554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51c4107df0 0x7f51c419ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:44.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:44.147+0000 7f51c9ed6700 1 -- 172.21.15.1:0/2431774554 >> 172.21.15.1:0/2431774554 conn(0x7f51c40fedc0 msgr2=0x7f51c41096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:44.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:44.147+0000 7f51c9ed6700 1 -- 172.21.15.1:0/2431774554 shutdown_connections 2024-05-04T03:16:44.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:44.147+0000 7f51c9ed6700 1 -- 172.21.15.1:0/2431774554 wait complete. 2024-05-04T03:16:44.589 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:44 smithi001 ceph-mon[29731]: pgmap v318: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 416 B/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-04T03:16:44.589 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:44 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2431774554' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:16:44.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:44 smithi077 ceph-mon[33531]: pgmap v318: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 416 B/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-04T03:16:44.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:44 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/2431774554' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:16:44.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:44 smithi072 ceph-mon[33332]: pgmap v318: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 416 B/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-04T03:16:44.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:44 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/2431774554' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:16:45.085 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-04T03:16:45.085 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-04T03:16:45.085 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-04T03:16:45.086 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-04T03:16:45.086 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-04T03:16:45.087 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-04T03:16:45.087 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-04T03:16:45.087 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-04T03:16:45.088 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-04T03:16:45.088 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-04T03:16:45.089 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-04T03:16:45.089 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-04T03:16:45.775 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:45.784 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:46.112 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:46.113 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:46.113 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:46.115 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:46.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:45 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:16:46.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:45 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:16:46.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:45 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:16:46.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:45 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:16:46.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:45 smithi001 ceph-mon[29731]: pgmap v319: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 378 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-04T03:16:46.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:45 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:16:46.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:45 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:16:46.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:45 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:16:46.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:45 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:16:46.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:45 smithi077 ceph-mon[33531]: pgmap v319: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 378 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-04T03:16:46.246 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:45 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:16:46.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:45 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:16:46.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:45 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:16:46.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:45 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:16:46.247 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:45 smithi072 ceph-mon[33332]: pgmap v319: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 378 B/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-04T03:16:46.548 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:46.548 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:46.548 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:46.549 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:46.549 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:47.024 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:16:48.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:48 smithi072 ceph-mon[33332]: pgmap v320: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-04T03:16:48.503 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:48 smithi001 ceph-mon[29731]: pgmap v320: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-04T03:16:48.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:48 smithi077 ceph-mon[33531]: pgmap v320: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-04T03:16:50.355 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:50 smithi001 ceph-mon[29731]: pgmap v321: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 341 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-04T03:16:50.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:50 smithi072 ceph-mon[33332]: pgmap v321: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 341 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-04T03:16:50.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:50 smithi077 ceph-mon[33531]: pgmap v321: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 341 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-04T03:16:52.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:52 smithi072 ceph-mon[33332]: pgmap v322: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-04T03:16:52.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:52 smithi001 ceph-mon[29731]: pgmap v322: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-04T03:16:52.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:52 smithi077 ceph-mon[33531]: pgmap v322: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-04T03:16:52.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.900+0000 7ffabe1d6700 1 -- 172.21.15.1:0/2755681952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffab8109870 msgr2=0x7ffab81111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:52.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.900+0000 7ffabe1d6700 1 --2- 172.21.15.1:0/2755681952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffab8109870 0x7ffab81111d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ffab000b0d0 tx=0x7ffab000b3e0 comp rx=0 tx=0).stop 2024-05-04T03:16:52.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.900+0000 7ffabe1d6700 1 -- 172.21.15.1:0/2755681952 shutdown_connections 2024-05-04T03:16:52.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.900+0000 7ffabe1d6700 1 --2- 172.21.15.1:0/2755681952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffab8109870 0x7ffab81111d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:52.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.900+0000 7ffabe1d6700 1 --2- 172.21.15.1:0/2755681952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffab81045f0 0x7ffab8104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:52.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.900+0000 7ffabe1d6700 1 --2- 172.21.15.1:0/2755681952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffab8108f20 0x7ffab8109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:52.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.900+0000 7ffabe1d6700 1 -- 172.21.15.1:0/2755681952 >> 172.21.15.1:0/2755681952 conn(0x7ffab80ffec0 msgr2=0x7ffab8102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:52.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.900+0000 7ffabe1d6700 1 -- 172.21.15.1:0/2755681952 shutdown_connections 2024-05-04T03:16:52.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.900+0000 7ffabe1d6700 1 -- 172.21.15.1:0/2755681952 wait complete. 2024-05-04T03:16:52.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.901+0000 7ffabe1d6700 1 Processor -- start 2024-05-04T03:16:52.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.901+0000 7ffabe1d6700 1 -- start start 2024-05-04T03:16:52.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.901+0000 7ffabe1d6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffab81045f0 0x7ffab8076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:52.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.901+0000 7ffabe1d6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffab8108f20 0x7ffab8077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:52.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.901+0000 7ffabe1d6700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffab8109870 0x7ffab8077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:52.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.901+0000 7ffabe1d6700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffab81144a0 con 0x7ffab8108f20 2024-05-04T03:16:52.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.901+0000 7ffabe1d6700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ffab8114320 con 0x7ffab8109870 2024-05-04T03:16:52.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.901+0000 7ffabe1d6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffab8114620 con 0x7ffab81045f0 2024-05-04T03:16:52.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.901+0000 7ffab6ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffab8108f20 0x7ffab8077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:52.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.901+0000 7ffab6ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffab8108f20 0x7ffab8077110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41842/0 (socket says 172.21.15.1:41842) 2024-05-04T03:16:52.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.901+0000 7ffab6ffd700 1 -- 172.21.15.1:0/1496339643 learned_addr learned my addr 172.21.15.1:0/1496339643 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:16:52.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.901+0000 7ffab77fe700 1 --2- 172.21.15.1:0/1496339643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffab81045f0 0x7ffab8076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:52.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.901+0000 7ffab7fff700 1 --2- 172.21.15.1:0/1496339643 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffab8109870 0x7ffab8077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:52.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.902+0000 7ffab6ffd700 1 -- 172.21.15.1:0/1496339643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffab81045f0 msgr2=0x7ffab8076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:52.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.902+0000 7ffab6ffd700 1 --2- 172.21.15.1:0/1496339643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffab81045f0 0x7ffab8076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:52.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.902+0000 7ffab6ffd700 1 -- 172.21.15.1:0/1496339643 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffab8109870 msgr2=0x7ffab8077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:52.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.902+0000 7ffab6ffd700 1 --2- 172.21.15.1:0/1496339643 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffab8109870 0x7ffab8077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:52.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.902+0000 7ffab6ffd700 1 -- 172.21.15.1:0/1496339643 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab81a9270 con 0x7ffab8108f20 2024-05-04T03:16:52.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.902+0000 7ffab6ffd700 1 --2- 172.21.15.1:0/1496339643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffab8108f20 0x7ffab8077110 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00d950 tx=0x7ffaac00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:52.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.902+0000 7ffab4ff9700 1 -- 172.21.15.1:0/1496339643 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac016070 con 0x7ffab8108f20 2024-05-04T03:16:52.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.902+0000 7ffab4ff9700 1 -- 172.21.15.1:0/1496339643 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaac004830 con 0x7ffab8108f20 2024-05-04T03:16:52.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.902+0000 7ffabe1d6700 1 -- 172.21.15.1:0/1496339643 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffab81a9560 con 0x7ffab8108f20 2024-05-04T03:16:52.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.902+0000 7ffab4ff9700 1 -- 172.21.15.1:0/1496339643 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac005520 con 0x7ffab8108f20 2024-05-04T03:16:52.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.902+0000 7ffabe1d6700 1 -- 172.21.15.1:0/1496339643 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffab81a9af0 con 0x7ffab8108f20 2024-05-04T03:16:52.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.903+0000 7ffabe1d6700 1 -- 172.21.15.1:0/1496339643 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffab8063ed0 con 0x7ffab8108f20 2024-05-04T03:16:52.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.905+0000 7ffab4ff9700 1 -- 172.21.15.1:0/1496339643 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7ffaac004220 con 0x7ffab8108f20 2024-05-04T03:16:52.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.905+0000 7ffab4ff9700 1 --2- 172.21.15.1:0/1496339643 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ffaa00b2990 0x7ffaa00b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:52.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.906+0000 7ffab77fe700 1 --2- 172.21.15.1:0/1496339643 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ffaa00b2990 0x7ffaa00b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:52.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.906+0000 7ffab4ff9700 1 -- 172.21.15.1:0/1496339643 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7ffaac0e2530 con 0x7ffab8108f20 2024-05-04T03:16:52.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.906+0000 7ffab4ff9700 1 --2- 172.21.15.1:0/1496339643 >> [v2:172.21.15.1:6810/4050439365,v1:172.21.15.1:6811/4050439365] conn(0x7ffaa00bc3a0 0x7ffaa00be890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:52.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.906+0000 7ffab4ff9700 1 -- 172.21.15.1:0/1496339643 --> [v2:172.21.15.1:6810/4050439365,v1:172.21.15.1:6811/4050439365] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffaac015ec0 con 0x7ffaa00bc3a0 2024-05-04T03:16:52.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.906+0000 7ffab4ff9700 1 -- 172.21.15.1:0/1496339643 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7ffaac0e2a10 con 0x7ffab8108f20 2024-05-04T03:16:52.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.906+0000 7ffab7fff700 1 --2- 172.21.15.1:0/1496339643 >> [v2:172.21.15.1:6810/4050439365,v1:172.21.15.1:6811/4050439365] conn(0x7ffaa00bc3a0 0x7ffaa00be890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:52.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.906+0000 7ffab77fe700 1 --2- 172.21.15.1:0/1496339643 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ffaa00b2990 0x7ffaa00b4ea0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8009c60 tx=0x7ffaa8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:52.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.906+0000 7ffab7fff700 1 --2- 172.21.15.1:0/1496339643 >> [v2:172.21.15.1:6810/4050439365,v1:172.21.15.1:6811/4050439365] conn(0x7ffaa00bc3a0 0x7ffaa00be890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:52.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.907+0000 7ffab4ff9700 1 -- 172.21.15.1:0/1496339643 <== osd.3 v2:172.21.15.1:6810/4050439365 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffaac015ec0 con 0x7ffaa00bc3a0 2024-05-04T03:16:52.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.924+0000 7ffabe1d6700 1 -- 172.21.15.1:0/1496339643 --> [v2:172.21.15.1:6810/4050439365,v1:172.21.15.1:6811/4050439365] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffab80642d0 con 0x7ffaa00bc3a0 2024-05-04T03:16:52.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.924+0000 7ffab4ff9700 1 -- 172.21.15.1:0/1496339643 <== osd.3 v2:172.21.15.1:6810/4050439365 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ffab80642d0 con 0x7ffaa00bc3a0 2024-05-04T03:16:52.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.925+0000 7ffabe1d6700 1 -- 172.21.15.1:0/1496339643 >> [v2:172.21.15.1:6810/4050439365,v1:172.21.15.1:6811/4050439365] conn(0x7ffaa00bc3a0 msgr2=0x7ffaa00be890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:52.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.925+0000 7ffabe1d6700 1 --2- 172.21.15.1:0/1496339643 >> [v2:172.21.15.1:6810/4050439365,v1:172.21.15.1:6811/4050439365] conn(0x7ffaa00bc3a0 0x7ffaa00be890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:52.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.925+0000 7ffabe1d6700 1 -- 172.21.15.1:0/1496339643 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ffaa00b2990 msgr2=0x7ffaa00b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:52.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.925+0000 7ffabe1d6700 1 --2- 172.21.15.1:0/1496339643 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ffaa00b2990 0x7ffaa00b4ea0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8009c60 tx=0x7ffaa8009260 comp rx=0 tx=0).stop 2024-05-04T03:16:52.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.925+0000 7ffabe1d6700 1 -- 172.21.15.1:0/1496339643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffab8108f20 msgr2=0x7ffab8077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:52.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.925+0000 7ffabe1d6700 1 --2- 172.21.15.1:0/1496339643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffab8108f20 0x7ffab8077110 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00d950 tx=0x7ffaac00dc60 comp rx=0 tx=0).stop 2024-05-04T03:16:52.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.925+0000 7ffab7fff700 1 -- 172.21.15.1:0/1496339643 reap_dead start 2024-05-04T03:16:52.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.925+0000 7ffabe1d6700 1 -- 172.21.15.1:0/1496339643 shutdown_connections 2024-05-04T03:16:52.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.925+0000 7ffabe1d6700 1 -- 172.21.15.1:0/1496339643 >> 172.21.15.1:0/1496339643 conn(0x7ffab80ffec0 msgr2=0x7ffab8108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:52.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.925+0000 7ffabe1d6700 1 -- 172.21.15.1:0/1496339643 shutdown_connections 2024-05-04T03:16:52.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:52.925+0000 7ffabe1d6700 1 -- 172.21.15.1:0/1496339643 wait complete. 2024-05-04T03:16:53.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.871+0000 7f8543d98700 1 -- 172.21.15.1:0/3937463706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f853c1045e0 msgr2=0x7f853c1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:53.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.871+0000 7f8543d98700 1 --2- 172.21.15.1:0/3937463706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f853c1045e0 0x7f853c1049f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f85380099c0 tx=0x7f8538009cd0 comp rx=0 tx=0).stop 2024-05-04T03:16:53.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.871+0000 7f8543d98700 1 -- 172.21.15.1:0/3937463706 shutdown_connections 2024-05-04T03:16:53.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.871+0000 7f8543d98700 1 --2- 172.21.15.1:0/3937463706 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f853c109860 0x7f853c1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:53.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.871+0000 7f8543d98700 1 --2- 172.21.15.1:0/3937463706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f853c1045e0 0x7f853c1049f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:53.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.871+0000 7f8543d98700 1 --2- 172.21.15.1:0/3937463706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f853c108f10 0x7f853c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:53.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.871+0000 7f8543d98700 1 -- 172.21.15.1:0/3937463706 >> 172.21.15.1:0/3937463706 conn(0x7f853c0ffed0 msgr2=0x7f853c102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:53.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.871+0000 7f8543d98700 1 -- 172.21.15.1:0/3937463706 shutdown_connections 2024-05-04T03:16:53.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.871+0000 7f8543d98700 1 -- 172.21.15.1:0/3937463706 wait complete. 2024-05-04T03:16:53.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.872+0000 7f8543d98700 1 Processor -- start 2024-05-04T03:16:53.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.872+0000 7f8543d98700 1 -- start start 2024-05-04T03:16:53.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.872+0000 7f8543d98700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f853c1045e0 0x7f853c19af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:53.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.872+0000 7f8543d98700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f853c108f10 0x7f853c19b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:53.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.872+0000 7f8543d98700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f853c109860 0x7f853c19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:53.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.872+0000 7f8543d98700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f853c114510 con 0x7f853c108f10 2024-05-04T03:16:53.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.872+0000 7f8543d98700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f853c114390 con 0x7f853c1045e0 2024-05-04T03:16:53.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.872+0000 7f8543d98700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f853c114690 con 0x7f853c109860 2024-05-04T03:16:53.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.872+0000 7f8541333700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f853c108f10 0x7f853c19b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:53.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.872+0000 7f8541333700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f853c108f10 0x7f853c19b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41858/0 (socket says 172.21.15.1:41858) 2024-05-04T03:16:53.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.872+0000 7f8541333700 1 -- 172.21.15.1:0/3537606430 learned_addr learned my addr 172.21.15.1:0/3537606430 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:16:53.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.872+0000 7f8541b34700 1 --2- 172.21.15.1:0/3537606430 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f853c1045e0 0x7f853c19af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:53.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.873+0000 7f8542335700 1 --2- 172.21.15.1:0/3537606430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f853c109860 0x7f853c19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:53.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.873+0000 7f8541333700 1 -- 172.21.15.1:0/3537606430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f853c109860 msgr2=0x7f853c19f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:53.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.873+0000 7f8541333700 1 --2- 172.21.15.1:0/3537606430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f853c109860 0x7f853c19f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:53.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.873+0000 7f8541333700 1 -- 172.21.15.1:0/3537606430 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f853c1045e0 msgr2=0x7f853c19af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:53.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.873+0000 7f8541333700 1 --2- 172.21.15.1:0/3537606430 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f853c1045e0 0x7f853c19af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:53.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.873+0000 7f8541333700 1 -- 172.21.15.1:0/3537606430 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853c19fd20 con 0x7f853c108f10 2024-05-04T03:16:53.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.873+0000 7f8541333700 1 --2- 172.21.15.1:0/3537606430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f853c108f10 0x7f853c19b4a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f852c00c9e0 tx=0x7f852c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:53.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.873+0000 7f8532ffd700 1 -- 172.21.15.1:0/3537606430 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c007aa0 con 0x7f853c108f10 2024-05-04T03:16:53.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.873+0000 7f8532ffd700 1 -- 172.21.15.1:0/3537606430 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f852c007c40 con 0x7f853c108f10 2024-05-04T03:16:53.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.873+0000 7f8543d98700 1 -- 172.21.15.1:0/3537606430 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f853c1a0010 con 0x7f853c108f10 2024-05-04T03:16:53.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.873+0000 7f8532ffd700 1 -- 172.21.15.1:0/3537606430 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c00fe60 con 0x7f853c108f10 2024-05-04T03:16:53.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.873+0000 7f8543d98700 1 -- 172.21.15.1:0/3537606430 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f853c1a79e0 con 0x7f853c108f10 2024-05-04T03:16:53.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.874+0000 7f8543d98700 1 -- 172.21.15.1:0/3537606430 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f853c063ed0 con 0x7f853c108f10 2024-05-04T03:16:53.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.876+0000 7f8532ffd700 1 -- 172.21.15.1:0/3537606430 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f852c00f450 con 0x7f853c108f10 2024-05-04T03:16:53.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.876+0000 7f8532ffd700 1 --2- 172.21.15.1:0/3537606430 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f85280b2990 0x7f85280b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:53.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.876+0000 7f8541b34700 1 --2- 172.21.15.1:0/3537606430 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f85280b2990 0x7f85280b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:53.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.877+0000 7f8532ffd700 1 -- 172.21.15.1:0/3537606430 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f852c0e2520 con 0x7f853c108f10 2024-05-04T03:16:53.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.877+0000 7f8532ffd700 1 --2- 172.21.15.1:0/3537606430 >> [v2:172.21.15.1:6826/2681322272,v1:172.21.15.1:6827/2681322272] conn(0x7f85280bc3a0 0x7f85280be890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:53.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.877+0000 7f8532ffd700 1 -- 172.21.15.1:0/3537606430 --> [v2:172.21.15.1:6826/2681322272,v1:172.21.15.1:6827/2681322272] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f85280bef60 con 0x7f85280bc3a0 2024-05-04T03:16:53.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.877+0000 7f8532ffd700 1 -- 172.21.15.1:0/3537606430 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f852c0e2a00 con 0x7f853c108f10 2024-05-04T03:16:53.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.877+0000 7f8542335700 1 --2- 172.21.15.1:0/3537606430 >> [v2:172.21.15.1:6826/2681322272,v1:172.21.15.1:6827/2681322272] conn(0x7f85280bc3a0 0x7f85280be890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:53.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.877+0000 7f8541b34700 1 --2- 172.21.15.1:0/3537606430 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f85280b2990 0x7f85280b4ea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8538009990 tx=0x7f8538000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:53.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.877+0000 7f8542335700 1 --2- 172.21.15.1:0/3537606430 >> [v2:172.21.15.1:6826/2681322272,v1:172.21.15.1:6827/2681322272] conn(0x7f85280bc3a0 0x7f85280be890 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:53.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.878+0000 7f8532ffd700 1 -- 172.21.15.1:0/3537606430 <== osd.9 v2:172.21.15.1:6826/2681322272 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f85280bef60 con 0x7f85280bc3a0 2024-05-04T03:16:53.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.895+0000 7f8543d98700 1 -- 172.21.15.1:0/3537606430 --> [v2:172.21.15.1:6826/2681322272,v1:172.21.15.1:6827/2681322272] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f853c0642d0 con 0x7f85280bc3a0 2024-05-04T03:16:53.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.895+0000 7f8532ffd700 1 -- 172.21.15.1:0/3537606430 <== osd.9 v2:172.21.15.1:6826/2681322272 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f853c0642d0 con 0x7f85280bc3a0 2024-05-04T03:16:53.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.896+0000 7f8543d98700 1 -- 172.21.15.1:0/3537606430 >> [v2:172.21.15.1:6826/2681322272,v1:172.21.15.1:6827/2681322272] conn(0x7f85280bc3a0 msgr2=0x7f85280be890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:53.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.896+0000 7f8543d98700 1 --2- 172.21.15.1:0/3537606430 >> [v2:172.21.15.1:6826/2681322272,v1:172.21.15.1:6827/2681322272] conn(0x7f85280bc3a0 0x7f85280be890 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:53.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.896+0000 7f8543d98700 1 -- 172.21.15.1:0/3537606430 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f85280b2990 msgr2=0x7f85280b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:53.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.896+0000 7f8543d98700 1 --2- 172.21.15.1:0/3537606430 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f85280b2990 0x7f85280b4ea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8538009990 tx=0x7f8538000bc0 comp rx=0 tx=0).stop 2024-05-04T03:16:53.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.896+0000 7f8543d98700 1 -- 172.21.15.1:0/3537606430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f853c108f10 msgr2=0x7f853c19b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:53.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.896+0000 7f8543d98700 1 --2- 172.21.15.1:0/3537606430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f853c108f10 0x7f853c19b4a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f852c00c9e0 tx=0x7f852c00ccf0 comp rx=0 tx=0).stop 2024-05-04T03:16:53.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.896+0000 7f8542335700 1 -- 172.21.15.1:0/3537606430 reap_dead start 2024-05-04T03:16:53.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.896+0000 7f8543d98700 1 -- 172.21.15.1:0/3537606430 shutdown_connections 2024-05-04T03:16:53.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.896+0000 7f8543d98700 1 -- 172.21.15.1:0/3537606430 >> 172.21.15.1:0/3537606430 conn(0x7f853c0ffed0 msgr2=0x7f853c102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:53.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.896+0000 7f8543d98700 1 -- 172.21.15.1:0/3537606430 shutdown_connections 2024-05-04T03:16:53.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:53.896+0000 7f8543d98700 1 -- 172.21.15.1:0/3537606430 wait complete. 2024-05-04T03:16:54.231 INFO:teuthology.orchestra.run.smithi001.stdout:107374182468 2024-05-04T03:16:54.231 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-04T03:16:54.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.439+0000 7f1c42db0700 1 -- 172.21.15.1:0/880693027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c3c107df0 msgr2=0x7f1c3c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:54.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.439+0000 7f1c42db0700 1 --2- 172.21.15.1:0/880693027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c3c107df0 0x7f1c3c108200 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1c240099c0 tx=0x7f1c24009cd0 comp rx=0 tx=0).stop 2024-05-04T03:16:54.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.439+0000 7f1c42db0700 1 -- 172.21.15.1:0/880693027 shutdown_connections 2024-05-04T03:16:54.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.439+0000 7f1c42db0700 1 --2- 172.21.15.1:0/880693027 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c3c1034c0 0x7f1c3c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:54.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.439+0000 7f1c42db0700 1 --2- 172.21.15.1:0/880693027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c3c108740 0x7f1c3c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:54.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.439+0000 7f1c42db0700 1 --2- 172.21.15.1:0/880693027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c3c107df0 0x7f1c3c108200 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:54.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.439+0000 7f1c42db0700 1 -- 172.21.15.1:0/880693027 >> 172.21.15.1:0/880693027 conn(0x7f1c3c0fedc0 msgr2=0x7f1c3c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:54.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.439+0000 7f1c42db0700 1 -- 172.21.15.1:0/880693027 shutdown_connections 2024-05-04T03:16:54.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.439+0000 7f1c42db0700 1 -- 172.21.15.1:0/880693027 wait complete. 2024-05-04T03:16:54.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.440+0000 7f1c42db0700 1 Processor -- start 2024-05-04T03:16:54.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.440+0000 7f1c42db0700 1 -- start start 2024-05-04T03:16:54.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.441+0000 7f1c42db0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c3c1034c0 0x7f1c3c19f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:54.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.441+0000 7f1c42db0700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c3c107df0 0x7f1c3c19ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:54.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.441+0000 7f1c42db0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c3c108740 0x7f1c3c199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:54.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.441+0000 7f1c42db0700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3c113350 con 0x7f1c3c108740 2024-05-04T03:16:54.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.441+0000 7f1c42db0700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3c1131d0 con 0x7f1c3c107df0 2024-05-04T03:16:54.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.441+0000 7f1c42db0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3c1134d0 con 0x7f1c3c1034c0 2024-05-04T03:16:54.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.441+0000 7f1c4134d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c3c108740 0x7f1c3c199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:54.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.441+0000 7f1c4134d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c3c108740 0x7f1c3c199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41876/0 (socket says 172.21.15.1:41876) 2024-05-04T03:16:54.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.441+0000 7f1c4134d700 1 -- 172.21.15.1:0/2594258652 learned_addr learned my addr 172.21.15.1:0/2594258652 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:16:54.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.441+0000 7f1c40b4c700 1 --2- 172.21.15.1:0/2594258652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c3c1034c0 0x7f1c3c19f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:54.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.441+0000 7f1c3bfff700 1 --2- 172.21.15.1:0/2594258652 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c3c107df0 0x7f1c3c19ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:54.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.441+0000 7f1c4134d700 1 -- 172.21.15.1:0/2594258652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c3c1034c0 msgr2=0x7f1c3c19f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:54.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.441+0000 7f1c4134d700 1 --2- 172.21.15.1:0/2594258652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c3c1034c0 0x7f1c3c19f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:54.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.441+0000 7f1c4134d700 1 -- 172.21.15.1:0/2594258652 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c3c107df0 msgr2=0x7f1c3c19ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:54.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.441+0000 7f1c4134d700 1 --2- 172.21.15.1:0/2594258652 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c3c107df0 0x7f1c3c19ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:54.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.441+0000 7f1c4134d700 1 -- 172.21.15.1:0/2594258652 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c3c19a280 con 0x7f1c3c108740 2024-05-04T03:16:54.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.442+0000 7f1c4134d700 1 --2- 172.21.15.1:0/2594258652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c3c108740 0x7f1c3c199b50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1c34007f10 tx=0x7f1c3400f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:54.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.442+0000 7f1c39ffb700 1 -- 172.21.15.1:0/2594258652 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c34016030 con 0x7f1c3c108740 2024-05-04T03:16:54.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.442+0000 7f1c39ffb700 1 -- 172.21.15.1:0/2594258652 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c3400fd30 con 0x7f1c3c108740 2024-05-04T03:16:54.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.442+0000 7f1c42db0700 1 -- 172.21.15.1:0/2594258652 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c3c19a570 con 0x7f1c3c108740 2024-05-04T03:16:54.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.442+0000 7f1c39ffb700 1 -- 172.21.15.1:0/2594258652 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c34012e50 con 0x7f1c3c108740 2024-05-04T03:16:54.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.442+0000 7f1c42db0700 1 -- 172.21.15.1:0/2594258652 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c3c06b610 con 0x7f1c3c108740 2024-05-04T03:16:54.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.443+0000 7f1c42db0700 1 -- 172.21.15.1:0/2594258652 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1c3c06bb50 con 0x7f1c3c108740 2024-05-04T03:16:54.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.445+0000 7f1c39ffb700 1 -- 172.21.15.1:0/2594258652 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f1c34004c40 con 0x7f1c3c108740 2024-05-04T03:16:54.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.446+0000 7f1c39ffb700 1 --2- 172.21.15.1:0/2594258652 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1c280b2b30 0x7f1c280b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:54.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.446+0000 7f1c39ffb700 1 -- 172.21.15.1:0/2594258652 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f1c340e4790 con 0x7f1c3c108740 2024-05-04T03:16:54.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.446+0000 7f1c40b4c700 1 --2- 172.21.15.1:0/2594258652 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1c280b2b30 0x7f1c280b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:54.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.446+0000 7f1c39ffb700 1 --2- 172.21.15.1:0/2594258652 >> [v2:172.21.15.1:6802/3479011484,v1:172.21.15.1:6803/3479011484] conn(0x7f1c280bc540 0x7f1c280bea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:54.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.446+0000 7f1c39ffb700 1 -- 172.21.15.1:0/2594258652 --> [v2:172.21.15.1:6802/3479011484,v1:172.21.15.1:6803/3479011484] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1c3401ee70 con 0x7f1c280bc540 2024-05-04T03:16:54.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.446+0000 7f1c39ffb700 1 -- 172.21.15.1:0/2594258652 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f1c340e4c70 con 0x7f1c3c108740 2024-05-04T03:16:54.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.446+0000 7f1c3bfff700 1 --2- 172.21.15.1:0/2594258652 >> [v2:172.21.15.1:6802/3479011484,v1:172.21.15.1:6803/3479011484] conn(0x7f1c280bc540 0x7f1c280bea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:54.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.447+0000 7f1c40b4c700 1 --2- 172.21.15.1:0/2594258652 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1c280b2b30 0x7f1c280b5040 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c005950 tx=0x7f1c2c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:54.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.447+0000 7f1c3bfff700 1 --2- 172.21.15.1:0/2594258652 >> [v2:172.21.15.1:6802/3479011484,v1:172.21.15.1:6803/3479011484] conn(0x7f1c280bc540 0x7f1c280bea30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:54.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.448+0000 7f1c39ffb700 1 -- 172.21.15.1:0/2594258652 <== osd.0 v2:172.21.15.1:6802/3479011484 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1c3401ee70 con 0x7f1c280bc540 2024-05-04T03:16:54.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.464+0000 7f1c42db0700 1 -- 172.21.15.1:0/2594258652 --> [v2:172.21.15.1:6802/3479011484,v1:172.21.15.1:6803/3479011484] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1c3c06bfd0 con 0x7f1c280bc540 2024-05-04T03:16:54.466 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:54 smithi001 ceph-mon[29731]: pgmap v323: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-04T03:16:54.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.465+0000 7f1c39ffb700 1 -- 172.21.15.1:0/2594258652 <== osd.0 v2:172.21.15.1:6802/3479011484 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1c3c06bfd0 con 0x7f1c280bc540 2024-05-04T03:16:54.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.465+0000 7f1c42db0700 1 -- 172.21.15.1:0/2594258652 >> [v2:172.21.15.1:6802/3479011484,v1:172.21.15.1:6803/3479011484] conn(0x7f1c280bc540 msgr2=0x7f1c280bea30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:54.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.465+0000 7f1c42db0700 1 --2- 172.21.15.1:0/2594258652 >> [v2:172.21.15.1:6802/3479011484,v1:172.21.15.1:6803/3479011484] conn(0x7f1c280bc540 0x7f1c280bea30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:54.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.465+0000 7f1c42db0700 1 -- 172.21.15.1:0/2594258652 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1c280b2b30 msgr2=0x7f1c280b5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:54.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.465+0000 7f1c42db0700 1 --2- 172.21.15.1:0/2594258652 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1c280b2b30 0x7f1c280b5040 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c005950 tx=0x7f1c2c00b410 comp rx=0 tx=0).stop 2024-05-04T03:16:54.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.465+0000 7f1c42db0700 1 -- 172.21.15.1:0/2594258652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c3c108740 msgr2=0x7f1c3c199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:54.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.465+0000 7f1c42db0700 1 --2- 172.21.15.1:0/2594258652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c3c108740 0x7f1c3c199b50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1c34007f10 tx=0x7f1c3400f5c0 comp rx=0 tx=0).stop 2024-05-04T03:16:54.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.465+0000 7f1c4134d700 1 -- 172.21.15.1:0/2594258652 reap_dead start 2024-05-04T03:16:54.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.465+0000 7f1c42db0700 1 -- 172.21.15.1:0/2594258652 shutdown_connections 2024-05-04T03:16:54.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.465+0000 7f1c42db0700 1 -- 172.21.15.1:0/2594258652 >> 172.21.15.1:0/2594258652 conn(0x7f1c3c0fedc0 msgr2=0x7f1c3c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:54.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.465+0000 7f1c42db0700 1 -- 172.21.15.1:0/2594258652 shutdown_connections 2024-05-04T03:16:54.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:54.465+0000 7f1c42db0700 1 -- 172.21.15.1:0/2594258652 wait complete. 2024-05-04T03:16:54.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:54 smithi072 ceph-mon[33332]: pgmap v323: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-04T03:16:54.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:54 smithi077 ceph-mon[33531]: pgmap v323: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-04T03:16:56.160 INFO:teuthology.orchestra.run.smithi001.stdout:34359738452 2024-05-04T03:16:56.160 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-04T03:16:56.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.366+0000 7fac443f9700 1 -- 172.21.15.1:0/4007219023 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac3c108740 msgr2=0x7fac3c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:56.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.366+0000 7fac443f9700 1 --2- 172.21.15.1:0/4007219023 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac3c108740 0x7fac3c1100a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fac340099c0 tx=0x7fac34009cd0 comp rx=0 tx=0).stop 2024-05-04T03:16:56.369 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:56 smithi001 ceph-mon[29731]: pgmap v324: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-04T03:16:56.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.367+0000 7fac443f9700 1 -- 172.21.15.1:0/4007219023 shutdown_connections 2024-05-04T03:16:56.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.367+0000 7fac443f9700 1 --2- 172.21.15.1:0/4007219023 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fac3c1034c0 0x7fac3c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:56.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.367+0000 7fac443f9700 1 --2- 172.21.15.1:0/4007219023 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac3c108740 0x7fac3c1100a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:56.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.367+0000 7fac443f9700 1 --2- 172.21.15.1:0/4007219023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac3c107df0 0x7fac3c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:56.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.367+0000 7fac443f9700 1 -- 172.21.15.1:0/4007219023 >> 172.21.15.1:0/4007219023 conn(0x7fac3c0fedf0 msgr2=0x7fac3c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:56.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.367+0000 7fac443f9700 1 -- 172.21.15.1:0/4007219023 shutdown_connections 2024-05-04T03:16:56.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.367+0000 7fac443f9700 1 -- 172.21.15.1:0/4007219023 wait complete. 2024-05-04T03:16:56.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.368+0000 7fac443f9700 1 Processor -- start 2024-05-04T03:16:56.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.368+0000 7fac443f9700 1 -- start start 2024-05-04T03:16:56.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.368+0000 7fac443f9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fac3c1034c0 0x7fac3c19bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:56.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.368+0000 7fac443f9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac3c107df0 0x7fac3c19c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:56.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.368+0000 7fac443f9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac3c108740 0x7fac3c19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:56.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.368+0000 7fac443f9700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c1133c0 con 0x7fac3c107df0 2024-05-04T03:16:56.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.369+0000 7fac41994700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac3c107df0 0x7fac3c19c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:56.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.369+0000 7fac443f9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c113240 con 0x7fac3c1034c0 2024-05-04T03:16:56.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.369+0000 7fac443f9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c113540 con 0x7fac3c108740 2024-05-04T03:16:56.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.369+0000 7fac41994700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac3c107df0 0x7fac3c19c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41886/0 (socket says 172.21.15.1:41886) 2024-05-04T03:16:56.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.369+0000 7fac42195700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fac3c1034c0 0x7fac3c19bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:56.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.369+0000 7fac41994700 1 -- 172.21.15.1:0/1846473307 learned_addr learned my addr 172.21.15.1:0/1846473307 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:16:56.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.369+0000 7fac42996700 1 --2- 172.21.15.1:0/1846473307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac3c108740 0x7fac3c19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:56.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.369+0000 7fac41994700 1 -- 172.21.15.1:0/1846473307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac3c108740 msgr2=0x7fac3c19c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:56.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.369+0000 7fac41994700 1 --2- 172.21.15.1:0/1846473307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac3c108740 0x7fac3c19c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:56.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.369+0000 7fac41994700 1 -- 172.21.15.1:0/1846473307 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fac3c1034c0 msgr2=0x7fac3c19bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:56.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.369+0000 7fac41994700 1 --2- 172.21.15.1:0/1846473307 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fac3c1034c0 0x7fac3c19bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:56.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.369+0000 7fac41994700 1 -- 172.21.15.1:0/1846473307 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3c1a6e80 con 0x7fac3c107df0 2024-05-04T03:16:56.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.369+0000 7fac42195700 1 --2- 172.21.15.1:0/1846473307 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fac3c1034c0 0x7fac3c19bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:16:56.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.369+0000 7fac41994700 1 --2- 172.21.15.1:0/1846473307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac3c107df0 0x7fac3c19c100 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fac3000c9e0 tx=0x7fac3000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:56.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.370+0000 7fac3b7fe700 1 -- 172.21.15.1:0/1846473307 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac30007aa0 con 0x7fac3c107df0 2024-05-04T03:16:56.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.370+0000 7fac3b7fe700 1 -- 172.21.15.1:0/1846473307 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac30007c40 con 0x7fac3c107df0 2024-05-04T03:16:56.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.370+0000 7fac443f9700 1 -- 172.21.15.1:0/1846473307 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac3c1a7170 con 0x7fac3c107df0 2024-05-04T03:16:56.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.370+0000 7fac3b7fe700 1 -- 172.21.15.1:0/1846473307 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3000fe60 con 0x7fac3c107df0 2024-05-04T03:16:56.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.370+0000 7fac443f9700 1 -- 172.21.15.1:0/1846473307 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac3c1a76d0 con 0x7fac3c107df0 2024-05-04T03:16:56.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.370+0000 7fac443f9700 1 -- 172.21.15.1:0/1846473307 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fac3c063ed0 con 0x7fac3c107df0 2024-05-04T03:16:56.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.372+0000 7fac3b7fe700 1 -- 172.21.15.1:0/1846473307 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fac3000f450 con 0x7fac3c107df0 2024-05-04T03:16:56.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.373+0000 7fac3b7fe700 1 --2- 172.21.15.1:0/1846473307 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fac240b2990 0x7fac240b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:56.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.373+0000 7fac42195700 1 --2- 172.21.15.1:0/1846473307 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fac240b2990 0x7fac240b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:56.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.373+0000 7fac3b7fe700 1 -- 172.21.15.1:0/1846473307 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fac300e24f0 con 0x7fac3c107df0 2024-05-04T03:16:56.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.374+0000 7fac3b7fe700 1 --2- 172.21.15.1:0/1846473307 >> [v2:172.21.15.1:6818/1799033649,v1:172.21.15.1:6819/1799033649] conn(0x7fac240bc3a0 0x7fac240be890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:16:56.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.374+0000 7fac3b7fe700 1 -- 172.21.15.1:0/1846473307 --> [v2:172.21.15.1:6818/1799033649,v1:172.21.15.1:6819/1799033649] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fac30010e50 con 0x7fac240bc3a0 2024-05-04T03:16:56.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.374+0000 7fac3b7fe700 1 -- 172.21.15.1:0/1846473307 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fac300e29d0 con 0x7fac3c107df0 2024-05-04T03:16:56.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.374+0000 7fac42195700 1 --2- 172.21.15.1:0/1846473307 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fac240b2990 0x7fac240b4ea0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fac2c00c720 tx=0x7fac2c009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:56.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.374+0000 7fac42996700 1 --2- 172.21.15.1:0/1846473307 >> [v2:172.21.15.1:6818/1799033649,v1:172.21.15.1:6819/1799033649] conn(0x7fac240bc3a0 0x7fac240be890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:16:56.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.374+0000 7fac42996700 1 --2- 172.21.15.1:0/1846473307 >> [v2:172.21.15.1:6818/1799033649,v1:172.21.15.1:6819/1799033649] conn(0x7fac240bc3a0 0x7fac240be890 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:16:56.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.375+0000 7fac3b7fe700 1 -- 172.21.15.1:0/1846473307 <== osd.6 v2:172.21.15.1:6818/1799033649 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fac30010e50 con 0x7fac240bc3a0 2024-05-04T03:16:56.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.392+0000 7fac443f9700 1 -- 172.21.15.1:0/1846473307 --> [v2:172.21.15.1:6818/1799033649,v1:172.21.15.1:6819/1799033649] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fac3c064300 con 0x7fac240bc3a0 2024-05-04T03:16:56.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.392+0000 7fac3b7fe700 1 -- 172.21.15.1:0/1846473307 <== osd.6 v2:172.21.15.1:6818/1799033649 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fac3c064300 con 0x7fac240bc3a0 2024-05-04T03:16:56.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.392+0000 7fac443f9700 1 -- 172.21.15.1:0/1846473307 >> [v2:172.21.15.1:6818/1799033649,v1:172.21.15.1:6819/1799033649] conn(0x7fac240bc3a0 msgr2=0x7fac240be890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.392+0000 7fac443f9700 1 --2- 172.21.15.1:0/1846473307 >> [v2:172.21.15.1:6818/1799033649,v1:172.21.15.1:6819/1799033649] conn(0x7fac240bc3a0 0x7fac240be890 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:16:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.392+0000 7fac443f9700 1 -- 172.21.15.1:0/1846473307 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fac240b2990 msgr2=0x7fac240b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.392+0000 7fac443f9700 1 --2- 172.21.15.1:0/1846473307 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fac240b2990 0x7fac240b4ea0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fac2c00c720 tx=0x7fac2c009040 comp rx=0 tx=0).stop 2024-05-04T03:16:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.392+0000 7fac443f9700 1 -- 172.21.15.1:0/1846473307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac3c107df0 msgr2=0x7fac3c19c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:16:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.392+0000 7fac443f9700 1 --2- 172.21.15.1:0/1846473307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac3c107df0 0x7fac3c19c100 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fac3000c9e0 tx=0x7fac3000ccf0 comp rx=0 tx=0).stop 2024-05-04T03:16:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.392+0000 7fac42996700 1 -- 172.21.15.1:0/1846473307 reap_dead start 2024-05-04T03:16:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.393+0000 7fac443f9700 1 -- 172.21.15.1:0/1846473307 shutdown_connections 2024-05-04T03:16:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.393+0000 7fac443f9700 1 -- 172.21.15.1:0/1846473307 >> 172.21.15.1:0/1846473307 conn(0x7fac3c0fedf0 msgr2=0x7fac3c107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:16:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.393+0000 7fac443f9700 1 -- 172.21.15.1:0/1846473307 shutdown_connections 2024-05-04T03:16:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:16:56.393+0000 7fac443f9700 1 -- 172.21.15.1:0/1846473307 wait complete. 2024-05-04T03:16:56.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:56 smithi072 ceph-mon[33332]: pgmap v324: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-04T03:16:56.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:56 smithi077 ceph-mon[33531]: pgmap v324: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-04T03:16:58.455 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:16:58 smithi001 ceph-mon[29731]: pgmap v325: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-04T03:16:58.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:16:58 smithi072 ceph-mon[33332]: pgmap v325: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-04T03:16:58.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:16:58 smithi077 ceph-mon[33531]: pgmap v325: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-04T03:16:58.815 INFO:teuthology.orchestra.run.smithi001.stdout:184683593783 2024-05-04T03:16:58.816 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-04T03:16:59.984 INFO:teuthology.orchestra.run.smithi001.stdout:266287972391 2024-05-04T03:16:59.984 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-04T03:17:00.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:00 smithi072 ceph-mon[33332]: pgmap v326: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-04T03:17:00.552 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:00 smithi001 ceph-mon[29731]: pgmap v326: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-04T03:17:00.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:00 smithi077 ceph-mon[33531]: pgmap v326: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-04T03:17:01.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:01 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:17:01.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:01 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:17:01.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:01 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:17:01.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:01 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:17:01.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:01 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:17:01.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:01 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:17:01.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:01 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:17:01.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:01 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:17:01.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:01 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:17:01.929 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:01 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:17:01.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:01 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:17:01.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:01 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:17:01.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:01 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:17:01.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:01 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:17:01.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:01 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:17:02.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.247+0000 7f959fb3a700 1 -- 172.21.15.1:0/200798661 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95981034e0 msgr2=0x7f95981038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:02.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.247+0000 7f959fb3a700 1 --2- 172.21.15.1:0/200798661 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95981034e0 0x7f95981038f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f95900099c0 tx=0x7f9590009cd0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.248+0000 7f959fb3a700 1 -- 172.21.15.1:0/200798661 shutdown_connections 2024-05-04T03:17:02.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.248+0000 7f959fb3a700 1 --2- 172.21.15.1:0/200798661 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9598108760 0x7f95981100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.248+0000 7f959fb3a700 1 --2- 172.21.15.1:0/200798661 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95981034e0 0x7f95981038f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.248+0000 7f959fb3a700 1 --2- 172.21.15.1:0/200798661 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9598107e10 0x7f9598108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.248+0000 7f959fb3a700 1 -- 172.21.15.1:0/200798661 >> 172.21.15.1:0/200798661 conn(0x7f95980fef30 msgr2=0x7f9598101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:02.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.248+0000 7f959fb3a700 1 -- 172.21.15.1:0/200798661 shutdown_connections 2024-05-04T03:17:02.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.248+0000 7f959fb3a700 1 -- 172.21.15.1:0/200798661 wait complete. 2024-05-04T03:17:02.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.248+0000 7f959fb3a700 1 Processor -- start 2024-05-04T03:17:02.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.248+0000 7f959fb3a700 1 -- start start 2024-05-04T03:17:02.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.249+0000 7f959fb3a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95981034e0 0x7f959819bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:02.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.249+0000 7f959fb3a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9598107e10 0x7f959819c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:02.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.249+0000 7f959fb3a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9598108760 0x7f959819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:02.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.249+0000 7f959fb3a700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9598113390 con 0x7f9598107e10 2024-05-04T03:17:02.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.249+0000 7f959fb3a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9598113210 con 0x7f95981034e0 2024-05-04T03:17:02.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.249+0000 7f959fb3a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9598113510 con 0x7f9598108760 2024-05-04T03:17:02.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.249+0000 7f959d0d5700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9598107e10 0x7f959819c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:02.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.249+0000 7f959d0d5700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9598107e10 0x7f959819c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46958/0 (socket says 172.21.15.1:46958) 2024-05-04T03:17:02.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.249+0000 7f959d0d5700 1 -- 172.21.15.1:0/398639401 learned_addr learned my addr 172.21.15.1:0/398639401 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:02.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.249+0000 7f959d8d6700 1 --2- 172.21.15.1:0/398639401 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95981034e0 0x7f959819bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:02.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.249+0000 7f959e0d7700 1 --2- 172.21.15.1:0/398639401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9598108760 0x7f959819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:02.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.249+0000 7f959d0d5700 1 -- 172.21.15.1:0/398639401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9598108760 msgr2=0x7f959819c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:02.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.249+0000 7f959d0d5700 1 --2- 172.21.15.1:0/398639401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9598108760 0x7f959819c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.249+0000 7f959d0d5700 1 -- 172.21.15.1:0/398639401 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95981034e0 msgr2=0x7f959819bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:02.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.249+0000 7f959d0d5700 1 --2- 172.21.15.1:0/398639401 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95981034e0 0x7f959819bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.249+0000 7f959d0d5700 1 -- 172.21.15.1:0/398639401 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95981a7090 con 0x7f9598107e10 2024-05-04T03:17:02.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.249+0000 7f959e0d7700 1 --2- 172.21.15.1:0/398639401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9598108760 0x7f959819c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:17:02.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.249+0000 7f959d0d5700 1 --2- 172.21.15.1:0/398639401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9598107e10 0x7f959819c2b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f958c00c8f0 tx=0x7f958c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:02.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.249+0000 7f9596ffd700 1 -- 172.21.15.1:0/398639401 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c007940 con 0x7f9598107e10 2024-05-04T03:17:02.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.249+0000 7f9596ffd700 1 -- 172.21.15.1:0/398639401 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f958c00f450 con 0x7f9598107e10 2024-05-04T03:17:02.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.250+0000 7f959fb3a700 1 -- 172.21.15.1:0/398639401 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95981a7380 con 0x7f9598107e10 2024-05-04T03:17:02.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.250+0000 7f9596ffd700 1 -- 172.21.15.1:0/398639401 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c018660 con 0x7f9598107e10 2024-05-04T03:17:02.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.250+0000 7f959fb3a700 1 -- 172.21.15.1:0/398639401 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95981a7910 con 0x7f9598107e10 2024-05-04T03:17:02.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.250+0000 7f959fb3a700 1 -- 172.21.15.1:0/398639401 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9598063ed0 con 0x7f9598107e10 2024-05-04T03:17:02.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.252+0000 7f9596ffd700 1 -- 172.21.15.1:0/398639401 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f958c00f890 con 0x7f9598107e10 2024-05-04T03:17:02.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.253+0000 7f9596ffd700 1 --2- 172.21.15.1:0/398639401 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f95800b2990 0x7f95800b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:02.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.253+0000 7f959d8d6700 1 --2- 172.21.15.1:0/398639401 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f95800b2990 0x7f95800b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:02.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.253+0000 7f9596ffd700 1 -- 172.21.15.1:0/398639401 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f958c0e2600 con 0x7f9598107e10 2024-05-04T03:17:02.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.253+0000 7f9596ffd700 1 --2- 172.21.15.1:0/398639401 >> [v2:172.21.15.72:6808/2501732391,v1:172.21.15.72:6809/2501732391] conn(0x7f95800bc3a0 0x7f95800be890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:02.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.253+0000 7f9596ffd700 1 -- 172.21.15.1:0/398639401 --> [v2:172.21.15.72:6808/2501732391,v1:172.21.15.72:6809/2501732391] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f958c019ec0 con 0x7f95800bc3a0 2024-05-04T03:17:02.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.253+0000 7f9596ffd700 1 -- 172.21.15.1:0/398639401 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f958c0e29b0 con 0x7f9598107e10 2024-05-04T03:17:02.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.253+0000 7f959d8d6700 1 --2- 172.21.15.1:0/398639401 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f95800b2990 0x7f95800b4ea0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9590009990 tx=0x7f95900036e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:02.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.253+0000 7f959e0d7700 1 --2- 172.21.15.1:0/398639401 >> [v2:172.21.15.72:6808/2501732391,v1:172.21.15.72:6809/2501732391] conn(0x7f95800bc3a0 0x7f95800be890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:02.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.254+0000 7f959e0d7700 1 --2- 172.21.15.1:0/398639401 >> [v2:172.21.15.72:6808/2501732391,v1:172.21.15.72:6809/2501732391] conn(0x7f95800bc3a0 0x7f95800be890 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:02.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.255+0000 7f9596ffd700 1 -- 172.21.15.1:0/398639401 <== osd.4 v2:172.21.15.72:6808/2501732391 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f958c019ec0 con 0x7f95800bc3a0 2024-05-04T03:17:02.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.271+0000 7f959fb3a700 1 -- 172.21.15.1:0/398639401 --> [v2:172.21.15.72:6808/2501732391,v1:172.21.15.72:6809/2501732391] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f95980642d0 con 0x7f95800bc3a0 2024-05-04T03:17:02.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.272+0000 7f9596ffd700 1 -- 172.21.15.1:0/398639401 <== osd.4 v2:172.21.15.72:6808/2501732391 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f95980642d0 con 0x7f95800bc3a0 2024-05-04T03:17:02.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.272+0000 7f959fb3a700 1 -- 172.21.15.1:0/398639401 >> [v2:172.21.15.72:6808/2501732391,v1:172.21.15.72:6809/2501732391] conn(0x7f95800bc3a0 msgr2=0x7f95800be890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:02.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.272+0000 7f959fb3a700 1 --2- 172.21.15.1:0/398639401 >> [v2:172.21.15.72:6808/2501732391,v1:172.21.15.72:6809/2501732391] conn(0x7f95800bc3a0 0x7f95800be890 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.272+0000 7f959fb3a700 1 -- 172.21.15.1:0/398639401 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f95800b2990 msgr2=0x7f95800b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:02.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.272+0000 7f959fb3a700 1 --2- 172.21.15.1:0/398639401 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f95800b2990 0x7f95800b4ea0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9590009990 tx=0x7f95900036e0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.272+0000 7f959fb3a700 1 -- 172.21.15.1:0/398639401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9598107e10 msgr2=0x7f959819c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:02.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.272+0000 7f959fb3a700 1 --2- 172.21.15.1:0/398639401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9598107e10 0x7f959819c2b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f958c00c8f0 tx=0x7f958c00cc00 comp rx=0 tx=0).stop 2024-05-04T03:17:02.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.272+0000 7f959e0d7700 1 -- 172.21.15.1:0/398639401 reap_dead start 2024-05-04T03:17:02.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.272+0000 7f959fb3a700 1 -- 172.21.15.1:0/398639401 shutdown_connections 2024-05-04T03:17:02.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.272+0000 7f959fb3a700 1 -- 172.21.15.1:0/398639401 >> 172.21.15.1:0/398639401 conn(0x7f95980fef30 msgr2=0x7f9598101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:02.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.272+0000 7f959fb3a700 1 -- 172.21.15.1:0/398639401 shutdown_connections 2024-05-04T03:17:02.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.272+0000 7f959fb3a700 1 -- 172.21.15.1:0/398639401 wait complete. 2024-05-04T03:17:02.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.127+0000 7f2767ce7700 1 -- 172.21.15.1:0/2086250424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27601034c0 msgr2=0x7f27601038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:02.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.127+0000 7f2767ce7700 1 --2- 172.21.15.1:0/2086250424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27601034c0 0x7f27601038d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f275c0099c0 tx=0x7f275c009cd0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.127+0000 7f2767ce7700 1 -- 172.21.15.1:0/2086250424 shutdown_connections 2024-05-04T03:17:02.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.127+0000 7f2767ce7700 1 --2- 172.21.15.1:0/2086250424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2760108740 0x7f27601100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.127+0000 7f2767ce7700 1 --2- 172.21.15.1:0/2086250424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27601034c0 0x7f27601038d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.127+0000 7f2767ce7700 1 --2- 172.21.15.1:0/2086250424 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2760107df0 0x7f2760108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.127+0000 7f2767ce7700 1 -- 172.21.15.1:0/2086250424 >> 172.21.15.1:0/2086250424 conn(0x7f27600fedc0 msgr2=0x7f27601011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:02.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.127+0000 7f2767ce7700 1 -- 172.21.15.1:0/2086250424 shutdown_connections 2024-05-04T03:17:02.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.127+0000 7f2767ce7700 1 -- 172.21.15.1:0/2086250424 wait complete. 2024-05-04T03:17:02.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.128+0000 7f2767ce7700 1 Processor -- start 2024-05-04T03:17:02.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.128+0000 7f2767ce7700 1 -- start start 2024-05-04T03:17:02.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.128+0000 7f2767ce7700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27601034c0 0x7f276019ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:02.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.128+0000 7f2767ce7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2760107df0 0x7f276019b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:02.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.128+0000 7f2767ce7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2760108740 0x7f276019f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:02.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.128+0000 7f2767ce7700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2760113370 con 0x7f27601034c0 2024-05-04T03:17:02.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.128+0000 7f2767ce7700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f27601131f0 con 0x7f2760107df0 2024-05-04T03:17:02.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.128+0000 7f2767ce7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f27601134f0 con 0x7f2760108740 2024-05-04T03:17:02.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.128+0000 7f2765a83700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27601034c0 0x7f276019ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:02.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.128+0000 7f2765a83700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27601034c0 0x7f276019ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46932/0 (socket says 172.21.15.1:46932) 2024-05-04T03:17:02.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.128+0000 7f2765a83700 1 -- 172.21.15.1:0/1013536803 learned_addr learned my addr 172.21.15.1:0/1013536803 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:02.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.128+0000 7f2766284700 1 --2- 172.21.15.1:0/1013536803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2760108740 0x7f276019f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:02.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.128+0000 7f2765282700 1 --2- 172.21.15.1:0/1013536803 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2760107df0 0x7f276019b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:02.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.128+0000 7f2765a83700 1 -- 172.21.15.1:0/1013536803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2760108740 msgr2=0x7f276019f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:02.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.128+0000 7f2765a83700 1 --2- 172.21.15.1:0/1013536803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2760108740 0x7f276019f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.128+0000 7f2765a83700 1 -- 172.21.15.1:0/1013536803 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2760107df0 msgr2=0x7f276019b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:02.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.128+0000 7f2765a83700 1 --2- 172.21.15.1:0/1013536803 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2760107df0 0x7f276019b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.128+0000 7f2765a83700 1 -- 172.21.15.1:0/1013536803 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276019fc70 con 0x7f27601034c0 2024-05-04T03:17:02.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.128+0000 7f2766284700 1 --2- 172.21.15.1:0/1013536803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2760108740 0x7f276019f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:17:02.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.128+0000 7f2765282700 1 --2- 172.21.15.1:0/1013536803 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2760107df0 0x7f276019b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:17:02.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.129+0000 7f2765a83700 1 --2- 172.21.15.1:0/1013536803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27601034c0 0x7f276019ae50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f275c0098e0 tx=0x7f275c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:02.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.129+0000 7f2756ffd700 1 -- 172.21.15.1:0/1013536803 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c027070 con 0x7f27601034c0 2024-05-04T03:17:02.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.129+0000 7f2756ffd700 1 -- 172.21.15.1:0/1013536803 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f275c003fa0 con 0x7f27601034c0 2024-05-04T03:17:02.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.129+0000 7f2767ce7700 1 -- 172.21.15.1:0/1013536803 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f276019ff00 con 0x7f27601034c0 2024-05-04T03:17:02.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.129+0000 7f2756ffd700 1 -- 172.21.15.1:0/1013536803 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c022d40 con 0x7f27601034c0 2024-05-04T03:17:02.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.129+0000 7f2767ce7700 1 -- 172.21.15.1:0/1013536803 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27601a0430 con 0x7f27601034c0 2024-05-04T03:17:02.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.130+0000 7f2767ce7700 1 -- 172.21.15.1:0/1013536803 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2760107870 con 0x7f27601034c0 2024-05-04T03:17:02.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.132+0000 7f2756ffd700 1 -- 172.21.15.1:0/1013536803 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f275c02a070 con 0x7f27601034c0 2024-05-04T03:17:02.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.133+0000 7f2756ffd700 1 --2- 172.21.15.1:0/1013536803 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f274c0b2990 0x7f274c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:02.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.133+0000 7f2765282700 1 --2- 172.21.15.1:0/1013536803 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f274c0b2990 0x7f274c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:02.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.133+0000 7f2756ffd700 1 -- 172.21.15.1:0/1013536803 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f275c0e3a70 con 0x7f27601034c0 2024-05-04T03:17:02.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.134+0000 7f2756ffd700 1 --2- 172.21.15.1:0/1013536803 >> [v2:172.21.15.72:6800/3725706149,v1:172.21.15.72:6801/3725706149] conn(0x7f274c0bc3a0 0x7f274c0be890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:02.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.134+0000 7f2756ffd700 1 -- 172.21.15.1:0/1013536803 --> [v2:172.21.15.72:6800/3725706149,v1:172.21.15.72:6801/3725706149] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f274c0bef60 con 0x7f274c0bc3a0 2024-05-04T03:17:02.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.134+0000 7f2756ffd700 1 -- 172.21.15.1:0/1013536803 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f275c0e3f50 con 0x7f27601034c0 2024-05-04T03:17:02.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.134+0000 7f2765282700 1 --2- 172.21.15.1:0/1013536803 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f274c0b2990 0x7f274c0b4ea0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2750005950 tx=0x7f27500058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:02.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.134+0000 7f2766284700 1 --2- 172.21.15.1:0/1013536803 >> [v2:172.21.15.72:6800/3725706149,v1:172.21.15.72:6801/3725706149] conn(0x7f274c0bc3a0 0x7f274c0be890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:02.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.134+0000 7f2766284700 1 --2- 172.21.15.1:0/1013536803 >> [v2:172.21.15.72:6800/3725706149,v1:172.21.15.72:6801/3725706149] conn(0x7f274c0bc3a0 0x7f274c0be890 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:02.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.135+0000 7f2756ffd700 1 -- 172.21.15.1:0/1013536803 <== osd.1 v2:172.21.15.72:6800/3725706149 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f274c0bef60 con 0x7f274c0bc3a0 2024-05-04T03:17:02.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.152+0000 7f2767ce7700 1 -- 172.21.15.1:0/1013536803 --> [v2:172.21.15.72:6800/3725706149,v1:172.21.15.72:6801/3725706149] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2760104970 con 0x7f274c0bc3a0 2024-05-04T03:17:02.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.153+0000 7f2756ffd700 1 -- 172.21.15.1:0/1013536803 <== osd.1 v2:172.21.15.72:6800/3725706149 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2760104970 con 0x7f274c0bc3a0 2024-05-04T03:17:02.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.153+0000 7f2767ce7700 1 -- 172.21.15.1:0/1013536803 >> [v2:172.21.15.72:6800/3725706149,v1:172.21.15.72:6801/3725706149] conn(0x7f274c0bc3a0 msgr2=0x7f274c0be890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:02.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.153+0000 7f2767ce7700 1 --2- 172.21.15.1:0/1013536803 >> [v2:172.21.15.72:6800/3725706149,v1:172.21.15.72:6801/3725706149] conn(0x7f274c0bc3a0 0x7f274c0be890 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.153+0000 7f2767ce7700 1 -- 172.21.15.1:0/1013536803 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f274c0b2990 msgr2=0x7f274c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:02.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.153+0000 7f2767ce7700 1 --2- 172.21.15.1:0/1013536803 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f274c0b2990 0x7f274c0b4ea0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2750005950 tx=0x7f27500058e0 comp rx=0 tx=0).stop 2024-05-04T03:17:02.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.153+0000 7f2767ce7700 1 -- 172.21.15.1:0/1013536803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27601034c0 msgr2=0x7f276019ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:02.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.153+0000 7f2767ce7700 1 --2- 172.21.15.1:0/1013536803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27601034c0 0x7f276019ae50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f275c0098e0 tx=0x7f275c005e80 comp rx=0 tx=0).stop 2024-05-04T03:17:02.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.153+0000 7f2766284700 1 -- 172.21.15.1:0/1013536803 reap_dead start 2024-05-04T03:17:02.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.153+0000 7f2767ce7700 1 -- 172.21.15.1:0/1013536803 shutdown_connections 2024-05-04T03:17:02.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.153+0000 7f2767ce7700 1 -- 172.21.15.1:0/1013536803 >> 172.21.15.1:0/1013536803 conn(0x7f27600fedc0 msgr2=0x7f27601011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:02.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.153+0000 7f2767ce7700 1 -- 172.21.15.1:0/1013536803 shutdown_connections 2024-05-04T03:17:02.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:02.153+0000 7f2767ce7700 1 -- 172.21.15.1:0/1013536803 wait complete. 2024-05-04T03:17:02.819 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:02 smithi001 ceph-mon[29731]: pgmap v327: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:17:02.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:02 smithi077 ceph-mon[33531]: pgmap v327: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:17:02.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:02 smithi072 ceph-mon[33332]: pgmap v327: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:17:03.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.314+0000 7f36dd04e700 1 -- 172.21.15.1:0/1229252133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36d8108740 msgr2=0x7f36d81100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:03.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.314+0000 7f36dd04e700 1 --2- 172.21.15.1:0/1229252133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36d8108740 0x7f36d81100a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f36d000b0d0 tx=0x7f36d000b3e0 comp rx=0 tx=0).stop 2024-05-04T03:17:03.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.314+0000 7f36dd04e700 1 -- 172.21.15.1:0/1229252133 shutdown_connections 2024-05-04T03:17:03.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.314+0000 7f36dd04e700 1 --2- 172.21.15.1:0/1229252133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36d8108740 0x7f36d81100a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:03.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.314+0000 7f36dd04e700 1 --2- 172.21.15.1:0/1229252133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36d81034c0 0x7f36d81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:03.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.314+0000 7f36dd04e700 1 --2- 172.21.15.1:0/1229252133 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36d8107df0 0x7f36d8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:03.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.314+0000 7f36dd04e700 1 -- 172.21.15.1:0/1229252133 >> 172.21.15.1:0/1229252133 conn(0x7f36d80fedc0 msgr2=0x7f36d81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:03.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.315+0000 7f36dd04e700 1 -- 172.21.15.1:0/1229252133 shutdown_connections 2024-05-04T03:17:03.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.315+0000 7f36dd04e700 1 -- 172.21.15.1:0/1229252133 wait complete. 2024-05-04T03:17:03.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.315+0000 7f36dd04e700 1 Processor -- start 2024-05-04T03:17:03.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.316+0000 7f36dd04e700 1 -- start start 2024-05-04T03:17:03.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.316+0000 7f36dd04e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36d81034c0 0x7f36d819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:03.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.316+0000 7f36dd04e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36d8107df0 0x7f36d819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:03.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.316+0000 7f36dd04e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36d8108740 0x7f36d819f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:03.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.316+0000 7f36dd04e700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f36d8113370 con 0x7f36d81034c0 2024-05-04T03:17:03.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.316+0000 7f36dd04e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f36d81131f0 con 0x7f36d8108740 2024-05-04T03:17:03.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.316+0000 7f36dd04e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f36d81134f0 con 0x7f36d8107df0 2024-05-04T03:17:03.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.316+0000 7f36d6d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36d81034c0 0x7f36d819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:03.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.316+0000 7f36d6d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36d81034c0 0x7f36d819add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46972/0 (socket says 172.21.15.1:46972) 2024-05-04T03:17:03.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.316+0000 7f36d6d9d700 1 -- 172.21.15.1:0/1509118998 learned_addr learned my addr 172.21.15.1:0/1509118998 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:03.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.316+0000 7f36d759e700 1 --2- 172.21.15.1:0/1509118998 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36d8108740 0x7f36d819f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:03.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.316+0000 7f36d659c700 1 --2- 172.21.15.1:0/1509118998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36d8107df0 0x7f36d819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:03.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.316+0000 7f36d6d9d700 1 -- 172.21.15.1:0/1509118998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36d8107df0 msgr2=0x7f36d819b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:03.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.316+0000 7f36d6d9d700 1 --2- 172.21.15.1:0/1509118998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f36d8107df0 0x7f36d819b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:03.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.316+0000 7f36d6d9d700 1 -- 172.21.15.1:0/1509118998 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36d8108740 msgr2=0x7f36d819f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:03.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.316+0000 7f36d6d9d700 1 --2- 172.21.15.1:0/1509118998 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36d8108740 0x7f36d819f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:03.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.316+0000 7f36d6d9d700 1 -- 172.21.15.1:0/1509118998 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d819fc50 con 0x7f36d81034c0 2024-05-04T03:17:03.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.317+0000 7f36d6d9d700 1 --2- 172.21.15.1:0/1509118998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36d81034c0 0x7f36d819add0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f36c800d950 tx=0x7f36c800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:03.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.317+0000 7f36bffff700 1 -- 172.21.15.1:0/1509118998 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c8016070 con 0x7f36d81034c0 2024-05-04T03:17:03.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.317+0000 7f36dd04e700 1 -- 172.21.15.1:0/1509118998 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36d819ff40 con 0x7f36d81034c0 2024-05-04T03:17:03.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.317+0000 7f36dd04e700 1 -- 172.21.15.1:0/1509118998 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36d81a7860 con 0x7f36d81034c0 2024-05-04T03:17:03.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.317+0000 7f36bffff700 1 -- 172.21.15.1:0/1509118998 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36c8009df0 con 0x7f36d81034c0 2024-05-04T03:17:03.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.317+0000 7f36bffff700 1 -- 172.21.15.1:0/1509118998 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c800f3c0 con 0x7f36d81034c0 2024-05-04T03:17:03.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.317+0000 7f36dd04e700 1 -- 172.21.15.1:0/1509118998 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f36d8063f50 con 0x7f36d81034c0 2024-05-04T03:17:03.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.319+0000 7f36bffff700 1 -- 172.21.15.1:0/1509118998 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f36c8019070 con 0x7f36d81034c0 2024-05-04T03:17:03.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.320+0000 7f36bffff700 1 --2- 172.21.15.1:0/1509118998 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f36c00b2910 0x7f36c00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:03.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.320+0000 7f36d659c700 1 --2- 172.21.15.1:0/1509118998 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f36c00b2910 0x7f36c00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:03.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.320+0000 7f36bffff700 1 -- 172.21.15.1:0/1509118998 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f36c80e2430 con 0x7f36d81034c0 2024-05-04T03:17:03.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.320+0000 7f36bffff700 1 --2- 172.21.15.1:0/1509118998 >> [v2:172.21.15.77:6800/515038343,v1:172.21.15.77:6801/515038343] conn(0x7f36c00bc320 0x7f36c00be810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:03.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.320+0000 7f36bffff700 1 -- 172.21.15.1:0/1509118998 --> [v2:172.21.15.77:6800/515038343,v1:172.21.15.77:6801/515038343] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f36c00beee0 con 0x7f36c00bc320 2024-05-04T03:17:03.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.320+0000 7f36bffff700 1 -- 172.21.15.1:0/1509118998 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f36c80e2910 con 0x7f36d81034c0 2024-05-04T03:17:03.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.321+0000 7f36d659c700 1 --2- 172.21.15.1:0/1509118998 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f36c00b2910 0x7f36c00b4e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f36cc005950 tx=0x7f36cc0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:03.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.321+0000 7f36d759e700 1 --2- 172.21.15.1:0/1509118998 >> [v2:172.21.15.77:6800/515038343,v1:172.21.15.77:6801/515038343] conn(0x7f36c00bc320 0x7f36c00be810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:03.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.321+0000 7f36d759e700 1 --2- 172.21.15.1:0/1509118998 >> [v2:172.21.15.77:6800/515038343,v1:172.21.15.77:6801/515038343] conn(0x7f36c00bc320 0x7f36c00be810 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:03.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.322+0000 7f36bffff700 1 -- 172.21.15.1:0/1509118998 <== osd.2 v2:172.21.15.77:6800/515038343 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f36c00beee0 con 0x7f36c00bc320 2024-05-04T03:17:03.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.338+0000 7f36dd04e700 1 -- 172.21.15.1:0/1509118998 --> [v2:172.21.15.77:6800/515038343,v1:172.21.15.77:6801/515038343] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f36d8064350 con 0x7f36c00bc320 2024-05-04T03:17:03.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.339+0000 7f36bffff700 1 -- 172.21.15.1:0/1509118998 <== osd.2 v2:172.21.15.77:6800/515038343 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f36d8064350 con 0x7f36c00bc320 2024-05-04T03:17:03.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.340+0000 7f36dd04e700 1 -- 172.21.15.1:0/1509118998 >> [v2:172.21.15.77:6800/515038343,v1:172.21.15.77:6801/515038343] conn(0x7f36c00bc320 msgr2=0x7f36c00be810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:03.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.340+0000 7f36dd04e700 1 --2- 172.21.15.1:0/1509118998 >> [v2:172.21.15.77:6800/515038343,v1:172.21.15.77:6801/515038343] conn(0x7f36c00bc320 0x7f36c00be810 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:03.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.340+0000 7f36dd04e700 1 -- 172.21.15.1:0/1509118998 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f36c00b2910 msgr2=0x7f36c00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:03.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.340+0000 7f36dd04e700 1 --2- 172.21.15.1:0/1509118998 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f36c00b2910 0x7f36c00b4e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f36cc005950 tx=0x7f36cc0058e0 comp rx=0 tx=0).stop 2024-05-04T03:17:03.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.340+0000 7f36dd04e700 1 -- 172.21.15.1:0/1509118998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36d81034c0 msgr2=0x7f36d819add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:03.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.340+0000 7f36dd04e700 1 --2- 172.21.15.1:0/1509118998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36d81034c0 0x7f36d819add0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f36c800d950 tx=0x7f36c800dc60 comp rx=0 tx=0).stop 2024-05-04T03:17:03.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.340+0000 7f36d759e700 1 -- 172.21.15.1:0/1509118998 reap_dead start 2024-05-04T03:17:03.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.340+0000 7f36dd04e700 1 -- 172.21.15.1:0/1509118998 shutdown_connections 2024-05-04T03:17:03.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.340+0000 7f36dd04e700 1 -- 172.21.15.1:0/1509118998 >> 172.21.15.1:0/1509118998 conn(0x7f36d80fedc0 msgr2=0x7f36d806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:03.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.340+0000 7f36dd04e700 1 -- 172.21.15.1:0/1509118998 shutdown_connections 2024-05-04T03:17:03.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.340+0000 7f36dd04e700 1 -- 172.21.15.1:0/1509118998 wait complete. 2024-05-04T03:17:03.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.907+0000 7f3dfc2ae700 1 -- 172.21.15.1:0/10886966 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3df4105590 msgr2=0x7f3df41059a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:03.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.907+0000 7f3dfc2ae700 1 --2- 172.21.15.1:0/10886966 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3df4105590 0x7f3df41059a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3df0009990 tx=0x7f3df0009ca0 comp rx=0 tx=0).stop 2024-05-04T03:17:03.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.907+0000 7f3dfc2ae700 1 -- 172.21.15.1:0/10886966 shutdown_connections 2024-05-04T03:17:03.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.907+0000 7f3dfc2ae700 1 --2- 172.21.15.1:0/10886966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3df410e970 0x7f3df4110e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:03.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.907+0000 7f3dfc2ae700 1 --2- 172.21.15.1:0/10886966 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3df4105590 0x7f3df41059a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:03.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.907+0000 7f3dfc2ae700 1 --2- 172.21.15.1:0/10886966 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3df4105ee0 0x7f3df410e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:03.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.907+0000 7f3dfc2ae700 1 -- 172.21.15.1:0/10886966 >> 172.21.15.1:0/10886966 conn(0x7f3df4078e00 msgr2=0x7f3df4079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:03.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.907+0000 7f3dfc2ae700 1 -- 172.21.15.1:0/10886966 shutdown_connections 2024-05-04T03:17:03.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.907+0000 7f3dfc2ae700 1 -- 172.21.15.1:0/10886966 wait complete. 2024-05-04T03:17:03.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.908+0000 7f3dfc2ae700 1 Processor -- start 2024-05-04T03:17:03.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.908+0000 7f3dfc2ae700 1 -- start start 2024-05-04T03:17:03.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.908+0000 7f3dfc2ae700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3df4105590 0x7f3df419ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:03.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.908+0000 7f3dfc2ae700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3df4105ee0 0x7f3df419b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:03.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.908+0000 7f3dfc2ae700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3df410e970 0x7f3df419f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:03.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.908+0000 7f3dfc2ae700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3df4114500 con 0x7f3df4105ee0 2024-05-04T03:17:03.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.908+0000 7f3dfc2ae700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3df4114380 con 0x7f3df4105590 2024-05-04T03:17:03.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.908+0000 7f3dfc2ae700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3df4114680 con 0x7f3df410e970 2024-05-04T03:17:03.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.908+0000 7f3df9849700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3df4105ee0 0x7f3df419b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:03.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.908+0000 7f3df9849700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3df4105ee0 0x7f3df419b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47002/0 (socket says 172.21.15.1:47002) 2024-05-04T03:17:03.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.908+0000 7f3df9849700 1 -- 172.21.15.1:0/1026031975 learned_addr learned my addr 172.21.15.1:0/1026031975 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:03.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.908+0000 7f3dfa04a700 1 --2- 172.21.15.1:0/1026031975 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3df4105590 0x7f3df419ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:03.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.908+0000 7f3dfa84b700 1 --2- 172.21.15.1:0/1026031975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3df410e970 0x7f3df419f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:03.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.909+0000 7f3df9849700 1 -- 172.21.15.1:0/1026031975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3df410e970 msgr2=0x7f3df419f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:03.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.909+0000 7f3df9849700 1 --2- 172.21.15.1:0/1026031975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3df410e970 0x7f3df419f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:03.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.909+0000 7f3df9849700 1 -- 172.21.15.1:0/1026031975 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3df4105590 msgr2=0x7f3df419ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:03.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.909+0000 7f3df9849700 1 --2- 172.21.15.1:0/1026031975 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3df4105590 0x7f3df419ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:03.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.909+0000 7f3df9849700 1 -- 172.21.15.1:0/1026031975 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df419fd20 con 0x7f3df4105ee0 2024-05-04T03:17:03.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.909+0000 7f3dfa84b700 1 --2- 172.21.15.1:0/1026031975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3df410e970 0x7f3df419f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:17:03.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.909+0000 7f3dfa04a700 1 --2- 172.21.15.1:0/1026031975 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3df4105590 0x7f3df419ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:17:03.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.909+0000 7f3df9849700 1 --2- 172.21.15.1:0/1026031975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3df4105ee0 0x7f3df419b3f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3de400b530 tx=0x7f3de400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:03.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.909+0000 7f3deb7fe700 1 -- 172.21.15.1:0/1026031975 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de4015020 con 0x7f3df4105ee0 2024-05-04T03:17:03.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.909+0000 7f3dfc2ae700 1 -- 172.21.15.1:0/1026031975 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df41a0010 con 0x7f3df4105ee0 2024-05-04T03:17:03.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.909+0000 7f3deb7fe700 1 -- 172.21.15.1:0/1026031975 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3de4011ce0 con 0x7f3df4105ee0 2024-05-04T03:17:03.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.909+0000 7f3deb7fe700 1 -- 172.21.15.1:0/1026031975 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de4010d20 con 0x7f3df4105ee0 2024-05-04T03:17:03.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.909+0000 7f3dfc2ae700 1 -- 172.21.15.1:0/1026031975 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df41a7930 con 0x7f3df4105ee0 2024-05-04T03:17:03.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.910+0000 7f3dfc2ae700 1 -- 172.21.15.1:0/1026031975 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3df4063f50 con 0x7f3df4105ee0 2024-05-04T03:17:03.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.913+0000 7f3deb7fe700 1 -- 172.21.15.1:0/1026031975 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f3de4018020 con 0x7f3df4105ee0 2024-05-04T03:17:03.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.914+0000 7f3deb7fe700 1 --2- 172.21.15.1:0/1026031975 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f3de00b2910 0x7f3de00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:03.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.914+0000 7f3dfa04a700 1 --2- 172.21.15.1:0/1026031975 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f3de00b2910 0x7f3de00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:03.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.914+0000 7f3deb7fe700 1 -- 172.21.15.1:0/1026031975 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f3de40e21c0 con 0x7f3df4105ee0 2024-05-04T03:17:03.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.914+0000 7f3deb7fe700 1 --2- 172.21.15.1:0/1026031975 >> [v2:172.21.15.72:6824/3293703569,v1:172.21.15.72:6825/3293703569] conn(0x7f3de00bc320 0x7f3de00be810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:03.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.914+0000 7f3deb7fe700 1 -- 172.21.15.1:0/1026031975 --> [v2:172.21.15.72:6824/3293703569,v1:172.21.15.72:6825/3293703569] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3de401be50 con 0x7f3de00bc320 2024-05-04T03:17:03.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.914+0000 7f3deb7fe700 1 -- 172.21.15.1:0/1026031975 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f3de40e2570 con 0x7f3df4105ee0 2024-05-04T03:17:03.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.914+0000 7f3dfa04a700 1 --2- 172.21.15.1:0/1026031975 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f3de00b2910 0x7f3de00b4e20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3df0003800 tx=0x7f3df001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:03.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.914+0000 7f3dfa84b700 1 --2- 172.21.15.1:0/1026031975 >> [v2:172.21.15.72:6824/3293703569,v1:172.21.15.72:6825/3293703569] conn(0x7f3de00bc320 0x7f3de00be810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:03.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.915+0000 7f3dfa84b700 1 --2- 172.21.15.1:0/1026031975 >> [v2:172.21.15.72:6824/3293703569,v1:172.21.15.72:6825/3293703569] conn(0x7f3de00bc320 0x7f3de00be810 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:03.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.916+0000 7f3deb7fe700 1 -- 172.21.15.1:0/1026031975 <== osd.10 v2:172.21.15.72:6824/3293703569 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3de401be50 con 0x7f3de00bc320 2024-05-04T03:17:03.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.932+0000 7f3dfc2ae700 1 -- 172.21.15.1:0/1026031975 --> [v2:172.21.15.72:6824/3293703569,v1:172.21.15.72:6825/3293703569] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3df4064350 con 0x7f3de00bc320 2024-05-04T03:17:03.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.933+0000 7f3deb7fe700 1 -- 172.21.15.1:0/1026031975 <== osd.10 v2:172.21.15.72:6824/3293703569 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3df4064350 con 0x7f3de00bc320 2024-05-04T03:17:03.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.934+0000 7f3dfc2ae700 1 -- 172.21.15.1:0/1026031975 >> [v2:172.21.15.72:6824/3293703569,v1:172.21.15.72:6825/3293703569] conn(0x7f3de00bc320 msgr2=0x7f3de00be810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:03.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.934+0000 7f3dfc2ae700 1 --2- 172.21.15.1:0/1026031975 >> [v2:172.21.15.72:6824/3293703569,v1:172.21.15.72:6825/3293703569] conn(0x7f3de00bc320 0x7f3de00be810 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:03.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.934+0000 7f3dfc2ae700 1 -- 172.21.15.1:0/1026031975 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f3de00b2910 msgr2=0x7f3de00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:03.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.934+0000 7f3dfc2ae700 1 --2- 172.21.15.1:0/1026031975 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f3de00b2910 0x7f3de00b4e20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3df0003800 tx=0x7f3df001a040 comp rx=0 tx=0).stop 2024-05-04T03:17:03.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.934+0000 7f3dfc2ae700 1 -- 172.21.15.1:0/1026031975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3df4105ee0 msgr2=0x7f3df419b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:03.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.934+0000 7f3dfc2ae700 1 --2- 172.21.15.1:0/1026031975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3df4105ee0 0x7f3df419b3f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3de400b530 tx=0x7f3de400b8f0 comp rx=0 tx=0).stop 2024-05-04T03:17:03.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.934+0000 7f3dfa84b700 1 -- 172.21.15.1:0/1026031975 reap_dead start 2024-05-04T03:17:03.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.934+0000 7f3dfc2ae700 1 -- 172.21.15.1:0/1026031975 shutdown_connections 2024-05-04T03:17:03.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.934+0000 7f3dfc2ae700 1 -- 172.21.15.1:0/1026031975 >> 172.21.15.1:0/1026031975 conn(0x7f3df4078e00 msgr2=0x7f3df4100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:03.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.934+0000 7f3dfc2ae700 1 -- 172.21.15.1:0/1026031975 shutdown_connections 2024-05-04T03:17:03.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:03.934+0000 7f3dfc2ae700 1 -- 172.21.15.1:0/1026031975 wait complete. 2024-05-04T03:17:04.492 INFO:teuthology.orchestra.run.smithi001.stdout:77309411405 2024-05-04T03:17:04.492 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-04T03:17:04.749 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:04 smithi001 ceph-mon[29731]: pgmap v328: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:17:04.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.864+0000 7f0a4cf9f700 1 -- 172.21.15.1:0/170892597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a4810e740 msgr2=0x7f0a48110be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:04.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.864+0000 7f0a4cf9f700 1 --2- 172.21.15.1:0/170892597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a4810e740 0x7f0a48110be0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000b0a0 tx=0x7f0a4000b3b0 comp rx=0 tx=0).stop 2024-05-04T03:17:04.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.864+0000 7f0a4cf9f700 1 -- 172.21.15.1:0/170892597 shutdown_connections 2024-05-04T03:17:04.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.864+0000 7f0a4cf9f700 1 --2- 172.21.15.1:0/170892597 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a48105310 0x7f0a48105720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:04.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.864+0000 7f0a4cf9f700 1 --2- 172.21.15.1:0/170892597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a4810e740 0x7f0a48110be0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:04.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.864+0000 7f0a4cf9f700 1 --2- 172.21.15.1:0/170892597 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0a48105c60 0x7f0a4810e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:04.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.864+0000 7f0a4cf9f700 1 -- 172.21.15.1:0/170892597 >> 172.21.15.1:0/170892597 conn(0x7f0a48078ac0 msgr2=0x7f0a48078f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:04.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.864+0000 7f0a4cf9f700 1 -- 172.21.15.1:0/170892597 shutdown_connections 2024-05-04T03:17:04.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.864+0000 7f0a4cf9f700 1 -- 172.21.15.1:0/170892597 wait complete. 2024-05-04T03:17:04.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.865+0000 7f0a4cf9f700 1 Processor -- start 2024-05-04T03:17:04.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.865+0000 7f0a4cf9f700 1 -- start start 2024-05-04T03:17:04.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.866+0000 7f0a4cf9f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0a48105310 0x7f0a481a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:04.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.866+0000 7f0a4cf9f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a48105c60 0x7f0a481a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:04.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.866+0000 7f0a4cf9f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a4810e740 0x7f0a4819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:04.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.866+0000 7f0a4cf9f700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0a48114260 con 0x7f0a4810e740 2024-05-04T03:17:04.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.866+0000 7f0a4cf9f700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0a481140e0 con 0x7f0a48105c60 2024-05-04T03:17:04.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.866+0000 7f0a4cf9f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0a481143e0 con 0x7f0a48105310 2024-05-04T03:17:04.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.866+0000 7f0a47fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a4810e740 0x7f0a4819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:04.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.866+0000 7f0a47fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a4810e740 0x7f0a4819a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47006/0 (socket says 172.21.15.1:47006) 2024-05-04T03:17:04.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.866+0000 7f0a47fff700 1 -- 172.21.15.1:0/2134731431 learned_addr learned my addr 172.21.15.1:0/2134731431 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:04.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.866+0000 7f0a3ffff700 1 --2- 172.21.15.1:0/2134731431 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a48105c60 0x7f0a481a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:04.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.866+0000 7f0a477fe700 1 --2- 172.21.15.1:0/2134731431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0a48105310 0x7f0a481a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:04.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.866+0000 7f0a47fff700 1 -- 172.21.15.1:0/2134731431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0a48105310 msgr2=0x7f0a481a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:04.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.866+0000 7f0a47fff700 1 --2- 172.21.15.1:0/2134731431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0a48105310 0x7f0a481a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:04.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.866+0000 7f0a47fff700 1 -- 172.21.15.1:0/2134731431 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a48105c60 msgr2=0x7f0a481a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:04.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.866+0000 7f0a47fff700 1 --2- 172.21.15.1:0/2134731431 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0a48105c60 0x7f0a481a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:04.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.866+0000 7f0a47fff700 1 -- 172.21.15.1:0/2134731431 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4819b020 con 0x7f0a4810e740 2024-05-04T03:17:04.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.866+0000 7f0a477fe700 1 --2- 172.21.15.1:0/2134731431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0a48105310 0x7f0a481a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:17:04.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.866+0000 7f0a47fff700 1 --2- 172.21.15.1:0/2134731431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a4810e740 0x7f0a4819a9e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0a400062a0 tx=0x7f0a40004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:04.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.866+0000 7f0a457fa700 1 -- 172.21.15.1:0/2134731431 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40021070 con 0x7f0a4810e740 2024-05-04T03:17:04.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.866+0000 7f0a4cf9f700 1 -- 172.21.15.1:0/2134731431 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a4819b2b0 con 0x7f0a4810e740 2024-05-04T03:17:04.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.866+0000 7f0a457fa700 1 -- 172.21.15.1:0/2134731431 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a40007c80 con 0x7f0a4810e740 2024-05-04T03:17:04.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.866+0000 7f0a457fa700 1 -- 172.21.15.1:0/2134731431 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4001d450 con 0x7f0a4810e740 2024-05-04T03:17:04.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.867+0000 7f0a4cf9f700 1 -- 172.21.15.1:0/2134731431 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a4819b7f0 con 0x7f0a4810e740 2024-05-04T03:17:04.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.867+0000 7f0a4cf9f700 1 -- 172.21.15.1:0/2134731431 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0a48063f50 con 0x7f0a4810e740 2024-05-04T03:17:04.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.869+0000 7f0a457fa700 1 -- 172.21.15.1:0/2134731431 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f0a40004020 con 0x7f0a4810e740 2024-05-04T03:17:04.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.870+0000 7f0a457fa700 1 --2- 172.21.15.1:0/2134731431 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0a340b2910 0x7f0a340b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:04.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.870+0000 7f0a477fe700 1 --2- 172.21.15.1:0/2134731431 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0a340b2910 0x7f0a340b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:04.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.870+0000 7f0a457fa700 1 -- 172.21.15.1:0/2134731431 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f0a4000f020 con 0x7f0a4810e740 2024-05-04T03:17:04.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.870+0000 7f0a457fa700 1 --2- 172.21.15.1:0/2134731431 >> [v2:172.21.15.77:6824/4155947548,v1:172.21.15.77:6825/4155947548] conn(0x7f0a340bc1f0 0x7f0a340be6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:04.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.870+0000 7f0a457fa700 1 -- 172.21.15.1:0/2134731431 --> [v2:172.21.15.77:6824/4155947548,v1:172.21.15.77:6825/4155947548] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0a40032ea0 con 0x7f0a340bc1f0 2024-05-04T03:17:04.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.870+0000 7f0a457fa700 1 -- 172.21.15.1:0/2134731431 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f0a400e42a0 con 0x7f0a4810e740 2024-05-04T03:17:04.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.870+0000 7f0a477fe700 1 --2- 172.21.15.1:0/2134731431 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0a340b2910 0x7f0a340b4e20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0a38005fd0 tx=0x7f0a38005e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:04.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.871+0000 7f0a3ffff700 1 --2- 172.21.15.1:0/2134731431 >> [v2:172.21.15.77:6824/4155947548,v1:172.21.15.77:6825/4155947548] conn(0x7f0a340bc1f0 0x7f0a340be6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:04.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.871+0000 7f0a3ffff700 1 --2- 172.21.15.1:0/2134731431 >> [v2:172.21.15.77:6824/4155947548,v1:172.21.15.77:6825/4155947548] conn(0x7f0a340bc1f0 0x7f0a340be6e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:04.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.872+0000 7f0a457fa700 1 -- 172.21.15.1:0/2134731431 <== osd.11 v2:172.21.15.77:6824/4155947548 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0a40032ea0 con 0x7f0a340bc1f0 2024-05-04T03:17:04.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.888+0000 7f0a4cf9f700 1 -- 172.21.15.1:0/2134731431 --> [v2:172.21.15.77:6824/4155947548,v1:172.21.15.77:6825/4155947548] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0a480643d0 con 0x7f0a340bc1f0 2024-05-04T03:17:04.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.889+0000 7f0a457fa700 1 -- 172.21.15.1:0/2134731431 <== osd.11 v2:172.21.15.77:6824/4155947548 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0a480643d0 con 0x7f0a340bc1f0 2024-05-04T03:17:04.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.889+0000 7f0a4cf9f700 1 -- 172.21.15.1:0/2134731431 >> [v2:172.21.15.77:6824/4155947548,v1:172.21.15.77:6825/4155947548] conn(0x7f0a340bc1f0 msgr2=0x7f0a340be6e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:04.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.889+0000 7f0a4cf9f700 1 --2- 172.21.15.1:0/2134731431 >> [v2:172.21.15.77:6824/4155947548,v1:172.21.15.77:6825/4155947548] conn(0x7f0a340bc1f0 0x7f0a340be6e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:04.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.889+0000 7f0a4cf9f700 1 -- 172.21.15.1:0/2134731431 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0a340b2910 msgr2=0x7f0a340b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:04.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.889+0000 7f0a4cf9f700 1 --2- 172.21.15.1:0/2134731431 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0a340b2910 0x7f0a340b4e20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0a38005fd0 tx=0x7f0a38005e40 comp rx=0 tx=0).stop 2024-05-04T03:17:04.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.889+0000 7f0a4cf9f700 1 -- 172.21.15.1:0/2134731431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a4810e740 msgr2=0x7f0a4819a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:04.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.889+0000 7f0a4cf9f700 1 --2- 172.21.15.1:0/2134731431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a4810e740 0x7f0a4819a9e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0a400062a0 tx=0x7f0a40004e20 comp rx=0 tx=0).stop 2024-05-04T03:17:04.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.889+0000 7f0a47fff700 1 -- 172.21.15.1:0/2134731431 reap_dead start 2024-05-04T03:17:04.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.889+0000 7f0a4cf9f700 1 -- 172.21.15.1:0/2134731431 shutdown_connections 2024-05-04T03:17:04.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.889+0000 7f0a4cf9f700 1 -- 172.21.15.1:0/2134731431 >> 172.21.15.1:0/2134731431 conn(0x7f0a48078ac0 msgr2=0x7f0a480ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:04.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.889+0000 7f0a4cf9f700 1 -- 172.21.15.1:0/2134731431 shutdown_connections 2024-05-04T03:17:04.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:04.889+0000 7f0a4cf9f700 1 -- 172.21.15.1:0/2134731431 wait complete. 2024-05-04T03:17:04.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:04 smithi077 ceph-mon[33531]: pgmap v328: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:17:04.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:04 smithi072 ceph-mon[33332]: pgmap v328: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:17:05.485 INFO:teuthology.orchestra.run.smithi001.stdout:133143986242 2024-05-04T03:17:05.511 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-04T03:17:05.874 INFO:teuthology.orchestra.run.smithi001.stdout:287762808868 2024-05-04T03:17:05.875 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-04T03:17:06.691 INFO:teuthology.orchestra.run.smithi001.stdout:313532612639 2024-05-04T03:17:06.691 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-04T03:17:06.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:06 smithi077 ceph-mon[33531]: pgmap v329: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-04T03:17:06.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:06 smithi072 ceph-mon[33332]: pgmap v329: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-04T03:17:07.055 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:06 smithi001 ceph-mon[29731]: pgmap v329: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-05-04T03:17:07.086 INFO:teuthology.orchestra.run.smithi001.stdout:55834574929 2024-05-04T03:17:07.087 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-04T03:17:07.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.615+0000 7f1b99a9e700 1 -- 172.21.15.1:0/1156432803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b94107df0 msgr2=0x7f1b94108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:07.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.615+0000 7f1b99a9e700 1 --2- 172.21.15.1:0/1156432803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b94107df0 0x7f1b94108200 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c0099c0 tx=0x7f1b7c009cd0 comp rx=0 tx=0).stop 2024-05-04T03:17:07.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.615+0000 7f1b99a9e700 1 -- 172.21.15.1:0/1156432803 shutdown_connections 2024-05-04T03:17:07.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.615+0000 7f1b99a9e700 1 --2- 172.21.15.1:0/1156432803 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b941034c0 0x7f1b941038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:07.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.615+0000 7f1b99a9e700 1 --2- 172.21.15.1:0/1156432803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b94108740 0x7f1b941100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:07.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.615+0000 7f1b99a9e700 1 --2- 172.21.15.1:0/1156432803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b94107df0 0x7f1b94108200 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:07.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.615+0000 7f1b99a9e700 1 -- 172.21.15.1:0/1156432803 >> 172.21.15.1:0/1156432803 conn(0x7f1b940fedf0 msgr2=0x7f1b94101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:07.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.615+0000 7f1b99a9e700 1 -- 172.21.15.1:0/1156432803 shutdown_connections 2024-05-04T03:17:07.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.616+0000 7f1b99a9e700 1 -- 172.21.15.1:0/1156432803 wait complete. 2024-05-04T03:17:07.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.616+0000 7f1b99a9e700 1 Processor -- start 2024-05-04T03:17:07.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.617+0000 7f1b99a9e700 1 -- start start 2024-05-04T03:17:07.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.617+0000 7f1b99a9e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b941034c0 0x7f1b9419d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:07.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.617+0000 7f1b99a9e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b94107df0 0x7f1b9419d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:07.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.617+0000 7f1b99a9e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b94108740 0x7f1b941a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:07.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.617+0000 7f1b99a9e700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1b94113350 con 0x7f1b941034c0 2024-05-04T03:17:07.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.617+0000 7f1b99a9e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1b941131d0 con 0x7f1b94107df0 2024-05-04T03:17:07.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.617+0000 7f1b99a9e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1b941134d0 con 0x7f1b94108740 2024-05-04T03:17:07.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.617+0000 7f1b937fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b941034c0 0x7f1b9419d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:07.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.617+0000 7f1b937fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b941034c0 0x7f1b9419d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47016/0 (socket says 172.21.15.1:47016) 2024-05-04T03:17:07.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.617+0000 7f1b937fe700 1 -- 172.21.15.1:0/422001466 learned_addr learned my addr 172.21.15.1:0/422001466 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:07.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.617+0000 7f1b93fff700 1 --2- 172.21.15.1:0/422001466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b94108740 0x7f1b941a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:07.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.617+0000 7f1b92ffd700 1 --2- 172.21.15.1:0/422001466 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b94107df0 0x7f1b9419d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:07.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.617+0000 7f1b937fe700 1 -- 172.21.15.1:0/422001466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b94108740 msgr2=0x7f1b941a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:07.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.617+0000 7f1b937fe700 1 --2- 172.21.15.1:0/422001466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1b94108740 0x7f1b941a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:07.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.617+0000 7f1b937fe700 1 -- 172.21.15.1:0/422001466 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b94107df0 msgr2=0x7f1b9419d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:07.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.617+0000 7f1b937fe700 1 --2- 172.21.15.1:0/422001466 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1b94107df0 0x7f1b9419d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:07.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.617+0000 7f1b937fe700 1 -- 172.21.15.1:0/422001466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b941a1ed0 con 0x7f1b941034c0 2024-05-04T03:17:07.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.618+0000 7f1b937fe700 1 --2- 172.21.15.1:0/422001466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b941034c0 0x7f1b9419d000 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c0060b0 tx=0x7f1b8c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:07.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.618+0000 7f1b90ff9700 1 -- 172.21.15.1:0/422001466 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c01a070 con 0x7f1b941034c0 2024-05-04T03:17:07.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.618+0000 7f1b99a9e700 1 -- 172.21.15.1:0/422001466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b941a21c0 con 0x7f1b941034c0 2024-05-04T03:17:07.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.618+0000 7f1b90ff9700 1 -- 172.21.15.1:0/422001466 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b8c00dd20 con 0x7f1b941034c0 2024-05-04T03:17:07.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.618+0000 7f1b90ff9700 1 -- 172.21.15.1:0/422001466 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c014600 con 0x7f1b941034c0 2024-05-04T03:17:07.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.618+0000 7f1b99a9e700 1 -- 172.21.15.1:0/422001466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b941a9ae0 con 0x7f1b941034c0 2024-05-04T03:17:07.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.619+0000 7f1b99a9e700 1 -- 172.21.15.1:0/422001466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1b94063e90 con 0x7f1b941034c0 2024-05-04T03:17:07.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.620+0000 7f1b90ff9700 1 -- 172.21.15.1:0/422001466 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f1b8c01d070 con 0x7f1b941034c0 2024-05-04T03:17:07.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.621+0000 7f1b90ff9700 1 --2- 172.21.15.1:0/422001466 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1b800b2910 0x7f1b800b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:07.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.621+0000 7f1b92ffd700 1 --2- 172.21.15.1:0/422001466 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1b800b2910 0x7f1b800b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:07.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.621+0000 7f1b90ff9700 1 -- 172.21.15.1:0/422001466 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f1b8c0e2610 con 0x7f1b941034c0 2024-05-04T03:17:07.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.621+0000 7f1b90ff9700 1 --2- 172.21.15.1:0/422001466 >> [v2:172.21.15.77:6808/3582852165,v1:172.21.15.77:6809/3582852165] conn(0x7f1b800bc320 0x7f1b800be810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:07.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.621+0000 7f1b90ff9700 1 -- 172.21.15.1:0/422001466 --> [v2:172.21.15.77:6808/3582852165,v1:172.21.15.77:6809/3582852165] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1b800beee0 con 0x7f1b800bc320 2024-05-04T03:17:07.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.621+0000 7f1b90ff9700 1 -- 172.21.15.1:0/422001466 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f1b8c0e2af0 con 0x7f1b941034c0 2024-05-04T03:17:07.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.622+0000 7f1b92ffd700 1 --2- 172.21.15.1:0/422001466 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1b800b2910 0x7f1b800b4e20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c006010 tx=0x7f1b7c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:07.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.622+0000 7f1b93fff700 1 --2- 172.21.15.1:0/422001466 >> [v2:172.21.15.77:6808/3582852165,v1:172.21.15.77:6809/3582852165] conn(0x7f1b800bc320 0x7f1b800be810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:07.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.622+0000 7f1b93fff700 1 --2- 172.21.15.1:0/422001466 >> [v2:172.21.15.77:6808/3582852165,v1:172.21.15.77:6809/3582852165] conn(0x7f1b800bc320 0x7f1b800be810 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:07.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.623+0000 7f1b90ff9700 1 -- 172.21.15.1:0/422001466 <== osd.5 v2:172.21.15.77:6808/3582852165 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1b800beee0 con 0x7f1b800bc320 2024-05-04T03:17:07.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.640+0000 7f1b99a9e700 1 -- 172.21.15.1:0/422001466 --> [v2:172.21.15.77:6808/3582852165,v1:172.21.15.77:6809/3582852165] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1b94064290 con 0x7f1b800bc320 2024-05-04T03:17:07.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.641+0000 7f1b90ff9700 1 -- 172.21.15.1:0/422001466 <== osd.5 v2:172.21.15.77:6808/3582852165 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1b94064290 con 0x7f1b800bc320 2024-05-04T03:17:07.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.641+0000 7f1b99a9e700 1 -- 172.21.15.1:0/422001466 >> [v2:172.21.15.77:6808/3582852165,v1:172.21.15.77:6809/3582852165] conn(0x7f1b800bc320 msgr2=0x7f1b800be810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:07.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.641+0000 7f1b99a9e700 1 --2- 172.21.15.1:0/422001466 >> [v2:172.21.15.77:6808/3582852165,v1:172.21.15.77:6809/3582852165] conn(0x7f1b800bc320 0x7f1b800be810 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:07.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.641+0000 7f1b99a9e700 1 -- 172.21.15.1:0/422001466 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1b800b2910 msgr2=0x7f1b800b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:07.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.641+0000 7f1b99a9e700 1 --2- 172.21.15.1:0/422001466 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1b800b2910 0x7f1b800b4e20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c006010 tx=0x7f1b7c000bc0 comp rx=0 tx=0).stop 2024-05-04T03:17:07.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.642+0000 7f1b99a9e700 1 -- 172.21.15.1:0/422001466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b941034c0 msgr2=0x7f1b9419d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:07.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.642+0000 7f1b99a9e700 1 --2- 172.21.15.1:0/422001466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b941034c0 0x7f1b9419d000 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c0060b0 tx=0x7f1b8c00d3b0 comp rx=0 tx=0).stop 2024-05-04T03:17:07.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.642+0000 7f1b93fff700 1 -- 172.21.15.1:0/422001466 reap_dead start 2024-05-04T03:17:07.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.642+0000 7f1b99a9e700 1 -- 172.21.15.1:0/422001466 shutdown_connections 2024-05-04T03:17:07.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.642+0000 7f1b99a9e700 1 -- 172.21.15.1:0/422001466 >> 172.21.15.1:0/422001466 conn(0x7f1b940fedf0 msgr2=0x7f1b941096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:07.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.642+0000 7f1b99a9e700 1 -- 172.21.15.1:0/422001466 shutdown_connections 2024-05-04T03:17:07.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.642+0000 7f1b99a9e700 1 -- 172.21.15.1:0/422001466 wait complete. 2024-05-04T03:17:07.803 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:17:07.803 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:17:07.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.844+0000 7f6fb0e79700 1 -- 172.21.15.1:0/3281101285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fac108740 msgr2=0x7f6fac1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:07.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.844+0000 7f6fb0e79700 1 --2- 172.21.15.1:0/3281101285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fac108740 0x7f6fac1100a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c00b0d0 tx=0x7f6f9c00b3e0 comp rx=0 tx=0).stop 2024-05-04T03:17:07.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.844+0000 7f6fb0e79700 1 -- 172.21.15.1:0/3281101285 shutdown_connections 2024-05-04T03:17:07.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.844+0000 7f6fb0e79700 1 --2- 172.21.15.1:0/3281101285 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6fac1034c0 0x7f6fac1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:07.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.845+0000 7f6fb0e79700 1 --2- 172.21.15.1:0/3281101285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fac108740 0x7f6fac1100a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:07.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.845+0000 7f6fb0e79700 1 --2- 172.21.15.1:0/3281101285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fac107df0 0x7f6fac108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:07.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.845+0000 7f6fb0e79700 1 -- 172.21.15.1:0/3281101285 >> 172.21.15.1:0/3281101285 conn(0x7f6fac0fedc0 msgr2=0x7f6fac1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:07.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.845+0000 7f6fb0e79700 1 -- 172.21.15.1:0/3281101285 shutdown_connections 2024-05-04T03:17:07.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.845+0000 7f6fb0e79700 1 -- 172.21.15.1:0/3281101285 wait complete. 2024-05-04T03:17:07.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.845+0000 7f6fb0e79700 1 Processor -- start 2024-05-04T03:17:07.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.846+0000 7f6fb0e79700 1 -- start start 2024-05-04T03:17:07.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.846+0000 7f6fb0e79700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fac1034c0 0x7f6fac19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:07.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.846+0000 7f6fb0e79700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6fac107df0 0x7f6fac19b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:07.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.846+0000 7f6fb0e79700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fac108740 0x7f6fac19f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:07.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.846+0000 7f6fb0e79700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac1133c0 con 0x7f6fac108740 2024-05-04T03:17:07.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.846+0000 7f6fb0e79700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac113240 con 0x7f6fac107df0 2024-05-04T03:17:07.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.846+0000 7f6fb0e79700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac113540 con 0x7f6fac1034c0 2024-05-04T03:17:07.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.846+0000 7f6faad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fac108740 0x7f6fac19f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:07.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.846+0000 7f6faad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fac108740 0x7f6fac19f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47032/0 (socket says 172.21.15.1:47032) 2024-05-04T03:17:07.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.846+0000 7f6faad9d700 1 -- 172.21.15.1:0/3354627091 learned_addr learned my addr 172.21.15.1:0/3354627091 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:07.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.846+0000 7f6fa9d9b700 1 --2- 172.21.15.1:0/3354627091 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6fac107df0 0x7f6fac19b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:07.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.846+0000 7f6faa59c700 1 --2- 172.21.15.1:0/3354627091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fac1034c0 0x7f6fac19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:07.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.846+0000 7f6faad9d700 1 -- 172.21.15.1:0/3354627091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fac1034c0 msgr2=0x7f6fac19ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:07.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.846+0000 7f6faad9d700 1 --2- 172.21.15.1:0/3354627091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6fac1034c0 0x7f6fac19ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:07.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.846+0000 7f6faad9d700 1 -- 172.21.15.1:0/3354627091 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6fac107df0 msgr2=0x7f6fac19b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:07.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.846+0000 7f6faad9d700 1 --2- 172.21.15.1:0/3354627091 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6fac107df0 0x7f6fac19b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:07.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.846+0000 7f6faad9d700 1 -- 172.21.15.1:0/3354627091 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fac19fc20 con 0x7f6fac108740 2024-05-04T03:17:07.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.847+0000 7f6faad9d700 1 --2- 172.21.15.1:0/3354627091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fac108740 0x7f6fac19f630 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0079a0 tx=0x7f6f9c0079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:07.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.847+0000 7f6fa37fe700 1 -- 172.21.15.1:0/3354627091 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c01f070 con 0x7f6fac108740 2024-05-04T03:17:07.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.847+0000 7f6fb0e79700 1 -- 172.21.15.1:0/3354627091 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fac19feb0 con 0x7f6fac108740 2024-05-04T03:17:07.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.847+0000 7f6fa37fe700 1 -- 172.21.15.1:0/3354627091 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f9c005070 con 0x7f6fac108740 2024-05-04T03:17:07.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.847+0000 7f6fb0e79700 1 -- 172.21.15.1:0/3354627091 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fac1a03e0 con 0x7f6fac108740 2024-05-04T03:17:07.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.847+0000 7f6fa37fe700 1 -- 172.21.15.1:0/3354627091 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c01b680 con 0x7f6fac108740 2024-05-04T03:17:07.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.848+0000 7f6fb0e79700 1 -- 172.21.15.1:0/3354627091 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6fac101730 con 0x7f6fac108740 2024-05-04T03:17:07.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.849+0000 7f6fa37fe700 1 -- 172.21.15.1:0/3354627091 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f6f9c022070 con 0x7f6fac108740 2024-05-04T03:17:07.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.850+0000 7f6fa37fe700 1 --2- 172.21.15.1:0/3354627091 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6f980b2990 0x7f6f980b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:07.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.850+0000 7f6faa59c700 1 --2- 172.21.15.1:0/3354627091 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6f980b2990 0x7f6f980b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:07.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.850+0000 7f6fa37fe700 1 -- 172.21.15.1:0/3354627091 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f6f9c00f020 con 0x7f6fac108740 2024-05-04T03:17:07.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.850+0000 7f6fa37fe700 1 --2- 172.21.15.1:0/3354627091 >> [v2:172.21.15.77:6816/1124479323,v1:172.21.15.77:6817/1124479323] conn(0x7f6f980bc270 0x7f6f980be760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:07.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.851+0000 7f6fa37fe700 1 -- 172.21.15.1:0/3354627091 --> [v2:172.21.15.77:6816/1124479323,v1:172.21.15.77:6817/1124479323] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6f9c0075c0 con 0x7f6f980bc270 2024-05-04T03:17:07.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.851+0000 7f6fa37fe700 1 -- 172.21.15.1:0/3354627091 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f6f9c0e3d40 con 0x7f6fac108740 2024-05-04T03:17:07.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.851+0000 7f6faa59c700 1 --2- 172.21.15.1:0/3354627091 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6f980b2990 0x7f6f980b4ea0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400ac50 tx=0x7f6fa400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:07.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.851+0000 7f6fa9d9b700 1 --2- 172.21.15.1:0/3354627091 >> [v2:172.21.15.77:6816/1124479323,v1:172.21.15.77:6817/1124479323] conn(0x7f6f980bc270 0x7f6f980be760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:07.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.851+0000 7f6fa9d9b700 1 --2- 172.21.15.1:0/3354627091 >> [v2:172.21.15.77:6816/1124479323,v1:172.21.15.77:6817/1124479323] conn(0x7f6f980bc270 0x7f6f980be760 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:07.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.852+0000 7f6fa37fe700 1 -- 172.21.15.1:0/3354627091 <== osd.8 v2:172.21.15.77:6816/1124479323 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f9c0075c0 con 0x7f6f980bc270 2024-05-04T03:17:07.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.869+0000 7f6fb0e79700 1 -- 172.21.15.1:0/3354627091 --> [v2:172.21.15.77:6816/1124479323,v1:172.21.15.77:6817/1124479323] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6fac104970 con 0x7f6f980bc270 2024-05-04T03:17:07.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.870+0000 7f6fa37fe700 1 -- 172.21.15.1:0/3354627091 <== osd.8 v2:172.21.15.77:6816/1124479323 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6fac104970 con 0x7f6f980bc270 2024-05-04T03:17:07.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.870+0000 7f6fb0e79700 1 -- 172.21.15.1:0/3354627091 >> [v2:172.21.15.77:6816/1124479323,v1:172.21.15.77:6817/1124479323] conn(0x7f6f980bc270 msgr2=0x7f6f980be760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:07.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.870+0000 7f6fb0e79700 1 --2- 172.21.15.1:0/3354627091 >> [v2:172.21.15.77:6816/1124479323,v1:172.21.15.77:6817/1124479323] conn(0x7f6f980bc270 0x7f6f980be760 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:07.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.870+0000 7f6fb0e79700 1 -- 172.21.15.1:0/3354627091 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6f980b2990 msgr2=0x7f6f980b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:07.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.870+0000 7f6fb0e79700 1 --2- 172.21.15.1:0/3354627091 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6f980b2990 0x7f6f980b4ea0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400ac50 tx=0x7f6fa400a380 comp rx=0 tx=0).stop 2024-05-04T03:17:07.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.870+0000 7f6fb0e79700 1 -- 172.21.15.1:0/3354627091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fac108740 msgr2=0x7f6fac19f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:07.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.870+0000 7f6fb0e79700 1 --2- 172.21.15.1:0/3354627091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6fac108740 0x7f6fac19f630 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0079a0 tx=0x7f6f9c0079d0 comp rx=0 tx=0).stop 2024-05-04T03:17:07.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.871+0000 7f6faad9d700 1 -- 172.21.15.1:0/3354627091 reap_dead start 2024-05-04T03:17:07.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.871+0000 7f6fb0e79700 1 -- 172.21.15.1:0/3354627091 shutdown_connections 2024-05-04T03:17:07.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.871+0000 7f6fb0e79700 1 -- 172.21.15.1:0/3354627091 >> 172.21.15.1:0/3354627091 conn(0x7f6fac0fedc0 msgr2=0x7f6fac107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:07.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.871+0000 7f6fb0e79700 1 -- 172.21.15.1:0/3354627091 shutdown_connections 2024-05-04T03:17:07.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:07.871+0000 7f6fb0e79700 1 -- 172.21.15.1:0/3354627091 wait complete. 2024-05-04T03:17:08.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.480+0000 7fa5f3f19700 1 -- 172.21.15.1:0/3840272192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5ec1045e0 msgr2=0x7fa5ec1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:08.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.480+0000 7fa5f3f19700 1 --2- 172.21.15.1:0/3840272192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5ec1045e0 0x7fa5ec1049f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80099c0 tx=0x7fa5e8009cd0 comp rx=0 tx=0).stop 2024-05-04T03:17:08.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.480+0000 7fa5f3f19700 1 -- 172.21.15.1:0/3840272192 shutdown_connections 2024-05-04T03:17:08.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.480+0000 7fa5f3f19700 1 --2- 172.21.15.1:0/3840272192 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5ec109860 0x7fa5ec1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:08.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.480+0000 7fa5f3f19700 1 --2- 172.21.15.1:0/3840272192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5ec1045e0 0x7fa5ec1049f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:08.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.480+0000 7fa5f3f19700 1 --2- 172.21.15.1:0/3840272192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5ec108f10 0x7fa5ec109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:08.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.480+0000 7fa5f3f19700 1 -- 172.21.15.1:0/3840272192 >> 172.21.15.1:0/3840272192 conn(0x7fa5ec0ffef0 msgr2=0x7fa5ec102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:08.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.480+0000 7fa5f3f19700 1 -- 172.21.15.1:0/3840272192 shutdown_connections 2024-05-04T03:17:08.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.480+0000 7fa5f3f19700 1 -- 172.21.15.1:0/3840272192 wait complete. 2024-05-04T03:17:08.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.481+0000 7fa5f3f19700 1 Processor -- start 2024-05-04T03:17:08.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.481+0000 7fa5f3f19700 1 -- start start 2024-05-04T03:17:08.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.481+0000 7fa5f3f19700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5ec1045e0 0x7fa5ec19d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:08.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.481+0000 7fa5f3f19700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5ec108f10 0x7fa5ec19d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:08.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.481+0000 7fa5f3f19700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5ec109860 0x7fa5ec1a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:08.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.481+0000 7fa5f3f19700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec114490 con 0x7fa5ec109860 2024-05-04T03:17:08.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.481+0000 7fa5f3f19700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec114310 con 0x7fa5ec108f10 2024-05-04T03:17:08.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.481+0000 7fa5f3f19700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ec114610 con 0x7fa5ec1045e0 2024-05-04T03:17:08.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.481+0000 7fa5f24b6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5ec109860 0x7fa5ec1a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:08.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.481+0000 7fa5f24b6700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5ec109860 0x7fa5ec1a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47050/0 (socket says 172.21.15.1:47050) 2024-05-04T03:17:08.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.482+0000 7fa5f24b6700 1 -- 172.21.15.1:0/1949794739 learned_addr learned my addr 172.21.15.1:0/1949794739 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:08.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.482+0000 7fa5f14b4700 1 --2- 172.21.15.1:0/1949794739 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5ec108f10 0x7fa5ec19d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:08.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.482+0000 7fa5f1cb5700 1 --2- 172.21.15.1:0/1949794739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5ec1045e0 0x7fa5ec19d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:08.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.482+0000 7fa5f24b6700 1 -- 172.21.15.1:0/1949794739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5ec1045e0 msgr2=0x7fa5ec19d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:08.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.482+0000 7fa5f24b6700 1 --2- 172.21.15.1:0/1949794739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5ec1045e0 0x7fa5ec19d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:08.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.482+0000 7fa5f24b6700 1 -- 172.21.15.1:0/1949794739 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5ec108f10 msgr2=0x7fa5ec19d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:08.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.482+0000 7fa5f24b6700 1 --2- 172.21.15.1:0/1949794739 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5ec108f10 0x7fa5ec19d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:08.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.482+0000 7fa5f24b6700 1 -- 172.21.15.1:0/1949794739 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ec1a1e40 con 0x7fa5ec109860 2024-05-04T03:17:08.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.482+0000 7fa5f1cb5700 1 --2- 172.21.15.1:0/1949794739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5ec1045e0 0x7fa5ec19d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:17:08.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.482+0000 7fa5f14b4700 1 --2- 172.21.15.1:0/1949794739 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5ec108f10 0x7fa5ec19d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:17:08.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.482+0000 7fa5f24b6700 1 --2- 172.21.15.1:0/1949794739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5ec109860 0x7fa5ec1a1850 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4007ee0 tx=0x7fa5e400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:08.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.482+0000 7fa5e2ffd700 1 -- 172.21.15.1:0/1949794739 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e400dbd0 con 0x7fa5ec109860 2024-05-04T03:17:08.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.482+0000 7fa5e2ffd700 1 -- 172.21.15.1:0/1949794739 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5e400dd70 con 0x7fa5ec109860 2024-05-04T03:17:08.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.482+0000 7fa5f3f19700 1 -- 172.21.15.1:0/1949794739 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5ec1a2130 con 0x7fa5ec109860 2024-05-04T03:17:08.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.482+0000 7fa5e2ffd700 1 -- 172.21.15.1:0/1949794739 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4005990 con 0x7fa5ec109860 2024-05-04T03:17:08.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.482+0000 7fa5f3f19700 1 -- 172.21.15.1:0/1949794739 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5ec1a2670 con 0x7fa5ec109860 2024-05-04T03:17:08.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.483+0000 7fa5f3f19700 1 -- 172.21.15.1:0/1949794739 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa5ec078700 con 0x7fa5ec109860 2024-05-04T03:17:08.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.485+0000 7fa5e2ffd700 1 -- 172.21.15.1:0/1949794739 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fa5e401a020 con 0x7fa5ec109860 2024-05-04T03:17:08.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.485+0000 7fa5e2ffd700 1 --2- 172.21.15.1:0/1949794739 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa5d80b2990 0x7fa5d80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:08.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.485+0000 7fa5f1cb5700 1 --2- 172.21.15.1:0/1949794739 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa5d80b2990 0x7fa5d80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:08.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.486+0000 7fa5e2ffd700 1 -- 172.21.15.1:0/1949794739 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fa5e40e20e0 con 0x7fa5ec109860 2024-05-04T03:17:08.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.486+0000 7fa5e2ffd700 1 --2- 172.21.15.1:0/1949794739 >> [v2:172.21.15.72:6816/3460835447,v1:172.21.15.72:6817/3460835447] conn(0x7fa5d80bc3a0 0x7fa5d80be890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:08.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.486+0000 7fa5e2ffd700 1 -- 172.21.15.1:0/1949794739 --> [v2:172.21.15.72:6816/3460835447,v1:172.21.15.72:6817/3460835447] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa5d80bef60 con 0x7fa5d80bc3a0 2024-05-04T03:17:08.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.486+0000 7fa5e2ffd700 1 -- 172.21.15.1:0/1949794739 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fa5e40e25c0 con 0x7fa5ec109860 2024-05-04T03:17:08.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.486+0000 7fa5f1cb5700 1 --2- 172.21.15.1:0/1949794739 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa5d80b2990 0x7fa5d80b4ea0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8000c00 tx=0x7fa5e801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:08.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.486+0000 7fa5f14b4700 1 --2- 172.21.15.1:0/1949794739 >> [v2:172.21.15.72:6816/3460835447,v1:172.21.15.72:6817/3460835447] conn(0x7fa5d80bc3a0 0x7fa5d80be890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:08.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.486+0000 7fa5f14b4700 1 --2- 172.21.15.1:0/1949794739 >> [v2:172.21.15.72:6816/3460835447,v1:172.21.15.72:6817/3460835447] conn(0x7fa5d80bc3a0 0x7fa5d80be890 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:08.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.487+0000 7fa5e2ffd700 1 -- 172.21.15.1:0/1949794739 <== osd.7 v2:172.21.15.72:6816/3460835447 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa5d80bef60 con 0x7fa5d80bc3a0 2024-05-04T03:17:08.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.504+0000 7fa5f3f19700 1 -- 172.21.15.1:0/1949794739 --> [v2:172.21.15.72:6816/3460835447,v1:172.21.15.72:6817/3460835447] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa5ec105a90 con 0x7fa5d80bc3a0 2024-05-04T03:17:08.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.505+0000 7fa5e2ffd700 1 -- 172.21.15.1:0/1949794739 <== osd.7 v2:172.21.15.72:6816/3460835447 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa5ec105a90 con 0x7fa5d80bc3a0 2024-05-04T03:17:08.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.505+0000 7fa5f3f19700 1 -- 172.21.15.1:0/1949794739 >> [v2:172.21.15.72:6816/3460835447,v1:172.21.15.72:6817/3460835447] conn(0x7fa5d80bc3a0 msgr2=0x7fa5d80be890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:08.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.505+0000 7fa5f3f19700 1 --2- 172.21.15.1:0/1949794739 >> [v2:172.21.15.72:6816/3460835447,v1:172.21.15.72:6817/3460835447] conn(0x7fa5d80bc3a0 0x7fa5d80be890 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:08.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.505+0000 7fa5f3f19700 1 -- 172.21.15.1:0/1949794739 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa5d80b2990 msgr2=0x7fa5d80b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:08.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.505+0000 7fa5f3f19700 1 --2- 172.21.15.1:0/1949794739 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa5d80b2990 0x7fa5d80b4ea0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8000c00 tx=0x7fa5e801a040 comp rx=0 tx=0).stop 2024-05-04T03:17:08.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.505+0000 7fa5f3f19700 1 -- 172.21.15.1:0/1949794739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5ec109860 msgr2=0x7fa5ec1a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:08.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.505+0000 7fa5f3f19700 1 --2- 172.21.15.1:0/1949794739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa5ec109860 0x7fa5ec1a1850 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4007ee0 tx=0x7fa5e400d3b0 comp rx=0 tx=0).stop 2024-05-04T03:17:08.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.505+0000 7fa5f24b6700 1 -- 172.21.15.1:0/1949794739 reap_dead start 2024-05-04T03:17:08.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.505+0000 7fa5f3f19700 1 -- 172.21.15.1:0/1949794739 shutdown_connections 2024-05-04T03:17:08.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.505+0000 7fa5f3f19700 1 -- 172.21.15.1:0/1949794739 >> 172.21.15.1:0/1949794739 conn(0x7fa5ec0ffef0 msgr2=0x7fa5ec110490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:08.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.505+0000 7fa5f3f19700 1 -- 172.21.15.1:0/1949794739 shutdown_connections 2024-05-04T03:17:08.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:08.505+0000 7fa5f3f19700 1 -- 172.21.15.1:0/1949794739 wait complete. 2024-05-04T03:17:08.894 INFO:teuthology.orchestra.run.smithi001.stdout:240518168624 2024-05-04T03:17:08.895 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-04T03:17:08.919 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:08 smithi001 ceph-mon[29731]: pgmap v330: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-04T03:17:08.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:08 smithi077 ceph-mon[33531]: pgmap v330: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-04T03:17:08.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:08 smithi072 ceph-mon[33332]: pgmap v330: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-04T03:17:10.782 INFO:teuthology.orchestra.run.smithi001.stdout:158913790015 2024-05-04T03:17:10.782 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-04T03:17:10.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:10 smithi077 ceph-mon[33531]: pgmap v331: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-04T03:17:10.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:10 smithi072 ceph-mon[33332]: pgmap v331: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-04T03:17:11.100 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:10 smithi001 ceph-mon[29731]: pgmap v331: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-04T03:17:11.446 INFO:teuthology.orchestra.run.smithi001.stdout:210453397556 2024-05-04T03:17:11.447 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-04T03:17:12.302 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:17:12.303 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:17:12.303 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:17:12.311 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:17:12.311 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:17:12.827 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:12 smithi001 ceph-mon[29731]: pgmap v332: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:17:12.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:12 smithi077 ceph-mon[33531]: pgmap v332: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:17:12.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:12 smithi072 ceph-mon[33332]: pgmap v332: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:17:13.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.069+0000 7f082b5e9700 1 -- 172.21.15.1:0/477334633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0824105e40 msgr2=0x7f082410e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:13.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.069+0000 7f082b5e9700 1 --2- 172.21.15.1:0/477334633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0824105e40 0x7f082410e390 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0818009990 tx=0x7f0818009ca0 comp rx=0 tx=0).stop 2024-05-04T03:17:13.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.069+0000 7f082b5e9700 1 -- 172.21.15.1:0/477334633 shutdown_connections 2024-05-04T03:17:13.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.069+0000 7f082b5e9700 1 --2- 172.21.15.1:0/477334633 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f082410e920 0x7f0824110e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:13.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.069+0000 7f082b5e9700 1 --2- 172.21.15.1:0/477334633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08241054f0 0x7f0824105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:13.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.069+0000 7f082b5e9700 1 --2- 172.21.15.1:0/477334633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0824105e40 0x7f082410e390 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:13.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.069+0000 7f082b5e9700 1 -- 172.21.15.1:0/477334633 >> 172.21.15.1:0/477334633 conn(0x7f0824078e00 msgr2=0x7f0824079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:13.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.070+0000 7f082b5e9700 1 -- 172.21.15.1:0/477334633 shutdown_connections 2024-05-04T03:17:13.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.070+0000 7f082b5e9700 1 -- 172.21.15.1:0/477334633 wait complete. 2024-05-04T03:17:13.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.070+0000 7f082b5e9700 1 Processor -- start 2024-05-04T03:17:13.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.070+0000 7f082b5e9700 1 -- start start 2024-05-04T03:17:13.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.071+0000 7f082b5e9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f08241054f0 0x7f082419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:13.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.071+0000 7f082b5e9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0824105e40 0x7f082419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:13.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.071+0000 7f082b5e9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f082410e920 0x7f082419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:13.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.071+0000 7f082b5e9700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f08241144a0 con 0x7f082410e920 2024-05-04T03:17:13.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.071+0000 7f082b5e9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0824114320 con 0x7f08241054f0 2024-05-04T03:17:13.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.071+0000 7f082b5e9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0824114620 con 0x7f0824105e40 2024-05-04T03:17:13.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.071+0000 7f0829b86700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f082410e920 0x7f082419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:13.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.071+0000 7f0829b86700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f082410e920 0x7f082419f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38386/0 (socket says 172.21.15.1:38386) 2024-05-04T03:17:13.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.071+0000 7f0829b86700 1 -- 172.21.15.1:0/4202073514 learned_addr learned my addr 172.21.15.1:0/4202073514 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:13.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.071+0000 7f0829385700 1 --2- 172.21.15.1:0/4202073514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f08241054f0 0x7f082419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:13.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.071+0000 7f0828b84700 1 --2- 172.21.15.1:0/4202073514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0824105e40 0x7f082419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:13.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.071+0000 7f0829b86700 1 -- 172.21.15.1:0/4202073514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0824105e40 msgr2=0x7f082419b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:13.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.071+0000 7f0829b86700 1 --2- 172.21.15.1:0/4202073514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0824105e40 0x7f082419b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:13.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.071+0000 7f0829b86700 1 -- 172.21.15.1:0/4202073514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f08241054f0 msgr2=0x7f082419ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:13.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.071+0000 7f0829b86700 1 --2- 172.21.15.1:0/4202073514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f08241054f0 0x7f082419ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:13.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.071+0000 7f0829b86700 1 -- 172.21.15.1:0/4202073514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082419fce0 con 0x7f082410e920 2024-05-04T03:17:13.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.071+0000 7f0829385700 1 --2- 172.21.15.1:0/4202073514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f08241054f0 0x7f082419ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:17:13.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.071+0000 7f0828b84700 1 --2- 172.21.15.1:0/4202073514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0824105e40 0x7f082419b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:17:13.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.071+0000 7f0829b86700 1 --2- 172.21.15.1:0/4202073514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f082410e920 0x7f082419f640 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f081400ada0 tx=0x7f081400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:13.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.071+0000 7f08227fc700 1 -- 172.21.15.1:0/4202073514 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814017020 con 0x7f082410e920 2024-05-04T03:17:13.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.071+0000 7f08227fc700 1 -- 172.21.15.1:0/4202073514 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f081400ce30 con 0x7f082410e920 2024-05-04T03:17:13.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.071+0000 7f082b5e9700 1 -- 172.21.15.1:0/4202073514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f082419ffd0 con 0x7f082410e920 2024-05-04T03:17:13.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.071+0000 7f08227fc700 1 -- 172.21.15.1:0/4202073514 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08140054a0 con 0x7f082410e920 2024-05-04T03:17:13.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.072+0000 7f082b5e9700 1 -- 172.21.15.1:0/4202073514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08241a78f0 con 0x7f082410e920 2024-05-04T03:17:13.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.072+0000 7f082b5e9700 1 -- 172.21.15.1:0/4202073514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08240512c0 con 0x7f082410e920 2024-05-04T03:17:13.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.074+0000 7f08227fc700 1 -- 172.21.15.1:0/4202073514 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f081400c980 con 0x7f082410e920 2024-05-04T03:17:13.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.075+0000 7f08227fc700 1 --2- 172.21.15.1:0/4202073514 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f080c0b2990 0x7f080c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:13.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.075+0000 7f0829385700 1 --2- 172.21.15.1:0/4202073514 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f080c0b2990 0x7f080c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:13.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.075+0000 7f08227fc700 1 -- 172.21.15.1:0/4202073514 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f08140e1f10 con 0x7f082410e920 2024-05-04T03:17:13.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.075+0000 7f0829385700 1 --2- 172.21.15.1:0/4202073514 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f080c0b2990 0x7f080c0b4ea0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f081c005fd0 tx=0x7f081c005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:13.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.078+0000 7f08227fc700 1 -- 172.21.15.1:0/4202073514 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0814021020 con 0x7f082410e920 2024-05-04T03:17:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.298+0000 7f082b5e9700 1 -- 172.21.15.1:0/4202073514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f082419c170 con 0x7f082410e920 2024-05-04T03:17:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.299+0000 7f08227fc700 1 -- 172.21.15.1:0/4202073514 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f08140a9c80 con 0x7f082410e920 2024-05-04T03:17:13.301 INFO:teuthology.orchestra.run.smithi001.stdout:34359738455 2024-05-04T03:17:13.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.301+0000 7f082b5e9700 1 -- 172.21.15.1:0/4202073514 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f080c0b2990 msgr2=0x7f080c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:13.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.301+0000 7f082b5e9700 1 --2- 172.21.15.1:0/4202073514 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f080c0b2990 0x7f080c0b4ea0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f081c005fd0 tx=0x7f081c005dc0 comp rx=0 tx=0).stop 2024-05-04T03:17:13.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.301+0000 7f082b5e9700 1 -- 172.21.15.1:0/4202073514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f082410e920 msgr2=0x7f082419f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:13.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.301+0000 7f082b5e9700 1 --2- 172.21.15.1:0/4202073514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f082410e920 0x7f082419f640 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f081400ada0 tx=0x7f081400c330 comp rx=0 tx=0).stop 2024-05-04T03:17:13.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.301+0000 7f082b5e9700 1 -- 172.21.15.1:0/4202073514 shutdown_connections 2024-05-04T03:17:13.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.301+0000 7f082b5e9700 1 --2- 172.21.15.1:0/4202073514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f08241054f0 0x7f082419ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:13.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.301+0000 7f082b5e9700 1 --2- 172.21.15.1:0/4202073514 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f080c0b2990 0x7f080c0b4ea0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:13.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.301+0000 7f082b5e9700 1 --2- 172.21.15.1:0/4202073514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f082410e920 0x7f082419f640 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:13.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.301+0000 7f082b5e9700 1 --2- 172.21.15.1:0/4202073514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0824105e40 0x7f082419b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:13.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.301+0000 7f082b5e9700 1 -- 172.21.15.1:0/4202073514 >> 172.21.15.1:0/4202073514 conn(0x7f0824078e00 msgr2=0x7f08241000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:13.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.301+0000 7f082b5e9700 1 -- 172.21.15.1:0/4202073514 shutdown_connections 2024-05-04T03:17:13.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:13.301+0000 7f082b5e9700 1 -- 172.21.15.1:0/4202073514 wait complete. 2024-05-04T03:17:13.714 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:17:13.715 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:17:13.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:13 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/4202073514' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-04T03:17:14.002 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:13 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/4202073514' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-04T03:17:14.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:13 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/4202073514' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-04T03:17:14.829 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:17:14.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:14 smithi001 ceph-mon[29731]: pgmap v333: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:17:14.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:14 smithi072 ceph-mon[33332]: pgmap v333: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:17:15.079 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:14 smithi077 ceph-mon[33531]: pgmap v333: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:17:15.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:15 smithi001 ceph-mon[29731]: pgmap v334: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-04T03:17:15.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:15 smithi072 ceph-mon[33332]: pgmap v334: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-04T03:17:16.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:15 smithi077 ceph-mon[33531]: pgmap v334: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-04T03:17:16.191 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:17:16.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.222+0000 7fc2d0ee0700 1 -- 172.21.15.1:0/1343372633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2cc107e10 msgr2=0x7fc2cc108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:16.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.222+0000 7fc2d0ee0700 1 --2- 172.21.15.1:0/1343372633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2cc107e10 0x7fc2cc108220 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc2c00099c0 tx=0x7fc2c0009cd0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.222+0000 7fc2d0ee0700 1 -- 172.21.15.1:0/1343372633 shutdown_connections 2024-05-04T03:17:16.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.222+0000 7fc2d0ee0700 1 --2- 172.21.15.1:0/1343372633 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc2cc108760 0x7fc2cc1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.222+0000 7fc2d0ee0700 1 --2- 172.21.15.1:0/1343372633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2cc1034e0 0x7fc2cc1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.222+0000 7fc2d0ee0700 1 --2- 172.21.15.1:0/1343372633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2cc107e10 0x7fc2cc108220 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.222+0000 7fc2d0ee0700 1 -- 172.21.15.1:0/1343372633 >> 172.21.15.1:0/1343372633 conn(0x7fc2cc0fef30 msgr2=0x7fc2cc101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:16.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.222+0000 7fc2d0ee0700 1 -- 172.21.15.1:0/1343372633 shutdown_connections 2024-05-04T03:17:16.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.222+0000 7fc2d0ee0700 1 -- 172.21.15.1:0/1343372633 wait complete. 2024-05-04T03:17:16.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.223+0000 7fc2d0ee0700 1 Processor -- start 2024-05-04T03:17:16.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.223+0000 7fc2d0ee0700 1 -- start start 2024-05-04T03:17:16.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.224+0000 7fc2d0ee0700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc2cc1034e0 0x7fc2cc19aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:16.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.224+0000 7fc2d0ee0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2cc107e10 0x7fc2cc19b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:16.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.224+0000 7fc2d0ee0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2cc108760 0x7fc2cc19f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:16.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.224+0000 7fc2d0ee0700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc1133e0 con 0x7fc2cc108760 2024-05-04T03:17:16.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.224+0000 7fc2d0ee0700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc113260 con 0x7fc2cc1034e0 2024-05-04T03:17:16.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.224+0000 7fc2d0ee0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc113560 con 0x7fc2cc107e10 2024-05-04T03:17:16.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.224+0000 7fc2cad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2cc108760 0x7fc2cc19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:16.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.224+0000 7fc2cad9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2cc108760 0x7fc2cc19f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38402/0 (socket says 172.21.15.1:38402) 2024-05-04T03:17:16.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.224+0000 7fc2cad9d700 1 -- 172.21.15.1:0/2232911731 learned_addr learned my addr 172.21.15.1:0/2232911731 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:16.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.224+0000 7fc2ca59c700 1 --2- 172.21.15.1:0/2232911731 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc2cc1034e0 0x7fc2cc19aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:16.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.224+0000 7fc2c9d9b700 1 --2- 172.21.15.1:0/2232911731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2cc107e10 0x7fc2cc19b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:16.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.224+0000 7fc2cad9d700 1 -- 172.21.15.1:0/2232911731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2cc107e10 msgr2=0x7fc2cc19b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:16.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.224+0000 7fc2cad9d700 1 --2- 172.21.15.1:0/2232911731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2cc107e10 0x7fc2cc19b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.224+0000 7fc2cad9d700 1 -- 172.21.15.1:0/2232911731 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc2cc1034e0 msgr2=0x7fc2cc19aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:16.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.224+0000 7fc2cad9d700 1 --2- 172.21.15.1:0/2232911731 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc2cc1034e0 0x7fc2cc19aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.224+0000 7fc2cad9d700 1 -- 172.21.15.1:0/2232911731 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2cc19fdb0 con 0x7fc2cc108760 2024-05-04T03:17:16.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.224+0000 7fc2c9d9b700 1 --2- 172.21.15.1:0/2232911731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2cc107e10 0x7fc2cc19b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:17:16.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.224+0000 7fc2cad9d700 1 --2- 172.21.15.1:0/2232911731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2cc108760 0x7fc2cc19f710 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4007ee0 tx=0x7fc2c400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:16.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.224+0000 7fc2bb7fe700 1 -- 172.21.15.1:0/2232911731 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c400ddd0 con 0x7fc2cc108760 2024-05-04T03:17:16.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.224+0000 7fc2d0ee0700 1 -- 172.21.15.1:0/2232911731 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2cc1a00a0 con 0x7fc2cc108760 2024-05-04T03:17:16.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.224+0000 7fc2bb7fe700 1 -- 172.21.15.1:0/2232911731 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c4004020 con 0x7fc2cc108760 2024-05-04T03:17:16.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.224+0000 7fc2bb7fe700 1 -- 172.21.15.1:0/2232911731 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4015660 con 0x7fc2cc108760 2024-05-04T03:17:16.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.225+0000 7fc2d0ee0700 1 -- 172.21.15.1:0/2232911731 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2cc1a79c0 con 0x7fc2cc108760 2024-05-04T03:17:16.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.226+0000 7fc2d0ee0700 1 -- 172.21.15.1:0/2232911731 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2cc0512c0 con 0x7fc2cc108760 2024-05-04T03:17:16.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.227+0000 7fc2bb7fe700 1 -- 172.21.15.1:0/2232911731 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fc2c40044f0 con 0x7fc2cc108760 2024-05-04T03:17:16.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.228+0000 7fc2bb7fe700 1 --2- 172.21.15.1:0/2232911731 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc2b40b2910 0x7fc2b40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:16.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.228+0000 7fc2ca59c700 1 --2- 172.21.15.1:0/2232911731 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc2b40b2910 0x7fc2b40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:16.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.228+0000 7fc2bb7fe700 1 -- 172.21.15.1:0/2232911731 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fc2c40e2710 con 0x7fc2cc108760 2024-05-04T03:17:16.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.228+0000 7fc2ca59c700 1 --2- 172.21.15.1:0/2232911731 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc2b40b2910 0x7fc2b40b4e20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc006fd0 tx=0x7fc2bc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:16.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.232+0000 7fc2bb7fe700 1 -- 172.21.15.1:0/2232911731 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc2c4010070 con 0x7fc2cc108760 2024-05-04T03:17:16.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.449+0000 7fc2d0ee0700 1 -- 172.21.15.1:0/2232911731 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fc2cc02e5f0 con 0x7fc2cc108760 2024-05-04T03:17:16.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.449+0000 7fc2bb7fe700 1 -- 172.21.15.1:0/2232911731 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc2c40aa2f0 con 0x7fc2cc108760 2024-05-04T03:17:16.451 INFO:teuthology.orchestra.run.smithi001.stdout:107374182473 2024-05-04T03:17:16.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.451+0000 7fc2d0ee0700 1 -- 172.21.15.1:0/2232911731 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc2b40b2910 msgr2=0x7fc2b40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:16.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.451+0000 7fc2d0ee0700 1 --2- 172.21.15.1:0/2232911731 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc2b40b2910 0x7fc2b40b4e20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc006fd0 tx=0x7fc2bc008040 comp rx=0 tx=0).stop 2024-05-04T03:17:16.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.451+0000 7fc2d0ee0700 1 -- 172.21.15.1:0/2232911731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2cc108760 msgr2=0x7fc2cc19f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:16.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.451+0000 7fc2d0ee0700 1 --2- 172.21.15.1:0/2232911731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2cc108760 0x7fc2cc19f710 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4007ee0 tx=0x7fc2c400d3b0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.451+0000 7fc2d0ee0700 1 -- 172.21.15.1:0/2232911731 shutdown_connections 2024-05-04T03:17:16.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.451+0000 7fc2d0ee0700 1 --2- 172.21.15.1:0/2232911731 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc2cc1034e0 0x7fc2cc19aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.451+0000 7fc2d0ee0700 1 --2- 172.21.15.1:0/2232911731 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc2b40b2910 0x7fc2b40b4e20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.451+0000 7fc2d0ee0700 1 --2- 172.21.15.1:0/2232911731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc2cc108760 0x7fc2cc19f710 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.451+0000 7fc2d0ee0700 1 --2- 172.21.15.1:0/2232911731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2cc107e10 0x7fc2cc19b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:16.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.451+0000 7fc2d0ee0700 1 -- 172.21.15.1:0/2232911731 >> 172.21.15.1:0/2232911731 conn(0x7fc2cc0fef30 msgr2=0x7fc2cc06c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:16.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.451+0000 7fc2d0ee0700 1 -- 172.21.15.1:0/2232911731 shutdown_connections 2024-05-04T03:17:16.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:16.451+0000 7fc2d0ee0700 1 -- 172.21.15.1:0/2232911731 wait complete. 2024-05-04T03:17:16.698 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738452 got 34359738455 for osd.0 2024-05-04T03:17:16.698 DEBUG:teuthology.parallel:result is None 2024-05-04T03:17:16.958 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:16 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2232911731' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-04T03:17:16.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:16 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/2232911731' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-04T03:17:17.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:16 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/2232911731' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-04T03:17:17.244 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182468 got 107374182473 for osd.3 2024-05-04T03:17:17.244 DEBUG:teuthology.parallel:result is None 2024-05-04T03:17:17.805 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:17 smithi001 ceph-mon[29731]: pgmap v335: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-04T03:17:17.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:17 smithi072 ceph-mon[33332]: pgmap v335: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-04T03:17:18.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:17 smithi077 ceph-mon[33531]: pgmap v335: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-04T03:17:20.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.178+0000 7fd860b7f700 1 -- 172.21.15.1:0/3196912232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd85c1034c0 msgr2=0x7fd85c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:20.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.178+0000 7fd860b7f700 1 --2- 172.21.15.1:0/3196912232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd85c1034c0 0x7fd85c1038d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0099c0 tx=0x7fd84c009cd0 comp rx=0 tx=0).stop 2024-05-04T03:17:20.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.179+0000 7fd860b7f700 1 -- 172.21.15.1:0/3196912232 shutdown_connections 2024-05-04T03:17:20.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.179+0000 7fd860b7f700 1 --2- 172.21.15.1:0/3196912232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd85c108740 0x7fd85c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:20.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.179+0000 7fd860b7f700 1 --2- 172.21.15.1:0/3196912232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd85c1034c0 0x7fd85c1038d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:20.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.179+0000 7fd860b7f700 1 --2- 172.21.15.1:0/3196912232 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd85c107df0 0x7fd85c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:20.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.179+0000 7fd860b7f700 1 -- 172.21.15.1:0/3196912232 >> 172.21.15.1:0/3196912232 conn(0x7fd85c0fedf0 msgr2=0x7fd85c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:20.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.179+0000 7fd860b7f700 1 -- 172.21.15.1:0/3196912232 shutdown_connections 2024-05-04T03:17:20.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.179+0000 7fd860b7f700 1 -- 172.21.15.1:0/3196912232 wait complete. 2024-05-04T03:17:20.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.179+0000 7fd860b7f700 1 Processor -- start 2024-05-04T03:17:20.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.180+0000 7fd860b7f700 1 -- start start 2024-05-04T03:17:20.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.180+0000 7fd860b7f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd85c1034c0 0x7fd85c19ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:20.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.180+0000 7fd860b7f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd85c107df0 0x7fd85c19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:20.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.180+0000 7fd860b7f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd85c108740 0x7fd85c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:20.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.180+0000 7fd860b7f700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c1144e0 con 0x7fd85c1034c0 2024-05-04T03:17:20.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.180+0000 7fd860b7f700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c114360 con 0x7fd85c107df0 2024-05-04T03:17:20.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.180+0000 7fd860b7f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c114660 con 0x7fd85c108740 2024-05-04T03:17:20.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.180+0000 7fd85a59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd85c1034c0 0x7fd85c19ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:20.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.180+0000 7fd85a59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd85c1034c0 0x7fd85c19ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38432/0 (socket says 172.21.15.1:38432) 2024-05-04T03:17:20.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.180+0000 7fd85a59c700 1 -- 172.21.15.1:0/47972530 learned_addr learned my addr 172.21.15.1:0/47972530 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:20.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.180+0000 7fd859d9b700 1 --2- 172.21.15.1:0/47972530 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd85c107df0 0x7fd85c19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:20.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.180+0000 7fd85ad9d700 1 --2- 172.21.15.1:0/47972530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd85c108740 0x7fd85c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:20.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.180+0000 7fd85a59c700 1 -- 172.21.15.1:0/47972530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd85c108740 msgr2=0x7fd85c19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:20.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.180+0000 7fd85a59c700 1 --2- 172.21.15.1:0/47972530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd85c108740 0x7fd85c19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:20.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.180+0000 7fd85a59c700 1 -- 172.21.15.1:0/47972530 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd85c107df0 msgr2=0x7fd85c19b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:20.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.180+0000 7fd85a59c700 1 --2- 172.21.15.1:0/47972530 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd85c107df0 0x7fd85c19b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:20.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.180+0000 7fd85a59c700 1 -- 172.21.15.1:0/47972530 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd85c19fd20 con 0x7fd85c1034c0 2024-05-04T03:17:20.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.180+0000 7fd85a59c700 1 --2- 172.21.15.1:0/47972530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd85c1034c0 0x7fd85c19ae50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00b580 tx=0x7fd84c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:20.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.181+0000 7fd84b7fe700 1 -- 172.21.15.1:0/47972530 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c027070 con 0x7fd85c1034c0 2024-05-04T03:17:20.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.181+0000 7fd860b7f700 1 -- 172.21.15.1:0/47972530 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd85c19ffb0 con 0x7fd85c1034c0 2024-05-04T03:17:20.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.181+0000 7fd84b7fe700 1 -- 172.21.15.1:0/47972530 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd84c003f20 con 0x7fd85c1034c0 2024-05-04T03:17:20.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.181+0000 7fd84b7fe700 1 -- 172.21.15.1:0/47972530 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c022d10 con 0x7fd85c1034c0 2024-05-04T03:17:20.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.181+0000 7fd860b7f700 1 -- 172.21.15.1:0/47972530 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd85c1a0490 con 0x7fd85c1034c0 2024-05-04T03:17:20.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.182+0000 7fd860b7f700 1 -- 172.21.15.1:0/47972530 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd85c0512c0 con 0x7fd85c1034c0 2024-05-04T03:17:20.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.184+0000 7fd84b7fe700 1 -- 172.21.15.1:0/47972530 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fd84c022470 con 0x7fd85c1034c0 2024-05-04T03:17:20.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.184+0000 7fd84b7fe700 1 --2- 172.21.15.1:0/47972530 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd8440b2910 0x7fd8440b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:20.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.184+0000 7fd859d9b700 1 --2- 172.21.15.1:0/47972530 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd8440b2910 0x7fd8440b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:20.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.184+0000 7fd84b7fe700 1 -- 172.21.15.1:0/47972530 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fd84c0e31d0 con 0x7fd85c1034c0 2024-05-04T03:17:20.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.185+0000 7fd859d9b700 1 --2- 172.21.15.1:0/47972530 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd8440b2910 0x7fd8440b4e20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd85c102400 tx=0x7fd850009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:20.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.188+0000 7fd84b7fe700 1 -- 172.21.15.1:0/47972530 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd84c013070 con 0x7fd85c1034c0 2024-05-04T03:17:20.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:20 smithi001 ceph-mon[29731]: pgmap v336: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-05-04T03:17:20.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:20 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/547093836' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-04T03:17:20.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.402+0000 7fd860b7f700 1 -- 172.21.15.1:0/47972530 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fd85c02e5f0 con 0x7fd85c1034c0 2024-05-04T03:17:20.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.403+0000 7fd84b7fe700 1 -- 172.21.15.1:0/47972530 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd84c0aaf40 con 0x7fd85c1034c0 2024-05-04T03:17:20.405 INFO:teuthology.orchestra.run.smithi001.stdout:287762808871 2024-05-04T03:17:20.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.405+0000 7fd860b7f700 1 -- 172.21.15.1:0/47972530 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd8440b2910 msgr2=0x7fd8440b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:20.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.405+0000 7fd860b7f700 1 --2- 172.21.15.1:0/47972530 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd8440b2910 0x7fd8440b4e20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd85c102400 tx=0x7fd850009190 comp rx=0 tx=0).stop 2024-05-04T03:17:20.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.405+0000 7fd860b7f700 1 -- 172.21.15.1:0/47972530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd85c1034c0 msgr2=0x7fd85c19ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:20.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.405+0000 7fd860b7f700 1 --2- 172.21.15.1:0/47972530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd85c1034c0 0x7fd85c19ae50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00b580 tx=0x7fd84c0047b0 comp rx=0 tx=0).stop 2024-05-04T03:17:20.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.405+0000 7fd860b7f700 1 -- 172.21.15.1:0/47972530 shutdown_connections 2024-05-04T03:17:20.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.405+0000 7fd860b7f700 1 --2- 172.21.15.1:0/47972530 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd8440b2910 0x7fd8440b4e20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:20.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.405+0000 7fd860b7f700 1 --2- 172.21.15.1:0/47972530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd85c108740 0x7fd85c19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:20.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.405+0000 7fd860b7f700 1 --2- 172.21.15.1:0/47972530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd85c1034c0 0x7fd85c19ae50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:20.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.405+0000 7fd860b7f700 1 --2- 172.21.15.1:0/47972530 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd85c107df0 0x7fd85c19b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:20.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.405+0000 7fd860b7f700 1 -- 172.21.15.1:0/47972530 >> 172.21.15.1:0/47972530 conn(0x7fd85c0fedf0 msgr2=0x7fd85c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:20.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.405+0000 7fd860b7f700 1 -- 172.21.15.1:0/47972530 shutdown_connections 2024-05-04T03:17:20.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:20.405+0000 7fd860b7f700 1 -- 172.21.15.1:0/47972530 wait complete. 2024-05-04T03:17:20.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:20 smithi072 ceph-mon[33332]: pgmap v336: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-05-04T03:17:20.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:20 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/547093836' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-04T03:17:20.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:20 smithi077 ceph-mon[33531]: pgmap v336: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-05-04T03:17:20.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:20 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/547093836' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-04T03:17:21.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.592+0000 7f2024520700 1 -- 172.21.15.1:0/1014043634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f201c107df0 msgr2=0x7f201c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:21.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.592+0000 7f2024520700 1 --2- 172.21.15.1:0/1014043634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f201c107df0 0x7f201c108200 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f200c0099c0 tx=0x7f200c009cd0 comp rx=0 tx=0).stop 2024-05-04T03:17:21.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.592+0000 7f2024520700 1 -- 172.21.15.1:0/1014043634 shutdown_connections 2024-05-04T03:17:21.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.592+0000 7f2024520700 1 --2- 172.21.15.1:0/1014043634 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f201c1034c0 0x7f201c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:21.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.592+0000 7f2024520700 1 --2- 172.21.15.1:0/1014043634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f201c108740 0x7f201c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:21.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.592+0000 7f2024520700 1 --2- 172.21.15.1:0/1014043634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f201c107df0 0x7f201c108200 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:21.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.592+0000 7f2024520700 1 -- 172.21.15.1:0/1014043634 >> 172.21.15.1:0/1014043634 conn(0x7f201c0fedc0 msgr2=0x7f201c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:21.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.592+0000 7f2024520700 1 -- 172.21.15.1:0/1014043634 shutdown_connections 2024-05-04T03:17:21.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.592+0000 7f2024520700 1 -- 172.21.15.1:0/1014043634 wait complete. 2024-05-04T03:17:21.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.593+0000 7f2024520700 1 Processor -- start 2024-05-04T03:17:21.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.593+0000 7f2024520700 1 -- start start 2024-05-04T03:17:21.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.593+0000 7f2024520700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f201c1034c0 0x7f201c19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:21.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.593+0000 7f2024520700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f201c107df0 0x7f201c19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:21.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.593+0000 7f2024520700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f201c108740 0x7f201c19f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:21.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.593+0000 7f2024520700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f201c1144c0 con 0x7f201c1034c0 2024-05-04T03:17:21.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.593+0000 7f2024520700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f201c114340 con 0x7f201c108740 2024-05-04T03:17:21.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.593+0000 7f2024520700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f201c114640 con 0x7f201c107df0 2024-05-04T03:17:21.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.593+0000 7f20222bc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f201c1034c0 0x7f201c19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:21.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.593+0000 7f20222bc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f201c1034c0 0x7f201c19adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38424/0 (socket says 172.21.15.1:38424) 2024-05-04T03:17:21.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.593+0000 7f20222bc700 1 -- 172.21.15.1:0/547093836 learned_addr learned my addr 172.21.15.1:0/547093836 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:21.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.594+0000 7f2021abb700 1 --2- 172.21.15.1:0/547093836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f201c107df0 0x7f201c19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:21.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.594+0000 7f2022abd700 1 --2- 172.21.15.1:0/547093836 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f201c108740 0x7f201c19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:21.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.594+0000 7f20222bc700 1 -- 172.21.15.1:0/547093836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f201c107df0 msgr2=0x7f201c19b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:21.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.594+0000 7f20222bc700 1 --2- 172.21.15.1:0/547093836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f201c107df0 0x7f201c19b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:21.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.594+0000 7f20222bc700 1 -- 172.21.15.1:0/547093836 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f201c108740 msgr2=0x7f201c19f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:21.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.594+0000 7f20222bc700 1 --2- 172.21.15.1:0/547093836 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f201c108740 0x7f201c19f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:21.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.594+0000 7f20222bc700 1 -- 172.21.15.1:0/547093836 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f201c19fcc0 con 0x7f201c1034c0 2024-05-04T03:17:21.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.594+0000 7f20222bc700 1 --2- 172.21.15.1:0/547093836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f201c1034c0 0x7f201c19adf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f201800d980 tx=0x7f201800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:21.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.594+0000 7f20137fe700 1 -- 172.21.15.1:0/547093836 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2018017070 con 0x7f201c1034c0 2024-05-04T03:17:21.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.594+0000 7f2024520700 1 -- 172.21.15.1:0/547093836 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f201c19ffb0 con 0x7f201c1034c0 2024-05-04T03:17:21.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.594+0000 7f20137fe700 1 -- 172.21.15.1:0/547093836 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2018005020 con 0x7f201c1034c0 2024-05-04T03:17:21.205 INFO:teuthology.orchestra.run.smithi001.stdout:184683593788 2024-05-04T03:17:21.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.594+0000 7f20137fe700 1 -- 172.21.15.1:0/547093836 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2018009b50 con 0x7f201c1034c0 2024-05-04T03:17:21.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.594+0000 7f2024520700 1 -- 172.21.15.1:0/547093836 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f201c1a78d0 con 0x7f201c1034c0 2024-05-04T03:17:21.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.595+0000 7f2024520700 1 -- 172.21.15.1:0/547093836 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f201c104970 con 0x7f201c1034c0 2024-05-04T03:17:21.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.597+0000 7f20137fe700 1 -- 172.21.15.1:0/547093836 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f20180051d0 con 0x7f201c1034c0 2024-05-04T03:17:21.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.597+0000 7f20137fe700 1 --2- 172.21.15.1:0/547093836 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f20080b2910 0x7f20080b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:21.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.597+0000 7f20137fe700 1 -- 172.21.15.1:0/547093836 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f20180e25b0 con 0x7f201c1034c0 2024-05-04T03:17:21.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.597+0000 7f2021abb700 1 --2- 172.21.15.1:0/547093836 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f20080b2910 0x7f20080b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:21.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.598+0000 7f2021abb700 1 --2- 172.21.15.1:0/547093836 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f20080b2910 0x7f20080b4e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f200c006010 tx=0x7f200c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:21.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.601+0000 7f20137fe700 1 -- 172.21.15.1:0/547093836 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2018020020 con 0x7f201c1034c0 2024-05-04T03:17:21.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.818+0000 7f2024520700 1 -- 172.21.15.1:0/547093836 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f201c02e5f0 con 0x7f201c1034c0 2024-05-04T03:17:21.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.818+0000 7f20137fe700 1 -- 172.21.15.1:0/547093836 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f20180aa290 con 0x7f201c1034c0 2024-05-04T03:17:21.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.820+0000 7f2024520700 1 -- 172.21.15.1:0/547093836 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f20080b2910 msgr2=0x7f20080b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:21.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.820+0000 7f2024520700 1 --2- 172.21.15.1:0/547093836 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f20080b2910 0x7f20080b4e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f200c006010 tx=0x7f200c000bc0 comp rx=0 tx=0).stop 2024-05-04T03:17:21.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.820+0000 7f2024520700 1 -- 172.21.15.1:0/547093836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f201c1034c0 msgr2=0x7f201c19adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:21.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.820+0000 7f2024520700 1 --2- 172.21.15.1:0/547093836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f201c1034c0 0x7f201c19adf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f201800d980 tx=0x7f201800dc90 comp rx=0 tx=0).stop 2024-05-04T03:17:21.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.820+0000 7f2024520700 1 -- 172.21.15.1:0/547093836 shutdown_connections 2024-05-04T03:17:21.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.820+0000 7f2024520700 1 --2- 172.21.15.1:0/547093836 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f201c108740 0x7f201c19f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:21.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.820+0000 7f2024520700 1 --2- 172.21.15.1:0/547093836 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f20080b2910 0x7f20080b4e20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:21.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.820+0000 7f2024520700 1 --2- 172.21.15.1:0/547093836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f201c1034c0 0x7f201c19adf0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:21.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.820+0000 7f2024520700 1 --2- 172.21.15.1:0/547093836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f201c107df0 0x7f201c19b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:21.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.820+0000 7f2024520700 1 -- 172.21.15.1:0/547093836 >> 172.21.15.1:0/547093836 conn(0x7f201c0fedc0 msgr2=0x7f201c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:21.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.820+0000 7f2024520700 1 -- 172.21.15.1:0/547093836 shutdown_connections 2024-05-04T03:17:21.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:19.820+0000 7f2024520700 1 -- 172.21.15.1:0/547093836 wait complete. 2024-05-04T03:17:21.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:21 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/47972530' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-04T03:17:21.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:21 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/47972530' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-04T03:17:21.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:21 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/47972530' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-04T03:17:21.883 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593783 got 184683593788 for osd.6 2024-05-04T03:17:21.884 DEBUG:teuthology.parallel:result is None 2024-05-04T03:17:22.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:22 smithi001 ceph-mon[29731]: pgmap v337: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:17:22.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:22 smithi072 ceph-mon[33332]: pgmap v337: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:17:22.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:22 smithi077 ceph-mon[33531]: pgmap v337: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:17:23.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.013+0000 7fd1ac90a700 1 -- 172.21.15.1:0/4174072554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd1a8108f30 msgr2=0x7fd1a8109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:23.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.013+0000 7fd1ac90a700 1 --2- 172.21.15.1:0/4174072554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd1a8108f30 0x7fd1a8109340 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0099c0 tx=0x7fd19c009cd0 comp rx=0 tx=0).stop 2024-05-04T03:17:23.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.013+0000 7fd1ac90a700 1 -- 172.21.15.1:0/4174072554 shutdown_connections 2024-05-04T03:17:23.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.013+0000 7fd1ac90a700 1 --2- 172.21.15.1:0/4174072554 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1a8104650 0x7fd1a8104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:23.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.013+0000 7fd1ac90a700 1 --2- 172.21.15.1:0/4174072554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1a8109880 0x7fd1a81111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:23.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.013+0000 7fd1ac90a700 1 --2- 172.21.15.1:0/4174072554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd1a8108f30 0x7fd1a8109340 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:23.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.013+0000 7fd1ac90a700 1 -- 172.21.15.1:0/4174072554 >> 172.21.15.1:0/4174072554 conn(0x7fd1a8079f00 msgr2=0x7fd1a807a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:23.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.013+0000 7fd1ac90a700 1 -- 172.21.15.1:0/4174072554 shutdown_connections 2024-05-04T03:17:23.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.013+0000 7fd1ac90a700 1 -- 172.21.15.1:0/4174072554 wait complete. 2024-05-04T03:17:23.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.014+0000 7fd1ac90a700 1 Processor -- start 2024-05-04T03:17:23.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.014+0000 7fd1ac90a700 1 -- start start 2024-05-04T03:17:23.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.014+0000 7fd1ac90a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1a8104650 0x7fd1a819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:23.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.014+0000 7fd1ac90a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1a8108f30 0x7fd1a819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:23.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.014+0000 7fd1ac90a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd1a8109880 0x7fd1a819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:23.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.014+0000 7fd1ac90a700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a8114490 con 0x7fd1a8109880 2024-05-04T03:17:23.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.014+0000 7fd1ac90a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a8114310 con 0x7fd1a8104650 2024-05-04T03:17:23.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.014+0000 7fd1ac90a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a8114610 con 0x7fd1a8108f30 2024-05-04T03:17:23.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.014+0000 7fd1a6d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd1a8109880 0x7fd1a819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:23.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.014+0000 7fd1a6d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd1a8109880 0x7fd1a819f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59276/0 (socket says 172.21.15.1:59276) 2024-05-04T03:17:23.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.014+0000 7fd1a6d9d700 1 -- 172.21.15.1:0/2165080014 learned_addr learned my addr 172.21.15.1:0/2165080014 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:23.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.014+0000 7fd1a659c700 1 --2- 172.21.15.1:0/2165080014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1a8104650 0x7fd1a819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:23.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.014+0000 7fd1a5d9b700 1 --2- 172.21.15.1:0/2165080014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1a8108f30 0x7fd1a819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:23.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.015+0000 7fd1a6d9d700 1 -- 172.21.15.1:0/2165080014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1a8108f30 msgr2=0x7fd1a819b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:23.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.015+0000 7fd1a6d9d700 1 --2- 172.21.15.1:0/2165080014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1a8108f30 0x7fd1a819b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:23.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.015+0000 7fd1a6d9d700 1 -- 172.21.15.1:0/2165080014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1a8104650 msgr2=0x7fd1a819ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:23.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.015+0000 7fd1a6d9d700 1 --2- 172.21.15.1:0/2165080014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1a8104650 0x7fd1a819ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:23.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.015+0000 7fd1a6d9d700 1 -- 172.21.15.1:0/2165080014 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1a819fd10 con 0x7fd1a8109880 2024-05-04T03:17:23.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.015+0000 7fd1a6d9d700 1 --2- 172.21.15.1:0/2165080014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd1a8109880 0x7fd1a819f670 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0007ee0 tx=0x7fd1a000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:23.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.015+0000 7fd1977fe700 1 -- 172.21.15.1:0/2165080014 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a000dd20 con 0x7fd1a8109880 2024-05-04T03:17:23.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.015+0000 7fd1ac90a700 1 -- 172.21.15.1:0/2165080014 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1a81a0000 con 0x7fd1a8109880 2024-05-04T03:17:23.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.015+0000 7fd1977fe700 1 -- 172.21.15.1:0/2165080014 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1a0004020 con 0x7fd1a8109880 2024-05-04T03:17:23.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.015+0000 7fd1977fe700 1 -- 172.21.15.1:0/2165080014 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0015660 con 0x7fd1a8109880 2024-05-04T03:17:23.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.015+0000 7fd1ac90a700 1 -- 172.21.15.1:0/2165080014 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1a81a7920 con 0x7fd1a8109880 2024-05-04T03:17:23.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.016+0000 7fd1ac90a700 1 -- 172.21.15.1:0/2165080014 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1a8105ab0 con 0x7fd1a8109880 2024-05-04T03:17:23.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.018+0000 7fd1977fe700 1 -- 172.21.15.1:0/2165080014 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fd1a00041d0 con 0x7fd1a8109880 2024-05-04T03:17:23.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.018+0000 7fd1977fe700 1 --2- 172.21.15.1:0/2165080014 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd1900b2910 0x7fd1900b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:23.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.018+0000 7fd1a659c700 1 --2- 172.21.15.1:0/2165080014 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd1900b2910 0x7fd1900b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:23.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.019+0000 7fd1977fe700 1 -- 172.21.15.1:0/2165080014 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fd1a00e23c0 con 0x7fd1a8109880 2024-05-04T03:17:23.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.019+0000 7fd1a659c700 1 --2- 172.21.15.1:0/2165080014 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd1900b2910 0x7fd1900b4e20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd198005950 tx=0x7fd19800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:23.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.022+0000 7fd1977fe700 1 -- 172.21.15.1:0/2165080014 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd1a0010070 con 0x7fd1a8109880 2024-05-04T03:17:23.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.203+0000 7ffae5a81700 1 -- 172.21.15.1:0/1830664669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae01045f0 msgr2=0x7ffae0104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:23.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.203+0000 7ffae5a81700 1 --2- 172.21.15.1:0/1830664669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae01045f0 0x7ffae0104a00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ffad00099c0 tx=0x7ffad0009cd0 comp rx=0 tx=0).stop 2024-05-04T03:17:23.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.203+0000 7ffae5a81700 1 -- 172.21.15.1:0/1830664669 shutdown_connections 2024-05-04T03:17:23.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.203+0000 7ffae5a81700 1 --2- 172.21.15.1:0/1830664669 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffae0109870 0x7ffae01111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:23.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.203+0000 7ffae5a81700 1 --2- 172.21.15.1:0/1830664669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae01045f0 0x7ffae0104a00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:23.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.203+0000 7ffae5a81700 1 --2- 172.21.15.1:0/1830664669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffae0108f20 0x7ffae0109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:23.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.203+0000 7ffae5a81700 1 -- 172.21.15.1:0/1830664669 >> 172.21.15.1:0/1830664669 conn(0x7ffae0100040 msgr2=0x7ffae0102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:23.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.203+0000 7ffae5a81700 1 -- 172.21.15.1:0/1830664669 shutdown_connections 2024-05-04T03:17:23.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.203+0000 7ffae5a81700 1 -- 172.21.15.1:0/1830664669 wait complete. 2024-05-04T03:17:23.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.204+0000 7ffae5a81700 1 Processor -- start 2024-05-04T03:17:23.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.204+0000 7ffae5a81700 1 -- start start 2024-05-04T03:17:23.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.204+0000 7ffae5a81700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae01045f0 0x7ffae019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:23.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.204+0000 7ffae5a81700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffae0108f20 0x7ffae019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:23.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.204+0000 7ffae5a81700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffae0109870 0x7ffae019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:23.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.204+0000 7ffae5a81700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffae01144a0 con 0x7ffae01045f0 2024-05-04T03:17:23.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.204+0000 7ffae5a81700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ffae0114320 con 0x7ffae0108f20 2024-05-04T03:17:23.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.204+0000 7ffae5a81700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffae0114620 con 0x7ffae0109870 2024-05-04T03:17:23.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.205+0000 7ffadf7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae01045f0 0x7ffae019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:23.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.205+0000 7ffadf7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae01045f0 0x7ffae019ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59288/0 (socket says 172.21.15.1:59288) 2024-05-04T03:17:23.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.205+0000 7ffadf7fe700 1 -- 172.21.15.1:0/1158778345 learned_addr learned my addr 172.21.15.1:0/1158778345 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:23.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.205+0000 7ffadeffd700 1 --2- 172.21.15.1:0/1158778345 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffae0108f20 0x7ffae019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:23.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.205+0000 7ffadffff700 1 --2- 172.21.15.1:0/1158778345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffae0109870 0x7ffae019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:23.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.205+0000 7ffadf7fe700 1 -- 172.21.15.1:0/1158778345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffae0109870 msgr2=0x7ffae019f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:23.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.205+0000 7ffadf7fe700 1 --2- 172.21.15.1:0/1158778345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffae0109870 0x7ffae019f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:23.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.205+0000 7ffadf7fe700 1 -- 172.21.15.1:0/1158778345 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffae0108f20 msgr2=0x7ffae019b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:23.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.205+0000 7ffadf7fe700 1 --2- 172.21.15.1:0/1158778345 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffae0108f20 0x7ffae019b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:23.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.205+0000 7ffadf7fe700 1 -- 172.21.15.1:0/1158778345 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae019fd00 con 0x7ffae01045f0 2024-05-04T03:17:23.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.205+0000 7ffadf7fe700 1 --2- 172.21.15.1:0/1158778345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae01045f0 0x7ffae019ae30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffad0006010 tx=0x7ffad00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:23.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.205+0000 7ffadcff9700 1 -- 172.21.15.1:0/1158778345 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad0026070 con 0x7ffae01045f0 2024-05-04T03:17:23.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.205+0000 7ffadcff9700 1 -- 172.21.15.1:0/1158778345 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffad000f460 con 0x7ffae01045f0 2024-05-04T03:17:23.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.205+0000 7ffae5a81700 1 -- 172.21.15.1:0/1158778345 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffae019ff90 con 0x7ffae01045f0 2024-05-04T03:17:23.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.205+0000 7ffadcff9700 1 -- 172.21.15.1:0/1158778345 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad0009e60 con 0x7ffae01045f0 2024-05-04T03:17:23.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.205+0000 7ffae5a81700 1 -- 172.21.15.1:0/1158778345 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffae01a0470 con 0x7ffae01045f0 2024-05-04T03:17:23.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.206+0000 7ffae5a81700 1 -- 172.21.15.1:0/1158778345 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffae00512c0 con 0x7ffae01045f0 2024-05-04T03:17:23.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.208+0000 7ffadcff9700 1 -- 172.21.15.1:0/1158778345 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7ffad0029070 con 0x7ffae01045f0 2024-05-04T03:17:23.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.209+0000 7ffadcff9700 1 --2- 172.21.15.1:0/1158778345 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ffac80b2910 0x7ffac80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:23.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.209+0000 7ffadeffd700 1 --2- 172.21.15.1:0/1158778345 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ffac80b2910 0x7ffac80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:23.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.209+0000 7ffadcff9700 1 -- 172.21.15.1:0/1158778345 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7ffad00e31b0 con 0x7ffae01045f0 2024-05-04T03:17:23.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.209+0000 7ffadeffd700 1 --2- 172.21.15.1:0/1158778345 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ffac80b2910 0x7ffac80b4e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffae0107580 tx=0x7ffad400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:23.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.212+0000 7ffadcff9700 1 -- 172.21.15.1:0/1158778345 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffad0013070 con 0x7ffae01045f0 2024-05-04T03:17:23.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.239+0000 7fd1ac90a700 1 -- 172.21.15.1:0/2165080014 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fd1a802e5f0 con 0x7fd1a8109880 2024-05-04T03:17:23.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.239+0000 7fd1977fe700 1 -- 172.21.15.1:0/2165080014 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd1a00aa0a0 con 0x7fd1a8109880 2024-05-04T03:17:23.242 INFO:teuthology.orchestra.run.smithi001.stdout:266287972397 2024-05-04T03:17:23.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.241+0000 7fd1ac90a700 1 -- 172.21.15.1:0/2165080014 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd1900b2910 msgr2=0x7fd1900b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:23.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.242+0000 7fd1ac90a700 1 --2- 172.21.15.1:0/2165080014 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd1900b2910 0x7fd1900b4e20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd198005950 tx=0x7fd19800a300 comp rx=0 tx=0).stop 2024-05-04T03:17:23.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.242+0000 7fd1ac90a700 1 -- 172.21.15.1:0/2165080014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd1a8109880 msgr2=0x7fd1a819f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:23.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.242+0000 7fd1ac90a700 1 --2- 172.21.15.1:0/2165080014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd1a8109880 0x7fd1a819f670 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0007ee0 tx=0x7fd1a000d3b0 comp rx=0 tx=0).stop 2024-05-04T03:17:23.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.242+0000 7fd1ac90a700 1 -- 172.21.15.1:0/2165080014 shutdown_connections 2024-05-04T03:17:23.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.242+0000 7fd1ac90a700 1 --2- 172.21.15.1:0/2165080014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1a8104650 0x7fd1a819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:23.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.242+0000 7fd1ac90a700 1 --2- 172.21.15.1:0/2165080014 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd1900b2910 0x7fd1900b4e20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:23.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.242+0000 7fd1ac90a700 1 --2- 172.21.15.1:0/2165080014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd1a8109880 0x7fd1a819f670 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:23.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.242+0000 7fd1ac90a700 1 --2- 172.21.15.1:0/2165080014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1a8108f30 0x7fd1a819b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:23.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.242+0000 7fd1ac90a700 1 -- 172.21.15.1:0/2165080014 >> 172.21.15.1:0/2165080014 conn(0x7fd1a8079f00 msgr2=0x7fd1a8102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:23.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.242+0000 7fd1ac90a700 1 -- 172.21.15.1:0/2165080014 shutdown_connections 2024-05-04T03:17:23.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.242+0000 7fd1ac90a700 1 -- 172.21.15.1:0/2165080014 wait complete. 2024-05-04T03:17:23.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.432+0000 7ffae5a81700 1 -- 172.21.15.1:0/1158778345 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7ffae002e5f0 con 0x7ffae01045f0 2024-05-04T03:17:23.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.432+0000 7ffadcff9700 1 -- 172.21.15.1:0/1158778345 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffad00aaf20 con 0x7ffae01045f0 2024-05-04T03:17:23.435 INFO:teuthology.orchestra.run.smithi001.stdout:77309411409 2024-05-04T03:17:23.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.434+0000 7ffae5a81700 1 -- 172.21.15.1:0/1158778345 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ffac80b2910 msgr2=0x7ffac80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:23.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.434+0000 7ffae5a81700 1 --2- 172.21.15.1:0/1158778345 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ffac80b2910 0x7ffac80b4e20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffae0107580 tx=0x7ffad400b410 comp rx=0 tx=0).stop 2024-05-04T03:17:23.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.434+0000 7ffae5a81700 1 -- 172.21.15.1:0/1158778345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae01045f0 msgr2=0x7ffae019ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:23.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.434+0000 7ffae5a81700 1 --2- 172.21.15.1:0/1158778345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae01045f0 0x7ffae019ae30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffad0006010 tx=0x7ffad00045e0 comp rx=0 tx=0).stop 2024-05-04T03:17:23.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.434+0000 7ffae5a81700 1 -- 172.21.15.1:0/1158778345 shutdown_connections 2024-05-04T03:17:23.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.434+0000 7ffae5a81700 1 --2- 172.21.15.1:0/1158778345 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ffac80b2910 0x7ffac80b4e20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:23.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.434+0000 7ffae5a81700 1 --2- 172.21.15.1:0/1158778345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffae0109870 0x7ffae019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:23.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.434+0000 7ffae5a81700 1 --2- 172.21.15.1:0/1158778345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffae01045f0 0x7ffae019ae30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:23.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.434+0000 7ffae5a81700 1 --2- 172.21.15.1:0/1158778345 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffae0108f20 0x7ffae019b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:23.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.434+0000 7ffae5a81700 1 -- 172.21.15.1:0/1158778345 >> 172.21.15.1:0/1158778345 conn(0x7ffae0100040 msgr2=0x7ffae0102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:23.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.434+0000 7ffae5a81700 1 -- 172.21.15.1:0/1158778345 shutdown_connections 2024-05-04T03:17:23.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:23.434+0000 7ffae5a81700 1 -- 172.21.15.1:0/1158778345 wait complete. 2024-05-04T03:17:24.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:24 smithi072 ceph-mon[33332]: pgmap v338: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:17:24.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:24 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/2165080014' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-04T03:17:24.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:24 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/1158778345' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-04T03:17:24.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:24 smithi001 ceph-mon[29731]: pgmap v338: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:17:24.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:24 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2165080014' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-04T03:17:24.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:24 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/1158778345' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-04T03:17:24.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:24 smithi077 ceph-mon[33531]: pgmap v338: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:17:24.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:24 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/2165080014' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-04T03:17:24.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:24 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/1158778345' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-04T03:17:25.135 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808868 got 287762808871 for osd.10 2024-05-04T03:17:25.135 DEBUG:teuthology.parallel:result is None 2024-05-04T03:17:25.952 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411405 got 77309411409 for osd.2 2024-05-04T03:17:25.973 DEBUG:teuthology.parallel:result is None 2024-05-04T03:17:26.496 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:26 smithi072 ceph-mon[33332]: pgmap v339: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-04T03:17:26.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:26 smithi001 ceph-mon[29731]: pgmap v339: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-04T03:17:26.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:26 smithi077 ceph-mon[33531]: pgmap v339: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-04T03:17:26.935 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972391 got 266287972397 for osd.9 2024-05-04T03:17:26.935 DEBUG:teuthology.parallel:result is None 2024-05-04T03:17:27.250 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:17:27.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:27 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/649371169' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-04T03:17:27.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:27 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/649371169' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-04T03:17:27.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:27 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/649371169' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-04T03:17:28.389 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:28 smithi001 ceph-mon[29731]: pgmap v340: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-04T03:17:28.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.632+0000 7fc9d67e7700 1 -- 172.21.15.1:0/1645486144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9d0108780 msgr2=0x7fc9d01100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:28.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.632+0000 7fc9d67e7700 1 --2- 172.21.15.1:0/1645486144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9d0108780 0x7fc9d01100e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc9c800b0d0 tx=0x7fc9c800b3e0 comp rx=0 tx=0).stop 2024-05-04T03:17:28.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.632+0000 7fc9d67e7700 1 -- 172.21.15.1:0/1645486144 shutdown_connections 2024-05-04T03:17:28.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.632+0000 7fc9d67e7700 1 --2- 172.21.15.1:0/1645486144 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9d0103550 0x7fc9d0103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:28.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.632+0000 7fc9d67e7700 1 --2- 172.21.15.1:0/1645486144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9d0108780 0x7fc9d01100e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:28.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.632+0000 7fc9d67e7700 1 --2- 172.21.15.1:0/1645486144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9d0107e30 0x7fc9d0108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:28.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.632+0000 7fc9d67e7700 1 -- 172.21.15.1:0/1645486144 >> 172.21.15.1:0/1645486144 conn(0x7fc9d0078e00 msgr2=0x7fc9d0079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:28.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.632+0000 7fc9d67e7700 1 -- 172.21.15.1:0/1645486144 shutdown_connections 2024-05-04T03:17:28.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.632+0000 7fc9d67e7700 1 -- 172.21.15.1:0/1645486144 wait complete. 2024-05-04T03:17:28.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.633+0000 7fc9d67e7700 1 Processor -- start 2024-05-04T03:17:28.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.633+0000 7fc9d67e7700 1 -- start start 2024-05-04T03:17:28.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.634+0000 7fc9d67e7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9d0103550 0x7fc9d019f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:28.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.634+0000 7fc9d67e7700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9d0107e30 0x7fc9d019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:28.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.634+0000 7fc9d67e7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9d0108780 0x7fc9d0199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:28.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.634+0000 7fc9d67e7700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01133b0 con 0x7fc9d0107e30 2024-05-04T03:17:28.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.634+0000 7fc9d67e7700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d0113230 con 0x7fc9d0103550 2024-05-04T03:17:28.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.634+0000 7fc9d67e7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d0113530 con 0x7fc9d0108780 2024-05-04T03:17:28.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.634+0000 7fc9cf7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9d0107e30 0x7fc9d019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:28.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.634+0000 7fc9cf7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9d0107e30 0x7fc9d019fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59314/0 (socket says 172.21.15.1:59314) 2024-05-04T03:17:28.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.634+0000 7fc9cf7fe700 1 -- 172.21.15.1:0/3731825953 learned_addr learned my addr 172.21.15.1:0/3731825953 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:28.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.634+0000 7fc9cffff700 1 --2- 172.21.15.1:0/3731825953 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9d0103550 0x7fc9d019f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:28.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.634+0000 7fc9d4d84700 1 --2- 172.21.15.1:0/3731825953 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9d0108780 0x7fc9d0199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:28.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.634+0000 7fc9cf7fe700 1 -- 172.21.15.1:0/3731825953 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9d0108780 msgr2=0x7fc9d0199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:28.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.634+0000 7fc9cf7fe700 1 --2- 172.21.15.1:0/3731825953 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9d0108780 0x7fc9d0199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:28.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.634+0000 7fc9cf7fe700 1 -- 172.21.15.1:0/3731825953 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9d0103550 msgr2=0x7fc9d019f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:28.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.634+0000 7fc9cf7fe700 1 --2- 172.21.15.1:0/3731825953 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9d0103550 0x7fc9d019f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:28.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.634+0000 7fc9cf7fe700 1 -- 172.21.15.1:0/3731825953 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d019a160 con 0x7fc9d0107e30 2024-05-04T03:17:28.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.634+0000 7fc9d4d84700 1 --2- 172.21.15.1:0/3731825953 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9d0108780 0x7fc9d0199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:17:28.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.634+0000 7fc9cf7fe700 1 --2- 172.21.15.1:0/3731825953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9d0107e30 0x7fc9d019fe10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400c9e0 tx=0x7fc9c400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:28.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.635+0000 7fc9cd7fa700 1 -- 172.21.15.1:0/3731825953 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4007aa0 con 0x7fc9d0107e30 2024-05-04T03:17:28.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.635+0000 7fc9cd7fa700 1 -- 172.21.15.1:0/3731825953 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9c4007c40 con 0x7fc9d0107e30 2024-05-04T03:17:28.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.635+0000 7fc9d67e7700 1 -- 172.21.15.1:0/3731825953 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d019a450 con 0x7fc9d0107e30 2024-05-04T03:17:28.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.635+0000 7fc9cd7fa700 1 -- 172.21.15.1:0/3731825953 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4018660 con 0x7fc9d0107e30 2024-05-04T03:17:28.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.635+0000 7fc9d67e7700 1 -- 172.21.15.1:0/3731825953 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d01a6760 con 0x7fc9d0107e30 2024-05-04T03:17:28.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.636+0000 7fc9d67e7700 1 -- 172.21.15.1:0/3731825953 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9d00512c0 con 0x7fc9d0107e30 2024-05-04T03:17:28.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.638+0000 7fc9cd7fa700 1 -- 172.21.15.1:0/3731825953 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fc9c4018800 con 0x7fc9d0107e30 2024-05-04T03:17:28.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.639+0000 7fc9cd7fa700 1 --2- 172.21.15.1:0/3731825953 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc9b80b2990 0x7fc9b80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:28.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.639+0000 7fc9cffff700 1 --2- 172.21.15.1:0/3731825953 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc9b80b2990 0x7fc9b80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:28.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.639+0000 7fc9cd7fa700 1 -- 172.21.15.1:0/3731825953 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fc9c40e24d0 con 0x7fc9d0107e30 2024-05-04T03:17:28.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.639+0000 7fc9cffff700 1 --2- 172.21.15.1:0/3731825953 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc9b80b2990 0x7fc9b80b4ea0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc9d019ac30 tx=0x7fc9c000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:28.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.644+0000 7fc9cd7fa700 1 -- 172.21.15.1:0/3731825953 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc9c4014030 con 0x7fc9d0107e30 2024-05-04T03:17:28.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:28 smithi077 ceph-mon[33531]: pgmap v340: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-04T03:17:28.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:28 smithi072 ceph-mon[33332]: pgmap v340: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-04T03:17:28.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.866+0000 7fc9d67e7700 1 -- 172.21.15.1:0/3731825953 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fc9d01a6930 con 0x7fc9d0107e30 2024-05-04T03:17:28.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.867+0000 7fc9cd7fa700 1 -- 172.21.15.1:0/3731825953 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc9c40aa1c0 con 0x7fc9d0107e30 2024-05-04T03:17:28.870 INFO:teuthology.orchestra.run.smithi001.stdout:133143986247 2024-05-04T03:17:28.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.869+0000 7fc9d67e7700 1 -- 172.21.15.1:0/3731825953 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc9b80b2990 msgr2=0x7fc9b80b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:28.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.869+0000 7fc9d67e7700 1 --2- 172.21.15.1:0/3731825953 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc9b80b2990 0x7fc9b80b4ea0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc9d019ac30 tx=0x7fc9c000b410 comp rx=0 tx=0).stop 2024-05-04T03:17:28.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.869+0000 7fc9d67e7700 1 -- 172.21.15.1:0/3731825953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9d0107e30 msgr2=0x7fc9d019fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:28.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.869+0000 7fc9d67e7700 1 --2- 172.21.15.1:0/3731825953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9d0107e30 0x7fc9d019fe10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400c9e0 tx=0x7fc9c400ccf0 comp rx=0 tx=0).stop 2024-05-04T03:17:28.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.869+0000 7fc9d67e7700 1 -- 172.21.15.1:0/3731825953 shutdown_connections 2024-05-04T03:17:28.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.869+0000 7fc9d67e7700 1 --2- 172.21.15.1:0/3731825953 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9d0103550 0x7fc9d019f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:28.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.869+0000 7fc9d67e7700 1 --2- 172.21.15.1:0/3731825953 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc9b80b2990 0x7fc9b80b4ea0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:28.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.869+0000 7fc9d67e7700 1 --2- 172.21.15.1:0/3731825953 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9d0108780 0x7fc9d0199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:28.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.869+0000 7fc9d67e7700 1 --2- 172.21.15.1:0/3731825953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9d0107e30 0x7fc9d019fe10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:28.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.869+0000 7fc9d67e7700 1 -- 172.21.15.1:0/3731825953 >> 172.21.15.1:0/3731825953 conn(0x7fc9d0078e00 msgr2=0x7fc9d0101e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:28.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.869+0000 7fc9d67e7700 1 -- 172.21.15.1:0/3731825953 shutdown_connections 2024-05-04T03:17:28.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:28.869+0000 7fc9d67e7700 1 -- 172.21.15.1:0/3731825953 wait complete. 2024-05-04T03:17:29.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:29 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3731825953' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-04T03:17:29.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:29 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/3731825953' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-04T03:17:29.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:29 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/3731825953' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-04T03:17:30.497 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:30 smithi001 ceph-mon[29731]: pgmap v341: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-04T03:17:30.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.510+0000 7f2024ba8700 1 -- 172.21.15.1:0/1959703339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20201034e0 msgr2=0x7f20201038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:30.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.510+0000 7f2024ba8700 1 --2- 172.21.15.1:0/1959703339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20201034e0 0x7f20201038f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f20100099c0 tx=0x7f2010009cd0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.510+0000 7f2024ba8700 1 -- 172.21.15.1:0/1959703339 shutdown_connections 2024-05-04T03:17:30.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.510+0000 7f2024ba8700 1 --2- 172.21.15.1:0/1959703339 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2020108760 0x7f20201100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.510+0000 7f2024ba8700 1 --2- 172.21.15.1:0/1959703339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20201034e0 0x7f20201038f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.510+0000 7f2024ba8700 1 --2- 172.21.15.1:0/1959703339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2020107e10 0x7f2020108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.510+0000 7f2024ba8700 1 -- 172.21.15.1:0/1959703339 >> 172.21.15.1:0/1959703339 conn(0x7f20200fef30 msgr2=0x7f2020101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:30.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.510+0000 7f2024ba8700 1 -- 172.21.15.1:0/1959703339 shutdown_connections 2024-05-04T03:17:30.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.510+0000 7f2024ba8700 1 -- 172.21.15.1:0/1959703339 wait complete. 2024-05-04T03:17:30.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.511+0000 7f2024ba8700 1 Processor -- start 2024-05-04T03:17:30.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.511+0000 7f2024ba8700 1 -- start start 2024-05-04T03:17:30.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.511+0000 7f2024ba8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20201034e0 0x7f202019bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:30.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.511+0000 7f2024ba8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2020107e10 0x7f202019c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:30.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.511+0000 7f2024ba8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2020108760 0x7f202019c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:30.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.512+0000 7f2024ba8700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2020113390 con 0x7f2020108760 2024-05-04T03:17:30.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.512+0000 7f2024ba8700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2020113210 con 0x7f20201034e0 2024-05-04T03:17:30.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.512+0000 7f2024ba8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2020113510 con 0x7f2020107e10 2024-05-04T03:17:30.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.512+0000 7f201ed9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2020108760 0x7f202019c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:30.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.512+0000 7f201ed9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2020108760 0x7f202019c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59322/0 (socket says 172.21.15.1:59322) 2024-05-04T03:17:30.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.512+0000 7f201ed9d700 1 -- 172.21.15.1:0/4015430035 learned_addr learned my addr 172.21.15.1:0/4015430035 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:30.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.512+0000 7f201e59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20201034e0 0x7f202019bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:30.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.512+0000 7f201dd9b700 1 --2- 172.21.15.1:0/4015430035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2020107e10 0x7f202019c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:30.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.512+0000 7f201ed9d700 1 -- 172.21.15.1:0/4015430035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2020107e10 msgr2=0x7f202019c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:30.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.512+0000 7f201ed9d700 1 --2- 172.21.15.1:0/4015430035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2020107e10 0x7f202019c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.512+0000 7f201ed9d700 1 -- 172.21.15.1:0/4015430035 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20201034e0 msgr2=0x7f202019bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:30.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.512+0000 7f201ed9d700 1 --2- 172.21.15.1:0/4015430035 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20201034e0 0x7f202019bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.512+0000 7f201ed9d700 1 -- 172.21.15.1:0/4015430035 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20201a7090 con 0x7f2020108760 2024-05-04T03:17:30.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.512+0000 7f201e59c700 1 --2- 172.21.15.1:0/4015430035 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20201034e0 0x7f202019bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:17:30.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.512+0000 7f201ed9d700 1 --2- 172.21.15.1:0/4015430035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2020108760 0x7f202019c7f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f201800ef30 tx=0x7f201800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:30.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.512+0000 7f200f7fe700 1 -- 172.21.15.1:0/4015430035 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20180092e0 con 0x7f2020108760 2024-05-04T03:17:30.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.512+0000 7f2024ba8700 1 -- 172.21.15.1:0/4015430035 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20201a7320 con 0x7f2020108760 2024-05-04T03:17:30.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.512+0000 7f200f7fe700 1 -- 172.21.15.1:0/4015430035 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2018004bb0 con 0x7f2020108760 2024-05-04T03:17:30.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.512+0000 7f200f7fe700 1 -- 172.21.15.1:0/4015430035 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2018004280 con 0x7f2020108760 2024-05-04T03:17:30.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.513+0000 7f2024ba8700 1 -- 172.21.15.1:0/4015430035 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20201a7860 con 0x7f2020108760 2024-05-04T03:17:30.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.514+0000 7f2024ba8700 1 -- 172.21.15.1:0/4015430035 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20200512c0 con 0x7f2020108760 2024-05-04T03:17:30.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.515+0000 7f200f7fe700 1 -- 172.21.15.1:0/4015430035 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f201801b030 con 0x7f2020108760 2024-05-04T03:17:30.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.516+0000 7f200f7fe700 1 --2- 172.21.15.1:0/4015430035 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f20080b2910 0x7f20080b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:30.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.516+0000 7f201e59c700 1 --2- 172.21.15.1:0/4015430035 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f20080b2910 0x7f20080b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:30.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.516+0000 7f200f7fe700 1 -- 172.21.15.1:0/4015430035 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f20180e1d80 con 0x7f2020108760 2024-05-04T03:17:30.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.516+0000 7f201e59c700 1 --2- 172.21.15.1:0/4015430035 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f20080b2910 0x7f20080b4e20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2010000c00 tx=0x7f201001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:30.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.521+0000 7f200f7fe700 1 -- 172.21.15.1:0/4015430035 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2018022020 con 0x7f2020108760 2024-05-04T03:17:30.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:30 smithi077 ceph-mon[33531]: pgmap v341: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-04T03:17:30.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.719+0000 7f0348b57700 1 -- 172.21.15.1:0/2944491535 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03441045e0 msgr2=0x7f03441049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:30.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.719+0000 7f0348b57700 1 --2- 172.21.15.1:0/2944491535 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03441045e0 0x7f03441049f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f03340099c0 tx=0x7f0334009cd0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.719+0000 7f0348b57700 1 -- 172.21.15.1:0/2944491535 shutdown_connections 2024-05-04T03:17:30.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.719+0000 7f0348b57700 1 --2- 172.21.15.1:0/2944491535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0344109860 0x7f03441111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.719+0000 7f0348b57700 1 --2- 172.21.15.1:0/2944491535 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03441045e0 0x7f03441049f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.719+0000 7f0348b57700 1 --2- 172.21.15.1:0/2944491535 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0344108f10 0x7f0344109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.719+0000 7f0348b57700 1 -- 172.21.15.1:0/2944491535 >> 172.21.15.1:0/2944491535 conn(0x7f03440ffe70 msgr2=0x7f03441022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:30.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.719+0000 7f0348b57700 1 -- 172.21.15.1:0/2944491535 shutdown_connections 2024-05-04T03:17:30.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.719+0000 7f0348b57700 1 -- 172.21.15.1:0/2944491535 wait complete. 2024-05-04T03:17:30.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.720+0000 7f0348b57700 1 Processor -- start 2024-05-04T03:17:30.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.720+0000 7f0348b57700 1 -- start start 2024-05-04T03:17:30.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.720+0000 7f0348b57700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03441045e0 0x7f0344075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:30.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.720+0000 7f0348b57700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0344108f10 0x7f0344076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:30.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.720+0000 7f0348b57700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0344109860 0x7f0344076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.720+0000 7f0348b57700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0344114490 con 0x7f0344109860 2024-05-04T03:17:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.720+0000 7f0348b57700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0344114310 con 0x7f03441045e0 2024-05-04T03:17:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.720+0000 7f0348b57700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0344114610 con 0x7f0344108f10 2024-05-04T03:17:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.720+0000 7f034259c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03441045e0 0x7f0344075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.720+0000 7f034259c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03441045e0 0x7f0344075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:42756/0 (socket says 172.21.15.1:42756) 2024-05-04T03:17:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.720+0000 7f034259c700 1 -- 172.21.15.1:0/3634754408 learned_addr learned my addr 172.21.15.1:0/3634754408 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.720+0000 7f0342d9d700 1 --2- 172.21.15.1:0/3634754408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0344109860 0x7f0344076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.721+0000 7f0341d9b700 1 --2- 172.21.15.1:0/3634754408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0344108f10 0x7f0344076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.721+0000 7f034259c700 1 -- 172.21.15.1:0/3634754408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0344108f10 msgr2=0x7f0344076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.721+0000 7f034259c700 1 --2- 172.21.15.1:0/3634754408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0344108f10 0x7f0344076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.721+0000 7f034259c700 1 -- 172.21.15.1:0/3634754408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0344109860 msgr2=0x7f0344076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.721+0000 7f034259c700 1 --2- 172.21.15.1:0/3634754408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0344109860 0x7f0344076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.721+0000 7f034259c700 1 -- 172.21.15.1:0/3634754408 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03441a9220 con 0x7f03441045e0 2024-05-04T03:17:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.721+0000 7f034259c700 1 --2- 172.21.15.1:0/3634754408 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03441045e0 0x7f0344075ac0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f03340098e0 tx=0x7f03340057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.721+0000 7f03337fe700 1 -- 172.21.15.1:0/3634754408 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0334027070 con 0x7f03441045e0 2024-05-04T03:17:30.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.721+0000 7f0348b57700 1 -- 172.21.15.1:0/3634754408 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03441a94b0 con 0x7f03441045e0 2024-05-04T03:17:30.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.721+0000 7f03337fe700 1 -- 172.21.15.1:0/3634754408 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0334003f20 con 0x7f03441045e0 2024-05-04T03:17:30.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.721+0000 7f03337fe700 1 -- 172.21.15.1:0/3634754408 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0334022da0 con 0x7f03441045e0 2024-05-04T03:17:30.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.722+0000 7f0348b57700 1 -- 172.21.15.1:0/3634754408 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03441a9960 con 0x7f03441045e0 2024-05-04T03:17:30.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.722+0000 7f0348b57700 1 -- 172.21.15.1:0/3634754408 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03440512c0 con 0x7f03441045e0 2024-05-04T03:17:30.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.724+0000 7f03337fe700 1 -- 172.21.15.1:0/3634754408 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f033402a070 con 0x7f03441045e0 2024-05-04T03:17:30.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.725+0000 7f03337fe700 1 --2- 172.21.15.1:0/3634754408 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f032c0b2910 0x7f032c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:30.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.725+0000 7f0341d9b700 1 --2- 172.21.15.1:0/3634754408 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f032c0b2910 0x7f032c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:30.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.725+0000 7f03337fe700 1 -- 172.21.15.1:0/3634754408 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f03340e37e0 con 0x7f03441045e0 2024-05-04T03:17:30.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.725+0000 7f0341d9b700 1 --2- 172.21.15.1:0/3634754408 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f032c0b2910 0x7f032c0b4e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0338005950 tx=0x7f03380058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:30.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.729+0000 7f03337fe700 1 -- 172.21.15.1:0/3634754408 <== mon.2 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0334031080 con 0x7f03441045e0 2024-05-04T03:17:30.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.739+0000 7f2024ba8700 1 -- 172.21.15.1:0/4015430035 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f202002e5f0 con 0x7f2020108760 2024-05-04T03:17:30.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.740+0000 7f200f7fe700 1 -- 172.21.15.1:0/4015430035 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2018010040 con 0x7f2020108760 2024-05-04T03:17:30.742 INFO:teuthology.orchestra.run.smithi001.stdout:158913790019 2024-05-04T03:17:30.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.742+0000 7f2024ba8700 1 -- 172.21.15.1:0/4015430035 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f20080b2910 msgr2=0x7f20080b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:30.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.742+0000 7f2024ba8700 1 --2- 172.21.15.1:0/4015430035 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f20080b2910 0x7f20080b4e20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2010000c00 tx=0x7f201001a040 comp rx=0 tx=0).stop 2024-05-04T03:17:30.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.742+0000 7f2024ba8700 1 -- 172.21.15.1:0/4015430035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2020108760 msgr2=0x7f202019c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:30.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.742+0000 7f2024ba8700 1 --2- 172.21.15.1:0/4015430035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2020108760 0x7f202019c7f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f201800ef30 tx=0x7f201800c3a0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.742+0000 7f2024ba8700 1 -- 172.21.15.1:0/4015430035 shutdown_connections 2024-05-04T03:17:30.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.742+0000 7f2024ba8700 1 --2- 172.21.15.1:0/4015430035 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20201034e0 0x7f202019bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.742+0000 7f2024ba8700 1 --2- 172.21.15.1:0/4015430035 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f20080b2910 0x7f20080b4e20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.742+0000 7f2024ba8700 1 --2- 172.21.15.1:0/4015430035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2020108760 0x7f202019c7f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.742+0000 7f2024ba8700 1 --2- 172.21.15.1:0/4015430035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2020107e10 0x7f202019c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.742+0000 7f2024ba8700 1 -- 172.21.15.1:0/4015430035 >> 172.21.15.1:0/4015430035 conn(0x7f20200fef30 msgr2=0x7f2020101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:30.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.742+0000 7f2024ba8700 1 -- 172.21.15.1:0/4015430035 shutdown_connections 2024-05-04T03:17:30.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.742+0000 7f2024ba8700 1 -- 172.21.15.1:0/4015430035 wait complete. 2024-05-04T03:17:30.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:30 smithi072 ceph-mon[33332]: pgmap v341: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-04T03:17:30.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.409+0000 7fe701bce700 1 -- 172.21.15.1:0/1535877436 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6fc1034c0 msgr2=0x7fe6fc1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:30.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.409+0000 7fe701bce700 1 --2- 172.21.15.1:0/1535877436 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6fc1034c0 0x7fe6fc1038d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00b0d0 tx=0x7fe6ec00b3e0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.410+0000 7fe701bce700 1 -- 172.21.15.1:0/1535877436 shutdown_connections 2024-05-04T03:17:30.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.410+0000 7fe701bce700 1 --2- 172.21.15.1:0/1535877436 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe6fc108740 0x7fe6fc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.410+0000 7fe701bce700 1 --2- 172.21.15.1:0/1535877436 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6fc1034c0 0x7fe6fc1038d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.410+0000 7fe701bce700 1 --2- 172.21.15.1:0/1535877436 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6fc107df0 0x7fe6fc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.410+0000 7fe701bce700 1 -- 172.21.15.1:0/1535877436 >> 172.21.15.1:0/1535877436 conn(0x7fe6fc0fedf0 msgr2=0x7fe6fc101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:30.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.410+0000 7fe701bce700 1 -- 172.21.15.1:0/1535877436 shutdown_connections 2024-05-04T03:17:30.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.410+0000 7fe701bce700 1 -- 172.21.15.1:0/1535877436 wait complete. 2024-05-04T03:17:30.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.410+0000 7fe701bce700 1 Processor -- start 2024-05-04T03:17:30.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.411+0000 7fe701bce700 1 -- start start 2024-05-04T03:17:30.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.411+0000 7fe701bce700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6fc1034c0 0x7fe6fc19bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:30.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.411+0000 7fe701bce700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6fc107df0 0x7fe6fc19c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:30.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.411+0000 7fe701bce700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe6fc108740 0x7fe6fc19c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:30.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.411+0000 7fe701bce700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc1133c0 con 0x7fe6fc107df0 2024-05-04T03:17:30.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.411+0000 7fe701bce700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc113240 con 0x7fe6fc108740 2024-05-04T03:17:30.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.411+0000 7fe701bce700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc113540 con 0x7fe6fc1034c0 2024-05-04T03:17:30.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.411+0000 7fe6faffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6fc107df0 0x7fe6fc19c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:30.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.411+0000 7fe6faffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6fc107df0 0x7fe6fc19c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59310/0 (socket says 172.21.15.1:59310) 2024-05-04T03:17:30.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.411+0000 7fe6faffd700 1 -- 172.21.15.1:0/649371169 learned_addr learned my addr 172.21.15.1:0/649371169 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:30.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.411+0000 7fe6fb7fe700 1 --2- 172.21.15.1:0/649371169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6fc1034c0 0x7fe6fc19bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:30.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.411+0000 7fe6fbfff700 1 --2- 172.21.15.1:0/649371169 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe6fc108740 0x7fe6fc19c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:30.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.411+0000 7fe6faffd700 1 -- 172.21.15.1:0/649371169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6fc1034c0 msgr2=0x7fe6fc19bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:30.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.411+0000 7fe6faffd700 1 --2- 172.21.15.1:0/649371169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6fc1034c0 0x7fe6fc19bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.411+0000 7fe6faffd700 1 -- 172.21.15.1:0/649371169 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe6fc108740 msgr2=0x7fe6fc19c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:30.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.411+0000 7fe6faffd700 1 --2- 172.21.15.1:0/649371169 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe6fc108740 0x7fe6fc19c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.411+0000 7fe6faffd700 1 -- 172.21.15.1:0/649371169 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6fc1a6f40 con 0x7fe6fc107df0 2024-05-04T03:17:30.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.411+0000 7fe6fbfff700 1 --2- 172.21.15.1:0/649371169 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe6fc108740 0x7fe6fc19c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:17:30.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.412+0000 7fe6faffd700 1 --2- 172.21.15.1:0/649371169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6fc107df0 0x7fe6fc19c1c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000c9e0 tx=0x7fe6f000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:30.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.412+0000 7fe6f8ff9700 1 -- 172.21.15.1:0/649371169 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0007aa0 con 0x7fe6fc107df0 2024-05-04T03:17:30.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.412+0000 7fe6f8ff9700 1 -- 172.21.15.1:0/649371169 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6f0007c40 con 0x7fe6fc107df0 2024-05-04T03:17:30.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.412+0000 7fe701bce700 1 -- 172.21.15.1:0/649371169 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6fc1a7230 con 0x7fe6fc107df0 2024-05-04T03:17:30.868 INFO:teuthology.orchestra.run.smithi001.stdout:210453397560 2024-05-04T03:17:30.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.412+0000 7fe6f8ff9700 1 -- 172.21.15.1:0/649371169 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0018660 con 0x7fe6fc107df0 2024-05-04T03:17:30.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.412+0000 7fe701bce700 1 -- 172.21.15.1:0/649371169 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6fc1a77c0 con 0x7fe6fc107df0 2024-05-04T03:17:30.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.413+0000 7fe701bce700 1 -- 172.21.15.1:0/649371169 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6fc0512c0 con 0x7fe6fc107df0 2024-05-04T03:17:30.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.414+0000 7fe6f8ff9700 1 -- 172.21.15.1:0/649371169 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fe6f0018800 con 0x7fe6fc107df0 2024-05-04T03:17:30.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.415+0000 7fe6f8ff9700 1 --2- 172.21.15.1:0/649371169 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe6e40b2990 0x7fe6e40b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:30.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.415+0000 7fe6fb7fe700 1 --2- 172.21.15.1:0/649371169 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe6e40b2990 0x7fe6e40b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:30.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.415+0000 7fe6f8ff9700 1 -- 172.21.15.1:0/649371169 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fe6f00e2300 con 0x7fe6fc107df0 2024-05-04T03:17:30.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.416+0000 7fe6fb7fe700 1 --2- 172.21.15.1:0/649371169 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe6e40b2990 0x7fe6e40b4ea0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec009900 tx=0x7fe6ec00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:30.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.420+0000 7fe6f8ff9700 1 -- 172.21.15.1:0/649371169 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe6f0014030 con 0x7fe6fc107df0 2024-05-04T03:17:30.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.638+0000 7fe701bce700 1 -- 172.21.15.1:0/649371169 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fe6fc075fe0 con 0x7fe6fc107df0 2024-05-04T03:17:30.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.638+0000 7fe6f8ff9700 1 -- 172.21.15.1:0/649371169 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe6f00a9ee0 con 0x7fe6fc107df0 2024-05-04T03:17:30.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.640+0000 7fe701bce700 1 -- 172.21.15.1:0/649371169 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe6e40b2990 msgr2=0x7fe6e40b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:30.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.640+0000 7fe701bce700 1 --2- 172.21.15.1:0/649371169 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe6e40b2990 0x7fe6e40b4ea0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec009900 tx=0x7fe6ec00b040 comp rx=0 tx=0).stop 2024-05-04T03:17:30.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.640+0000 7fe701bce700 1 -- 172.21.15.1:0/649371169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6fc107df0 msgr2=0x7fe6fc19c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:30.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.640+0000 7fe701bce700 1 --2- 172.21.15.1:0/649371169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6fc107df0 0x7fe6fc19c1c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000c9e0 tx=0x7fe6f000ccf0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.641+0000 7fe701bce700 1 -- 172.21.15.1:0/649371169 shutdown_connections 2024-05-04T03:17:30.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.641+0000 7fe701bce700 1 --2- 172.21.15.1:0/649371169 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe6fc108740 0x7fe6fc19c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.641+0000 7fe701bce700 1 --2- 172.21.15.1:0/649371169 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe6e40b2990 0x7fe6e40b4ea0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.641+0000 7fe701bce700 1 --2- 172.21.15.1:0/649371169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6fc1034c0 0x7fe6fc19bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.641+0000 7fe701bce700 1 --2- 172.21.15.1:0/649371169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6fc107df0 0x7fe6fc19c1c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.641+0000 7fe701bce700 1 -- 172.21.15.1:0/649371169 >> 172.21.15.1:0/649371169 conn(0x7fe6fc0fedf0 msgr2=0x7fe6fc1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:30.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.641+0000 7fe701bce700 1 -- 172.21.15.1:0/649371169 shutdown_connections 2024-05-04T03:17:30.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:26.641+0000 7fe701bce700 1 -- 172.21.15.1:0/649371169 wait complete. 2024-05-04T03:17:30.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.952+0000 7f0348b57700 1 -- 172.21.15.1:0/3634754408 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f034402e5f0 con 0x7f03441045e0 2024-05-04T03:17:30.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.953+0000 7f03337fe700 1 -- 172.21.15.1:0/3634754408 <== mon.2 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0334013070 con 0x7f03441045e0 2024-05-04T03:17:30.956 INFO:teuthology.orchestra.run.smithi001.stdout:240518168629 2024-05-04T03:17:30.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.956+0000 7f0348b57700 1 -- 172.21.15.1:0/3634754408 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f032c0b2910 msgr2=0x7f032c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:30.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.956+0000 7f0348b57700 1 --2- 172.21.15.1:0/3634754408 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f032c0b2910 0x7f032c0b4e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0338005950 tx=0x7f03380058e0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.956+0000 7f0348b57700 1 -- 172.21.15.1:0/3634754408 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03441045e0 msgr2=0x7f0344075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:30.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.956+0000 7f0348b57700 1 --2- 172.21.15.1:0/3634754408 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03441045e0 0x7f0344075ac0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f03340098e0 tx=0x7f03340057c0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.956+0000 7f0348b57700 1 -- 172.21.15.1:0/3634754408 shutdown_connections 2024-05-04T03:17:30.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.956+0000 7f0348b57700 1 --2- 172.21.15.1:0/3634754408 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03441045e0 0x7f0344075ac0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.956+0000 7f0348b57700 1 --2- 172.21.15.1:0/3634754408 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f032c0b2910 0x7f032c0b4e20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.956+0000 7f0348b57700 1 --2- 172.21.15.1:0/3634754408 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0344109860 0x7f0344076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.956+0000 7f0348b57700 1 --2- 172.21.15.1:0/3634754408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0344108f10 0x7f0344076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:30.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.956+0000 7f0348b57700 1 -- 172.21.15.1:0/3634754408 >> 172.21.15.1:0/3634754408 conn(0x7f03440ffe70 msgr2=0x7f0344107140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:30.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.956+0000 7f0348b57700 1 -- 172.21.15.1:0/3634754408 shutdown_connections 2024-05-04T03:17:30.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:30.956+0000 7f0348b57700 1 -- 172.21.15.1:0/3634754408 wait complete. 2024-05-04T03:17:31.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:31 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/4015430035' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-04T03:17:31.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:31 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3634754408' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-04T03:17:31.456 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397556 got 210453397560 for osd.7 2024-05-04T03:17:31.456 DEBUG:teuthology.parallel:result is None 2024-05-04T03:17:31.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.653+0000 7f364c6ae700 1 -- 172.21.15.1:0/853020217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3644109880 msgr2=0x7f36441111e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:31.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.653+0000 7f364c6ae700 1 --2- 172.21.15.1:0/853020217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3644109880 0x7f36441111e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f363c00b0d0 tx=0x7f363c00b3e0 comp rx=0 tx=0).stop 2024-05-04T03:17:31.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.653+0000 7f364c6ae700 1 -- 172.21.15.1:0/853020217 shutdown_connections 2024-05-04T03:17:31.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.653+0000 7f364c6ae700 1 --2- 172.21.15.1:0/853020217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3644109880 0x7f36441111e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:31.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.653+0000 7f364c6ae700 1 --2- 172.21.15.1:0/853020217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3644104650 0x7f3644104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:31.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.653+0000 7f364c6ae700 1 --2- 172.21.15.1:0/853020217 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3644108f30 0x7f3644109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:31.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.653+0000 7f364c6ae700 1 -- 172.21.15.1:0/853020217 >> 172.21.15.1:0/853020217 conn(0x7f3644079f00 msgr2=0x7f364407a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:31.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.653+0000 7f364c6ae700 1 -- 172.21.15.1:0/853020217 shutdown_connections 2024-05-04T03:17:31.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.653+0000 7f364c6ae700 1 -- 172.21.15.1:0/853020217 wait complete. 2024-05-04T03:17:31.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.654+0000 7f364c6ae700 1 Processor -- start 2024-05-04T03:17:31.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.654+0000 7f364c6ae700 1 -- start start 2024-05-04T03:17:31.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.654+0000 7f364c6ae700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3644104650 0x7f364419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:31.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.654+0000 7f364c6ae700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3644108f30 0x7f364419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:31.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.654+0000 7f364c6ae700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3644109880 0x7f364419f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:31.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.654+0000 7f364c6ae700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f36441144b0 con 0x7f3644104650 2024-05-04T03:17:31.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.654+0000 7f364c6ae700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3644114330 con 0x7f3644109880 2024-05-04T03:17:31.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.654+0000 7f364c6ae700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3644114630 con 0x7f3644108f30 2024-05-04T03:17:31.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.654+0000 7f364a44a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3644104650 0x7f364419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:31.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.654+0000 7f364a44a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3644104650 0x7f364419ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52432/0 (socket says 172.21.15.1:52432) 2024-05-04T03:17:31.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.654+0000 7f364a44a700 1 -- 172.21.15.1:0/2945338740 learned_addr learned my addr 172.21.15.1:0/2945338740 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:31.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.654+0000 7f3649c49700 1 --2- 172.21.15.1:0/2945338740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3644108f30 0x7f364419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:31.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.654+0000 7f364ac4b700 1 --2- 172.21.15.1:0/2945338740 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3644109880 0x7f364419f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:31.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.654+0000 7f364a44a700 1 -- 172.21.15.1:0/2945338740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3644108f30 msgr2=0x7f364419b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:31.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.654+0000 7f364a44a700 1 --2- 172.21.15.1:0/2945338740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3644108f30 0x7f364419b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:31.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.654+0000 7f364a44a700 1 -- 172.21.15.1:0/2945338740 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3644109880 msgr2=0x7f364419f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:31.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.654+0000 7f364a44a700 1 --2- 172.21.15.1:0/2945338740 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3644109880 0x7f364419f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:31.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.654+0000 7f364a44a700 1 -- 172.21.15.1:0/2945338740 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364419fc40 con 0x7f3644104650 2024-05-04T03:17:31.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.655+0000 7f364a44a700 1 --2- 172.21.15.1:0/2945338740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3644104650 0x7f364419ae20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f364000c9b0 tx=0x7f364000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:31.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.655+0000 7f363b7fe700 1 -- 172.21.15.1:0/2945338740 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3640007b00 con 0x7f3644104650 2024-05-04T03:17:31.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.655+0000 7f363b7fe700 1 -- 172.21.15.1:0/2945338740 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3640007ca0 con 0x7f3644104650 2024-05-04T03:17:31.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.655+0000 7f364c6ae700 1 -- 172.21.15.1:0/2945338740 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f364419ff30 con 0x7f3644104650 2024-05-04T03:17:31.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.655+0000 7f363b7fe700 1 -- 172.21.15.1:0/2945338740 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3640018660 con 0x7f3644104650 2024-05-04T03:17:31.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.655+0000 7f364c6ae700 1 -- 172.21.15.1:0/2945338740 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36441a7900 con 0x7f3644104650 2024-05-04T03:17:31.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.656+0000 7f364c6ae700 1 -- 172.21.15.1:0/2945338740 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36440512c0 con 0x7f3644104650 2024-05-04T03:17:31.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.658+0000 7f363b7fe700 1 -- 172.21.15.1:0/2945338740 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f364000f450 con 0x7f3644104650 2024-05-04T03:17:31.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.659+0000 7f363b7fe700 1 --2- 172.21.15.1:0/2945338740 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f36300b2990 0x7f36300b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:31.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.659+0000 7f3649c49700 1 --2- 172.21.15.1:0/2945338740 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f36300b2990 0x7f36300b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:31.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.659+0000 7f363b7fe700 1 -- 172.21.15.1:0/2945338740 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f36400e24a0 con 0x7f3644104650 2024-05-04T03:17:31.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.659+0000 7f3649c49700 1 --2- 172.21.15.1:0/2945338740 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f36300b2990 0x7f36300b4ea0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f36441097d0 tx=0x7f363400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:31.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.663+0000 7f363b7fe700 1 -- 172.21.15.1:0/2945338740 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3640014030 con 0x7f3644104650 2024-05-04T03:17:31.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:31 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/4015430035' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-04T03:17:31.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:31 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/3634754408' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-04T03:17:31.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:31 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/4015430035' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-04T03:17:31.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:31 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/3634754408' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-04T03:17:31.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.879+0000 7f364c6ae700 1 -- 172.21.15.1:0/2945338740 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f364419c180 con 0x7f3644104650 2024-05-04T03:17:31.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.880+0000 7f363b7fe700 1 -- 172.21.15.1:0/2945338740 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f36400aa290 con 0x7f3644104650 2024-05-04T03:17:31.883 INFO:teuthology.orchestra.run.smithi001.stdout:313532612645 2024-05-04T03:17:31.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.882+0000 7f364c6ae700 1 -- 172.21.15.1:0/2945338740 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f36300b2990 msgr2=0x7f36300b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:31.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.882+0000 7f364c6ae700 1 --2- 172.21.15.1:0/2945338740 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f36300b2990 0x7f36300b4ea0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f36441097d0 tx=0x7f363400b410 comp rx=0 tx=0).stop 2024-05-04T03:17:31.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.882+0000 7f364c6ae700 1 -- 172.21.15.1:0/2945338740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3644104650 msgr2=0x7f364419ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:31.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.882+0000 7f364c6ae700 1 --2- 172.21.15.1:0/2945338740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3644104650 0x7f364419ae20 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f364000c9b0 tx=0x7f364000ccc0 comp rx=0 tx=0).stop 2024-05-04T03:17:31.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.882+0000 7f364c6ae700 1 -- 172.21.15.1:0/2945338740 shutdown_connections 2024-05-04T03:17:31.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.882+0000 7f364c6ae700 1 --2- 172.21.15.1:0/2945338740 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3644109880 0x7f364419f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:31.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.882+0000 7f364c6ae700 1 --2- 172.21.15.1:0/2945338740 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f36300b2990 0x7f36300b4ea0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:31.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.882+0000 7f364c6ae700 1 --2- 172.21.15.1:0/2945338740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3644104650 0x7f364419ae20 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:31.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.882+0000 7f364c6ae700 1 --2- 172.21.15.1:0/2945338740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3644108f30 0x7f364419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:31.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.882+0000 7f364c6ae700 1 -- 172.21.15.1:0/2945338740 >> 172.21.15.1:0/2945338740 conn(0x7f3644079f00 msgr2=0x7f3644102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:31.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.882+0000 7f364c6ae700 1 -- 172.21.15.1:0/2945338740 shutdown_connections 2024-05-04T03:17:31.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:31.882+0000 7f364c6ae700 1 -- 172.21.15.1:0/2945338740 wait complete. 2024-05-04T03:17:32.509 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:32 smithi001 ceph-mon[29731]: pgmap v342: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:17:32.509 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:32 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2945338740' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-04T03:17:32.542 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790015 got 158913790019 for osd.5 2024-05-04T03:17:32.542 DEBUG:teuthology.parallel:result is None 2024-05-04T03:17:32.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:32 smithi077 ceph-mon[33531]: pgmap v342: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:17:32.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:32 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/2945338740' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-04T03:17:32.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:32 smithi072 ceph-mon[33332]: pgmap v342: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:17:32.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:32 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/2945338740' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-04T03:17:32.981 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612639 got 313532612645 for osd.11 2024-05-04T03:17:32.981 DEBUG:teuthology.parallel:result is None 2024-05-04T03:17:33.738 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168624 got 240518168629 for osd.8 2024-05-04T03:17:33.738 DEBUG:teuthology.parallel:result is None 2024-05-04T03:17:34.068 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986242 got 133143986247 for osd.4 2024-05-04T03:17:34.068 DEBUG:teuthology.parallel:result is None 2024-05-04T03:17:34.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:34 smithi001 ceph-mon[29731]: pgmap v343: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:17:34.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:34 smithi001 ceph-mon[29731]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-04T03:17:34.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:34 smithi077 ceph-mon[33531]: pgmap v343: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:17:34.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:34 smithi077 ceph-mon[33531]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-04T03:17:34.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:34 smithi072 ceph-mon[33332]: pgmap v343: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:17:34.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:34 smithi072 ceph-mon[33332]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-04T03:17:35.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.038+0000 7f6b84939700 1 -- 172.21.15.1:0/3982232996 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b80107df0 msgr2=0x7f6b80108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:35.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.038+0000 7f6b84939700 1 --2- 172.21.15.1:0/3982232996 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b80107df0 0x7f6b80108200 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6b740099c0 tx=0x7f6b74009cd0 comp rx=0 tx=0).stop 2024-05-04T03:17:35.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.038+0000 7f6b84939700 1 -- 172.21.15.1:0/3982232996 shutdown_connections 2024-05-04T03:17:35.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.038+0000 7f6b84939700 1 --2- 172.21.15.1:0/3982232996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b80108740 0x7f6b801100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:35.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.038+0000 7f6b84939700 1 --2- 172.21.15.1:0/3982232996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b801034c0 0x7f6b801038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:35.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.038+0000 7f6b84939700 1 --2- 172.21.15.1:0/3982232996 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b80107df0 0x7f6b80108200 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:35.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.038+0000 7f6b84939700 1 -- 172.21.15.1:0/3982232996 >> 172.21.15.1:0/3982232996 conn(0x7f6b800fedc0 msgr2=0x7f6b801011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:35.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.038+0000 7f6b84939700 1 -- 172.21.15.1:0/3982232996 shutdown_connections 2024-05-04T03:17:35.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.038+0000 7f6b84939700 1 -- 172.21.15.1:0/3982232996 wait complete. 2024-05-04T03:17:35.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.039+0000 7f6b84939700 1 Processor -- start 2024-05-04T03:17:35.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.039+0000 7f6b84939700 1 -- start start 2024-05-04T03:17:35.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.039+0000 7f6b84939700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b801034c0 0x7f6b8019bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:35.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.039+0000 7f6b84939700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b80107df0 0x7f6b8019c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:35.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.039+0000 7f6b84939700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b80108740 0x7f6b8019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:35.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.039+0000 7f6b84939700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6b80113350 con 0x7f6b80107df0 2024-05-04T03:17:35.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.039+0000 7f6b84939700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6b801131d0 con 0x7f6b80108740 2024-05-04T03:17:35.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.039+0000 7f6b84939700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6b801134d0 con 0x7f6b801034c0 2024-05-04T03:17:35.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.039+0000 7f6b7dd9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b80107df0 0x7f6b8019c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:35.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.039+0000 7f6b7dd9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b80107df0 0x7f6b8019c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52454/0 (socket says 172.21.15.1:52454) 2024-05-04T03:17:35.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.039+0000 7f6b7dd9b700 1 -- 172.21.15.1:0/1951176404 learned_addr learned my addr 172.21.15.1:0/1951176404 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:35.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.040+0000 7f6b7e59c700 1 --2- 172.21.15.1:0/1951176404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b801034c0 0x7f6b8019bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:35.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.040+0000 7f6b7ed9d700 1 --2- 172.21.15.1:0/1951176404 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b80108740 0x7f6b8019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:35.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.040+0000 7f6b7dd9b700 1 -- 172.21.15.1:0/1951176404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b801034c0 msgr2=0x7f6b8019bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:35.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.040+0000 7f6b7dd9b700 1 --2- 172.21.15.1:0/1951176404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b801034c0 0x7f6b8019bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:35.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.040+0000 7f6b7dd9b700 1 -- 172.21.15.1:0/1951176404 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b80108740 msgr2=0x7f6b8019c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:35.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.040+0000 7f6b7dd9b700 1 --2- 172.21.15.1:0/1951176404 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b80108740 0x7f6b8019c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:35.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.040+0000 7f6b7dd9b700 1 -- 172.21.15.1:0/1951176404 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b801a7080 con 0x7f6b80107df0 2024-05-04T03:17:35.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.040+0000 7f6b7ed9d700 1 --2- 172.21.15.1:0/1951176404 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b80108740 0x7f6b8019c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:17:35.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.040+0000 7f6b7dd9b700 1 --2- 172.21.15.1:0/1951176404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b80107df0 0x7f6b8019c270 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009f50 tx=0x7f6b740037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:35.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.040+0000 7f6b6f7fe700 1 -- 172.21.15.1:0/1951176404 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74027070 con 0x7f6b80107df0 2024-05-04T03:17:35.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.040+0000 7f6b6f7fe700 1 -- 172.21.15.1:0/1951176404 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b74004140 con 0x7f6b80107df0 2024-05-04T03:17:35.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.040+0000 7f6b6f7fe700 1 -- 172.21.15.1:0/1951176404 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7400f460 con 0x7f6b80107df0 2024-05-04T03:17:35.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.040+0000 7f6b84939700 1 -- 172.21.15.1:0/1951176404 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b801a7310 con 0x7f6b80107df0 2024-05-04T03:17:35.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.040+0000 7f6b84939700 1 -- 172.21.15.1:0/1951176404 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b801a77c0 con 0x7f6b80107df0 2024-05-04T03:17:35.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.041+0000 7f6b84939700 1 -- 172.21.15.1:0/1951176404 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b800512c0 con 0x7f6b80107df0 2024-05-04T03:17:35.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.043+0000 7f6b6f7fe700 1 -- 172.21.15.1:0/1951176404 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f6b7402a070 con 0x7f6b80107df0 2024-05-04T03:17:35.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.043+0000 7f6b6f7fe700 1 --2- 172.21.15.1:0/1951176404 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6b680b2900 0x7f6b680b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:35.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.043+0000 7f6b7e59c700 1 --2- 172.21.15.1:0/1951176404 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6b680b2900 0x7f6b680b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:35.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.044+0000 7f6b6f7fe700 1 -- 172.21.15.1:0/1951176404 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f6b740e3290 con 0x7f6b80107df0 2024-05-04T03:17:35.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.044+0000 7f6b7e59c700 1 --2- 172.21.15.1:0/1951176404 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6b680b2900 0x7f6b680b4e10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6b70005fd0 tx=0x7f6b7000a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:35.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.048+0000 7f6b6f7fe700 1 -- 172.21.15.1:0/1951176404 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6b74013070 con 0x7f6b80107df0 2024-05-04T03:17:35.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.260+0000 7f6b84939700 1 -- 172.21.15.1:0/1951176404 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f6b8002e5f0 con 0x7f6b80107df0 2024-05-04T03:17:35.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.260+0000 7f6b6f7fe700 1 -- 172.21.15.1:0/1951176404 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6b740ab100 con 0x7f6b80107df0 2024-05-04T03:17:35.262 INFO:teuthology.orchestra.run.smithi001.stdout:55834574936 2024-05-04T03:17:35.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.262+0000 7f6b84939700 1 -- 172.21.15.1:0/1951176404 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6b680b2900 msgr2=0x7f6b680b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:35.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.262+0000 7f6b84939700 1 --2- 172.21.15.1:0/1951176404 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6b680b2900 0x7f6b680b4e10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6b70005fd0 tx=0x7f6b7000a560 comp rx=0 tx=0).stop 2024-05-04T03:17:35.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.262+0000 7f6b84939700 1 -- 172.21.15.1:0/1951176404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b80107df0 msgr2=0x7f6b8019c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:35.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.262+0000 7f6b84939700 1 --2- 172.21.15.1:0/1951176404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b80107df0 0x7f6b8019c270 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009f50 tx=0x7f6b740037c0 comp rx=0 tx=0).stop 2024-05-04T03:17:35.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.262+0000 7f6b84939700 1 -- 172.21.15.1:0/1951176404 shutdown_connections 2024-05-04T03:17:35.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.262+0000 7f6b84939700 1 --2- 172.21.15.1:0/1951176404 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b80108740 0x7f6b8019c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:35.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.262+0000 7f6b84939700 1 --2- 172.21.15.1:0/1951176404 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6b680b2900 0x7f6b680b4e10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:35.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.262+0000 7f6b84939700 1 --2- 172.21.15.1:0/1951176404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b801034c0 0x7f6b8019bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:35.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.262+0000 7f6b84939700 1 --2- 172.21.15.1:0/1951176404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b80107df0 0x7f6b8019c270 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:35.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.262+0000 7f6b84939700 1 -- 172.21.15.1:0/1951176404 >> 172.21.15.1:0/1951176404 conn(0x7f6b800fedc0 msgr2=0x7f6b801011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:35.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.262+0000 7f6b84939700 1 -- 172.21.15.1:0/1951176404 shutdown_connections 2024-05-04T03:17:35.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:35.262+0000 7f6b84939700 1 -- 172.21.15.1:0/1951176404 wait complete. 2024-05-04T03:17:35.569 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:35 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/1951176404' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-04T03:17:35.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:35 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/1951176404' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-04T03:17:35.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:35 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/1951176404' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-04T03:17:35.942 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574929 got 55834574936 for osd.1 2024-05-04T03:17:35.942 DEBUG:teuthology.parallel:result is None 2024-05-04T03:17:35.942 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-04T03:17:35.943 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-04T03:17:36.195 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:17:36.564 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:36 smithi001 ceph-mon[29731]: pgmap v344: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-04T03:17:36.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:36 smithi077 ceph-mon[33531]: pgmap v344: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-04T03:17:36.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:36 smithi072 ceph-mon[33332]: pgmap v344: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-04T03:17:37.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.944+0000 7f0dfdb43700 1 -- 172.21.15.1:0/903812310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df8103310 msgr2=0x7f0df810f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:37.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.944+0000 7f0dfdb43700 1 --2- 172.21.15.1:0/903812310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df8103310 0x7f0df810f890 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0df000b0a0 tx=0x7f0df000b3b0 comp rx=0 tx=0).stop 2024-05-04T03:17:37.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.944+0000 7f0dfdb43700 1 -- 172.21.15.1:0/903812310 shutdown_connections 2024-05-04T03:17:37.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.944+0000 7f0dfdb43700 1 --2- 172.21.15.1:0/903812310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df8103310 0x7f0df810f890 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:37.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.944+0000 7f0dfdb43700 1 --2- 172.21.15.1:0/903812310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0df81089f0 0x7f0df8108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:37.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.944+0000 7f0dfdb43700 1 --2- 172.21.15.1:0/903812310 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0df81029c0 0x7f0df8102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:37.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.944+0000 7f0dfdb43700 1 -- 172.21.15.1:0/903812310 >> 172.21.15.1:0/903812310 conn(0x7f0df80fe6f0 msgr2=0x7f0df8100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:37.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.944+0000 7f0dfdb43700 1 -- 172.21.15.1:0/903812310 shutdown_connections 2024-05-04T03:17:37.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.944+0000 7f0dfdb43700 1 -- 172.21.15.1:0/903812310 wait complete. 2024-05-04T03:17:37.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.944+0000 7f0dfdb43700 1 Processor -- start 2024-05-04T03:17:37.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.944+0000 7f0dfdb43700 1 -- start start 2024-05-04T03:17:37.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.945+0000 7f0dfdb43700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df81029c0 0x7f0df819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:37.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.945+0000 7f0dfdb43700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0df8103310 0x7f0df819b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:37.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.945+0000 7f0dfdb43700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0df81089f0 0x7f0df819f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:37.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.945+0000 7f0dfdb43700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0df8112b20 con 0x7f0df81029c0 2024-05-04T03:17:37.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.945+0000 7f0dfdb43700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0df81129a0 con 0x7f0df81089f0 2024-05-04T03:17:37.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.945+0000 7f0dfdb43700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0df8112ca0 con 0x7f0df8103310 2024-05-04T03:17:37.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.945+0000 7f0df77fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df81029c0 0x7f0df819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:37.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.945+0000 7f0df77fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df81029c0 0x7f0df819ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52464/0 (socket says 172.21.15.1:52464) 2024-05-04T03:17:37.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.945+0000 7f0df77fe700 1 -- 172.21.15.1:0/408401160 learned_addr learned my addr 172.21.15.1:0/408401160 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:37.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.945+0000 7f0deffff700 1 --2- 172.21.15.1:0/408401160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0df8103310 0x7f0df819b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:37.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.945+0000 7f0df7fff700 1 --2- 172.21.15.1:0/408401160 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0df81089f0 0x7f0df819f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:37.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.945+0000 7f0df77fe700 1 -- 172.21.15.1:0/408401160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0df8103310 msgr2=0x7f0df819b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:37.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.945+0000 7f0df77fe700 1 --2- 172.21.15.1:0/408401160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0df8103310 0x7f0df819b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:37.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.945+0000 7f0df77fe700 1 -- 172.21.15.1:0/408401160 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0df81089f0 msgr2=0x7f0df819f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:37.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.945+0000 7f0df77fe700 1 --2- 172.21.15.1:0/408401160 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0df81089f0 0x7f0df819f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:37.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.945+0000 7f0df77fe700 1 -- 172.21.15.1:0/408401160 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df819fc20 con 0x7f0df81029c0 2024-05-04T03:17:37.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.945+0000 7f0deffff700 1 --2- 172.21.15.1:0/408401160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0df8103310 0x7f0df819b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:17:37.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.945+0000 7f0df7fff700 1 --2- 172.21.15.1:0/408401160 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0df81089f0 0x7f0df819f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:17:37.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.945+0000 7f0df77fe700 1 --2- 172.21.15.1:0/408401160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df81029c0 0x7f0df819ad50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0de800e970 tx=0x7f0de800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:37.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.946+0000 7f0df57fa700 1 -- 172.21.15.1:0/408401160 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de800cbd0 con 0x7f0df81029c0 2024-05-04T03:17:37.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.946+0000 7f0df57fa700 1 -- 172.21.15.1:0/408401160 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0de800cd70 con 0x7f0df81029c0 2024-05-04T03:17:37.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.946+0000 7f0df57fa700 1 -- 172.21.15.1:0/408401160 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de8010430 con 0x7f0df81029c0 2024-05-04T03:17:37.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.946+0000 7f0dfdb43700 1 -- 172.21.15.1:0/408401160 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df819ff10 con 0x7f0df81029c0 2024-05-04T03:17:37.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.946+0000 7f0dfdb43700 1 -- 172.21.15.1:0/408401160 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df81a7830 con 0x7f0df81029c0 2024-05-04T03:17:37.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.947+0000 7f0dfdb43700 1 -- 172.21.15.1:0/408401160 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0df80512c0 con 0x7f0df81029c0 2024-05-04T03:17:37.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.949+0000 7f0df57fa700 1 -- 172.21.15.1:0/408401160 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f0de8004d70 con 0x7f0df81029c0 2024-05-04T03:17:37.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.950+0000 7f0df57fa700 1 --2- 172.21.15.1:0/408401160 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0de40b2900 0x7f0de40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:37.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.950+0000 7f0deffff700 1 --2- 172.21.15.1:0/408401160 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0de40b2900 0x7f0de40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:37.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.950+0000 7f0df57fa700 1 -- 172.21.15.1:0/408401160 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f0de80e25d0 con 0x7f0df81029c0 2024-05-04T03:17:37.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.950+0000 7f0deffff700 1 --2- 172.21.15.1:0/408401160 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0de40b2900 0x7f0de40b4e10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0df8101f40 tx=0x7f0de0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:37.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:37.954+0000 7f0df57fa700 1 -- 172.21.15.1:0/408401160 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0de8054df0 con 0x7f0df81029c0 2024-05-04T03:17:38.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:38.162+0000 7f0dfdb43700 1 -- 172.21.15.1:0/408401160 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0df8101220 con 0x7f0de40b2900 2024-05-04T03:17:38.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:38.170+0000 7f0df57fa700 1 -- 172.21.15.1:0/408401160 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258907 (secure 0 0 0) 0x7f0df8101220 con 0x7f0de40b2900 2024-05-04T03:17:38.175 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:17:38.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:38.174+0000 7f0dfdb43700 1 -- 172.21.15.1:0/408401160 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0de40b2900 msgr2=0x7f0de40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:38.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:38.174+0000 7f0dfdb43700 1 --2- 172.21.15.1:0/408401160 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0de40b2900 0x7f0de40b4e10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0df8101f40 tx=0x7f0de0009260 comp rx=0 tx=0).stop 2024-05-04T03:17:38.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:38.174+0000 7f0dfdb43700 1 -- 172.21.15.1:0/408401160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df81029c0 msgr2=0x7f0df819ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:38.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:38.174+0000 7f0dfdb43700 1 --2- 172.21.15.1:0/408401160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df81029c0 0x7f0df819ad50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0de800e970 tx=0x7f0de800ed30 comp rx=0 tx=0).stop 2024-05-04T03:17:38.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:38.174+0000 7f0dfdb43700 1 -- 172.21.15.1:0/408401160 shutdown_connections 2024-05-04T03:17:38.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:38.174+0000 7f0dfdb43700 1 --2- 172.21.15.1:0/408401160 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0df81089f0 0x7f0df819f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:38.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:38.174+0000 7f0dfdb43700 1 --2- 172.21.15.1:0/408401160 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0de40b2900 0x7f0de40b4e10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:38.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:38.174+0000 7f0dfdb43700 1 --2- 172.21.15.1:0/408401160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0df81029c0 0x7f0df819ad50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:38.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:38.174+0000 7f0dfdb43700 1 --2- 172.21.15.1:0/408401160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0df8103310 0x7f0df819b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:38.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:38.174+0000 7f0dfdb43700 1 -- 172.21.15.1:0/408401160 >> 172.21.15.1:0/408401160 conn(0x7f0df80fe6f0 msgr2=0x7f0df8100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:38.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:38.174+0000 7f0dfdb43700 1 -- 172.21.15.1:0/408401160 shutdown_connections 2024-05-04T03:17:38.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:38.174+0000 7f0dfdb43700 1 -- 172.21.15.1:0/408401160 wait complete. 2024-05-04T03:17:38.179 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-04T03:17:38.501 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:38 smithi001 ceph-mon[29731]: pgmap v345: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-04T03:17:38.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:38 smithi077 ceph-mon[33531]: pgmap v345: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-04T03:17:38.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:38 smithi072 ceph-mon[33332]: pgmap v345: 97 pgs: 97 active+clean; 645 KiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-04T03:17:38.827 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-05-04T03:17:37.166301+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660876,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":634,"num_read_kb":415,"num_write":292,"num_write_kb":2271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":623,"ondisk_log_size":623,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":407628,"kb_used_data":12700,"kb_used_omap":18,"kb_used_meta":394797,"kb_avail":1124534196,"statfs":{"total":1151940427776,"available":1151523016704,"internally_reserved":0,"allocated":13004800,"data_stored":7261788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":404272441},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":31,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003678"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.171540+0000","last_change":"2024-05-04T03:15:48.049361+0000","last_active":"2024-05-04T03:17:33.171540+0000","last_peered":"2024-05-04T03:17:33.171540+0000","last_clean":"2024-05-04T03:17:33.171540+0000","last_became_active":"2024-05-04T03:15:48.048876+0000","last_became_peered":"2024-05-04T03:15:48.048876+0000","last_unstale":"2024-05-04T03:17:33.171540+0000","last_undegraded":"2024-05-04T03:17:33.171540+0000","last_fullsized":"2024-05-04T03:17:33.171540+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:50:52.568516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165716+0000","last_change":"2024-05-04T03:15:48.055554+0000","last_active":"2024-05-04T03:17:33.165716+0000","last_peered":"2024-05-04T03:17:33.165716+0000","last_clean":"2024-05-04T03:17:33.165716+0000","last_became_active":"2024-05-04T03:15:48.055155+0000","last_became_peered":"2024-05-04T03:15:48.055155+0000","last_unstale":"2024-05-04T03:17:33.165716+0000","last_undegraded":"2024-05-04T03:17:33.165716+0000","last_fullsized":"2024-05-04T03:17:33.165716+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:38:07.824031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166389+0000","last_change":"2024-05-04T03:15:48.047974+0000","last_active":"2024-05-04T03:17:33.166389+0000","last_peered":"2024-05-04T03:17:33.166389+0000","last_clean":"2024-05-04T03:17:33.166389+0000","last_became_active":"2024-05-04T03:15:48.044299+0000","last_became_peered":"2024-05-04T03:15:48.044299+0000","last_unstale":"2024-05-04T03:17:33.166389+0000","last_undegraded":"2024-05-04T03:17:33.166389+0000","last_fullsized":"2024-05-04T03:17:33.166389+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:49:08.014987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.155592+0000","last_change":"2024-05-04T03:15:48.050899+0000","last_active":"2024-05-04T03:17:33.155592+0000","last_peered":"2024-05-04T03:17:33.155592+0000","last_clean":"2024-05-04T03:17:33.155592+0000","last_became_active":"2024-05-04T03:15:48.048812+0000","last_became_peered":"2024-05-04T03:15:48.048812+0000","last_unstale":"2024-05-04T03:17:33.155592+0000","last_undegraded":"2024-05-04T03:17:33.155592+0000","last_fullsized":"2024-05-04T03:17:33.155592+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:21:01.829349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164977+0000","last_change":"2024-05-04T03:15:48.044403+0000","last_active":"2024-05-04T03:17:33.164977+0000","last_peered":"2024-05-04T03:17:33.164977+0000","last_clean":"2024-05-04T03:17:33.164977+0000","last_became_active":"2024-05-04T03:15:48.044021+0000","last_became_peered":"2024-05-04T03:15:48.044021+0000","last_unstale":"2024-05-04T03:17:33.164977+0000","last_undegraded":"2024-05-04T03:17:33.164977+0000","last_fullsized":"2024-05-04T03:17:33.164977+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:50:54.762070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334190+0000","last_change":"2024-05-04T03:15:48.053233+0000","last_active":"2024-05-04T03:16:34.334190+0000","last_peered":"2024-05-04T03:16:34.334190+0000","last_clean":"2024-05-04T03:16:34.334190+0000","last_became_active":"2024-05-04T03:15:48.052809+0000","last_became_peered":"2024-05-04T03:15:48.052809+0000","last_unstale":"2024-05-04T03:16:34.334190+0000","last_undegraded":"2024-05-04T03:16:34.334190+0000","last_fullsized":"2024-05-04T03:16:34.334190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:27:20.082694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.140939+0000","last_change":"2024-05-04T03:15:48.045108+0000","last_active":"2024-05-04T03:17:33.140939+0000","last_peered":"2024-05-04T03:17:33.140939+0000","last_clean":"2024-05-04T03:17:33.140939+0000","last_became_active":"2024-05-04T03:15:48.044719+0000","last_became_peered":"2024-05-04T03:15:48.044719+0000","last_unstale":"2024-05-04T03:17:33.140939+0000","last_undegraded":"2024-05-04T03:17:33.140939+0000","last_fullsized":"2024-05-04T03:17:33.140939+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:55:02.586294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164836+0000","last_change":"2024-05-04T03:15:48.043559+0000","last_active":"2024-05-04T03:17:33.164836+0000","last_peered":"2024-05-04T03:17:33.164836+0000","last_clean":"2024-05-04T03:17:33.164836+0000","last_became_active":"2024-05-04T03:15:48.043138+0000","last_became_peered":"2024-05-04T03:15:48.043138+0000","last_unstale":"2024-05-04T03:17:33.164836+0000","last_undegraded":"2024-05-04T03:17:33.164836+0000","last_fullsized":"2024-05-04T03:17:33.164836+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:05:25.411916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.333791+0000","last_change":"2024-05-04T03:15:48.039632+0000","last_active":"2024-05-04T03:16:34.333791+0000","last_peered":"2024-05-04T03:16:34.333791+0000","last_clean":"2024-05-04T03:16:34.333791+0000","last_became_active":"2024-05-04T03:15:48.039231+0000","last_became_peered":"2024-05-04T03:15:48.039231+0000","last_unstale":"2024-05-04T03:16:34.333791+0000","last_undegraded":"2024-05-04T03:16:34.333791+0000","last_fullsized":"2024-05-04T03:16:34.333791+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:33:12.853172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334645+0000","last_change":"2024-05-04T03:15:48.055138+0000","last_active":"2024-05-04T03:16:34.334645+0000","last_peered":"2024-05-04T03:16:34.334645+0000","last_clean":"2024-05-04T03:16:34.334645+0000","last_became_active":"2024-05-04T03:15:48.054791+0000","last_became_peered":"2024-05-04T03:15:48.054791+0000","last_unstale":"2024-05-04T03:16:34.334645+0000","last_undegraded":"2024-05-04T03:16:34.334645+0000","last_fullsized":"2024-05-04T03:16:34.334645+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:33:36.822549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165635+0000","last_change":"2024-05-04T03:15:48.050125+0000","last_active":"2024-05-04T03:17:33.165635+0000","last_peered":"2024-05-04T03:17:33.165635+0000","last_clean":"2024-05-04T03:17:33.165635+0000","last_became_active":"2024-05-04T03:15:48.049390+0000","last_became_peered":"2024-05-04T03:15:48.049390+0000","last_unstale":"2024-05-04T03:17:33.165635+0000","last_undegraded":"2024-05-04T03:17:33.165635+0000","last_fullsized":"2024-05-04T03:17:33.165635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:27:05.740383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.163273+0000","last_change":"2024-05-04T03:15:48.050700+0000","last_active":"2024-05-04T03:17:33.163273+0000","last_peered":"2024-05-04T03:17:33.163273+0000","last_clean":"2024-05-04T03:17:33.163273+0000","last_became_active":"2024-05-04T03:15:48.049288+0000","last_became_peered":"2024-05-04T03:15:48.049288+0000","last_unstale":"2024-05-04T03:17:33.163273+0000","last_undegraded":"2024-05-04T03:17:33.163273+0000","last_fullsized":"2024-05-04T03:17:33.163273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:40:07.072006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165193+0000","last_change":"2024-05-04T03:16:34.337123+0000","last_active":"2024-05-04T03:17:33.165193+0000","last_peered":"2024-05-04T03:17:33.165193+0000","last_clean":"2024-05-04T03:17:33.165193+0000","last_became_active":"2024-05-04T03:16:34.336657+0000","last_became_peered":"2024-05-04T03:16:34.336657+0000","last_unstale":"2024-05-04T03:17:33.165193+0000","last_undegraded":"2024-05-04T03:17:33.165193+0000","last_fullsized":"2024-05-04T03:17:33.165193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:58:15.939941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.162544+0000","last_change":"2024-05-04T03:15:48.047258+0000","last_active":"2024-05-04T03:17:33.162544+0000","last_peered":"2024-05-04T03:17:33.162544+0000","last_clean":"2024-05-04T03:17:33.162544+0000","last_became_active":"2024-05-04T03:15:48.046897+0000","last_became_peered":"2024-05-04T03:15:48.046897+0000","last_unstale":"2024-05-04T03:17:33.162544+0000","last_undegraded":"2024-05-04T03:17:33.162544+0000","last_fullsized":"2024-05-04T03:17:33.162544+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:14:39.217475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166174+0000","last_change":"2024-05-04T03:15:48.051240+0000","last_active":"2024-05-04T03:17:33.166174+0000","last_peered":"2024-05-04T03:17:33.166174+0000","last_clean":"2024-05-04T03:17:33.166174+0000","last_became_active":"2024-05-04T03:15:48.050081+0000","last_became_peered":"2024-05-04T03:15:48.050081+0000","last_unstale":"2024-05-04T03:17:33.166174+0000","last_undegraded":"2024-05-04T03:17:33.166174+0000","last_fullsized":"2024-05-04T03:17:33.166174+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:39:06.773121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.332461+0000","last_change":"2024-05-04T03:15:48.050936+0000","last_active":"2024-05-04T03:16:34.332461+0000","last_peered":"2024-05-04T03:16:34.332461+0000","last_clean":"2024-05-04T03:16:34.332461+0000","last_became_active":"2024-05-04T03:15:48.050567+0000","last_became_peered":"2024-05-04T03:15:48.050567+0000","last_unstale":"2024-05-04T03:16:34.332461+0000","last_undegraded":"2024-05-04T03:16:34.332461+0000","last_fullsized":"2024-05-04T03:16:34.332461+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:17:43.469889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.173119+0000","last_change":"2024-05-04T03:15:48.047200+0000","last_active":"2024-05-04T03:17:33.173119+0000","last_peered":"2024-05-04T03:17:33.173119+0000","last_clean":"2024-05-04T03:17:33.173119+0000","last_became_active":"2024-05-04T03:15:48.046782+0000","last_became_peered":"2024-05-04T03:15:48.046782+0000","last_unstale":"2024-05-04T03:17:33.173119+0000","last_undegraded":"2024-05-04T03:17:33.173119+0000","last_fullsized":"2024-05-04T03:17:33.173119+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:32:56.773916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.329252+0000","last_change":"2024-05-04T03:15:48.042114+0000","last_active":"2024-05-04T03:16:34.329252+0000","last_peered":"2024-05-04T03:16:34.329252+0000","last_clean":"2024-05-04T03:16:34.329252+0000","last_became_active":"2024-05-04T03:15:48.040921+0000","last_became_peered":"2024-05-04T03:15:48.040921+0000","last_unstale":"2024-05-04T03:16:34.329252+0000","last_undegraded":"2024-05-04T03:16:34.329252+0000","last_fullsized":"2024-05-04T03:16:34.329252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:16:20.130996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164290+0000","last_change":"2024-05-04T03:15:48.051060+0000","last_active":"2024-05-04T03:17:33.164290+0000","last_peered":"2024-05-04T03:17:33.164290+0000","last_clean":"2024-05-04T03:17:33.164290+0000","last_became_active":"2024-05-04T03:15:48.050642+0000","last_became_peered":"2024-05-04T03:15:48.050642+0000","last_unstale":"2024-05-04T03:17:33.164290+0000","last_undegraded":"2024-05-04T03:17:33.164290+0000","last_fullsized":"2024-05-04T03:17:33.164290+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:23:31.193072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.329322+0000","last_change":"2024-05-04T03:15:45.013477+0000","last_active":"2024-05-04T03:16:34.329322+0000","last_peered":"2024-05-04T03:16:34.329322+0000","last_clean":"2024-05-04T03:16:34.329322+0000","last_became_active":"2024-05-04T03:15:45.013161+0000","last_became_peered":"2024-05-04T03:15:45.013161+0000","last_unstale":"2024-05-04T03:16:34.329322+0000","last_undegraded":"2024-05-04T03:16:34.329322+0000","last_fullsized":"2024-05-04T03:16:34.329322+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:08:14.087090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164395+0000","last_change":"2024-05-04T03:15:48.045810+0000","last_active":"2024-05-04T03:17:33.164395+0000","last_peered":"2024-05-04T03:17:33.164395+0000","last_clean":"2024-05-04T03:17:33.164395+0000","last_became_active":"2024-05-04T03:15:48.045449+0000","last_became_peered":"2024-05-04T03:15:48.045449+0000","last_unstale":"2024-05-04T03:17:33.164395+0000","last_undegraded":"2024-05-04T03:17:33.164395+0000","last_fullsized":"2024-05-04T03:17:33.164395+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:48:05.084208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.417000+0000","last_change":"2024-05-04T03:15:45.204196+0000","last_active":"2024-05-04T03:16:34.417000+0000","last_peered":"2024-05-04T03:16:34.417000+0000","last_clean":"2024-05-04T03:16:34.417000+0000","last_became_active":"2024-05-04T03:15:45.203821+0000","last_became_peered":"2024-05-04T03:15:45.203821+0000","last_unstale":"2024-05-04T03:16:34.417000+0000","last_undegraded":"2024-05-04T03:16:34.417000+0000","last_fullsized":"2024-05-04T03:16:34.417000+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:17:05.074450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.167047+0000","last_change":"2024-05-04T03:15:48.045131+0000","last_active":"2024-05-04T03:17:33.167047+0000","last_peered":"2024-05-04T03:17:33.167047+0000","last_clean":"2024-05-04T03:17:33.167047+0000","last_became_active":"2024-05-04T03:15:48.044793+0000","last_became_peered":"2024-05-04T03:15:48.044793+0000","last_unstale":"2024-05-04T03:17:33.167047+0000","last_undegraded":"2024-05-04T03:17:33.167047+0000","last_fullsized":"2024-05-04T03:17:33.167047+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:03:37.769324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334692+0000","last_change":"2024-05-04T03:15:45.013644+0000","last_active":"2024-05-04T03:16:34.334692+0000","last_peered":"2024-05-04T03:16:34.334692+0000","last_clean":"2024-05-04T03:16:34.334692+0000","last_became_active":"2024-05-04T03:15:45.013282+0000","last_became_peered":"2024-05-04T03:15:45.013282+0000","last_unstale":"2024-05-04T03:16:34.334692+0000","last_undegraded":"2024-05-04T03:16:34.334692+0000","last_fullsized":"2024-05-04T03:16:34.334692+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:21:21.204433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.332993+0000","last_change":"2024-05-04T03:15:48.043046+0000","last_active":"2024-05-04T03:16:34.332993+0000","last_peered":"2024-05-04T03:16:34.332993+0000","last_clean":"2024-05-04T03:16:34.332993+0000","last_became_active":"2024-05-04T03:15:48.042621+0000","last_became_peered":"2024-05-04T03:15:48.042621+0000","last_unstale":"2024-05-04T03:16:34.332993+0000","last_undegraded":"2024-05-04T03:16:34.332993+0000","last_fullsized":"2024-05-04T03:16:34.332993+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:03:58.916423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334875+0000","last_change":"2024-05-04T03:15:45.010618+0000","last_active":"2024-05-04T03:16:34.334875+0000","last_peered":"2024-05-04T03:16:34.334875+0000","last_clean":"2024-05-04T03:16:34.334875+0000","last_became_active":"2024-05-04T03:15:45.009239+0000","last_became_peered":"2024-05-04T03:15:45.009239+0000","last_unstale":"2024-05-04T03:16:34.334875+0000","last_undegraded":"2024-05-04T03:16:34.334875+0000","last_fullsized":"2024-05-04T03:16:34.334875+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:49:46.777571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.145857+0000","last_change":"2024-05-04T03:15:48.049334+0000","last_active":"2024-05-04T03:17:33.145857+0000","last_peered":"2024-05-04T03:17:33.145857+0000","last_clean":"2024-05-04T03:17:33.145857+0000","last_became_active":"2024-05-04T03:15:48.048984+0000","last_became_peered":"2024-05-04T03:15:48.048984+0000","last_unstale":"2024-05-04T03:17:33.145857+0000","last_undegraded":"2024-05-04T03:17:33.145857+0000","last_fullsized":"2024-05-04T03:17:33.145857+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:28:25.366106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":123,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.393559+0000","last_change":"2024-05-04T03:15:45.015277+0000","last_active":"2024-05-04T03:17:33.393559+0000","last_peered":"2024-05-04T03:17:33.393559+0000","last_clean":"2024-05-04T03:17:33.393559+0000","last_became_active":"2024-05-04T03:15:45.014542+0000","last_became_peered":"2024-05-04T03:15:45.014542+0000","last_unstale":"2024-05-04T03:17:33.393559+0000","last_undegraded":"2024-05-04T03:17:33.393559+0000","last_fullsized":"2024-05-04T03:17:33.393559+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:22:15.594866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.146561+0000","last_change":"2024-05-04T03:15:48.045029+0000","last_active":"2024-05-04T03:17:33.146561+0000","last_peered":"2024-05-04T03:17:33.146561+0000","last_clean":"2024-05-04T03:17:33.146561+0000","last_became_active":"2024-05-04T03:15:48.041262+0000","last_became_peered":"2024-05-04T03:15:48.041262+0000","last_unstale":"2024-05-04T03:17:33.146561+0000","last_undegraded":"2024-05-04T03:17:33.146561+0000","last_fullsized":"2024-05-04T03:17:33.146561+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:45:56.882957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334143+0000","last_change":"2024-05-04T03:15:45.007530+0000","last_active":"2024-05-04T03:16:34.334143+0000","last_peered":"2024-05-04T03:16:34.334143+0000","last_clean":"2024-05-04T03:16:34.334143+0000","last_became_active":"2024-05-04T03:15:45.007054+0000","last_became_peered":"2024-05-04T03:15:45.007054+0000","last_unstale":"2024-05-04T03:16:34.334143+0000","last_undegraded":"2024-05-04T03:16:34.334143+0000","last_fullsized":"2024-05-04T03:16:34.334143+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:09:13.845679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.142587+0000","last_change":"2024-05-04T03:15:48.050956+0000","last_active":"2024-05-04T03:17:33.142587+0000","last_peered":"2024-05-04T03:17:33.142587+0000","last_clean":"2024-05-04T03:17:33.142587+0000","last_became_active":"2024-05-04T03:15:48.050597+0000","last_became_peered":"2024-05-04T03:15:48.050597+0000","last_unstale":"2024-05-04T03:17:33.142587+0000","last_undegraded":"2024-05-04T03:17:33.142587+0000","last_fullsized":"2024-05-04T03:17:33.142587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:27:34.127889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.331383+0000","last_change":"2024-05-04T03:15:45.007027+0000","last_active":"2024-05-04T03:16:34.331383+0000","last_peered":"2024-05-04T03:16:34.331383+0000","last_clean":"2024-05-04T03:16:34.331383+0000","last_became_active":"2024-05-04T03:15:45.006659+0000","last_became_peered":"2024-05-04T03:15:45.006659+0000","last_unstale":"2024-05-04T03:16:34.331383+0000","last_undegraded":"2024-05-04T03:16:34.331383+0000","last_fullsized":"2024-05-04T03:16:34.331383+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:06:11.827093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164076+0000","last_change":"2024-05-04T03:15:48.046196+0000","last_active":"2024-05-04T03:17:33.164076+0000","last_peered":"2024-05-04T03:17:33.164076+0000","last_clean":"2024-05-04T03:17:33.164076+0000","last_became_active":"2024-05-04T03:15:48.044316+0000","last_became_peered":"2024-05-04T03:15:48.044316+0000","last_unstale":"2024-05-04T03:17:33.164076+0000","last_undegraded":"2024-05-04T03:17:33.164076+0000","last_fullsized":"2024-05-04T03:17:33.164076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:04:39.301705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.416612+0000","last_change":"2024-05-04T03:15:45.015360+0000","last_active":"2024-05-04T03:16:34.416612+0000","last_peered":"2024-05-04T03:16:34.416612+0000","last_clean":"2024-05-04T03:16:34.416612+0000","last_became_active":"2024-05-04T03:15:45.014696+0000","last_became_peered":"2024-05-04T03:15:45.014696+0000","last_unstale":"2024-05-04T03:16:34.416612+0000","last_undegraded":"2024-05-04T03:16:34.416612+0000","last_fullsized":"2024-05-04T03:16:34.416612+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:17:28.838651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.328989+0000","last_change":"2024-05-04T03:15:48.039823+0000","last_active":"2024-05-04T03:16:34.328989+0000","last_peered":"2024-05-04T03:16:34.328989+0000","last_clean":"2024-05-04T03:16:34.328989+0000","last_became_active":"2024-05-04T03:15:48.039447+0000","last_became_peered":"2024-05-04T03:15:48.039447+0000","last_unstale":"2024-05-04T03:16:34.328989+0000","last_undegraded":"2024-05-04T03:16:34.328989+0000","last_fullsized":"2024-05-04T03:16:34.328989+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:45:55.136946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":117,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:31.389566+0000","last_change":"2024-05-04T03:15:45.004662+0000","last_active":"2024-05-04T03:17:31.389566+0000","last_peered":"2024-05-04T03:17:31.389566+0000","last_clean":"2024-05-04T03:17:31.389566+0000","last_became_active":"2024-05-04T03:15:45.004265+0000","last_became_peered":"2024-05-04T03:15:45.004265+0000","last_unstale":"2024-05-04T03:17:31.389566+0000","last_undegraded":"2024-05-04T03:17:31.389566+0000","last_fullsized":"2024-05-04T03:17:31.389566+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:14:03.065346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":88,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.333349+0000","last_change":"2024-05-04T03:15:48.042093+0000","last_active":"2024-05-04T03:16:34.333349+0000","last_peered":"2024-05-04T03:16:34.333349+0000","last_clean":"2024-05-04T03:16:34.333349+0000","last_became_active":"2024-05-04T03:15:48.041646+0000","last_became_peered":"2024-05-04T03:15:48.041646+0000","last_unstale":"2024-05-04T03:16:34.333349+0000","last_undegraded":"2024-05-04T03:16:34.333349+0000","last_fullsized":"2024-05-04T03:16:34.333349+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:06:32.083493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334519+0000","last_change":"2024-05-04T03:15:45.016731+0000","last_active":"2024-05-04T03:16:34.334519+0000","last_peered":"2024-05-04T03:16:34.334519+0000","last_clean":"2024-05-04T03:16:34.334519+0000","last_became_active":"2024-05-04T03:15:45.016313+0000","last_became_peered":"2024-05-04T03:15:45.016313+0000","last_unstale":"2024-05-04T03:16:34.334519+0000","last_undegraded":"2024-05-04T03:16:34.334519+0000","last_fullsized":"2024-05-04T03:16:34.334519+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:42:40.830986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.163387+0000","last_change":"2024-05-04T03:15:48.053649+0000","last_active":"2024-05-04T03:17:33.163387+0000","last_peered":"2024-05-04T03:17:33.163387+0000","last_clean":"2024-05-04T03:17:33.163387+0000","last_became_active":"2024-05-04T03:15:48.052407+0000","last_became_peered":"2024-05-04T03:15:48.052407+0000","last_unstale":"2024-05-04T03:17:33.163387+0000","last_undegraded":"2024-05-04T03:17:33.163387+0000","last_fullsized":"2024-05-04T03:17:33.163387+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:37:28.359272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.417235+0000","last_change":"2024-05-04T03:15:45.205180+0000","last_active":"2024-05-04T03:16:34.417235+0000","last_peered":"2024-05-04T03:16:34.417235+0000","last_clean":"2024-05-04T03:16:34.417235+0000","last_became_active":"2024-05-04T03:15:45.204828+0000","last_became_peered":"2024-05-04T03:15:45.204828+0000","last_unstale":"2024-05-04T03:16:34.417235+0000","last_undegraded":"2024-05-04T03:16:34.417235+0000","last_fullsized":"2024-05-04T03:16:34.417235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:27:34.063540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.333276+0000","last_change":"2024-05-04T03:15:48.047154+0000","last_active":"2024-05-04T03:16:34.333276+0000","last_peered":"2024-05-04T03:16:34.333276+0000","last_clean":"2024-05-04T03:16:34.333276+0000","last_became_active":"2024-05-04T03:15:48.046794+0000","last_became_peered":"2024-05-04T03:15:48.046794+0000","last_unstale":"2024-05-04T03:16:34.333276+0000","last_undegraded":"2024-05-04T03:16:34.333276+0000","last_fullsized":"2024-05-04T03:16:34.333276+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:21:42.723252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'32","reported_seq":243,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:32.391251+0000","last_change":"2024-05-04T03:15:45.008930+0000","last_active":"2024-05-04T03:17:32.391251+0000","last_peered":"2024-05-04T03:17:32.391251+0000","last_clean":"2024-05-04T03:17:32.391251+0000","last_became_active":"2024-05-04T03:15:45.008459+0000","last_became_peered":"2024-05-04T03:15:45.008459+0000","last_unstale":"2024-05-04T03:17:32.391251+0000","last_undegraded":"2024-05-04T03:17:32.391251+0000","last_fullsized":"2024-05-04T03:17:32.391251+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:07:52.609156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66132,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":185,"num_read_kb":157,"num_write":33,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.143616+0000","last_change":"2024-05-04T03:15:48.051848+0000","last_active":"2024-05-04T03:17:33.143616+0000","last_peered":"2024-05-04T03:17:33.143616+0000","last_clean":"2024-05-04T03:17:33.143616+0000","last_became_active":"2024-05-04T03:15:48.051171+0000","last_became_peered":"2024-05-04T03:15:48.051171+0000","last_unstale":"2024-05-04T03:17:33.143616+0000","last_undegraded":"2024-05-04T03:17:33.143616+0000","last_fullsized":"2024-05-04T03:17:33.143616+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:20:32.722845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.330336+0000","last_change":"2024-05-04T03:15:45.007970+0000","last_active":"2024-05-04T03:16:34.330336+0000","last_peered":"2024-05-04T03:16:34.330336+0000","last_clean":"2024-05-04T03:16:34.330336+0000","last_became_active":"2024-05-04T03:15:45.007605+0000","last_became_peered":"2024-05-04T03:15:45.007605+0000","last_unstale":"2024-05-04T03:16:34.330336+0000","last_undegraded":"2024-05-04T03:16:34.330336+0000","last_fullsized":"2024-05-04T03:16:34.330336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:52:08.466858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.168989+0000","last_change":"2024-05-04T03:15:48.050647+0000","last_active":"2024-05-04T03:17:33.168989+0000","last_peered":"2024-05-04T03:17:33.168989+0000","last_clean":"2024-05-04T03:17:33.168989+0000","last_became_active":"2024-05-04T03:15:48.050296+0000","last_became_peered":"2024-05-04T03:15:48.050296+0000","last_unstale":"2024-05-04T03:17:33.168989+0000","last_undegraded":"2024-05-04T03:17:33.168989+0000","last_fullsized":"2024-05-04T03:17:33.168989+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:14:23.902094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.416872+0000","last_change":"2024-05-04T03:15:45.014936+0000","last_active":"2024-05-04T03:16:34.416872+0000","last_peered":"2024-05-04T03:16:34.416872+0000","last_clean":"2024-05-04T03:16:34.416872+0000","last_became_active":"2024-05-04T03:15:45.014474+0000","last_became_peered":"2024-05-04T03:15:45.014474+0000","last_unstale":"2024-05-04T03:16:34.416872+0000","last_undegraded":"2024-05-04T03:16:34.416872+0000","last_fullsized":"2024-05-04T03:16:34.416872+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:26:37.972523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.169449+0000","last_change":"2024-05-04T03:16:34.336836+0000","last_active":"2024-05-04T03:17:33.169449+0000","last_peered":"2024-05-04T03:17:33.169449+0000","last_clean":"2024-05-04T03:17:33.169449+0000","last_became_active":"2024-05-04T03:16:34.336008+0000","last_became_peered":"2024-05-04T03:16:34.336008+0000","last_unstale":"2024-05-04T03:17:33.169449+0000","last_undegraded":"2024-05-04T03:17:33.169449+0000","last_fullsized":"2024-05-04T03:17:33.169449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:51:52.803138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'5","reported_seq":124,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.392762+0000","last_change":"2024-05-04T03:15:45.013177+0000","last_active":"2024-05-04T03:17:33.392762+0000","last_peered":"2024-05-04T03:17:33.392762+0000","last_clean":"2024-05-04T03:17:33.392762+0000","last_became_active":"2024-05-04T03:15:45.012764+0000","last_became_peered":"2024-05-04T03:15:45.012764+0000","last_unstale":"2024-05-04T03:17:33.392762+0000","last_undegraded":"2024-05-04T03:17:33.392762+0000","last_fullsized":"2024-05-04T03:17:33.392762+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:07:43.578443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.159529+0000","last_change":"2024-05-04T03:15:48.046453+0000","last_active":"2024-05-04T03:17:33.159529+0000","last_peered":"2024-05-04T03:17:33.159529+0000","last_clean":"2024-05-04T03:17:33.159529+0000","last_became_active":"2024-05-04T03:15:48.045990+0000","last_became_peered":"2024-05-04T03:15:48.045990+0000","last_unstale":"2024-05-04T03:17:33.159529+0000","last_undegraded":"2024-05-04T03:17:33.159529+0000","last_fullsized":"2024-05-04T03:17:33.159529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:54:57.285122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334287+0000","last_change":"2024-05-04T03:15:45.205539+0000","last_active":"2024-05-04T03:16:34.334287+0000","last_peered":"2024-05-04T03:16:34.334287+0000","last_clean":"2024-05-04T03:16:34.334287+0000","last_became_active":"2024-05-04T03:15:45.205174+0000","last_became_peered":"2024-05-04T03:15:45.205174+0000","last_unstale":"2024-05-04T03:16:34.334287+0000","last_undegraded":"2024-05-04T03:16:34.334287+0000","last_fullsized":"2024-05-04T03:16:34.334287+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:03:53.398368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.141711+0000","last_change":"2024-05-04T03:15:48.051824+0000","last_active":"2024-05-04T03:17:33.141711+0000","last_peered":"2024-05-04T03:17:33.141711+0000","last_clean":"2024-05-04T03:17:33.141711+0000","last_became_active":"2024-05-04T03:15:48.051483+0000","last_became_peered":"2024-05-04T03:15:48.051483+0000","last_unstale":"2024-05-04T03:17:33.141711+0000","last_undegraded":"2024-05-04T03:17:33.141711+0000","last_fullsized":"2024-05-04T03:17:33.141711+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:57:56.644486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334786+0000","last_change":"2024-05-04T03:15:45.011062+0000","last_active":"2024-05-04T03:16:34.334786+0000","last_peered":"2024-05-04T03:16:34.334786+0000","last_clean":"2024-05-04T03:16:34.334786+0000","last_became_active":"2024-05-04T03:15:45.010730+0000","last_became_peered":"2024-05-04T03:15:45.010730+0000","last_unstale":"2024-05-04T03:16:34.334786+0000","last_undegraded":"2024-05-04T03:16:34.334786+0000","last_fullsized":"2024-05-04T03:16:34.334786+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:45:55.979067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.167699+0000","last_change":"2024-05-04T03:15:48.045965+0000","last_active":"2024-05-04T03:17:33.167699+0000","last_peered":"2024-05-04T03:17:33.167699+0000","last_clean":"2024-05-04T03:17:33.167699+0000","last_became_active":"2024-05-04T03:15:48.045615+0000","last_became_peered":"2024-05-04T03:15:48.045615+0000","last_unstale":"2024-05-04T03:17:33.167699+0000","last_undegraded":"2024-05-04T03:17:33.167699+0000","last_fullsized":"2024-05-04T03:17:33.167699+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:14:23.497484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.329206+0000","last_change":"2024-05-04T03:15:45.204223+0000","last_active":"2024-05-04T03:16:34.329206+0000","last_peered":"2024-05-04T03:16:34.329206+0000","last_clean":"2024-05-04T03:16:34.329206+0000","last_became_active":"2024-05-04T03:15:45.203774+0000","last_became_peered":"2024-05-04T03:15:45.203774+0000","last_unstale":"2024-05-04T03:16:34.329206+0000","last_undegraded":"2024-05-04T03:16:34.329206+0000","last_fullsized":"2024-05-04T03:16:34.329206+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:02:34.456055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334351+0000","last_change":"2024-05-04T03:15:48.045574+0000","last_active":"2024-05-04T03:16:34.334351+0000","last_peered":"2024-05-04T03:16:34.334351+0000","last_clean":"2024-05-04T03:16:34.334351+0000","last_became_active":"2024-05-04T03:15:48.045237+0000","last_became_peered":"2024-05-04T03:15:48.045237+0000","last_unstale":"2024-05-04T03:16:34.334351+0000","last_undegraded":"2024-05-04T03:16:34.334351+0000","last_fullsized":"2024-05-04T03:16:34.334351+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:53:46.346199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.333866+0000","last_change":"2024-05-04T03:15:45.013895+0000","last_active":"2024-05-04T03:16:34.333866+0000","last_peered":"2024-05-04T03:16:34.333866+0000","last_clean":"2024-05-04T03:16:34.333866+0000","last_became_active":"2024-05-04T03:15:45.013306+0000","last_became_peered":"2024-05-04T03:15:45.013306+0000","last_unstale":"2024-05-04T03:16:34.333866+0000","last_undegraded":"2024-05-04T03:16:34.333866+0000","last_fullsized":"2024-05-04T03:16:34.333866+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:39:18.616432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.336997+0000","last_change":"2024-05-04T03:16:34.336997+0000","last_active":"2024-05-04T03:16:34.336997+0000","last_peered":"2024-05-04T03:16:34.336997+0000","last_clean":"2024-05-04T03:16:34.336997+0000","last_became_active":"2024-05-04T03:16:34.336505+0000","last_became_peered":"2024-05-04T03:16:34.336505+0000","last_unstale":"2024-05-04T03:16:34.336997+0000","last_undegraded":"2024-05-04T03:16:34.336997+0000","last_fullsized":"2024-05-04T03:16:34.336997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:23:38.979216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.149216+0000","last_change":"2024-05-04T03:15:48.050122+0000","last_active":"2024-05-04T03:17:33.149216+0000","last_peered":"2024-05-04T03:17:33.149216+0000","last_clean":"2024-05-04T03:17:33.149216+0000","last_became_active":"2024-05-04T03:15:48.049794+0000","last_became_peered":"2024-05-04T03:15:48.049794+0000","last_unstale":"2024-05-04T03:17:33.149216+0000","last_undegraded":"2024-05-04T03:17:33.149216+0000","last_fullsized":"2024-05-04T03:17:33.149216+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:29:11.066068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":119,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:32.390637+0000","last_change":"2024-05-04T03:15:45.205143+0000","last_active":"2024-05-04T03:17:32.390637+0000","last_peered":"2024-05-04T03:17:32.390637+0000","last_clean":"2024-05-04T03:17:32.390637+0000","last_became_active":"2024-05-04T03:15:45.204727+0000","last_became_peered":"2024-05-04T03:15:45.204727+0000","last_unstale":"2024-05-04T03:17:32.390637+0000","last_undegraded":"2024-05-04T03:17:32.390637+0000","last_fullsized":"2024-05-04T03:17:32.390637+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:15:05.799556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":88,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.163619+0000","last_change":"2024-05-04T03:15:48.042914+0000","last_active":"2024-05-04T03:17:33.163619+0000","last_peered":"2024-05-04T03:17:33.163619+0000","last_clean":"2024-05-04T03:17:33.163619+0000","last_became_active":"2024-05-04T03:15:48.041936+0000","last_became_peered":"2024-05-04T03:15:48.041936+0000","last_unstale":"2024-05-04T03:17:33.163619+0000","last_undegraded":"2024-05-04T03:17:33.163619+0000","last_fullsized":"2024-05-04T03:17:33.163619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:11:48.807602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.331225+0000","last_change":"2024-05-04T03:15:45.011045+0000","last_active":"2024-05-04T03:16:34.331225+0000","last_peered":"2024-05-04T03:16:34.331225+0000","last_clean":"2024-05-04T03:16:34.331225+0000","last_became_active":"2024-05-04T03:15:45.010608+0000","last_became_peered":"2024-05-04T03:15:45.010608+0000","last_unstale":"2024-05-04T03:16:34.331225+0000","last_undegraded":"2024-05-04T03:16:34.331225+0000","last_fullsized":"2024-05-04T03:16:34.331225+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:00:36.243588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":102,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.333935+0000","last_change":"2024-05-04T03:14:19.895070+0000","last_active":"2024-05-04T03:16:34.333935+0000","last_peered":"2024-05-04T03:16:34.333935+0000","last_clean":"2024-05-04T03:16:34.333935+0000","last_became_active":"2024-05-04T03:14:19.889987+0000","last_became_peered":"2024-05-04T03:14:19.889987+0000","last_unstale":"2024-05-04T03:16:34.333935+0000","last_undegraded":"2024-05-04T03:16:34.333935+0000","last_fullsized":"2024-05-04T03:16:34.333935+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:11:01.712629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:11:01.712629+0000","last_clean_scrub_stamp":"2024-05-04T03:11:01.712629+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:03:02.769719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165934+0000","last_change":"2024-05-04T03:15:48.044928+0000","last_active":"2024-05-04T03:17:33.165934+0000","last_peered":"2024-05-04T03:17:33.165934+0000","last_clean":"2024-05-04T03:17:33.165934+0000","last_became_active":"2024-05-04T03:15:48.044532+0000","last_became_peered":"2024-05-04T03:15:48.044532+0000","last_unstale":"2024-05-04T03:17:33.165934+0000","last_undegraded":"2024-05-04T03:17:33.165934+0000","last_fullsized":"2024-05-04T03:17:33.165934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:27:00.825692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334044+0000","last_change":"2024-05-04T03:15:45.014469+0000","last_active":"2024-05-04T03:16:34.334044+0000","last_peered":"2024-05-04T03:16:34.334044+0000","last_clean":"2024-05-04T03:16:34.334044+0000","last_became_active":"2024-05-04T03:15:45.013763+0000","last_became_peered":"2024-05-04T03:15:45.013763+0000","last_unstale":"2024-05-04T03:16:34.334044+0000","last_undegraded":"2024-05-04T03:16:34.334044+0000","last_fullsized":"2024-05-04T03:16:34.334044+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:19:36.379930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334548+0000","last_change":"2024-05-04T03:15:45.204536+0000","last_active":"2024-05-04T03:16:34.334548+0000","last_peered":"2024-05-04T03:16:34.334548+0000","last_clean":"2024-05-04T03:16:34.334548+0000","last_became_active":"2024-05-04T03:15:45.204072+0000","last_became_peered":"2024-05-04T03:15:45.204072+0000","last_unstale":"2024-05-04T03:16:34.334548+0000","last_undegraded":"2024-05-04T03:16:34.334548+0000","last_fullsized":"2024-05-04T03:16:34.334548+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:49:12.726220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.155914+0000","last_change":"2024-05-04T03:15:48.044982+0000","last_active":"2024-05-04T03:17:33.155914+0000","last_peered":"2024-05-04T03:17:33.155914+0000","last_clean":"2024-05-04T03:17:33.155914+0000","last_became_active":"2024-05-04T03:15:48.043996+0000","last_became_peered":"2024-05-04T03:15:48.043996+0000","last_unstale":"2024-05-04T03:17:33.155914+0000","last_undegraded":"2024-05-04T03:17:33.155914+0000","last_fullsized":"2024-05-04T03:17:33.155914+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:09:39.770023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'10","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.329189+0000","last_change":"2024-05-04T03:15:45.008153+0000","last_active":"2024-05-04T03:16:34.329189+0000","last_peered":"2024-05-04T03:16:34.329189+0000","last_clean":"2024-05-04T03:16:34.329189+0000","last_became_active":"2024-05-04T03:15:45.007698+0000","last_became_peered":"2024-05-04T03:15:45.007698+0000","last_unstale":"2024-05-04T03:16:34.329189+0000","last_undegraded":"2024-05-04T03:16:34.329189+0000","last_fullsized":"2024-05-04T03:16:34.329189+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:56:52.322049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.168628+0000","last_change":"2024-05-04T03:15:48.055767+0000","last_active":"2024-05-04T03:17:33.168628+0000","last_peered":"2024-05-04T03:17:33.168628+0000","last_clean":"2024-05-04T03:17:33.168628+0000","last_became_active":"2024-05-04T03:15:48.055252+0000","last_became_peered":"2024-05-04T03:15:48.055252+0000","last_unstale":"2024-05-04T03:17:33.168628+0000","last_undegraded":"2024-05-04T03:17:33.168628+0000","last_fullsized":"2024-05-04T03:17:33.168628+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:28:11.736803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.331482+0000","last_change":"2024-05-04T03:15:45.012838+0000","last_active":"2024-05-04T03:16:34.331482+0000","last_peered":"2024-05-04T03:16:34.331482+0000","last_clean":"2024-05-04T03:16:34.331482+0000","last_became_active":"2024-05-04T03:15:45.012429+0000","last_became_peered":"2024-05-04T03:15:45.012429+0000","last_unstale":"2024-05-04T03:16:34.331482+0000","last_undegraded":"2024-05-04T03:16:34.331482+0000","last_fullsized":"2024-05-04T03:16:34.331482+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:22:22.602551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165352+0000","last_change":"2024-05-04T03:15:48.056993+0000","last_active":"2024-05-04T03:17:33.165352+0000","last_peered":"2024-05-04T03:17:33.165352+0000","last_clean":"2024-05-04T03:17:33.165352+0000","last_became_active":"2024-05-04T03:15:48.055938+0000","last_became_peered":"2024-05-04T03:15:48.055938+0000","last_unstale":"2024-05-04T03:17:33.165352+0000","last_undegraded":"2024-05-04T03:17:33.165352+0000","last_fullsized":"2024-05-04T03:17:33.165352+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:23:59.521692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.340048+0000","last_change":"2024-05-04T03:16:34.340048+0000","last_active":"2024-05-04T03:16:34.340048+0000","last_peered":"2024-05-04T03:16:34.340048+0000","last_clean":"2024-05-04T03:16:34.340048+0000","last_became_active":"2024-05-04T03:16:34.336386+0000","last_became_peered":"2024-05-04T03:16:34.336386+0000","last_unstale":"2024-05-04T03:16:34.340048+0000","last_undegraded":"2024-05-04T03:16:34.340048+0000","last_fullsized":"2024-05-04T03:16:34.340048+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:45:43.734097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.128655+0000","last_change":"2024-05-04T03:15:48.043614+0000","last_active":"2024-05-04T03:17:33.128655+0000","last_peered":"2024-05-04T03:17:33.128655+0000","last_clean":"2024-05-04T03:17:33.128655+0000","last_became_active":"2024-05-04T03:15:48.042390+0000","last_became_peered":"2024-05-04T03:15:48.042390+0000","last_unstale":"2024-05-04T03:17:33.128655+0000","last_undegraded":"2024-05-04T03:17:33.128655+0000","last_fullsized":"2024-05-04T03:17:33.128655+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:56:22.791368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.416508+0000","last_change":"2024-05-04T03:15:45.005570+0000","last_active":"2024-05-04T03:16:34.416508+0000","last_peered":"2024-05-04T03:16:34.416508+0000","last_clean":"2024-05-04T03:16:34.416508+0000","last_became_active":"2024-05-04T03:15:45.005105+0000","last_became_peered":"2024-05-04T03:15:45.005105+0000","last_unstale":"2024-05-04T03:16:34.416508+0000","last_undegraded":"2024-05-04T03:16:34.416508+0000","last_fullsized":"2024-05-04T03:16:34.416508+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:40:17.275618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166777+0000","last_change":"2024-05-04T03:15:48.045081+0000","last_active":"2024-05-04T03:17:33.166777+0000","last_peered":"2024-05-04T03:17:33.166777+0000","last_clean":"2024-05-04T03:17:33.166777+0000","last_became_active":"2024-05-04T03:15:48.044745+0000","last_became_peered":"2024-05-04T03:15:48.044745+0000","last_unstale":"2024-05-04T03:17:33.166777+0000","last_undegraded":"2024-05-04T03:17:33.166777+0000","last_fullsized":"2024-05-04T03:17:33.166777+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:40:53.486222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.330010+0000","last_change":"2024-05-04T03:15:45.008926+0000","last_active":"2024-05-04T03:16:34.330010+0000","last_peered":"2024-05-04T03:16:34.330010+0000","last_clean":"2024-05-04T03:16:34.330010+0000","last_became_active":"2024-05-04T03:15:45.008412+0000","last_became_peered":"2024-05-04T03:15:45.008412+0000","last_unstale":"2024-05-04T03:16:34.330010+0000","last_undegraded":"2024-05-04T03:16:34.330010+0000","last_fullsized":"2024-05-04T03:16:34.330010+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:34:33.107866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.162308+0000","last_change":"2024-05-04T03:15:48.054580+0000","last_active":"2024-05-04T03:17:33.162308+0000","last_peered":"2024-05-04T03:17:33.162308+0000","last_clean":"2024-05-04T03:17:33.162308+0000","last_became_active":"2024-05-04T03:15:48.054242+0000","last_became_peered":"2024-05-04T03:15:48.054242+0000","last_unstale":"2024-05-04T03:17:33.162308+0000","last_undegraded":"2024-05-04T03:17:33.162308+0000","last_fullsized":"2024-05-04T03:17:33.162308+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:16:19.426571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.341316+0000","last_change":"2024-05-04T03:16:34.340794+0000","last_active":"2024-05-04T03:16:34.341316+0000","last_peered":"2024-05-04T03:16:34.341316+0000","last_clean":"2024-05-04T03:16:34.341316+0000","last_became_active":"2024-05-04T03:16:34.336038+0000","last_became_peered":"2024-05-04T03:16:34.336038+0000","last_unstale":"2024-05-04T03:16:34.341316+0000","last_undegraded":"2024-05-04T03:16:34.341316+0000","last_fullsized":"2024-05-04T03:16:34.341316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:45:19.323938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.172069+0000","last_change":"2024-05-04T03:15:48.045284+0000","last_active":"2024-05-04T03:17:33.172069+0000","last_peered":"2024-05-04T03:17:33.172069+0000","last_clean":"2024-05-04T03:17:33.172069+0000","last_became_active":"2024-05-04T03:15:48.044673+0000","last_became_peered":"2024-05-04T03:15:48.044673+0000","last_unstale":"2024-05-04T03:17:33.172069+0000","last_undegraded":"2024-05-04T03:17:33.172069+0000","last_fullsized":"2024-05-04T03:17:33.172069+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:51:52.389748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.330774+0000","last_change":"2024-05-04T03:15:45.205648+0000","last_active":"2024-05-04T03:16:34.330774+0000","last_peered":"2024-05-04T03:16:34.330774+0000","last_clean":"2024-05-04T03:16:34.330774+0000","last_became_active":"2024-05-04T03:15:45.204169+0000","last_became_peered":"2024-05-04T03:15:45.204169+0000","last_unstale":"2024-05-04T03:16:34.330774+0000","last_undegraded":"2024-05-04T03:16:34.330774+0000","last_fullsized":"2024-05-04T03:16:34.330774+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:20:48.859276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165884+0000","last_change":"2024-05-04T03:15:48.049808+0000","last_active":"2024-05-04T03:17:33.165884+0000","last_peered":"2024-05-04T03:17:33.165884+0000","last_clean":"2024-05-04T03:17:33.165884+0000","last_became_active":"2024-05-04T03:15:48.048998+0000","last_became_peered":"2024-05-04T03:15:48.048998+0000","last_unstale":"2024-05-04T03:17:33.165884+0000","last_undegraded":"2024-05-04T03:17:33.165884+0000","last_fullsized":"2024-05-04T03:17:33.165884+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:54:46.786406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.417058+0000","last_change":"2024-05-04T03:15:45.008874+0000","last_active":"2024-05-04T03:16:34.417058+0000","last_peered":"2024-05-04T03:16:34.417058+0000","last_clean":"2024-05-04T03:16:34.417058+0000","last_became_active":"2024-05-04T03:15:45.008530+0000","last_became_peered":"2024-05-04T03:15:45.008530+0000","last_unstale":"2024-05-04T03:16:34.417058+0000","last_undegraded":"2024-05-04T03:16:34.417058+0000","last_fullsized":"2024-05-04T03:16:34.417058+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:31:41.697386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.168741+0000","last_change":"2024-05-04T03:15:48.046042+0000","last_active":"2024-05-04T03:17:33.168741+0000","last_peered":"2024-05-04T03:17:33.168741+0000","last_clean":"2024-05-04T03:17:33.168741+0000","last_became_active":"2024-05-04T03:15:48.045387+0000","last_became_peered":"2024-05-04T03:15:48.045387+0000","last_unstale":"2024-05-04T03:17:33.168741+0000","last_undegraded":"2024-05-04T03:17:33.168741+0000","last_fullsized":"2024-05-04T03:17:33.168741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:43:50.782623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.331577+0000","last_change":"2024-05-04T03:15:45.009126+0000","last_active":"2024-05-04T03:16:34.331577+0000","last_peered":"2024-05-04T03:16:34.331577+0000","last_clean":"2024-05-04T03:16:34.331577+0000","last_became_active":"2024-05-04T03:15:45.008802+0000","last_became_peered":"2024-05-04T03:15:45.008802+0000","last_unstale":"2024-05-04T03:16:34.331577+0000","last_undegraded":"2024-05-04T03:16:34.331577+0000","last_fullsized":"2024-05-04T03:16:34.331577+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:25:30.867870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.163747+0000","last_change":"2024-05-04T03:15:48.050478+0000","last_active":"2024-05-04T03:17:33.163747+0000","last_peered":"2024-05-04T03:17:33.163747+0000","last_clean":"2024-05-04T03:17:33.163747+0000","last_became_active":"2024-05-04T03:15:48.045756+0000","last_became_peered":"2024-05-04T03:15:48.045756+0000","last_unstale":"2024-05-04T03:17:33.163747+0000","last_undegraded":"2024-05-04T03:17:33.163747+0000","last_fullsized":"2024-05-04T03:17:33.163747+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:05:11.098381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165760+0000","last_change":"2024-05-04T03:15:48.042778+0000","last_active":"2024-05-04T03:17:33.165760+0000","last_peered":"2024-05-04T03:17:33.165760+0000","last_clean":"2024-05-04T03:17:33.165760+0000","last_became_active":"2024-05-04T03:15:48.042349+0000","last_became_peered":"2024-05-04T03:15:48.042349+0000","last_unstale":"2024-05-04T03:17:33.165760+0000","last_undegraded":"2024-05-04T03:17:33.165760+0000","last_fullsized":"2024-05-04T03:17:33.165760+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:40:12.260026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.160850+0000","last_change":"2024-05-04T03:15:48.048544+0000","last_active":"2024-05-04T03:17:33.160850+0000","last_peered":"2024-05-04T03:17:33.160850+0000","last_clean":"2024-05-04T03:17:33.160850+0000","last_became_active":"2024-05-04T03:15:48.047873+0000","last_became_peered":"2024-05-04T03:15:48.047873+0000","last_unstale":"2024-05-04T03:17:33.160850+0000","last_undegraded":"2024-05-04T03:17:33.160850+0000","last_fullsized":"2024-05-04T03:17:33.160850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:48:02.290401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166521+0000","last_change":"2024-05-04T03:15:48.053467+0000","last_active":"2024-05-04T03:17:33.166521+0000","last_peered":"2024-05-04T03:17:33.166521+0000","last_clean":"2024-05-04T03:17:33.166521+0000","last_became_active":"2024-05-04T03:15:48.050796+0000","last_became_peered":"2024-05-04T03:15:48.050796+0000","last_unstale":"2024-05-04T03:17:33.166521+0000","last_undegraded":"2024-05-04T03:17:33.166521+0000","last_fullsized":"2024-05-04T03:17:33.166521+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:11:15.009467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.158905+0000","last_change":"2024-05-04T03:15:48.054616+0000","last_active":"2024-05-04T03:17:33.158905+0000","last_peered":"2024-05-04T03:17:33.158905+0000","last_clean":"2024-05-04T03:17:33.158905+0000","last_became_active":"2024-05-04T03:15:48.054142+0000","last_became_peered":"2024-05-04T03:15:48.054142+0000","last_unstale":"2024-05-04T03:17:33.158905+0000","last_undegraded":"2024-05-04T03:17:33.158905+0000","last_fullsized":"2024-05-04T03:17:33.158905+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:18:24.067666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166245+0000","last_change":"2024-05-04T03:15:48.046969+0000","last_active":"2024-05-04T03:17:33.166245+0000","last_peered":"2024-05-04T03:17:33.166245+0000","last_clean":"2024-05-04T03:17:33.166245+0000","last_became_active":"2024-05-04T03:15:48.046637+0000","last_became_peered":"2024-05-04T03:15:48.046637+0000","last_unstale":"2024-05-04T03:17:33.166245+0000","last_undegraded":"2024-05-04T03:17:33.166245+0000","last_fullsized":"2024-05-04T03:17:33.166245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:03:01.763037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.163594+0000","last_change":"2024-05-04T03:15:48.041753+0000","last_active":"2024-05-04T03:17:33.163594+0000","last_peered":"2024-05-04T03:17:33.163594+0000","last_clean":"2024-05-04T03:17:33.163594+0000","last_became_active":"2024-05-04T03:15:48.041236+0000","last_became_peered":"2024-05-04T03:15:48.041236+0000","last_unstale":"2024-05-04T03:17:33.163594+0000","last_undegraded":"2024-05-04T03:17:33.163594+0000","last_fullsized":"2024-05-04T03:17:33.163594+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:41:02.928257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.161902+0000","last_change":"2024-05-04T03:15:48.044192+0000","last_active":"2024-05-04T03:17:33.161902+0000","last_peered":"2024-05-04T03:17:33.161902+0000","last_clean":"2024-05-04T03:17:33.161902+0000","last_became_active":"2024-05-04T03:15:48.043849+0000","last_became_peered":"2024-05-04T03:15:48.043849+0000","last_unstale":"2024-05-04T03:17:33.161902+0000","last_undegraded":"2024-05-04T03:17:33.161902+0000","last_fullsized":"2024-05-04T03:17:33.161902+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:55:22.000788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.161015+0000","last_change":"2024-05-04T03:15:48.045675+0000","last_active":"2024-05-04T03:17:33.161015+0000","last_peered":"2024-05-04T03:17:33.161015+0000","last_clean":"2024-05-04T03:17:33.161015+0000","last_became_active":"2024-05-04T03:15:48.045321+0000","last_became_peered":"2024-05-04T03:15:48.045321+0000","last_unstale":"2024-05-04T03:17:33.161015+0000","last_undegraded":"2024-05-04T03:17:33.161015+0000","last_fullsized":"2024-05-04T03:17:33.161015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:12:06.060729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.142817+0000","last_change":"2024-05-04T03:15:48.040872+0000","last_active":"2024-05-04T03:17:33.142817+0000","last_peered":"2024-05-04T03:17:33.142817+0000","last_clean":"2024-05-04T03:17:33.142817+0000","last_became_active":"2024-05-04T03:15:48.040485+0000","last_became_peered":"2024-05-04T03:15:48.040485+0000","last_unstale":"2024-05-04T03:17:33.142817+0000","last_undegraded":"2024-05-04T03:17:33.142817+0000","last_fullsized":"2024-05-04T03:17:33.142817+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:58:01.789270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.161700+0000","last_change":"2024-05-04T03:15:48.049080+0000","last_active":"2024-05-04T03:17:33.161700+0000","last_peered":"2024-05-04T03:17:33.161700+0000","last_clean":"2024-05-04T03:17:33.161700+0000","last_became_active":"2024-05-04T03:15:48.044624+0000","last_became_peered":"2024-05-04T03:15:48.044624+0000","last_unstale":"2024-05-04T03:17:33.161700+0000","last_undegraded":"2024-05-04T03:17:33.161700+0000","last_fullsized":"2024-05-04T03:17:33.161700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:47:08.647401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.332733+0000","last_change":"2024-05-04T03:15:48.049394+0000","last_active":"2024-05-04T03:16:34.332733+0000","last_peered":"2024-05-04T03:16:34.332733+0000","last_clean":"2024-05-04T03:16:34.332733+0000","last_became_active":"2024-05-04T03:15:48.048980+0000","last_became_peered":"2024-05-04T03:15:48.048980+0000","last_unstale":"2024-05-04T03:16:34.332733+0000","last_undegraded":"2024-05-04T03:16:34.332733+0000","last_fullsized":"2024-05-04T03:16:34.332733+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:43:24.914665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.171090+0000","last_change":"2024-05-04T03:15:48.051672+0000","last_active":"2024-05-04T03:17:33.171090+0000","last_peered":"2024-05-04T03:17:33.171090+0000","last_clean":"2024-05-04T03:17:33.171090+0000","last_became_active":"2024-05-04T03:15:48.051314+0000","last_became_peered":"2024-05-04T03:15:48.051314+0000","last_unstale":"2024-05-04T03:17:33.171090+0000","last_undegraded":"2024-05-04T03:17:33.171090+0000","last_fullsized":"2024-05-04T03:17:33.171090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:19:22.808102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.167645+0000","last_change":"2024-05-04T03:15:48.053045+0000","last_active":"2024-05-04T03:17:33.167645+0000","last_peered":"2024-05-04T03:17:33.167645+0000","last_clean":"2024-05-04T03:17:33.167645+0000","last_became_active":"2024-05-04T03:15:48.052659+0000","last_became_peered":"2024-05-04T03:15:48.052659+0000","last_unstale":"2024-05-04T03:17:33.167645+0000","last_undegraded":"2024-05-04T03:17:33.167645+0000","last_fullsized":"2024-05-04T03:17:33.167645+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:53:25.624540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":388,"ondisk_log_size":388,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70274,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":558,"num_read_kb":351,"num_write":167,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":210930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612646,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":440503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808874,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1601536,"data_stored":1098507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972399,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":440739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168630,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":440659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397562,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32652,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712500,"statfs":{"total":95995035648,"available":95961600000,"internally_reserved":0,"allocated":1511424,"data_stored":1032730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593791,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":905216,"data_stored":440128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":970752,"data_stored":506053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40400,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704752,"statfs":{"total":95995035648,"available":95953666048,"internally_reserved":0,"allocated":925696,"data_stored":441399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":507652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40396,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704756,"statfs":{"total":95995035648,"available":95953670144,"internally_reserved":0,"allocated":921600,"data_stored":440439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986249,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":441097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790021,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36824,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708328,"statfs":{"total":95995035648,"available":95957327872,"internally_reserved":0,"allocated":1523712,"data_stored":1031882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":66132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":67731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":68101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-04T03:17:38.832 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-04T03:17:39.080 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:17:39.538 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:39 smithi001 ceph-mon[29731]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:17:39.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:39 smithi077 ceph-mon[33531]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:17:39.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:39 smithi072 ceph-mon[33332]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:17:40.461 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:40 smithi001 ceph-mon[29731]: pgmap v346: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-04T03:17:40.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:40 smithi077 ceph-mon[33531]: pgmap v346: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-04T03:17:40.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.691+0000 7fa1cd151700 1 -- 172.21.15.1:0/2213472020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1c81034c0 msgr2=0x7fa1c81038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:40.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.691+0000 7fa1cd151700 1 --2- 172.21.15.1:0/2213472020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1c81034c0 0x7fa1c81038d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80099c0 tx=0x7fa1b8009cd0 comp rx=0 tx=0).stop 2024-05-04T03:17:40.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.691+0000 7fa1cd151700 1 -- 172.21.15.1:0/2213472020 shutdown_connections 2024-05-04T03:17:40.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.691+0000 7fa1cd151700 1 --2- 172.21.15.1:0/2213472020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1c8108740 0x7fa1c81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:40.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.691+0000 7fa1cd151700 1 --2- 172.21.15.1:0/2213472020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1c81034c0 0x7fa1c81038d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:40.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.691+0000 7fa1cd151700 1 --2- 172.21.15.1:0/2213472020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1c8107df0 0x7fa1c8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:40.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.691+0000 7fa1cd151700 1 -- 172.21.15.1:0/2213472020 >> 172.21.15.1:0/2213472020 conn(0x7fa1c80fedc0 msgr2=0x7fa1c81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:40.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.691+0000 7fa1cd151700 1 -- 172.21.15.1:0/2213472020 shutdown_connections 2024-05-04T03:17:40.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.691+0000 7fa1cd151700 1 -- 172.21.15.1:0/2213472020 wait complete. 2024-05-04T03:17:40.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.692+0000 7fa1cd151700 1 Processor -- start 2024-05-04T03:17:40.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.692+0000 7fa1cd151700 1 -- start start 2024-05-04T03:17:40.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.692+0000 7fa1cd151700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1c81034c0 0x7fa1c819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:40.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.692+0000 7fa1cd151700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1c8107df0 0x7fa1c819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:40.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.692+0000 7fa1cd151700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1c8108740 0x7fa1c819f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:40.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.692+0000 7fa1cd151700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c8113370 con 0x7fa1c8108740 2024-05-04T03:17:40.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.692+0000 7fa1cd151700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c81131f0 con 0x7fa1c81034c0 2024-05-04T03:17:40.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.692+0000 7fa1cd151700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c81134f0 con 0x7fa1c8107df0 2024-05-04T03:17:40.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.692+0000 7fa1c759e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1c8108740 0x7fa1c819f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:40.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.692+0000 7fa1c759e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1c8108740 0x7fa1c819f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52478/0 (socket says 172.21.15.1:52478) 2024-05-04T03:17:40.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.692+0000 7fa1c759e700 1 -- 172.21.15.1:0/1234749551 learned_addr learned my addr 172.21.15.1:0/1234749551 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:40.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.692+0000 7fa1c6d9d700 1 --2- 172.21.15.1:0/1234749551 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1c81034c0 0x7fa1c819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:40.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.693+0000 7fa1c659c700 1 --2- 172.21.15.1:0/1234749551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1c8107df0 0x7fa1c819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:40.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.693+0000 7fa1c759e700 1 -- 172.21.15.1:0/1234749551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1c8107df0 msgr2=0x7fa1c819b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:40.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.693+0000 7fa1c759e700 1 --2- 172.21.15.1:0/1234749551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1c8107df0 0x7fa1c819b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:40.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.693+0000 7fa1c759e700 1 -- 172.21.15.1:0/1234749551 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1c81034c0 msgr2=0x7fa1c819adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:40.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.693+0000 7fa1c759e700 1 --2- 172.21.15.1:0/1234749551 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1c81034c0 0x7fa1c819adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:40.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.693+0000 7fa1c759e700 1 -- 172.21.15.1:0/1234749551 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c819fc10 con 0x7fa1c8108740 2024-05-04T03:17:40.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.693+0000 7fa1c659c700 1 --2- 172.21.15.1:0/1234749551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1c8107df0 0x7fa1c819b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:17:40.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.693+0000 7fa1c759e700 1 --2- 172.21.15.1:0/1234749551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1c8108740 0x7fa1c819f620 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000ef30 tx=0x7fa1c000c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:40.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.693+0000 7fa1affff700 1 -- 172.21.15.1:0/1234749551 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c00092e0 con 0x7fa1c8108740 2024-05-04T03:17:40.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.693+0000 7fa1cd151700 1 -- 172.21.15.1:0/1234749551 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1c819fea0 con 0x7fa1c8108740 2024-05-04T03:17:40.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.693+0000 7fa1affff700 1 -- 172.21.15.1:0/1234749551 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1c0004bb0 con 0x7fa1c8108740 2024-05-04T03:17:40.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.693+0000 7fa1affff700 1 -- 172.21.15.1:0/1234749551 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0004280 con 0x7fa1c8108740 2024-05-04T03:17:40.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.693+0000 7fa1cd151700 1 -- 172.21.15.1:0/1234749551 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1c81a0390 con 0x7fa1c8108740 2024-05-04T03:17:40.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.694+0000 7fa1cd151700 1 -- 172.21.15.1:0/1234749551 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1c80512c0 con 0x7fa1c8108740 2024-05-04T03:17:40.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.696+0000 7fa1affff700 1 -- 172.21.15.1:0/1234749551 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fa1c001b030 con 0x7fa1c8108740 2024-05-04T03:17:40.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.697+0000 7fa1affff700 1 --2- 172.21.15.1:0/1234749551 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa1b00b2880 0x7fa1b00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:40.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.697+0000 7fa1c6d9d700 1 --2- 172.21.15.1:0/1234749551 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa1b00b2880 0x7fa1b00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:40.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.697+0000 7fa1affff700 1 -- 172.21.15.1:0/1234749551 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fa1c00e2710 con 0x7fa1c8108740 2024-05-04T03:17:40.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.697+0000 7fa1c6d9d700 1 --2- 172.21.15.1:0/1234749551 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa1b00b2880 0x7fa1b00b4d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8003760 tx=0x7fa1b801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:40.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.701+0000 7fa1affff700 1 -- 172.21.15.1:0/1234749551 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa1c0022020 con 0x7fa1c8108740 2024-05-04T03:17:40.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:40 smithi072 ceph-mon[33332]: pgmap v346: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-04T03:17:40.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.909+0000 7fa1cd151700 1 -- 172.21.15.1:0/1234749551 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa1c8063c00 con 0x7fa1b00b2880 2024-05-04T03:17:40.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.917+0000 7fa1affff700 1 -- 172.21.15.1:0/1234749551 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258915 (secure 0 0 0) 0x7fa1c8063c00 con 0x7fa1b00b2880 2024-05-04T03:17:40.922 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:17:40.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.920+0000 7fa1cd151700 1 -- 172.21.15.1:0/1234749551 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa1b00b2880 msgr2=0x7fa1b00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:40.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.920+0000 7fa1cd151700 1 --2- 172.21.15.1:0/1234749551 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa1b00b2880 0x7fa1b00b4d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8003760 tx=0x7fa1b801a040 comp rx=0 tx=0).stop 2024-05-04T03:17:40.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.920+0000 7fa1cd151700 1 -- 172.21.15.1:0/1234749551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1c8108740 msgr2=0x7fa1c819f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:40.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.920+0000 7fa1cd151700 1 --2- 172.21.15.1:0/1234749551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1c8108740 0x7fa1c819f620 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000ef30 tx=0x7fa1c000c450 comp rx=0 tx=0).stop 2024-05-04T03:17:40.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.920+0000 7fa1cd151700 1 -- 172.21.15.1:0/1234749551 shutdown_connections 2024-05-04T03:17:40.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.920+0000 7fa1cd151700 1 --2- 172.21.15.1:0/1234749551 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1c81034c0 0x7fa1c819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:40.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.920+0000 7fa1cd151700 1 --2- 172.21.15.1:0/1234749551 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa1b00b2880 0x7fa1b00b4d90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:40.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.920+0000 7fa1cd151700 1 --2- 172.21.15.1:0/1234749551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa1c8108740 0x7fa1c819f620 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:40.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.920+0000 7fa1cd151700 1 --2- 172.21.15.1:0/1234749551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1c8107df0 0x7fa1c819b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:40.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.920+0000 7fa1cd151700 1 -- 172.21.15.1:0/1234749551 >> 172.21.15.1:0/1234749551 conn(0x7fa1c80fedc0 msgr2=0x7fa1c806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:40.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.921+0000 7fa1cd151700 1 -- 172.21.15.1:0/1234749551 shutdown_connections 2024-05-04T03:17:40.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:40.921+0000 7fa1cd151700 1 -- 172.21.15.1:0/1234749551 wait complete. 2024-05-04T03:17:40.925 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-04T03:17:41.544 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-05-04T03:17:39.166823+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661125,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":660,"num_read_kb":431,"num_write":297,"num_write_kb":2274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":741,"ondisk_log_size":741,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":411780,"kb_used_data":12708,"kb_used_omap":18,"kb_used_meta":398957,"kb_avail":1124530044,"statfs":{"total":1151940427776,"available":1151518765056,"internally_reserved":0,"allocated":13012992,"data_stored":7262427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":408532281},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":47,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003442"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.171540+0000","last_change":"2024-05-04T03:15:48.049361+0000","last_active":"2024-05-04T03:17:33.171540+0000","last_peered":"2024-05-04T03:17:33.171540+0000","last_clean":"2024-05-04T03:17:33.171540+0000","last_became_active":"2024-05-04T03:15:48.048876+0000","last_became_peered":"2024-05-04T03:15:48.048876+0000","last_unstale":"2024-05-04T03:17:33.171540+0000","last_undegraded":"2024-05-04T03:17:33.171540+0000","last_fullsized":"2024-05-04T03:17:33.171540+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:50:52.568516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165716+0000","last_change":"2024-05-04T03:15:48.055554+0000","last_active":"2024-05-04T03:17:33.165716+0000","last_peered":"2024-05-04T03:17:33.165716+0000","last_clean":"2024-05-04T03:17:33.165716+0000","last_became_active":"2024-05-04T03:15:48.055155+0000","last_became_peered":"2024-05-04T03:15:48.055155+0000","last_unstale":"2024-05-04T03:17:33.165716+0000","last_undegraded":"2024-05-04T03:17:33.165716+0000","last_fullsized":"2024-05-04T03:17:33.165716+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:38:07.824031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166389+0000","last_change":"2024-05-04T03:15:48.047974+0000","last_active":"2024-05-04T03:17:33.166389+0000","last_peered":"2024-05-04T03:17:33.166389+0000","last_clean":"2024-05-04T03:17:33.166389+0000","last_became_active":"2024-05-04T03:15:48.044299+0000","last_became_peered":"2024-05-04T03:15:48.044299+0000","last_unstale":"2024-05-04T03:17:33.166389+0000","last_undegraded":"2024-05-04T03:17:33.166389+0000","last_fullsized":"2024-05-04T03:17:33.166389+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:49:08.014987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.155592+0000","last_change":"2024-05-04T03:15:48.050899+0000","last_active":"2024-05-04T03:17:33.155592+0000","last_peered":"2024-05-04T03:17:33.155592+0000","last_clean":"2024-05-04T03:17:33.155592+0000","last_became_active":"2024-05-04T03:15:48.048812+0000","last_became_peered":"2024-05-04T03:15:48.048812+0000","last_unstale":"2024-05-04T03:17:33.155592+0000","last_undegraded":"2024-05-04T03:17:33.155592+0000","last_fullsized":"2024-05-04T03:17:33.155592+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:21:01.829349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164977+0000","last_change":"2024-05-04T03:15:48.044403+0000","last_active":"2024-05-04T03:17:33.164977+0000","last_peered":"2024-05-04T03:17:33.164977+0000","last_clean":"2024-05-04T03:17:33.164977+0000","last_became_active":"2024-05-04T03:15:48.044021+0000","last_became_peered":"2024-05-04T03:15:48.044021+0000","last_unstale":"2024-05-04T03:17:33.164977+0000","last_undegraded":"2024-05-04T03:17:33.164977+0000","last_fullsized":"2024-05-04T03:17:33.164977+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:50:54.762070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.158681+0000","last_change":"2024-05-04T03:15:48.053233+0000","last_active":"2024-05-04T03:17:33.158681+0000","last_peered":"2024-05-04T03:17:33.158681+0000","last_clean":"2024-05-04T03:17:33.158681+0000","last_became_active":"2024-05-04T03:15:48.052809+0000","last_became_peered":"2024-05-04T03:15:48.052809+0000","last_unstale":"2024-05-04T03:17:33.158681+0000","last_undegraded":"2024-05-04T03:17:33.158681+0000","last_fullsized":"2024-05-04T03:17:33.158681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:27:20.082694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.140939+0000","last_change":"2024-05-04T03:15:48.045108+0000","last_active":"2024-05-04T03:17:33.140939+0000","last_peered":"2024-05-04T03:17:33.140939+0000","last_clean":"2024-05-04T03:17:33.140939+0000","last_became_active":"2024-05-04T03:15:48.044719+0000","last_became_peered":"2024-05-04T03:15:48.044719+0000","last_unstale":"2024-05-04T03:17:33.140939+0000","last_undegraded":"2024-05-04T03:17:33.140939+0000","last_fullsized":"2024-05-04T03:17:33.140939+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:55:02.586294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164836+0000","last_change":"2024-05-04T03:15:48.043559+0000","last_active":"2024-05-04T03:17:33.164836+0000","last_peered":"2024-05-04T03:17:33.164836+0000","last_clean":"2024-05-04T03:17:33.164836+0000","last_became_active":"2024-05-04T03:15:48.043138+0000","last_became_peered":"2024-05-04T03:15:48.043138+0000","last_unstale":"2024-05-04T03:17:33.164836+0000","last_undegraded":"2024-05-04T03:17:33.164836+0000","last_fullsized":"2024-05-04T03:17:33.164836+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:05:25.411916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.173953+0000","last_change":"2024-05-04T03:15:48.039632+0000","last_active":"2024-05-04T03:17:33.173953+0000","last_peered":"2024-05-04T03:17:33.173953+0000","last_clean":"2024-05-04T03:17:33.173953+0000","last_became_active":"2024-05-04T03:15:48.039231+0000","last_became_peered":"2024-05-04T03:15:48.039231+0000","last_unstale":"2024-05-04T03:17:33.173953+0000","last_undegraded":"2024-05-04T03:17:33.173953+0000","last_fullsized":"2024-05-04T03:17:33.173953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:33:12.853172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.168057+0000","last_change":"2024-05-04T03:15:48.055138+0000","last_active":"2024-05-04T03:17:33.168057+0000","last_peered":"2024-05-04T03:17:33.168057+0000","last_clean":"2024-05-04T03:17:33.168057+0000","last_became_active":"2024-05-04T03:15:48.054791+0000","last_became_peered":"2024-05-04T03:15:48.054791+0000","last_unstale":"2024-05-04T03:17:33.168057+0000","last_undegraded":"2024-05-04T03:17:33.168057+0000","last_fullsized":"2024-05-04T03:17:33.168057+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:33:36.822549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165635+0000","last_change":"2024-05-04T03:15:48.050125+0000","last_active":"2024-05-04T03:17:33.165635+0000","last_peered":"2024-05-04T03:17:33.165635+0000","last_clean":"2024-05-04T03:17:33.165635+0000","last_became_active":"2024-05-04T03:15:48.049390+0000","last_became_peered":"2024-05-04T03:15:48.049390+0000","last_unstale":"2024-05-04T03:17:33.165635+0000","last_undegraded":"2024-05-04T03:17:33.165635+0000","last_fullsized":"2024-05-04T03:17:33.165635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:27:05.740383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.163273+0000","last_change":"2024-05-04T03:15:48.050700+0000","last_active":"2024-05-04T03:17:33.163273+0000","last_peered":"2024-05-04T03:17:33.163273+0000","last_clean":"2024-05-04T03:17:33.163273+0000","last_became_active":"2024-05-04T03:15:48.049288+0000","last_became_peered":"2024-05-04T03:15:48.049288+0000","last_unstale":"2024-05-04T03:17:33.163273+0000","last_undegraded":"2024-05-04T03:17:33.163273+0000","last_fullsized":"2024-05-04T03:17:33.163273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:40:07.072006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165193+0000","last_change":"2024-05-04T03:16:34.337123+0000","last_active":"2024-05-04T03:17:33.165193+0000","last_peered":"2024-05-04T03:17:33.165193+0000","last_clean":"2024-05-04T03:17:33.165193+0000","last_became_active":"2024-05-04T03:16:34.336657+0000","last_became_peered":"2024-05-04T03:16:34.336657+0000","last_unstale":"2024-05-04T03:17:33.165193+0000","last_undegraded":"2024-05-04T03:17:33.165193+0000","last_fullsized":"2024-05-04T03:17:33.165193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:58:15.939941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.162544+0000","last_change":"2024-05-04T03:15:48.047258+0000","last_active":"2024-05-04T03:17:33.162544+0000","last_peered":"2024-05-04T03:17:33.162544+0000","last_clean":"2024-05-04T03:17:33.162544+0000","last_became_active":"2024-05-04T03:15:48.046897+0000","last_became_peered":"2024-05-04T03:15:48.046897+0000","last_unstale":"2024-05-04T03:17:33.162544+0000","last_undegraded":"2024-05-04T03:17:33.162544+0000","last_fullsized":"2024-05-04T03:17:33.162544+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:14:39.217475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166174+0000","last_change":"2024-05-04T03:15:48.051240+0000","last_active":"2024-05-04T03:17:33.166174+0000","last_peered":"2024-05-04T03:17:33.166174+0000","last_clean":"2024-05-04T03:17:33.166174+0000","last_became_active":"2024-05-04T03:15:48.050081+0000","last_became_peered":"2024-05-04T03:15:48.050081+0000","last_unstale":"2024-05-04T03:17:33.166174+0000","last_undegraded":"2024-05-04T03:17:33.166174+0000","last_fullsized":"2024-05-04T03:17:33.166174+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:39:06.773121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.170623+0000","last_change":"2024-05-04T03:15:48.050936+0000","last_active":"2024-05-04T03:17:33.170623+0000","last_peered":"2024-05-04T03:17:33.170623+0000","last_clean":"2024-05-04T03:17:33.170623+0000","last_became_active":"2024-05-04T03:15:48.050567+0000","last_became_peered":"2024-05-04T03:15:48.050567+0000","last_unstale":"2024-05-04T03:17:33.170623+0000","last_undegraded":"2024-05-04T03:17:33.170623+0000","last_fullsized":"2024-05-04T03:17:33.170623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:17:43.469889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.173119+0000","last_change":"2024-05-04T03:15:48.047200+0000","last_active":"2024-05-04T03:17:33.173119+0000","last_peered":"2024-05-04T03:17:33.173119+0000","last_clean":"2024-05-04T03:17:33.173119+0000","last_became_active":"2024-05-04T03:15:48.046782+0000","last_became_peered":"2024-05-04T03:15:48.046782+0000","last_unstale":"2024-05-04T03:17:33.173119+0000","last_undegraded":"2024-05-04T03:17:33.173119+0000","last_fullsized":"2024-05-04T03:17:33.173119+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:32:56.773916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.152758+0000","last_change":"2024-05-04T03:15:48.042114+0000","last_active":"2024-05-04T03:17:33.152758+0000","last_peered":"2024-05-04T03:17:33.152758+0000","last_clean":"2024-05-04T03:17:33.152758+0000","last_became_active":"2024-05-04T03:15:48.040921+0000","last_became_peered":"2024-05-04T03:15:48.040921+0000","last_unstale":"2024-05-04T03:17:33.152758+0000","last_undegraded":"2024-05-04T03:17:33.152758+0000","last_fullsized":"2024-05-04T03:17:33.152758+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:16:20.130996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164290+0000","last_change":"2024-05-04T03:15:48.051060+0000","last_active":"2024-05-04T03:17:33.164290+0000","last_peered":"2024-05-04T03:17:33.164290+0000","last_clean":"2024-05-04T03:17:33.164290+0000","last_became_active":"2024-05-04T03:15:48.050642+0000","last_became_peered":"2024-05-04T03:15:48.050642+0000","last_unstale":"2024-05-04T03:17:33.164290+0000","last_undegraded":"2024-05-04T03:17:33.164290+0000","last_fullsized":"2024-05-04T03:17:33.164290+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:23:31.193072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.329322+0000","last_change":"2024-05-04T03:15:45.013477+0000","last_active":"2024-05-04T03:16:34.329322+0000","last_peered":"2024-05-04T03:16:34.329322+0000","last_clean":"2024-05-04T03:16:34.329322+0000","last_became_active":"2024-05-04T03:15:45.013161+0000","last_became_peered":"2024-05-04T03:15:45.013161+0000","last_unstale":"2024-05-04T03:16:34.329322+0000","last_undegraded":"2024-05-04T03:16:34.329322+0000","last_fullsized":"2024-05-04T03:16:34.329322+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:08:14.087090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164395+0000","last_change":"2024-05-04T03:15:48.045810+0000","last_active":"2024-05-04T03:17:33.164395+0000","last_peered":"2024-05-04T03:17:33.164395+0000","last_clean":"2024-05-04T03:17:33.164395+0000","last_became_active":"2024-05-04T03:15:48.045449+0000","last_became_peered":"2024-05-04T03:15:48.045449+0000","last_unstale":"2024-05-04T03:17:33.164395+0000","last_undegraded":"2024-05-04T03:17:33.164395+0000","last_fullsized":"2024-05-04T03:17:33.164395+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:48:05.084208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.417000+0000","last_change":"2024-05-04T03:15:45.204196+0000","last_active":"2024-05-04T03:16:34.417000+0000","last_peered":"2024-05-04T03:16:34.417000+0000","last_clean":"2024-05-04T03:16:34.417000+0000","last_became_active":"2024-05-04T03:15:45.203821+0000","last_became_peered":"2024-05-04T03:15:45.203821+0000","last_unstale":"2024-05-04T03:16:34.417000+0000","last_undegraded":"2024-05-04T03:16:34.417000+0000","last_fullsized":"2024-05-04T03:16:34.417000+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:17:05.074450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.167047+0000","last_change":"2024-05-04T03:15:48.045131+0000","last_active":"2024-05-04T03:17:33.167047+0000","last_peered":"2024-05-04T03:17:33.167047+0000","last_clean":"2024-05-04T03:17:33.167047+0000","last_became_active":"2024-05-04T03:15:48.044793+0000","last_became_peered":"2024-05-04T03:15:48.044793+0000","last_unstale":"2024-05-04T03:17:33.167047+0000","last_undegraded":"2024-05-04T03:17:33.167047+0000","last_fullsized":"2024-05-04T03:17:33.167047+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:03:37.769324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334692+0000","last_change":"2024-05-04T03:15:45.013644+0000","last_active":"2024-05-04T03:16:34.334692+0000","last_peered":"2024-05-04T03:16:34.334692+0000","last_clean":"2024-05-04T03:16:34.334692+0000","last_became_active":"2024-05-04T03:15:45.013282+0000","last_became_peered":"2024-05-04T03:15:45.013282+0000","last_unstale":"2024-05-04T03:16:34.334692+0000","last_undegraded":"2024-05-04T03:16:34.334692+0000","last_fullsized":"2024-05-04T03:16:34.334692+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:21:21.204433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.174503+0000","last_change":"2024-05-04T03:15:48.043046+0000","last_active":"2024-05-04T03:17:33.174503+0000","last_peered":"2024-05-04T03:17:33.174503+0000","last_clean":"2024-05-04T03:17:33.174503+0000","last_became_active":"2024-05-04T03:15:48.042621+0000","last_became_peered":"2024-05-04T03:15:48.042621+0000","last_unstale":"2024-05-04T03:17:33.174503+0000","last_undegraded":"2024-05-04T03:17:33.174503+0000","last_fullsized":"2024-05-04T03:17:33.174503+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:03:58.916423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334875+0000","last_change":"2024-05-04T03:15:45.010618+0000","last_active":"2024-05-04T03:16:34.334875+0000","last_peered":"2024-05-04T03:16:34.334875+0000","last_clean":"2024-05-04T03:16:34.334875+0000","last_became_active":"2024-05-04T03:15:45.009239+0000","last_became_peered":"2024-05-04T03:15:45.009239+0000","last_unstale":"2024-05-04T03:16:34.334875+0000","last_undegraded":"2024-05-04T03:16:34.334875+0000","last_fullsized":"2024-05-04T03:16:34.334875+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:49:46.777571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.145857+0000","last_change":"2024-05-04T03:15:48.049334+0000","last_active":"2024-05-04T03:17:33.145857+0000","last_peered":"2024-05-04T03:17:33.145857+0000","last_clean":"2024-05-04T03:17:33.145857+0000","last_became_active":"2024-05-04T03:15:48.048984+0000","last_became_peered":"2024-05-04T03:15:48.048984+0000","last_unstale":"2024-05-04T03:17:33.145857+0000","last_undegraded":"2024-05-04T03:17:33.145857+0000","last_fullsized":"2024-05-04T03:17:33.145857+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:28:25.366106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":123,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.393559+0000","last_change":"2024-05-04T03:15:45.015277+0000","last_active":"2024-05-04T03:17:33.393559+0000","last_peered":"2024-05-04T03:17:33.393559+0000","last_clean":"2024-05-04T03:17:33.393559+0000","last_became_active":"2024-05-04T03:15:45.014542+0000","last_became_peered":"2024-05-04T03:15:45.014542+0000","last_unstale":"2024-05-04T03:17:33.393559+0000","last_undegraded":"2024-05-04T03:17:33.393559+0000","last_fullsized":"2024-05-04T03:17:33.393559+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:22:15.594866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.146561+0000","last_change":"2024-05-04T03:15:48.045029+0000","last_active":"2024-05-04T03:17:33.146561+0000","last_peered":"2024-05-04T03:17:33.146561+0000","last_clean":"2024-05-04T03:17:33.146561+0000","last_became_active":"2024-05-04T03:15:48.041262+0000","last_became_peered":"2024-05-04T03:15:48.041262+0000","last_unstale":"2024-05-04T03:17:33.146561+0000","last_undegraded":"2024-05-04T03:17:33.146561+0000","last_fullsized":"2024-05-04T03:17:33.146561+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:45:56.882957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334143+0000","last_change":"2024-05-04T03:15:45.007530+0000","last_active":"2024-05-04T03:16:34.334143+0000","last_peered":"2024-05-04T03:16:34.334143+0000","last_clean":"2024-05-04T03:16:34.334143+0000","last_became_active":"2024-05-04T03:15:45.007054+0000","last_became_peered":"2024-05-04T03:15:45.007054+0000","last_unstale":"2024-05-04T03:16:34.334143+0000","last_undegraded":"2024-05-04T03:16:34.334143+0000","last_fullsized":"2024-05-04T03:16:34.334143+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:09:13.845679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.142587+0000","last_change":"2024-05-04T03:15:48.050956+0000","last_active":"2024-05-04T03:17:33.142587+0000","last_peered":"2024-05-04T03:17:33.142587+0000","last_clean":"2024-05-04T03:17:33.142587+0000","last_became_active":"2024-05-04T03:15:48.050597+0000","last_became_peered":"2024-05-04T03:15:48.050597+0000","last_unstale":"2024-05-04T03:17:33.142587+0000","last_undegraded":"2024-05-04T03:17:33.142587+0000","last_fullsized":"2024-05-04T03:17:33.142587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:27:34.127889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.331383+0000","last_change":"2024-05-04T03:15:45.007027+0000","last_active":"2024-05-04T03:16:34.331383+0000","last_peered":"2024-05-04T03:16:34.331383+0000","last_clean":"2024-05-04T03:16:34.331383+0000","last_became_active":"2024-05-04T03:15:45.006659+0000","last_became_peered":"2024-05-04T03:15:45.006659+0000","last_unstale":"2024-05-04T03:16:34.331383+0000","last_undegraded":"2024-05-04T03:16:34.331383+0000","last_fullsized":"2024-05-04T03:16:34.331383+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:06:11.827093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164076+0000","last_change":"2024-05-04T03:15:48.046196+0000","last_active":"2024-05-04T03:17:33.164076+0000","last_peered":"2024-05-04T03:17:33.164076+0000","last_clean":"2024-05-04T03:17:33.164076+0000","last_became_active":"2024-05-04T03:15:48.044316+0000","last_became_peered":"2024-05-04T03:15:48.044316+0000","last_unstale":"2024-05-04T03:17:33.164076+0000","last_undegraded":"2024-05-04T03:17:33.164076+0000","last_fullsized":"2024-05-04T03:17:33.164076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:04:39.301705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.416612+0000","last_change":"2024-05-04T03:15:45.015360+0000","last_active":"2024-05-04T03:16:34.416612+0000","last_peered":"2024-05-04T03:16:34.416612+0000","last_clean":"2024-05-04T03:16:34.416612+0000","last_became_active":"2024-05-04T03:15:45.014696+0000","last_became_peered":"2024-05-04T03:15:45.014696+0000","last_unstale":"2024-05-04T03:16:34.416612+0000","last_undegraded":"2024-05-04T03:16:34.416612+0000","last_fullsized":"2024-05-04T03:16:34.416612+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:17:28.838651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.162779+0000","last_change":"2024-05-04T03:15:48.039823+0000","last_active":"2024-05-04T03:17:33.162779+0000","last_peered":"2024-05-04T03:17:33.162779+0000","last_clean":"2024-05-04T03:17:33.162779+0000","last_became_active":"2024-05-04T03:15:48.039447+0000","last_became_peered":"2024-05-04T03:15:48.039447+0000","last_unstale":"2024-05-04T03:17:33.162779+0000","last_undegraded":"2024-05-04T03:17:33.162779+0000","last_fullsized":"2024-05-04T03:17:33.162779+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:45:55.136946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":122,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:36.397900+0000","last_change":"2024-05-04T03:15:45.004662+0000","last_active":"2024-05-04T03:17:36.397900+0000","last_peered":"2024-05-04T03:17:36.397900+0000","last_clean":"2024-05-04T03:17:36.397900+0000","last_became_active":"2024-05-04T03:15:45.004265+0000","last_became_peered":"2024-05-04T03:15:45.004265+0000","last_unstale":"2024-05-04T03:17:36.397900+0000","last_undegraded":"2024-05-04T03:17:36.397900+0000","last_fullsized":"2024-05-04T03:17:36.397900+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:14:03.065346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":93,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.173793+0000","last_change":"2024-05-04T03:15:48.042093+0000","last_active":"2024-05-04T03:17:33.173793+0000","last_peered":"2024-05-04T03:17:33.173793+0000","last_clean":"2024-05-04T03:17:33.173793+0000","last_became_active":"2024-05-04T03:15:48.041646+0000","last_became_peered":"2024-05-04T03:15:48.041646+0000","last_unstale":"2024-05-04T03:17:33.173793+0000","last_undegraded":"2024-05-04T03:17:33.173793+0000","last_fullsized":"2024-05-04T03:17:33.173793+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:06:32.083493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334519+0000","last_change":"2024-05-04T03:15:45.016731+0000","last_active":"2024-05-04T03:16:34.334519+0000","last_peered":"2024-05-04T03:16:34.334519+0000","last_clean":"2024-05-04T03:16:34.334519+0000","last_became_active":"2024-05-04T03:15:45.016313+0000","last_became_peered":"2024-05-04T03:15:45.016313+0000","last_unstale":"2024-05-04T03:16:34.334519+0000","last_undegraded":"2024-05-04T03:16:34.334519+0000","last_fullsized":"2024-05-04T03:16:34.334519+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:42:40.830986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.163387+0000","last_change":"2024-05-04T03:15:48.053649+0000","last_active":"2024-05-04T03:17:33.163387+0000","last_peered":"2024-05-04T03:17:33.163387+0000","last_clean":"2024-05-04T03:17:33.163387+0000","last_became_active":"2024-05-04T03:15:48.052407+0000","last_became_peered":"2024-05-04T03:15:48.052407+0000","last_unstale":"2024-05-04T03:17:33.163387+0000","last_undegraded":"2024-05-04T03:17:33.163387+0000","last_fullsized":"2024-05-04T03:17:33.163387+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:37:28.359272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.417235+0000","last_change":"2024-05-04T03:15:45.205180+0000","last_active":"2024-05-04T03:16:34.417235+0000","last_peered":"2024-05-04T03:16:34.417235+0000","last_clean":"2024-05-04T03:16:34.417235+0000","last_became_active":"2024-05-04T03:15:45.204828+0000","last_became_peered":"2024-05-04T03:15:45.204828+0000","last_unstale":"2024-05-04T03:16:34.417235+0000","last_undegraded":"2024-05-04T03:16:34.417235+0000","last_fullsized":"2024-05-04T03:16:34.417235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:27:34.063540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.171081+0000","last_change":"2024-05-04T03:15:48.047154+0000","last_active":"2024-05-04T03:17:33.171081+0000","last_peered":"2024-05-04T03:17:33.171081+0000","last_clean":"2024-05-04T03:17:33.171081+0000","last_became_active":"2024-05-04T03:15:48.046794+0000","last_became_peered":"2024-05-04T03:15:48.046794+0000","last_unstale":"2024-05-04T03:17:33.171081+0000","last_undegraded":"2024-05-04T03:17:33.171081+0000","last_fullsized":"2024-05-04T03:17:33.171081+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:21:42.723252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'35","reported_seq":257,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:37.399484+0000","last_change":"2024-05-04T03:15:45.008930+0000","last_active":"2024-05-04T03:17:37.399484+0000","last_peered":"2024-05-04T03:17:37.399484+0000","last_clean":"2024-05-04T03:17:37.399484+0000","last_became_active":"2024-05-04T03:15:45.008459+0000","last_became_peered":"2024-05-04T03:15:45.008459+0000","last_unstale":"2024-05-04T03:17:37.399484+0000","last_undegraded":"2024-05-04T03:17:37.399484+0000","last_fullsized":"2024-05-04T03:17:37.399484+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:07:52.609156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66381,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":196,"num_read_kb":163,"num_write":36,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.143616+0000","last_change":"2024-05-04T03:15:48.051848+0000","last_active":"2024-05-04T03:17:33.143616+0000","last_peered":"2024-05-04T03:17:33.143616+0000","last_clean":"2024-05-04T03:17:33.143616+0000","last_became_active":"2024-05-04T03:15:48.051171+0000","last_became_peered":"2024-05-04T03:15:48.051171+0000","last_unstale":"2024-05-04T03:17:33.143616+0000","last_undegraded":"2024-05-04T03:17:33.143616+0000","last_fullsized":"2024-05-04T03:17:33.143616+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:20:32.722845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.330336+0000","last_change":"2024-05-04T03:15:45.007970+0000","last_active":"2024-05-04T03:16:34.330336+0000","last_peered":"2024-05-04T03:16:34.330336+0000","last_clean":"2024-05-04T03:16:34.330336+0000","last_became_active":"2024-05-04T03:15:45.007605+0000","last_became_peered":"2024-05-04T03:15:45.007605+0000","last_unstale":"2024-05-04T03:16:34.330336+0000","last_undegraded":"2024-05-04T03:16:34.330336+0000","last_fullsized":"2024-05-04T03:16:34.330336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:52:08.466858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.168989+0000","last_change":"2024-05-04T03:15:48.050647+0000","last_active":"2024-05-04T03:17:33.168989+0000","last_peered":"2024-05-04T03:17:33.168989+0000","last_clean":"2024-05-04T03:17:33.168989+0000","last_became_active":"2024-05-04T03:15:48.050296+0000","last_became_peered":"2024-05-04T03:15:48.050296+0000","last_unstale":"2024-05-04T03:17:33.168989+0000","last_undegraded":"2024-05-04T03:17:33.168989+0000","last_fullsized":"2024-05-04T03:17:33.168989+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:14:23.902094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.416872+0000","last_change":"2024-05-04T03:15:45.014936+0000","last_active":"2024-05-04T03:16:34.416872+0000","last_peered":"2024-05-04T03:16:34.416872+0000","last_clean":"2024-05-04T03:16:34.416872+0000","last_became_active":"2024-05-04T03:15:45.014474+0000","last_became_peered":"2024-05-04T03:15:45.014474+0000","last_unstale":"2024-05-04T03:16:34.416872+0000","last_undegraded":"2024-05-04T03:16:34.416872+0000","last_fullsized":"2024-05-04T03:16:34.416872+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:26:37.972523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.169449+0000","last_change":"2024-05-04T03:16:34.336836+0000","last_active":"2024-05-04T03:17:33.169449+0000","last_peered":"2024-05-04T03:17:33.169449+0000","last_clean":"2024-05-04T03:17:33.169449+0000","last_became_active":"2024-05-04T03:16:34.336008+0000","last_became_peered":"2024-05-04T03:16:34.336008+0000","last_unstale":"2024-05-04T03:17:33.169449+0000","last_undegraded":"2024-05-04T03:17:33.169449+0000","last_fullsized":"2024-05-04T03:17:33.169449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:51:52.803138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'5","reported_seq":129,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:37.399773+0000","last_change":"2024-05-04T03:15:45.013177+0000","last_active":"2024-05-04T03:17:37.399773+0000","last_peered":"2024-05-04T03:17:37.399773+0000","last_clean":"2024-05-04T03:17:37.399773+0000","last_became_active":"2024-05-04T03:15:45.012764+0000","last_became_peered":"2024-05-04T03:15:45.012764+0000","last_unstale":"2024-05-04T03:17:37.399773+0000","last_undegraded":"2024-05-04T03:17:37.399773+0000","last_fullsized":"2024-05-04T03:17:37.399773+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:07:43.578443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.159529+0000","last_change":"2024-05-04T03:15:48.046453+0000","last_active":"2024-05-04T03:17:33.159529+0000","last_peered":"2024-05-04T03:17:33.159529+0000","last_clean":"2024-05-04T03:17:33.159529+0000","last_became_active":"2024-05-04T03:15:48.045990+0000","last_became_peered":"2024-05-04T03:15:48.045990+0000","last_unstale":"2024-05-04T03:17:33.159529+0000","last_undegraded":"2024-05-04T03:17:33.159529+0000","last_fullsized":"2024-05-04T03:17:33.159529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:54:57.285122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334287+0000","last_change":"2024-05-04T03:15:45.205539+0000","last_active":"2024-05-04T03:16:34.334287+0000","last_peered":"2024-05-04T03:16:34.334287+0000","last_clean":"2024-05-04T03:16:34.334287+0000","last_became_active":"2024-05-04T03:15:45.205174+0000","last_became_peered":"2024-05-04T03:15:45.205174+0000","last_unstale":"2024-05-04T03:16:34.334287+0000","last_undegraded":"2024-05-04T03:16:34.334287+0000","last_fullsized":"2024-05-04T03:16:34.334287+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:03:53.398368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.141711+0000","last_change":"2024-05-04T03:15:48.051824+0000","last_active":"2024-05-04T03:17:33.141711+0000","last_peered":"2024-05-04T03:17:33.141711+0000","last_clean":"2024-05-04T03:17:33.141711+0000","last_became_active":"2024-05-04T03:15:48.051483+0000","last_became_peered":"2024-05-04T03:15:48.051483+0000","last_unstale":"2024-05-04T03:17:33.141711+0000","last_undegraded":"2024-05-04T03:17:33.141711+0000","last_fullsized":"2024-05-04T03:17:33.141711+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:57:56.644486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334786+0000","last_change":"2024-05-04T03:15:45.011062+0000","last_active":"2024-05-04T03:16:34.334786+0000","last_peered":"2024-05-04T03:16:34.334786+0000","last_clean":"2024-05-04T03:16:34.334786+0000","last_became_active":"2024-05-04T03:15:45.010730+0000","last_became_peered":"2024-05-04T03:15:45.010730+0000","last_unstale":"2024-05-04T03:16:34.334786+0000","last_undegraded":"2024-05-04T03:16:34.334786+0000","last_fullsized":"2024-05-04T03:16:34.334786+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:45:55.979067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.167699+0000","last_change":"2024-05-04T03:15:48.045965+0000","last_active":"2024-05-04T03:17:33.167699+0000","last_peered":"2024-05-04T03:17:33.167699+0000","last_clean":"2024-05-04T03:17:33.167699+0000","last_became_active":"2024-05-04T03:15:48.045615+0000","last_became_peered":"2024-05-04T03:15:48.045615+0000","last_unstale":"2024-05-04T03:17:33.167699+0000","last_undegraded":"2024-05-04T03:17:33.167699+0000","last_fullsized":"2024-05-04T03:17:33.167699+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:14:23.497484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.329206+0000","last_change":"2024-05-04T03:15:45.204223+0000","last_active":"2024-05-04T03:16:34.329206+0000","last_peered":"2024-05-04T03:16:34.329206+0000","last_clean":"2024-05-04T03:16:34.329206+0000","last_became_active":"2024-05-04T03:15:45.203774+0000","last_became_peered":"2024-05-04T03:15:45.203774+0000","last_unstale":"2024-05-04T03:16:34.329206+0000","last_undegraded":"2024-05-04T03:16:34.329206+0000","last_fullsized":"2024-05-04T03:16:34.329206+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:02:34.456055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166817+0000","last_change":"2024-05-04T03:15:48.045574+0000","last_active":"2024-05-04T03:17:33.166817+0000","last_peered":"2024-05-04T03:17:33.166817+0000","last_clean":"2024-05-04T03:17:33.166817+0000","last_became_active":"2024-05-04T03:15:48.045237+0000","last_became_peered":"2024-05-04T03:15:48.045237+0000","last_unstale":"2024-05-04T03:17:33.166817+0000","last_undegraded":"2024-05-04T03:17:33.166817+0000","last_fullsized":"2024-05-04T03:17:33.166817+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:53:46.346199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.333866+0000","last_change":"2024-05-04T03:15:45.013895+0000","last_active":"2024-05-04T03:16:34.333866+0000","last_peered":"2024-05-04T03:16:34.333866+0000","last_clean":"2024-05-04T03:16:34.333866+0000","last_became_active":"2024-05-04T03:15:45.013306+0000","last_became_peered":"2024-05-04T03:15:45.013306+0000","last_unstale":"2024-05-04T03:16:34.333866+0000","last_undegraded":"2024-05-04T03:16:34.333866+0000","last_fullsized":"2024-05-04T03:16:34.333866+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:39:18.616432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.158810+0000","last_change":"2024-05-04T03:16:34.336997+0000","last_active":"2024-05-04T03:17:33.158810+0000","last_peered":"2024-05-04T03:17:33.158810+0000","last_clean":"2024-05-04T03:17:33.158810+0000","last_became_active":"2024-05-04T03:16:34.336505+0000","last_became_peered":"2024-05-04T03:16:34.336505+0000","last_unstale":"2024-05-04T03:17:33.158810+0000","last_undegraded":"2024-05-04T03:17:33.158810+0000","last_fullsized":"2024-05-04T03:17:33.158810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:23:38.979216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.149216+0000","last_change":"2024-05-04T03:15:48.050122+0000","last_active":"2024-05-04T03:17:33.149216+0000","last_peered":"2024-05-04T03:17:33.149216+0000","last_clean":"2024-05-04T03:17:33.149216+0000","last_became_active":"2024-05-04T03:15:48.049794+0000","last_became_peered":"2024-05-04T03:15:48.049794+0000","last_unstale":"2024-05-04T03:17:33.149216+0000","last_undegraded":"2024-05-04T03:17:33.149216+0000","last_fullsized":"2024-05-04T03:17:33.149216+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:29:11.066068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":124,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:36.397904+0000","last_change":"2024-05-04T03:15:45.205143+0000","last_active":"2024-05-04T03:17:36.397904+0000","last_peered":"2024-05-04T03:17:36.397904+0000","last_clean":"2024-05-04T03:17:36.397904+0000","last_became_active":"2024-05-04T03:15:45.204727+0000","last_became_peered":"2024-05-04T03:15:45.204727+0000","last_unstale":"2024-05-04T03:17:36.397904+0000","last_undegraded":"2024-05-04T03:17:36.397904+0000","last_fullsized":"2024-05-04T03:17:36.397904+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:15:05.799556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":93,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.163619+0000","last_change":"2024-05-04T03:15:48.042914+0000","last_active":"2024-05-04T03:17:33.163619+0000","last_peered":"2024-05-04T03:17:33.163619+0000","last_clean":"2024-05-04T03:17:33.163619+0000","last_became_active":"2024-05-04T03:15:48.041936+0000","last_became_peered":"2024-05-04T03:15:48.041936+0000","last_unstale":"2024-05-04T03:17:33.163619+0000","last_undegraded":"2024-05-04T03:17:33.163619+0000","last_fullsized":"2024-05-04T03:17:33.163619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:11:48.807602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.331225+0000","last_change":"2024-05-04T03:15:45.011045+0000","last_active":"2024-05-04T03:16:34.331225+0000","last_peered":"2024-05-04T03:16:34.331225+0000","last_clean":"2024-05-04T03:16:34.331225+0000","last_became_active":"2024-05-04T03:15:45.010608+0000","last_became_peered":"2024-05-04T03:15:45.010608+0000","last_unstale":"2024-05-04T03:16:34.331225+0000","last_undegraded":"2024-05-04T03:16:34.331225+0000","last_fullsized":"2024-05-04T03:16:34.331225+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:00:36.243588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":102,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.333935+0000","last_change":"2024-05-04T03:14:19.895070+0000","last_active":"2024-05-04T03:16:34.333935+0000","last_peered":"2024-05-04T03:16:34.333935+0000","last_clean":"2024-05-04T03:16:34.333935+0000","last_became_active":"2024-05-04T03:14:19.889987+0000","last_became_peered":"2024-05-04T03:14:19.889987+0000","last_unstale":"2024-05-04T03:16:34.333935+0000","last_undegraded":"2024-05-04T03:16:34.333935+0000","last_fullsized":"2024-05-04T03:16:34.333935+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:11:01.712629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:11:01.712629+0000","last_clean_scrub_stamp":"2024-05-04T03:11:01.712629+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:03:02.769719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165934+0000","last_change":"2024-05-04T03:15:48.044928+0000","last_active":"2024-05-04T03:17:33.165934+0000","last_peered":"2024-05-04T03:17:33.165934+0000","last_clean":"2024-05-04T03:17:33.165934+0000","last_became_active":"2024-05-04T03:15:48.044532+0000","last_became_peered":"2024-05-04T03:15:48.044532+0000","last_unstale":"2024-05-04T03:17:33.165934+0000","last_undegraded":"2024-05-04T03:17:33.165934+0000","last_fullsized":"2024-05-04T03:17:33.165934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:27:00.825692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334044+0000","last_change":"2024-05-04T03:15:45.014469+0000","last_active":"2024-05-04T03:16:34.334044+0000","last_peered":"2024-05-04T03:16:34.334044+0000","last_clean":"2024-05-04T03:16:34.334044+0000","last_became_active":"2024-05-04T03:15:45.013763+0000","last_became_peered":"2024-05-04T03:15:45.013763+0000","last_unstale":"2024-05-04T03:16:34.334044+0000","last_undegraded":"2024-05-04T03:16:34.334044+0000","last_fullsized":"2024-05-04T03:16:34.334044+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:19:36.379930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334548+0000","last_change":"2024-05-04T03:15:45.204536+0000","last_active":"2024-05-04T03:16:34.334548+0000","last_peered":"2024-05-04T03:16:34.334548+0000","last_clean":"2024-05-04T03:16:34.334548+0000","last_became_active":"2024-05-04T03:15:45.204072+0000","last_became_peered":"2024-05-04T03:15:45.204072+0000","last_unstale":"2024-05-04T03:16:34.334548+0000","last_undegraded":"2024-05-04T03:16:34.334548+0000","last_fullsized":"2024-05-04T03:16:34.334548+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:49:12.726220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.155914+0000","last_change":"2024-05-04T03:15:48.044982+0000","last_active":"2024-05-04T03:17:33.155914+0000","last_peered":"2024-05-04T03:17:33.155914+0000","last_clean":"2024-05-04T03:17:33.155914+0000","last_became_active":"2024-05-04T03:15:48.043996+0000","last_became_peered":"2024-05-04T03:15:48.043996+0000","last_unstale":"2024-05-04T03:17:33.155914+0000","last_undegraded":"2024-05-04T03:17:33.155914+0000","last_fullsized":"2024-05-04T03:17:33.155914+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:09:39.770023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'11","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.329254+0000","last_change":"2024-05-04T03:15:45.008153+0000","last_active":"2024-05-04T03:16:34.329254+0000","last_peered":"2024-05-04T03:16:34.329254+0000","last_clean":"2024-05-04T03:16:34.329254+0000","last_became_active":"2024-05-04T03:15:45.007698+0000","last_became_peered":"2024-05-04T03:15:45.007698+0000","last_unstale":"2024-05-04T03:16:34.329254+0000","last_undegraded":"2024-05-04T03:16:34.329254+0000","last_fullsized":"2024-05-04T03:16:34.329254+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:56:52.322049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.168628+0000","last_change":"2024-05-04T03:15:48.055767+0000","last_active":"2024-05-04T03:17:33.168628+0000","last_peered":"2024-05-04T03:17:33.168628+0000","last_clean":"2024-05-04T03:17:33.168628+0000","last_became_active":"2024-05-04T03:15:48.055252+0000","last_became_peered":"2024-05-04T03:15:48.055252+0000","last_unstale":"2024-05-04T03:17:33.168628+0000","last_undegraded":"2024-05-04T03:17:33.168628+0000","last_fullsized":"2024-05-04T03:17:33.168628+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:28:11.736803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.331482+0000","last_change":"2024-05-04T03:15:45.012838+0000","last_active":"2024-05-04T03:16:34.331482+0000","last_peered":"2024-05-04T03:16:34.331482+0000","last_clean":"2024-05-04T03:16:34.331482+0000","last_became_active":"2024-05-04T03:15:45.012429+0000","last_became_peered":"2024-05-04T03:15:45.012429+0000","last_unstale":"2024-05-04T03:16:34.331482+0000","last_undegraded":"2024-05-04T03:16:34.331482+0000","last_fullsized":"2024-05-04T03:16:34.331482+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:22:22.602551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165352+0000","last_change":"2024-05-04T03:15:48.056993+0000","last_active":"2024-05-04T03:17:33.165352+0000","last_peered":"2024-05-04T03:17:33.165352+0000","last_clean":"2024-05-04T03:17:33.165352+0000","last_became_active":"2024-05-04T03:15:48.055938+0000","last_became_peered":"2024-05-04T03:15:48.055938+0000","last_unstale":"2024-05-04T03:17:33.165352+0000","last_undegraded":"2024-05-04T03:17:33.165352+0000","last_fullsized":"2024-05-04T03:17:33.165352+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:23:59.521692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.340048+0000","last_change":"2024-05-04T03:16:34.340048+0000","last_active":"2024-05-04T03:16:34.340048+0000","last_peered":"2024-05-04T03:16:34.340048+0000","last_clean":"2024-05-04T03:16:34.340048+0000","last_became_active":"2024-05-04T03:16:34.336386+0000","last_became_peered":"2024-05-04T03:16:34.336386+0000","last_unstale":"2024-05-04T03:16:34.340048+0000","last_undegraded":"2024-05-04T03:16:34.340048+0000","last_fullsized":"2024-05-04T03:16:34.340048+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:45:43.734097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.128655+0000","last_change":"2024-05-04T03:15:48.043614+0000","last_active":"2024-05-04T03:17:33.128655+0000","last_peered":"2024-05-04T03:17:33.128655+0000","last_clean":"2024-05-04T03:17:33.128655+0000","last_became_active":"2024-05-04T03:15:48.042390+0000","last_became_peered":"2024-05-04T03:15:48.042390+0000","last_unstale":"2024-05-04T03:17:33.128655+0000","last_undegraded":"2024-05-04T03:17:33.128655+0000","last_fullsized":"2024-05-04T03:17:33.128655+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:56:22.791368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.416508+0000","last_change":"2024-05-04T03:15:45.005570+0000","last_active":"2024-05-04T03:16:34.416508+0000","last_peered":"2024-05-04T03:16:34.416508+0000","last_clean":"2024-05-04T03:16:34.416508+0000","last_became_active":"2024-05-04T03:15:45.005105+0000","last_became_peered":"2024-05-04T03:15:45.005105+0000","last_unstale":"2024-05-04T03:16:34.416508+0000","last_undegraded":"2024-05-04T03:16:34.416508+0000","last_fullsized":"2024-05-04T03:16:34.416508+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:40:17.275618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166777+0000","last_change":"2024-05-04T03:15:48.045081+0000","last_active":"2024-05-04T03:17:33.166777+0000","last_peered":"2024-05-04T03:17:33.166777+0000","last_clean":"2024-05-04T03:17:33.166777+0000","last_became_active":"2024-05-04T03:15:48.044745+0000","last_became_peered":"2024-05-04T03:15:48.044745+0000","last_unstale":"2024-05-04T03:17:33.166777+0000","last_undegraded":"2024-05-04T03:17:33.166777+0000","last_fullsized":"2024-05-04T03:17:33.166777+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:40:53.486222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.330010+0000","last_change":"2024-05-04T03:15:45.008926+0000","last_active":"2024-05-04T03:16:34.330010+0000","last_peered":"2024-05-04T03:16:34.330010+0000","last_clean":"2024-05-04T03:16:34.330010+0000","last_became_active":"2024-05-04T03:15:45.008412+0000","last_became_peered":"2024-05-04T03:15:45.008412+0000","last_unstale":"2024-05-04T03:16:34.330010+0000","last_undegraded":"2024-05-04T03:16:34.330010+0000","last_fullsized":"2024-05-04T03:16:34.330010+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:34:33.107866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.162308+0000","last_change":"2024-05-04T03:15:48.054580+0000","last_active":"2024-05-04T03:17:33.162308+0000","last_peered":"2024-05-04T03:17:33.162308+0000","last_clean":"2024-05-04T03:17:33.162308+0000","last_became_active":"2024-05-04T03:15:48.054242+0000","last_became_peered":"2024-05-04T03:15:48.054242+0000","last_unstale":"2024-05-04T03:17:33.162308+0000","last_undegraded":"2024-05-04T03:17:33.162308+0000","last_fullsized":"2024-05-04T03:17:33.162308+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:16:19.426571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.341316+0000","last_change":"2024-05-04T03:16:34.340794+0000","last_active":"2024-05-04T03:16:34.341316+0000","last_peered":"2024-05-04T03:16:34.341316+0000","last_clean":"2024-05-04T03:16:34.341316+0000","last_became_active":"2024-05-04T03:16:34.336038+0000","last_became_peered":"2024-05-04T03:16:34.336038+0000","last_unstale":"2024-05-04T03:16:34.341316+0000","last_undegraded":"2024-05-04T03:16:34.341316+0000","last_fullsized":"2024-05-04T03:16:34.341316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:45:19.323938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.172069+0000","last_change":"2024-05-04T03:15:48.045284+0000","last_active":"2024-05-04T03:17:33.172069+0000","last_peered":"2024-05-04T03:17:33.172069+0000","last_clean":"2024-05-04T03:17:33.172069+0000","last_became_active":"2024-05-04T03:15:48.044673+0000","last_became_peered":"2024-05-04T03:15:48.044673+0000","last_unstale":"2024-05-04T03:17:33.172069+0000","last_undegraded":"2024-05-04T03:17:33.172069+0000","last_fullsized":"2024-05-04T03:17:33.172069+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:51:52.389748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.330774+0000","last_change":"2024-05-04T03:15:45.205648+0000","last_active":"2024-05-04T03:16:34.330774+0000","last_peered":"2024-05-04T03:16:34.330774+0000","last_clean":"2024-05-04T03:16:34.330774+0000","last_became_active":"2024-05-04T03:15:45.204169+0000","last_became_peered":"2024-05-04T03:15:45.204169+0000","last_unstale":"2024-05-04T03:16:34.330774+0000","last_undegraded":"2024-05-04T03:16:34.330774+0000","last_fullsized":"2024-05-04T03:16:34.330774+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:20:48.859276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165884+0000","last_change":"2024-05-04T03:15:48.049808+0000","last_active":"2024-05-04T03:17:33.165884+0000","last_peered":"2024-05-04T03:17:33.165884+0000","last_clean":"2024-05-04T03:17:33.165884+0000","last_became_active":"2024-05-04T03:15:48.048998+0000","last_became_peered":"2024-05-04T03:15:48.048998+0000","last_unstale":"2024-05-04T03:17:33.165884+0000","last_undegraded":"2024-05-04T03:17:33.165884+0000","last_fullsized":"2024-05-04T03:17:33.165884+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:54:46.786406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.417058+0000","last_change":"2024-05-04T03:15:45.008874+0000","last_active":"2024-05-04T03:16:34.417058+0000","last_peered":"2024-05-04T03:16:34.417058+0000","last_clean":"2024-05-04T03:16:34.417058+0000","last_became_active":"2024-05-04T03:15:45.008530+0000","last_became_peered":"2024-05-04T03:15:45.008530+0000","last_unstale":"2024-05-04T03:16:34.417058+0000","last_undegraded":"2024-05-04T03:16:34.417058+0000","last_fullsized":"2024-05-04T03:16:34.417058+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:31:41.697386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.168741+0000","last_change":"2024-05-04T03:15:48.046042+0000","last_active":"2024-05-04T03:17:33.168741+0000","last_peered":"2024-05-04T03:17:33.168741+0000","last_clean":"2024-05-04T03:17:33.168741+0000","last_became_active":"2024-05-04T03:15:48.045387+0000","last_became_peered":"2024-05-04T03:15:48.045387+0000","last_unstale":"2024-05-04T03:17:33.168741+0000","last_undegraded":"2024-05-04T03:17:33.168741+0000","last_fullsized":"2024-05-04T03:17:33.168741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:43:50.782623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.331577+0000","last_change":"2024-05-04T03:15:45.009126+0000","last_active":"2024-05-04T03:16:34.331577+0000","last_peered":"2024-05-04T03:16:34.331577+0000","last_clean":"2024-05-04T03:16:34.331577+0000","last_became_active":"2024-05-04T03:15:45.008802+0000","last_became_peered":"2024-05-04T03:15:45.008802+0000","last_unstale":"2024-05-04T03:16:34.331577+0000","last_undegraded":"2024-05-04T03:16:34.331577+0000","last_fullsized":"2024-05-04T03:16:34.331577+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:25:30.867870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.163747+0000","last_change":"2024-05-04T03:15:48.050478+0000","last_active":"2024-05-04T03:17:33.163747+0000","last_peered":"2024-05-04T03:17:33.163747+0000","last_clean":"2024-05-04T03:17:33.163747+0000","last_became_active":"2024-05-04T03:15:48.045756+0000","last_became_peered":"2024-05-04T03:15:48.045756+0000","last_unstale":"2024-05-04T03:17:33.163747+0000","last_undegraded":"2024-05-04T03:17:33.163747+0000","last_fullsized":"2024-05-04T03:17:33.163747+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:05:11.098381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165760+0000","last_change":"2024-05-04T03:15:48.042778+0000","last_active":"2024-05-04T03:17:33.165760+0000","last_peered":"2024-05-04T03:17:33.165760+0000","last_clean":"2024-05-04T03:17:33.165760+0000","last_became_active":"2024-05-04T03:15:48.042349+0000","last_became_peered":"2024-05-04T03:15:48.042349+0000","last_unstale":"2024-05-04T03:17:33.165760+0000","last_undegraded":"2024-05-04T03:17:33.165760+0000","last_fullsized":"2024-05-04T03:17:33.165760+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:40:12.260026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.160850+0000","last_change":"2024-05-04T03:15:48.048544+0000","last_active":"2024-05-04T03:17:33.160850+0000","last_peered":"2024-05-04T03:17:33.160850+0000","last_clean":"2024-05-04T03:17:33.160850+0000","last_became_active":"2024-05-04T03:15:48.047873+0000","last_became_peered":"2024-05-04T03:15:48.047873+0000","last_unstale":"2024-05-04T03:17:33.160850+0000","last_undegraded":"2024-05-04T03:17:33.160850+0000","last_fullsized":"2024-05-04T03:17:33.160850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:48:02.290401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166521+0000","last_change":"2024-05-04T03:15:48.053467+0000","last_active":"2024-05-04T03:17:33.166521+0000","last_peered":"2024-05-04T03:17:33.166521+0000","last_clean":"2024-05-04T03:17:33.166521+0000","last_became_active":"2024-05-04T03:15:48.050796+0000","last_became_peered":"2024-05-04T03:15:48.050796+0000","last_unstale":"2024-05-04T03:17:33.166521+0000","last_undegraded":"2024-05-04T03:17:33.166521+0000","last_fullsized":"2024-05-04T03:17:33.166521+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:11:15.009467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.158905+0000","last_change":"2024-05-04T03:15:48.054616+0000","last_active":"2024-05-04T03:17:33.158905+0000","last_peered":"2024-05-04T03:17:33.158905+0000","last_clean":"2024-05-04T03:17:33.158905+0000","last_became_active":"2024-05-04T03:15:48.054142+0000","last_became_peered":"2024-05-04T03:15:48.054142+0000","last_unstale":"2024-05-04T03:17:33.158905+0000","last_undegraded":"2024-05-04T03:17:33.158905+0000","last_fullsized":"2024-05-04T03:17:33.158905+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:18:24.067666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166245+0000","last_change":"2024-05-04T03:15:48.046969+0000","last_active":"2024-05-04T03:17:33.166245+0000","last_peered":"2024-05-04T03:17:33.166245+0000","last_clean":"2024-05-04T03:17:33.166245+0000","last_became_active":"2024-05-04T03:15:48.046637+0000","last_became_peered":"2024-05-04T03:15:48.046637+0000","last_unstale":"2024-05-04T03:17:33.166245+0000","last_undegraded":"2024-05-04T03:17:33.166245+0000","last_fullsized":"2024-05-04T03:17:33.166245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:03:01.763037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.163594+0000","last_change":"2024-05-04T03:15:48.041753+0000","last_active":"2024-05-04T03:17:33.163594+0000","last_peered":"2024-05-04T03:17:33.163594+0000","last_clean":"2024-05-04T03:17:33.163594+0000","last_became_active":"2024-05-04T03:15:48.041236+0000","last_became_peered":"2024-05-04T03:15:48.041236+0000","last_unstale":"2024-05-04T03:17:33.163594+0000","last_undegraded":"2024-05-04T03:17:33.163594+0000","last_fullsized":"2024-05-04T03:17:33.163594+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:41:02.928257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.161902+0000","last_change":"2024-05-04T03:15:48.044192+0000","last_active":"2024-05-04T03:17:33.161902+0000","last_peered":"2024-05-04T03:17:33.161902+0000","last_clean":"2024-05-04T03:17:33.161902+0000","last_became_active":"2024-05-04T03:15:48.043849+0000","last_became_peered":"2024-05-04T03:15:48.043849+0000","last_unstale":"2024-05-04T03:17:33.161902+0000","last_undegraded":"2024-05-04T03:17:33.161902+0000","last_fullsized":"2024-05-04T03:17:33.161902+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:55:22.000788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.161015+0000","last_change":"2024-05-04T03:15:48.045675+0000","last_active":"2024-05-04T03:17:33.161015+0000","last_peered":"2024-05-04T03:17:33.161015+0000","last_clean":"2024-05-04T03:17:33.161015+0000","last_became_active":"2024-05-04T03:15:48.045321+0000","last_became_peered":"2024-05-04T03:15:48.045321+0000","last_unstale":"2024-05-04T03:17:33.161015+0000","last_undegraded":"2024-05-04T03:17:33.161015+0000","last_fullsized":"2024-05-04T03:17:33.161015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:12:06.060729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.142817+0000","last_change":"2024-05-04T03:15:48.040872+0000","last_active":"2024-05-04T03:17:33.142817+0000","last_peered":"2024-05-04T03:17:33.142817+0000","last_clean":"2024-05-04T03:17:33.142817+0000","last_became_active":"2024-05-04T03:15:48.040485+0000","last_became_peered":"2024-05-04T03:15:48.040485+0000","last_unstale":"2024-05-04T03:17:33.142817+0000","last_undegraded":"2024-05-04T03:17:33.142817+0000","last_fullsized":"2024-05-04T03:17:33.142817+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:58:01.789270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.161700+0000","last_change":"2024-05-04T03:15:48.049080+0000","last_active":"2024-05-04T03:17:33.161700+0000","last_peered":"2024-05-04T03:17:33.161700+0000","last_clean":"2024-05-04T03:17:33.161700+0000","last_became_active":"2024-05-04T03:15:48.044624+0000","last_became_peered":"2024-05-04T03:15:48.044624+0000","last_unstale":"2024-05-04T03:17:33.161700+0000","last_undegraded":"2024-05-04T03:17:33.161700+0000","last_fullsized":"2024-05-04T03:17:33.161700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:47:08.647401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.171257+0000","last_change":"2024-05-04T03:15:48.049394+0000","last_active":"2024-05-04T03:17:33.171257+0000","last_peered":"2024-05-04T03:17:33.171257+0000","last_clean":"2024-05-04T03:17:33.171257+0000","last_became_active":"2024-05-04T03:15:48.048980+0000","last_became_peered":"2024-05-04T03:15:48.048980+0000","last_unstale":"2024-05-04T03:17:33.171257+0000","last_undegraded":"2024-05-04T03:17:33.171257+0000","last_fullsized":"2024-05-04T03:17:33.171257+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:43:24.914665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.171090+0000","last_change":"2024-05-04T03:15:48.051672+0000","last_active":"2024-05-04T03:17:33.171090+0000","last_peered":"2024-05-04T03:17:33.171090+0000","last_clean":"2024-05-04T03:17:33.171090+0000","last_became_active":"2024-05-04T03:15:48.051314+0000","last_became_peered":"2024-05-04T03:15:48.051314+0000","last_unstale":"2024-05-04T03:17:33.171090+0000","last_undegraded":"2024-05-04T03:17:33.171090+0000","last_fullsized":"2024-05-04T03:17:33.171090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:19:22.808102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.167645+0000","last_change":"2024-05-04T03:15:48.053045+0000","last_active":"2024-05-04T03:17:33.167645+0000","last_peered":"2024-05-04T03:17:33.167645+0000","last_clean":"2024-05-04T03:17:33.167645+0000","last_became_active":"2024-05-04T03:15:48.052659+0000","last_became_peered":"2024-05-04T03:15:48.052659+0000","last_unstale":"2024-05-04T03:17:33.167645+0000","last_undegraded":"2024-05-04T03:17:33.167645+0000","last_fullsized":"2024-05-04T03:17:33.167645+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:53:25.624540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70523,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":584,"num_read_kb":367,"num_write":172,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":211569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612646,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":440503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808875,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1605632,"data_stored":1098897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972400,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":440739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168631,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":440659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397563,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32652,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712500,"statfs":{"total":95995035648,"available":95961600000,"internally_reserved":0,"allocated":1511424,"data_stored":1032730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593792,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40380,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704772,"statfs":{"total":95995035648,"available":95953686528,"internally_reserved":0,"allocated":905216,"data_stored":440128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":506302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40400,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704752,"statfs":{"total":95995035648,"available":95953666048,"internally_reserved":0,"allocated":925696,"data_stored":441399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":507652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40396,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704756,"statfs":{"total":95995035648,"available":95953670144,"internally_reserved":0,"allocated":921600,"data_stored":440439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986250,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":441097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790021,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36824,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708328,"statfs":{"total":95995035648,"available":95957327872,"internally_reserved":0,"allocated":1523712,"data_stored":1031882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":67731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":68491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-04T03:17:41.549 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-04T03:17:41.549 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-04T03:17:41.549 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-04T03:17:41.549 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph health --format=json 2024-05-04T03:17:41.799 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:17:42.604 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:42 smithi001 ceph-mon[29731]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:17:42.604 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:42 smithi001 ceph-mon[29731]: pgmap v347: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-04T03:17:42.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:42 smithi077 ceph-mon[33531]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:17:42.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:42 smithi077 ceph-mon[33531]: pgmap v347: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-04T03:17:42.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:42 smithi072 ceph-mon[33332]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:17:42.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:42 smithi072 ceph-mon[33332]: pgmap v347: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-04T03:17:43.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.362+0000 7f4e0caac700 1 -- 172.21.15.1:0/1899672989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e08110c30 msgr2=0x7f4e08113050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:43.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.362+0000 7f4e0caac700 1 --2- 172.21.15.1:0/1899672989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e08110c30 0x7f4e08113050 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000b0a0 tx=0x7f4e0000b3b0 comp rx=0 tx=0).stop 2024-05-04T03:17:43.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.362+0000 7f4e0caac700 1 -- 172.21.15.1:0/1899672989 shutdown_connections 2024-05-04T03:17:43.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.362+0000 7f4e0caac700 1 --2- 172.21.15.1:0/1899672989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e08110c30 0x7f4e08113050 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:43.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.362+0000 7f4e0caac700 1 --2- 172.21.15.1:0/1899672989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e08103080 0x7f4e08103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:43.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.362+0000 7f4e0caac700 1 --2- 172.21.15.1:0/1899672989 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e081039d0 0x7f4e081106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:43.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.362+0000 7f4e0caac700 1 -- 172.21.15.1:0/1899672989 >> 172.21.15.1:0/1899672989 conn(0x7f4e080fedf0 msgr2=0x7f4e08101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:43.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.362+0000 7f4e0caac700 1 -- 172.21.15.1:0/1899672989 shutdown_connections 2024-05-04T03:17:43.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.362+0000 7f4e0caac700 1 -- 172.21.15.1:0/1899672989 wait complete. 2024-05-04T03:17:43.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.363+0000 7f4e0caac700 1 Processor -- start 2024-05-04T03:17:43.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.363+0000 7f4e0caac700 1 -- start start 2024-05-04T03:17:43.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.363+0000 7f4e0caac700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e08103080 0x7f4e0819d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:43.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.363+0000 7f4e0caac700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e081039d0 0x7f4e0819d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:43.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.363+0000 7f4e0caac700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e08110c30 0x7f4e081a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:43.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.363+0000 7f4e0caac700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4e08116710 con 0x7f4e081039d0 2024-05-04T03:17:43.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.363+0000 7f4e0caac700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4e08116590 con 0x7f4e08103080 2024-05-04T03:17:43.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.363+0000 7f4e0caac700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4e08116890 con 0x7f4e08110c30 2024-05-04T03:17:43.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.364+0000 7f4e05d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e081039d0 0x7f4e0819d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:43.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.364+0000 7f4e05d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e081039d0 0x7f4e0819d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46026/0 (socket says 172.21.15.1:46026) 2024-05-04T03:17:43.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.364+0000 7f4e05d9b700 1 -- 172.21.15.1:0/2877783360 learned_addr learned my addr 172.21.15.1:0/2877783360 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:43.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.364+0000 7f4e0659c700 1 --2- 172.21.15.1:0/2877783360 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e08103080 0x7f4e0819d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:43.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.364+0000 7f4e06d9d700 1 --2- 172.21.15.1:0/2877783360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e08110c30 0x7f4e081a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:43.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.364+0000 7f4e05d9b700 1 -- 172.21.15.1:0/2877783360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e08110c30 msgr2=0x7f4e081a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:43.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.364+0000 7f4e05d9b700 1 --2- 172.21.15.1:0/2877783360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e08110c30 0x7f4e081a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:43.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.364+0000 7f4e05d9b700 1 -- 172.21.15.1:0/2877783360 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e08103080 msgr2=0x7f4e0819d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:43.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.364+0000 7f4e05d9b700 1 --2- 172.21.15.1:0/2877783360 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e08103080 0x7f4e0819d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:43.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.364+0000 7f4e05d9b700 1 -- 172.21.15.1:0/2877783360 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e081a1f10 con 0x7f4e081039d0 2024-05-04T03:17:43.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.364+0000 7f4e06d9d700 1 --2- 172.21.15.1:0/2877783360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e08110c30 0x7f4e081a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:17:43.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.364+0000 7f4e05d9b700 1 --2- 172.21.15.1:0/2877783360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e081039d0 0x7f4e0819d5e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc00d950 tx=0x7f4dfc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:43.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.364+0000 7f4df77fe700 1 -- 172.21.15.1:0/2877783360 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc017070 con 0x7f4e081039d0 2024-05-04T03:17:43.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.364+0000 7f4df77fe700 1 -- 172.21.15.1:0/2877783360 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dfc009de0 con 0x7f4e081039d0 2024-05-04T03:17:43.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.364+0000 7f4e0caac700 1 -- 172.21.15.1:0/2877783360 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e081a2200 con 0x7f4e081039d0 2024-05-04T03:17:43.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.365+0000 7f4df77fe700 1 -- 172.21.15.1:0/2877783360 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc00f3c0 con 0x7f4e081039d0 2024-05-04T03:17:43.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.365+0000 7f4e0caac700 1 -- 172.21.15.1:0/2877783360 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e081a9b20 con 0x7f4e081039d0 2024-05-04T03:17:43.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.366+0000 7f4e0caac700 1 -- 172.21.15.1:0/2877783360 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e080512c0 con 0x7f4e081039d0 2024-05-04T03:17:43.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.367+0000 7f4df77fe700 1 -- 172.21.15.1:0/2877783360 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f4dfc01a070 con 0x7f4e081039d0 2024-05-04T03:17:43.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.368+0000 7f4df77fe700 1 --2- 172.21.15.1:0/2877783360 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4df00b2880 0x7f4df00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:43.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.368+0000 7f4e0659c700 1 --2- 172.21.15.1:0/2877783360 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4df00b2880 0x7f4df00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:43.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.368+0000 7f4df77fe700 1 -- 172.21.15.1:0/2877783360 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f4dfc0e1a00 con 0x7f4e081039d0 2024-05-04T03:17:43.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.368+0000 7f4e0659c700 1 --2- 172.21.15.1:0/2877783360 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4df00b2880 0x7f4df00b4d90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4df8004630 tx=0x7f4df800b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:43.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.373+0000 7f4df77fe700 1 -- 172.21.15.1:0/2877783360 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4dfc0a9740 con 0x7f4e081039d0 2024-05-04T03:17:43.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.640+0000 7f4e0caac700 1 -- 172.21.15.1:0/2877783360 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4e0802e5f0 con 0x7f4e081039d0 2024-05-04T03:17:43.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.641+0000 7f4df77fe700 1 -- 172.21.15.1:0/2877783360 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f4dfc0a9040 con 0x7f4e081039d0 2024-05-04T03:17:43.643 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:17:43.643 INFO:teuthology.orchestra.run.smithi001.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-04T03:17:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.643+0000 7f4e0caac700 1 -- 172.21.15.1:0/2877783360 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4df00b2880 msgr2=0x7f4df00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.643+0000 7f4e0caac700 1 --2- 172.21.15.1:0/2877783360 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4df00b2880 0x7f4df00b4d90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4df8004630 tx=0x7f4df800b3c0 comp rx=0 tx=0).stop 2024-05-04T03:17:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.643+0000 7f4e0caac700 1 -- 172.21.15.1:0/2877783360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e081039d0 msgr2=0x7f4e0819d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.643+0000 7f4e0caac700 1 --2- 172.21.15.1:0/2877783360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e081039d0 0x7f4e0819d5e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc00d950 tx=0x7f4dfc00dc60 comp rx=0 tx=0).stop 2024-05-04T03:17:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.643+0000 7f4e0caac700 1 -- 172.21.15.1:0/2877783360 shutdown_connections 2024-05-04T03:17:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.643+0000 7f4e0caac700 1 --2- 172.21.15.1:0/2877783360 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e08103080 0x7f4e0819d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.643+0000 7f4e0caac700 1 --2- 172.21.15.1:0/2877783360 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4df00b2880 0x7f4df00b4d90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.643+0000 7f4e0caac700 1 --2- 172.21.15.1:0/2877783360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4e08110c30 0x7f4e081a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.643+0000 7f4e0caac700 1 --2- 172.21.15.1:0/2877783360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4e081039d0 0x7f4e0819d5e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.643+0000 7f4e0caac700 1 -- 172.21.15.1:0/2877783360 >> 172.21.15.1:0/2877783360 conn(0x7f4e080fedf0 msgr2=0x7f4e08112700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:43.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.643+0000 7f4e0caac700 1 -- 172.21.15.1:0/2877783360 shutdown_connections 2024-05-04T03:17:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:43.643+0000 7f4e0caac700 1 -- 172.21.15.1:0/2877783360 wait complete. 2024-05-04T03:17:44.264 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-04T03:17:44.265 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:17:44.513 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:17:44.549 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:44 smithi072 ceph-mon[33332]: pgmap v348: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-04T03:17:44.549 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:44 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/2877783360' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-04T03:17:44.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:44 smithi001 ceph-mon[29731]: pgmap v348: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-04T03:17:44.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:44 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2877783360' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-04T03:17:44.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:44 smithi077 ceph-mon[33531]: pgmap v348: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-04T03:17:44.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:44 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/2877783360' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-04T03:17:46.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.203+0000 7f7c7620f700 1 -- 172.21.15.1:0/1575503386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c701034c0 msgr2=0x7f7c701038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:46.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.203+0000 7f7c7620f700 1 --2- 172.21.15.1:0/1575503386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c701034c0 0x7f7c701038d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7c600099c0 tx=0x7f7c60009cd0 comp rx=0 tx=0).stop 2024-05-04T03:17:46.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.203+0000 7f7c7620f700 1 -- 172.21.15.1:0/1575503386 shutdown_connections 2024-05-04T03:17:46.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.203+0000 7f7c7620f700 1 --2- 172.21.15.1:0/1575503386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c70108740 0x7f7c701100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:46.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.203+0000 7f7c7620f700 1 --2- 172.21.15.1:0/1575503386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c701034c0 0x7f7c701038d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:46.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.203+0000 7f7c7620f700 1 --2- 172.21.15.1:0/1575503386 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7c70107df0 0x7f7c70108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:46.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.203+0000 7f7c7620f700 1 -- 172.21.15.1:0/1575503386 >> 172.21.15.1:0/1575503386 conn(0x7f7c700fedc0 msgr2=0x7f7c701011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:46.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.204+0000 7f7c7620f700 1 -- 172.21.15.1:0/1575503386 shutdown_connections 2024-05-04T03:17:46.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.204+0000 7f7c7620f700 1 -- 172.21.15.1:0/1575503386 wait complete. 2024-05-04T03:17:46.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.204+0000 7f7c7620f700 1 Processor -- start 2024-05-04T03:17:46.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.205+0000 7f7c7620f700 1 -- start start 2024-05-04T03:17:46.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.205+0000 7f7c7620f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c701034c0 0x7f7c70199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:46.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.205+0000 7f7c7620f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c70107df0 0x7f7c7019a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:46.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.205+0000 7f7c7620f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7c70108740 0x7f7c7019e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:46.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.205+0000 7f7c7620f700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7c701133e0 con 0x7f7c701034c0 2024-05-04T03:17:46.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.205+0000 7f7c7620f700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7c70113260 con 0x7f7c70108740 2024-05-04T03:17:46.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.205+0000 7f7c7620f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7c70113560 con 0x7f7c70107df0 2024-05-04T03:17:46.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.205+0000 7f7c6f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c701034c0 0x7f7c70199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:46.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.205+0000 7f7c6f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c701034c0 0x7f7c70199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46046/0 (socket says 172.21.15.1:46046) 2024-05-04T03:17:46.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.205+0000 7f7c6f7fe700 1 -- 172.21.15.1:0/138005620 learned_addr learned my addr 172.21.15.1:0/138005620 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:46.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.205+0000 7f7c6effd700 1 --2- 172.21.15.1:0/138005620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c70107df0 0x7f7c7019a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:46.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.205+0000 7f7c6ffff700 1 --2- 172.21.15.1:0/138005620 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7c70108740 0x7f7c7019e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:46.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.205+0000 7f7c6f7fe700 1 -- 172.21.15.1:0/138005620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c70107df0 msgr2=0x7f7c7019a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:46.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.206+0000 7f7c6f7fe700 1 --2- 172.21.15.1:0/138005620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c70107df0 0x7f7c7019a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:46.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.206+0000 7f7c6f7fe700 1 -- 172.21.15.1:0/138005620 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7c70108740 msgr2=0x7f7c7019e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:46.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.206+0000 7f7c6f7fe700 1 --2- 172.21.15.1:0/138005620 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7c70108740 0x7f7c7019e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:46.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.206+0000 7f7c6f7fe700 1 -- 172.21.15.1:0/138005620 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c7019ec40 con 0x7f7c701034c0 2024-05-04T03:17:46.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.206+0000 7f7c6effd700 1 --2- 172.21.15.1:0/138005620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c70107df0 0x7f7c7019a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:17:46.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.206+0000 7f7c6ffff700 1 --2- 172.21.15.1:0/138005620 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7c70108740 0x7f7c7019e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:17:46.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.206+0000 7f7c6f7fe700 1 --2- 172.21.15.1:0/138005620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c701034c0 0x7f7c70199d70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000b580 tx=0x7f7c600047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:46.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.206+0000 7f7c6cff9700 1 -- 172.21.15.1:0/138005620 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c60027070 con 0x7f7c701034c0 2024-05-04T03:17:46.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.206+0000 7f7c6cff9700 1 -- 172.21.15.1:0/138005620 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c60004020 con 0x7f7c701034c0 2024-05-04T03:17:46.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.206+0000 7f7c7620f700 1 -- 172.21.15.1:0/138005620 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c7019eed0 con 0x7f7c701034c0 2024-05-04T03:17:46.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.206+0000 7f7c7620f700 1 -- 172.21.15.1:0/138005620 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c7019f3b0 con 0x7f7c701034c0 2024-05-04T03:17:46.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.206+0000 7f7c6cff9700 1 -- 172.21.15.1:0/138005620 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c60022e30 con 0x7f7c701034c0 2024-05-04T03:17:46.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.207+0000 7f7c7620f700 1 -- 172.21.15.1:0/138005620 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c700512c0 con 0x7f7c701034c0 2024-05-04T03:17:46.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.209+0000 7f7c6cff9700 1 -- 172.21.15.1:0/138005620 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f7c60022590 con 0x7f7c701034c0 2024-05-04T03:17:46.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.209+0000 7f7c6cff9700 1 --2- 172.21.15.1:0/138005620 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f7c580b2880 0x7f7c580b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:46.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.209+0000 7f7c6effd700 1 --2- 172.21.15.1:0/138005620 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f7c580b2880 0x7f7c580b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:46.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.209+0000 7f7c6cff9700 1 -- 172.21.15.1:0/138005620 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f7c600e2f50 con 0x7f7c701034c0 2024-05-04T03:17:46.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.210+0000 7f7c6effd700 1 --2- 172.21.15.1:0/138005620 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f7c580b2880 0x7f7c580b4d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7c701023d0 tx=0x7f7c64008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:46.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.213+0000 7f7c6cff9700 1 -- 172.21.15.1:0/138005620 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7c60013070 con 0x7f7c701034c0 2024-05-04T03:17:46.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.474+0000 7f7c7620f700 1 -- 172.21.15.1:0/138005620 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c7002e5f0 con 0x7f7c701034c0 2024-05-04T03:17:46.477 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:46 smithi001 ceph-mon[29731]: pgmap v349: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-04T03:17:46.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.475+0000 7f7c6cff9700 1 -- 172.21.15.1:0/138005620 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11844 (secure 0 0 0) 0x7f7c600aadc0 con 0x7f7c701034c0 2024-05-04T03:17:46.477 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:17:46.479 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2186681910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2186681910},{"type":"v1","addr":"172.21.15.1:6835","nonce":2186681910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2747191047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2747191047},{"type":"v1","addr":"172.21.15.1:6837","nonce":2747191047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4115120540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4115120540},{"type":"v1","addr":"172.21.15.1:6839","nonce":4115120540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3405709780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3405709780},{"type":"v1","addr":"172.21.15.1:6841","nonce":3405709780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/3816098761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":3816098761},{"type":"v1","addr":"172.21.15.72:6833","nonce":3816098761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/379026128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":379026128},{"type":"v1","addr":"172.21.15.77:6833","nonce":379026128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:15:49.823471+0000","modified":"2024-05-04T03:16:07.918400+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24472,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/3930984976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3930984976},{"type":"v1","addr":"172.21.15.72:6835","nonce":3930984976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/4279951305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4279951305},{"type":"v1","addr":"172.21.15.77:6835","nonce":4279951305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2003521417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2003521417},{"type":"v1","addr":"172.21.15.77:6837","nonce":2003521417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/1417614203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1417614203},{"type":"v1","addr":"172.21.15.72:6837","nonce":1417614203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.77:6839/922723189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":922723189},{"type":"v1","addr":"172.21.15.77:6839","nonce":922723189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2976050650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2976050650},{"type":"v1","addr":"172.21.15.72:6839","nonce":2976050650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24472,24494,24467]},"id":1}]} 2024-05-04T03:17:46.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.477+0000 7f7c7620f700 1 -- 172.21.15.1:0/138005620 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f7c580b2880 msgr2=0x7f7c580b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:46.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.477+0000 7f7c7620f700 1 --2- 172.21.15.1:0/138005620 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f7c580b2880 0x7f7c580b4d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7c701023d0 tx=0x7f7c64008040 comp rx=0 tx=0).stop 2024-05-04T03:17:46.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.477+0000 7f7c7620f700 1 -- 172.21.15.1:0/138005620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c701034c0 msgr2=0x7f7c70199d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:46.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.477+0000 7f7c7620f700 1 --2- 172.21.15.1:0/138005620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c701034c0 0x7f7c70199d70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7c6000b580 tx=0x7f7c600047b0 comp rx=0 tx=0).stop 2024-05-04T03:17:46.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.477+0000 7f7c7620f700 1 -- 172.21.15.1:0/138005620 shutdown_connections 2024-05-04T03:17:46.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.477+0000 7f7c7620f700 1 --2- 172.21.15.1:0/138005620 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7c70108740 0x7f7c7019e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:46.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.477+0000 7f7c7620f700 1 --2- 172.21.15.1:0/138005620 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f7c580b2880 0x7f7c580b4d90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:46.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.477+0000 7f7c7620f700 1 --2- 172.21.15.1:0/138005620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c701034c0 0x7f7c70199d70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:46.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.477+0000 7f7c7620f700 1 --2- 172.21.15.1:0/138005620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7c70107df0 0x7f7c7019a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:46.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.477+0000 7f7c7620f700 1 -- 172.21.15.1:0/138005620 >> 172.21.15.1:0/138005620 conn(0x7f7c700fedc0 msgr2=0x7f7c701096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:46.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.478+0000 7f7c7620f700 1 -- 172.21.15.1:0/138005620 shutdown_connections 2024-05-04T03:17:46.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:46.478+0000 7f7c7620f700 1 -- 172.21.15.1:0/138005620 wait complete. 2024-05-04T03:17:46.481 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-05-04T03:17:46.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:46 smithi077 ceph-mon[33531]: pgmap v349: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-04T03:17:46.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:46 smithi072 ceph-mon[33332]: pgmap v349: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-04T03:17:47.138 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:17:47.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:47 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/138005620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:17:47.390 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:17:47.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:47 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/138005620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:17:47.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:47 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/138005620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:17:48.472 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:48 smithi001 ceph-mon[29731]: pgmap v350: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-04T03:17:48.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:48 smithi077 ceph-mon[33531]: pgmap v350: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-04T03:17:48.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:48 smithi072 ceph-mon[33332]: pgmap v350: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-04T03:17:49.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.022+0000 7fd396789700 1 -- 172.21.15.1:0/2665489113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3901045d0 msgr2=0x7fd3901049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:49.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.022+0000 7fd396789700 1 --2- 172.21.15.1:0/2665489113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3901045d0 0x7fd3901049e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd3800099c0 tx=0x7fd380009cd0 comp rx=0 tx=0).stop 2024-05-04T03:17:49.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.022+0000 7fd396789700 1 -- 172.21.15.1:0/2665489113 shutdown_connections 2024-05-04T03:17:49.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.022+0000 7fd396789700 1 --2- 172.21.15.1:0/2665489113 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd390109850 0x7fd3901111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:49.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.022+0000 7fd396789700 1 --2- 172.21.15.1:0/2665489113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3901045d0 0x7fd3901049e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:49.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.022+0000 7fd396789700 1 --2- 172.21.15.1:0/2665489113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd390108f00 0x7fd390109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:49.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.022+0000 7fd396789700 1 -- 172.21.15.1:0/2665489113 >> 172.21.15.1:0/2665489113 conn(0x7fd3900ffea0 msgr2=0x7fd3901022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:49.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.022+0000 7fd396789700 1 -- 172.21.15.1:0/2665489113 shutdown_connections 2024-05-04T03:17:49.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.023+0000 7fd396789700 1 -- 172.21.15.1:0/2665489113 wait complete. 2024-05-04T03:17:49.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.024+0000 7fd396789700 1 Processor -- start 2024-05-04T03:17:49.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.024+0000 7fd396789700 1 -- start start 2024-05-04T03:17:49.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.024+0000 7fd396789700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3901045d0 0x7fd39019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:49.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.024+0000 7fd396789700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd390108f00 0x7fd39019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:49.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.024+0000 7fd396789700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd390109850 0x7fd39019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:49.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.024+0000 7fd396789700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd390114500 con 0x7fd3901045d0 2024-05-04T03:17:49.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.024+0000 7fd396789700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd390114380 con 0x7fd390109850 2024-05-04T03:17:49.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.024+0000 7fd396789700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd390114680 con 0x7fd390108f00 2024-05-04T03:17:49.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.024+0000 7fd38ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3901045d0 0x7fd39019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:49.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.024+0000 7fd38ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3901045d0 0x7fd39019ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46060/0 (socket says 172.21.15.1:46060) 2024-05-04T03:17:49.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.024+0000 7fd38ffff700 1 -- 172.21.15.1:0/3225705984 learned_addr learned my addr 172.21.15.1:0/3225705984 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:49.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.024+0000 7fd38f7fe700 1 --2- 172.21.15.1:0/3225705984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd390108f00 0x7fd39019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:49.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.024+0000 7fd394d26700 1 --2- 172.21.15.1:0/3225705984 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd390109850 0x7fd39019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:49.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.025+0000 7fd38ffff700 1 -- 172.21.15.1:0/3225705984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd390108f00 msgr2=0x7fd39019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:49.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.025+0000 7fd38ffff700 1 --2- 172.21.15.1:0/3225705984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd390108f00 0x7fd39019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:49.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.025+0000 7fd38ffff700 1 -- 172.21.15.1:0/3225705984 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd390109850 msgr2=0x7fd39019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:49.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.025+0000 7fd38ffff700 1 --2- 172.21.15.1:0/3225705984 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd390109850 0x7fd39019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:49.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.025+0000 7fd38ffff700 1 -- 172.21.15.1:0/3225705984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39019fce0 con 0x7fd3901045d0 2024-05-04T03:17:49.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.025+0000 7fd394d26700 1 --2- 172.21.15.1:0/3225705984 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd390109850 0x7fd39019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:17:49.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.025+0000 7fd38ffff700 1 --2- 172.21.15.1:0/3225705984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3901045d0 0x7fd39019ae10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd38000b580 tx=0x7fd3800047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:49.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.025+0000 7fd38d7fa700 1 -- 172.21.15.1:0/3225705984 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380027070 con 0x7fd3901045d0 2024-05-04T03:17:49.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.025+0000 7fd38d7fa700 1 -- 172.21.15.1:0/3225705984 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3800040a0 con 0x7fd3901045d0 2024-05-04T03:17:49.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.025+0000 7fd396789700 1 -- 172.21.15.1:0/3225705984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd39019ff70 con 0x7fd3901045d0 2024-05-04T03:17:49.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.025+0000 7fd38d7fa700 1 -- 172.21.15.1:0/3225705984 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38000f460 con 0x7fd3901045d0 2024-05-04T03:17:49.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.025+0000 7fd396789700 1 -- 172.21.15.1:0/3225705984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3901a0450 con 0x7fd3901045d0 2024-05-04T03:17:49.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.026+0000 7fd396789700 1 -- 172.21.15.1:0/3225705984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3900512c0 con 0x7fd3901045d0 2024-05-04T03:17:49.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.028+0000 7fd38d7fa700 1 -- 172.21.15.1:0/3225705984 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fd38002a070 con 0x7fd3901045d0 2024-05-04T03:17:49.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.028+0000 7fd38d7fa700 1 --2- 172.21.15.1:0/3225705984 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd3780b2880 0x7fd3780b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:49.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.029+0000 7fd38f7fe700 1 --2- 172.21.15.1:0/3225705984 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd3780b2880 0x7fd3780b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:49.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.029+0000 7fd38d7fa700 1 -- 172.21.15.1:0/3225705984 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fd3800e3070 con 0x7fd3901045d0 2024-05-04T03:17:49.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.029+0000 7fd38f7fe700 1 --2- 172.21.15.1:0/3225705984 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd3780b2880 0x7fd3780b4d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd3901074f0 tx=0x7fd38400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:49.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.033+0000 7fd38d7fa700 1 -- 172.21.15.1:0/3225705984 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd38005fdf0 con 0x7fd3901045d0 2024-05-04T03:17:49.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.294+0000 7fd396789700 1 -- 172.21.15.1:0/3225705984 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd39002e5f0 con 0x7fd3901045d0 2024-05-04T03:17:49.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.294+0000 7fd38d7fa700 1 -- 172.21.15.1:0/3225705984 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11844 (secure 0 0 0) 0x7fd380013070 con 0x7fd3901045d0 2024-05-04T03:17:49.296 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:17:49.297 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2186681910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2186681910},{"type":"v1","addr":"172.21.15.1:6835","nonce":2186681910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2747191047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2747191047},{"type":"v1","addr":"172.21.15.1:6837","nonce":2747191047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4115120540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4115120540},{"type":"v1","addr":"172.21.15.1:6839","nonce":4115120540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3405709780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3405709780},{"type":"v1","addr":"172.21.15.1:6841","nonce":3405709780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/3816098761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":3816098761},{"type":"v1","addr":"172.21.15.72:6833","nonce":3816098761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/379026128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":379026128},{"type":"v1","addr":"172.21.15.77:6833","nonce":379026128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:15:49.823471+0000","modified":"2024-05-04T03:16:07.918400+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24472,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/3930984976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3930984976},{"type":"v1","addr":"172.21.15.72:6835","nonce":3930984976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/4279951305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4279951305},{"type":"v1","addr":"172.21.15.77:6835","nonce":4279951305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2003521417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2003521417},{"type":"v1","addr":"172.21.15.77:6837","nonce":2003521417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/1417614203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1417614203},{"type":"v1","addr":"172.21.15.72:6837","nonce":1417614203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.77:6839/922723189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":922723189},{"type":"v1","addr":"172.21.15.77:6839","nonce":922723189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2976050650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2976050650},{"type":"v1","addr":"172.21.15.72:6839","nonce":2976050650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24472,24494,24467]},"id":1}]} 2024-05-04T03:17:49.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.297+0000 7fd396789700 1 -- 172.21.15.1:0/3225705984 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd3780b2880 msgr2=0x7fd3780b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:49.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.297+0000 7fd396789700 1 --2- 172.21.15.1:0/3225705984 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd3780b2880 0x7fd3780b4d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd3901074f0 tx=0x7fd38400a300 comp rx=0 tx=0).stop 2024-05-04T03:17:49.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.297+0000 7fd396789700 1 -- 172.21.15.1:0/3225705984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3901045d0 msgr2=0x7fd39019ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:49.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.297+0000 7fd396789700 1 --2- 172.21.15.1:0/3225705984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3901045d0 0x7fd39019ae10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd38000b580 tx=0x7fd3800047b0 comp rx=0 tx=0).stop 2024-05-04T03:17:49.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.297+0000 7fd396789700 1 -- 172.21.15.1:0/3225705984 shutdown_connections 2024-05-04T03:17:49.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.297+0000 7fd396789700 1 --2- 172.21.15.1:0/3225705984 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd390109850 0x7fd39019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:49.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.297+0000 7fd396789700 1 --2- 172.21.15.1:0/3225705984 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd3780b2880 0x7fd3780b4d90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:49.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.297+0000 7fd396789700 1 --2- 172.21.15.1:0/3225705984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3901045d0 0x7fd39019ae10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:49.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.297+0000 7fd396789700 1 --2- 172.21.15.1:0/3225705984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd390108f00 0x7fd39019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:49.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.297+0000 7fd396789700 1 -- 172.21.15.1:0/3225705984 >> 172.21.15.1:0/3225705984 conn(0x7fd3900ffea0 msgr2=0x7fd390102d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:49.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.297+0000 7fd396789700 1 -- 172.21.15.1:0/3225705984 shutdown_connections 2024-05-04T03:17:49.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:49.297+0000 7fd396789700 1 -- 172.21.15.1:0/3225705984 wait complete. 2024-05-04T03:17:49.300 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-05-04T03:17:49.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:49 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3225705984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:17:49.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:49 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/3225705984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:17:49.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:49 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/3225705984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:17:49.925 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-04T03:17:50.173 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:17:50.606 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:50 smithi001 ceph-mon[29731]: pgmap v351: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-04T03:17:50.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:50 smithi077 ceph-mon[33531]: pgmap v351: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-04T03:17:50.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:50 smithi072 ceph-mon[33332]: pgmap v351: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-04T03:17:51.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.953+0000 7fc01962e700 1 -- 172.21.15.1:0/3763520462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0141034c0 msgr2=0x7fc0141038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:51.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.953+0000 7fc01962e700 1 --2- 172.21.15.1:0/3763520462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0141034c0 0x7fc0141038d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc0040099c0 tx=0x7fc004009cd0 comp rx=0 tx=0).stop 2024-05-04T03:17:51.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.953+0000 7fc01962e700 1 -- 172.21.15.1:0/3763520462 shutdown_connections 2024-05-04T03:17:51.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.953+0000 7fc01962e700 1 --2- 172.21.15.1:0/3763520462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc014108740 0x7fc0141100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:51.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.953+0000 7fc01962e700 1 --2- 172.21.15.1:0/3763520462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0141034c0 0x7fc0141038d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:51.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.953+0000 7fc01962e700 1 --2- 172.21.15.1:0/3763520462 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc014107df0 0x7fc014108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:51.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.953+0000 7fc01962e700 1 -- 172.21.15.1:0/3763520462 >> 172.21.15.1:0/3763520462 conn(0x7fc0140fedc0 msgr2=0x7fc0141011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:51.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.953+0000 7fc01962e700 1 -- 172.21.15.1:0/3763520462 shutdown_connections 2024-05-04T03:17:51.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.953+0000 7fc01962e700 1 -- 172.21.15.1:0/3763520462 wait complete. 2024-05-04T03:17:51.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.954+0000 7fc01962e700 1 Processor -- start 2024-05-04T03:17:51.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.954+0000 7fc01962e700 1 -- start start 2024-05-04T03:17:51.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.954+0000 7fc01962e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0141034c0 0x7fc01419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:51.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.954+0000 7fc01962e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc014107df0 0x7fc01419c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:51.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.954+0000 7fc01962e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc014108740 0x7fc01419c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:51.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.954+0000 7fc01962e700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc014113370 con 0x7fc0141034c0 2024-05-04T03:17:51.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.954+0000 7fc01962e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141131f0 con 0x7fc014107df0 2024-05-04T03:17:51.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.954+0000 7fc01962e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc0141134f0 con 0x7fc014108740 2024-05-04T03:17:51.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.954+0000 7fc012ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0141034c0 0x7fc01419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:51.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.954+0000 7fc012ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0141034c0 0x7fc01419bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33842/0 (socket says 172.21.15.1:33842) 2024-05-04T03:17:51.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.954+0000 7fc012ffd700 1 -- 172.21.15.1:0/3647792607 learned_addr learned my addr 172.21.15.1:0/3647792607 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:51.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.954+0000 7fc0127fc700 1 --2- 172.21.15.1:0/3647792607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc014107df0 0x7fc01419c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:51.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.954+0000 7fc0137fe700 1 --2- 172.21.15.1:0/3647792607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc014108740 0x7fc01419c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:51.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.955+0000 7fc012ffd700 1 -- 172.21.15.1:0/3647792607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc014108740 msgr2=0x7fc01419c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:51.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.955+0000 7fc012ffd700 1 --2- 172.21.15.1:0/3647792607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc014108740 0x7fc01419c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:51.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.955+0000 7fc012ffd700 1 -- 172.21.15.1:0/3647792607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc014107df0 msgr2=0x7fc01419c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:51.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.955+0000 7fc012ffd700 1 --2- 172.21.15.1:0/3647792607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc014107df0 0x7fc01419c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:51.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.955+0000 7fc012ffd700 1 -- 172.21.15.1:0/3647792607 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0141a6f30 con 0x7fc0141034c0 2024-05-04T03:17:51.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.955+0000 7fc0137fe700 1 --2- 172.21.15.1:0/3647792607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc014108740 0x7fc01419c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:17:51.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.955+0000 7fc012ffd700 1 --2- 172.21.15.1:0/3647792607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0141034c0 0x7fc01419bc70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc00400b580 tx=0x7fc0040047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:51.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.955+0000 7fc00bfff700 1 -- 172.21.15.1:0/3647792607 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004027070 con 0x7fc0141034c0 2024-05-04T03:17:51.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.955+0000 7fc00bfff700 1 -- 172.21.15.1:0/3647792607 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc004004020 con 0x7fc0141034c0 2024-05-04T03:17:51.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.955+0000 7fc01962e700 1 -- 172.21.15.1:0/3647792607 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0141a71c0 con 0x7fc0141034c0 2024-05-04T03:17:51.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.955+0000 7fc00bfff700 1 -- 172.21.15.1:0/3647792607 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004022e30 con 0x7fc0141034c0 2024-05-04T03:17:51.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.955+0000 7fc01962e700 1 -- 172.21.15.1:0/3647792607 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0141a7610 con 0x7fc0141034c0 2024-05-04T03:17:51.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.956+0000 7fc01962e700 1 -- 172.21.15.1:0/3647792607 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0140512c0 con 0x7fc0141034c0 2024-05-04T03:17:51.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.958+0000 7fc00bfff700 1 -- 172.21.15.1:0/3647792607 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fc004022590 con 0x7fc0141034c0 2024-05-04T03:17:51.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.958+0000 7fc00bfff700 1 --2- 172.21.15.1:0/3647792607 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc0000b2880 0x7fc0000b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:51.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.958+0000 7fc0127fc700 1 --2- 172.21.15.1:0/3647792607 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc0000b2880 0x7fc0000b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:51.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.958+0000 7fc00bfff700 1 -- 172.21.15.1:0/3647792607 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fc0040e2e70 con 0x7fc0141034c0 2024-05-04T03:17:51.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.959+0000 7fc0127fc700 1 --2- 172.21.15.1:0/3647792607 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc0000b2880 0x7fc0000b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbffc005ea0 tx=0x7fbffc005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:51.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:51.963+0000 7fc00bfff700 1 -- 172.21.15.1:0/3647792607 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc004013070 con 0x7fc0141034c0 2024-05-04T03:17:52.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:52.185+0000 7fc01962e700 1 -- 172.21.15.1:0/3647792607 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc01402e5f0 con 0x7fc0141034c0 2024-05-04T03:17:52.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:52.187+0000 7fc00bfff700 1 -- 172.21.15.1:0/3647792607 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22380 (secure 0 0 0) 0x7fc0040aace0 con 0x7fc0141034c0 2024-05-04T03:17:52.189 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:17:52.189 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":83,"fsid":"4a572aa8-09c3-11ef-bc97-c7b262605968","created":"2024-05-04T03:07:32.845547+0000","modified":"2024-05-04T03:16:34.321580+0000","last_up_change":"2024-05-04T03:14:44.192571+0000","last_in_change":"2024-05-04T03:14:23.353342+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T03:11:01.139189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T03:15:43.151696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T03:15:46.753539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"100fe035-2454-4c95-90c7-759956ec36a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6803","nonce":3479011484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6805","nonce":3479011484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6809","nonce":3479011484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6807","nonce":3479011484}]},"public_addr":"172.21.15.1:6803/3479011484","cluster_addr":"172.21.15.1:6805/3479011484","heartbeat_back_addr":"172.21.15.1:6809/3479011484","heartbeat_front_addr":"172.21.15.1:6807/3479011484","state":["exists","up"]},{"osd":1,"uuid":"5826ac76-ef6b-4040-8ad1-305db355af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6801","nonce":3725706149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6803","nonce":3725706149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6807","nonce":3725706149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6805","nonce":3725706149}]},"public_addr":"172.21.15.72:6801/3725706149","cluster_addr":"172.21.15.72:6803/3725706149","heartbeat_back_addr":"172.21.15.72:6807/3725706149","heartbeat_front_addr":"172.21.15.72:6805/3725706149","state":["exists","up"]},{"osd":2,"uuid":"923d76e5-b032-4190-9e35-546d27722a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6801","nonce":515038343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6803","nonce":515038343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6807","nonce":515038343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6805","nonce":515038343}]},"public_addr":"172.21.15.77:6801/515038343","cluster_addr":"172.21.15.77:6803/515038343","heartbeat_back_addr":"172.21.15.77:6807/515038343","heartbeat_front_addr":"172.21.15.77:6805/515038343","state":["exists","up"]},{"osd":3,"uuid":"495cc0c5-6a99-481f-ad9e-38f05fb84356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6811","nonce":4050439365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6813","nonce":4050439365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6817","nonce":4050439365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6815","nonce":4050439365}]},"public_addr":"172.21.15.1:6811/4050439365","cluster_addr":"172.21.15.1:6813/4050439365","heartbeat_back_addr":"172.21.15.1:6817/4050439365","heartbeat_front_addr":"172.21.15.1:6815/4050439365","state":["exists","up"]},{"osd":4,"uuid":"b86b8026-79c0-4056-9e67-a61f54b6ab5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6809","nonce":2501732391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6811","nonce":2501732391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6815","nonce":2501732391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6813","nonce":2501732391}]},"public_addr":"172.21.15.72:6809/2501732391","cluster_addr":"172.21.15.72:6811/2501732391","heartbeat_back_addr":"172.21.15.72:6815/2501732391","heartbeat_front_addr":"172.21.15.72:6813/2501732391","state":["exists","up"]},{"osd":5,"uuid":"80975cb9-39e0-46b8-a343-9ff69194a14e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6809","nonce":3582852165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6811","nonce":3582852165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6815","nonce":3582852165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6813","nonce":3582852165}]},"public_addr":"172.21.15.77:6809/3582852165","cluster_addr":"172.21.15.77:6811/3582852165","heartbeat_back_addr":"172.21.15.77:6815/3582852165","heartbeat_front_addr":"172.21.15.77:6813/3582852165","state":["exists","up"]},{"osd":6,"uuid":"aca3fac5-fbb5-42de-bf0e-e4a0db7381e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6819","nonce":1799033649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6821","nonce":1799033649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6825","nonce":1799033649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6823","nonce":1799033649}]},"public_addr":"172.21.15.1:6819/1799033649","cluster_addr":"172.21.15.1:6821/1799033649","heartbeat_back_addr":"172.21.15.1:6825/1799033649","heartbeat_front_addr":"172.21.15.1:6823/1799033649","state":["exists","up"]},{"osd":7,"uuid":"cbff6b8a-c463-4249-ae45-fdc6f99a5489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6817","nonce":3460835447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6819","nonce":3460835447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6823","nonce":3460835447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6821","nonce":3460835447}]},"public_addr":"172.21.15.72:6817/3460835447","cluster_addr":"172.21.15.72:6819/3460835447","heartbeat_back_addr":"172.21.15.72:6823/3460835447","heartbeat_front_addr":"172.21.15.72:6821/3460835447","state":["exists","up"]},{"osd":8,"uuid":"e140bf13-4887-4d68-9373-3d4fc1c4584a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6817","nonce":1124479323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6819","nonce":1124479323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6823","nonce":1124479323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6821","nonce":1124479323}]},"public_addr":"172.21.15.77:6817/1124479323","cluster_addr":"172.21.15.77:6819/1124479323","heartbeat_back_addr":"172.21.15.77:6823/1124479323","heartbeat_front_addr":"172.21.15.77:6821/1124479323","state":["exists","up"]},{"osd":9,"uuid":"f1e8a232-cb11-4b9d-8789-e1637fa5aaa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6827","nonce":2681322272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6829","nonce":2681322272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6833","nonce":2681322272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6831","nonce":2681322272}]},"public_addr":"172.21.15.1:6827/2681322272","cluster_addr":"172.21.15.1:6829/2681322272","heartbeat_back_addr":"172.21.15.1:6833/2681322272","heartbeat_front_addr":"172.21.15.1:6831/2681322272","state":["exists","up"]},{"osd":10,"uuid":"79b570af-3d1e-439d-a328-9e6bc7d67839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6825","nonce":3293703569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6827","nonce":3293703569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6831","nonce":3293703569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6829","nonce":3293703569}]},"public_addr":"172.21.15.72:6825/3293703569","cluster_addr":"172.21.15.72:6827/3293703569","heartbeat_back_addr":"172.21.15.72:6831/3293703569","heartbeat_front_addr":"172.21.15.72:6829/3293703569","state":["exists","up"]},{"osd":11,"uuid":"e35bf83d-9ee4-4745-96c2-1c35cc8eefba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6825","nonce":4155947548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6827","nonce":4155947548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6831","nonce":4155947548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6829","nonce":4155947548}]},"public_addr":"172.21.15.77:6825/4155947548","cluster_addr":"172.21.15.77:6827/4155947548","heartbeat_back_addr":"172.21.15.77:6831/4155947548","heartbeat_front_addr":"172.21.15.77:6829/4155947548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:09.467126+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:33.861139+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:57.741962+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:11:21.746021+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:11:46.323252+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:12:10.640180+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:12:35.750591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:00.306549+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:25.012495+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:51.185229+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:14:16.403850+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:14:41.653316+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/2029897106":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/402488450":"2024-05-05T03:08:33.045884+0000","172.21.15.1:6801/128793132":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/2233526606":"2024-05-05T03:08:06.135783+0000","172.21.15.1:0/2715541855":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/2415697325":"2024-05-05T03:08:06.135783+0000","172.21.15.1:0/2305298106":"2024-05-05T03:08:06.135783+0000","172.21.15.1:6800/128793132":"2024-05-05T03:08:33.045884+0000","172.21.15.1:6800/4200529863":"2024-05-05T03:08:06.135783+0000","172.21.15.1:6801/4200529863":"2024-05-05T03:08:06.135783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T03:17:52.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:52.189+0000 7fc01962e700 1 -- 172.21.15.1:0/3647792607 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc0000b2880 msgr2=0x7fc0000b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:52.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:52.190+0000 7fc01962e700 1 --2- 172.21.15.1:0/3647792607 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc0000b2880 0x7fc0000b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbffc005ea0 tx=0x7fbffc005e30 comp rx=0 tx=0).stop 2024-05-04T03:17:52.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:52.190+0000 7fc01962e700 1 -- 172.21.15.1:0/3647792607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0141034c0 msgr2=0x7fc01419bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:52.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:52.190+0000 7fc01962e700 1 --2- 172.21.15.1:0/3647792607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0141034c0 0x7fc01419bc70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc00400b580 tx=0x7fc0040047b0 comp rx=0 tx=0).stop 2024-05-04T03:17:52.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:52.190+0000 7fc01962e700 1 -- 172.21.15.1:0/3647792607 shutdown_connections 2024-05-04T03:17:52.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:52.190+0000 7fc01962e700 1 --2- 172.21.15.1:0/3647792607 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc0000b2880 0x7fc0000b4d90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:52.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:52.190+0000 7fc01962e700 1 --2- 172.21.15.1:0/3647792607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc014108740 0x7fc01419c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:52.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:52.190+0000 7fc01962e700 1 --2- 172.21.15.1:0/3647792607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0141034c0 0x7fc01419bc70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:52.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:52.190+0000 7fc01962e700 1 --2- 172.21.15.1:0/3647792607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc014107df0 0x7fc01419c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:52.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:52.190+0000 7fc01962e700 1 -- 172.21.15.1:0/3647792607 >> 172.21.15.1:0/3647792607 conn(0x7fc0140fedc0 msgr2=0x7fc0141096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:52.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:52.190+0000 7fc01962e700 1 -- 172.21.15.1:0/3647792607 shutdown_connections 2024-05-04T03:17:52.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:52.190+0000 7fc01962e700 1 -- 172.21.15.1:0/3647792607 wait complete. 2024-05-04T03:17:52.524 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:52 smithi001 ceph-mon[29731]: pgmap v352: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:17:52.524 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:52 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3647792607' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:17:52.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:52 smithi077 ceph-mon[33531]: pgmap v352: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:17:52.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:52 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/3647792607' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:17:52.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:52 smithi072 ceph-mon[33332]: pgmap v352: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:17:52.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:52 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/3647792607' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:17:52.860 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:17:53.123 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:17:54.560 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:54 smithi001 ceph-mon[29731]: pgmap v353: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:17:54.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:54 smithi077 ceph-mon[33531]: pgmap v353: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:17:54.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:54 smithi072 ceph-mon[33332]: pgmap v353: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:17:54.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.804+0000 7f00d6a58700 1 -- 172.21.15.1:0/713678858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00d0109870 msgr2=0x7f00d01111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:54.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.804+0000 7f00d6a58700 1 --2- 172.21.15.1:0/713678858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00d0109870 0x7f00d01111d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f00c800b0d0 tx=0x7f00c800b3e0 comp rx=0 tx=0).stop 2024-05-04T03:17:54.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.804+0000 7f00d6a58700 1 -- 172.21.15.1:0/713678858 shutdown_connections 2024-05-04T03:17:54.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.804+0000 7f00d6a58700 1 --2- 172.21.15.1:0/713678858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00d0109870 0x7f00d01111d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:54.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.804+0000 7f00d6a58700 1 --2- 172.21.15.1:0/713678858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00d01045f0 0x7f00d0104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:54.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.804+0000 7f00d6a58700 1 --2- 172.21.15.1:0/713678858 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00d0108f20 0x7f00d0109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:54.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.804+0000 7f00d6a58700 1 -- 172.21.15.1:0/713678858 >> 172.21.15.1:0/713678858 conn(0x7f00d0100040 msgr2=0x7f00d0102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:54.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.804+0000 7f00d6a58700 1 -- 172.21.15.1:0/713678858 shutdown_connections 2024-05-04T03:17:54.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.804+0000 7f00d6a58700 1 -- 172.21.15.1:0/713678858 wait complete. 2024-05-04T03:17:54.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.804+0000 7f00d6a58700 1 Processor -- start 2024-05-04T03:17:54.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.804+0000 7f00d6a58700 1 -- start start 2024-05-04T03:17:54.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.805+0000 7f00d6a58700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00d01045f0 0x7f00d019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.805+0000 7f00d6a58700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00d0108f20 0x7f00d019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.805+0000 7f00d6a58700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00d0109870 0x7f00d019f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.805+0000 7f00d6a58700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f00d01144a0 con 0x7f00d0109870 2024-05-04T03:17:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.805+0000 7f00d6a58700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f00d0114320 con 0x7f00d0108f20 2024-05-04T03:17:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.805+0000 7f00d6a58700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f00d0114620 con 0x7f00d01045f0 2024-05-04T03:17:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.805+0000 7f00d4ff5700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00d0109870 0x7f00d019f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.805+0000 7f00d4ff5700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00d0109870 0x7f00d019f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33870/0 (socket says 172.21.15.1:33870) 2024-05-04T03:17:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.805+0000 7f00d4ff5700 1 -- 172.21.15.1:0/1379598187 learned_addr learned my addr 172.21.15.1:0/1379598187 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.805+0000 7f00cf7fe700 1 --2- 172.21.15.1:0/1379598187 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00d0108f20 0x7f00d019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.805+0000 7f00cffff700 1 --2- 172.21.15.1:0/1379598187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00d01045f0 0x7f00d019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:54.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.805+0000 7f00d4ff5700 1 -- 172.21.15.1:0/1379598187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00d01045f0 msgr2=0x7f00d019adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:54.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.805+0000 7f00d4ff5700 1 --2- 172.21.15.1:0/1379598187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00d01045f0 0x7f00d019adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:54.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.805+0000 7f00d4ff5700 1 -- 172.21.15.1:0/1379598187 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00d0108f20 msgr2=0x7f00d019b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:54.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.805+0000 7f00d4ff5700 1 --2- 172.21.15.1:0/1379598187 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00d0108f20 0x7f00d019b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:54.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.805+0000 7f00d4ff5700 1 -- 172.21.15.1:0/1379598187 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d019fbd0 con 0x7f00d0109870 2024-05-04T03:17:54.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.805+0000 7f00d4ff5700 1 --2- 172.21.15.1:0/1379598187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00d0109870 0x7f00d019f5e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f00c8009850 tx=0x7f00c8007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:54.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.806+0000 7f00cd7fa700 1 -- 172.21.15.1:0/1379598187 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c801f070 con 0x7f00d0109870 2024-05-04T03:17:54.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.806+0000 7f00cd7fa700 1 -- 172.21.15.1:0/1379598187 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00c8004c50 con 0x7f00d0109870 2024-05-04T03:17:54.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.806+0000 7f00d6a58700 1 -- 172.21.15.1:0/1379598187 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00d019fe60 con 0x7f00d0109870 2024-05-04T03:17:54.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.806+0000 7f00cd7fa700 1 -- 172.21.15.1:0/1379598187 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c801b7b0 con 0x7f00d0109870 2024-05-04T03:17:54.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.806+0000 7f00d6a58700 1 -- 172.21.15.1:0/1379598187 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00d01a0340 con 0x7f00d0109870 2024-05-04T03:17:54.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.807+0000 7f00d6a58700 1 -- 172.21.15.1:0/1379598187 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00d00512c0 con 0x7f00d0109870 2024-05-04T03:17:54.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.808+0000 7f00cd7fa700 1 -- 172.21.15.1:0/1379598187 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f00c8022070 con 0x7f00d0109870 2024-05-04T03:17:54.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.809+0000 7f00cd7fa700 1 --2- 172.21.15.1:0/1379598187 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f00b80b2880 0x7f00b80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:54.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.809+0000 7f00cffff700 1 --2- 172.21.15.1:0/1379598187 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f00b80b2880 0x7f00b80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:54.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.809+0000 7f00cd7fa700 1 -- 172.21.15.1:0/1379598187 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f00c800f020 con 0x7f00d0109870 2024-05-04T03:17:54.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.810+0000 7f00cffff700 1 --2- 172.21.15.1:0/1379598187 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f00b80b2880 0x7f00b80b4d90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f00c0005ea0 tx=0x7f00c0005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:54.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:54.813+0000 7f00cd7fa700 1 -- 172.21.15.1:0/1379598187 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f00c8029020 con 0x7f00d0109870 2024-05-04T03:17:55.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:55.073+0000 7f00d6a58700 1 -- 172.21.15.1:0/1379598187 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00d002e5f0 con 0x7f00d0109870 2024-05-04T03:17:55.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:55.073+0000 7f00cd7fa700 1 -- 172.21.15.1:0/1379598187 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11844 (secure 0 0 0) 0x7f00c80ab540 con 0x7f00d0109870 2024-05-04T03:17:55.075 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:17:55.075 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2186681910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2186681910},{"type":"v1","addr":"172.21.15.1:6835","nonce":2186681910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2747191047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2747191047},{"type":"v1","addr":"172.21.15.1:6837","nonce":2747191047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4115120540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4115120540},{"type":"v1","addr":"172.21.15.1:6839","nonce":4115120540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3405709780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3405709780},{"type":"v1","addr":"172.21.15.1:6841","nonce":3405709780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/3816098761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":3816098761},{"type":"v1","addr":"172.21.15.72:6833","nonce":3816098761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/379026128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":379026128},{"type":"v1","addr":"172.21.15.77:6833","nonce":379026128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:15:49.823471+0000","modified":"2024-05-04T03:16:07.918400+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24472,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/3930984976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3930984976},{"type":"v1","addr":"172.21.15.72:6835","nonce":3930984976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/4279951305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4279951305},{"type":"v1","addr":"172.21.15.77:6835","nonce":4279951305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2003521417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2003521417},{"type":"v1","addr":"172.21.15.77:6837","nonce":2003521417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/1417614203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1417614203},{"type":"v1","addr":"172.21.15.72:6837","nonce":1417614203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.77:6839/922723189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":922723189},{"type":"v1","addr":"172.21.15.77:6839","nonce":922723189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2976050650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2976050650},{"type":"v1","addr":"172.21.15.72:6839","nonce":2976050650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24472,24494,24467]},"id":1}]} 2024-05-04T03:17:55.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:55.076+0000 7f00d6a58700 1 -- 172.21.15.1:0/1379598187 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f00b80b2880 msgr2=0x7f00b80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:55.076+0000 7f00d6a58700 1 --2- 172.21.15.1:0/1379598187 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f00b80b2880 0x7f00b80b4d90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f00c0005ea0 tx=0x7f00c0005e30 comp rx=0 tx=0).stop 2024-05-04T03:17:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:55.076+0000 7f00d6a58700 1 -- 172.21.15.1:0/1379598187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00d0109870 msgr2=0x7f00d019f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:55.076+0000 7f00d6a58700 1 --2- 172.21.15.1:0/1379598187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00d0109870 0x7f00d019f5e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f00c8009850 tx=0x7f00c8007a20 comp rx=0 tx=0).stop 2024-05-04T03:17:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:55.076+0000 7f00d6a58700 1 -- 172.21.15.1:0/1379598187 shutdown_connections 2024-05-04T03:17:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:55.076+0000 7f00d6a58700 1 --2- 172.21.15.1:0/1379598187 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f00b80b2880 0x7f00b80b4d90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:55.076+0000 7f00d6a58700 1 --2- 172.21.15.1:0/1379598187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00d0109870 0x7f00d019f5e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:55.076+0000 7f00d6a58700 1 --2- 172.21.15.1:0/1379598187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f00d01045f0 0x7f00d019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:55.076+0000 7f00d6a58700 1 --2- 172.21.15.1:0/1379598187 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f00d0108f20 0x7f00d019b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:55.076+0000 7f00d6a58700 1 -- 172.21.15.1:0/1379598187 >> 172.21.15.1:0/1379598187 conn(0x7f00d0100040 msgr2=0x7f00d0108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:55.076+0000 7f00d6a58700 1 -- 172.21.15.1:0/1379598187 shutdown_connections 2024-05-04T03:17:55.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:55.076+0000 7f00d6a58700 1 -- 172.21.15.1:0/1379598187 wait complete. 2024-05-04T03:17:55.079 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-05-04T03:17:55.360 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:55 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/1379598187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:17:55.665 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-04T03:15:49.823471+0000', 'modified': '2024-05-04T03:16:07.918400+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24494, 'mds_1': 24472, 'mds_2': 24467}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.72:6835/3930984976', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.72:6834', 'nonce': 3930984976}, {'type': 'v1', 'addr': '172.21.15.72:6835', 'nonce': 3930984976}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6835/4279951305', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6834', 'nonce': 4279951305}, {'type': 'v1', 'addr': '172.21.15.77:6835', 'nonce': 4279951305}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6837/2003521417', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6836', 'nonce': 2003521417}, {'type': 'v1', 'addr': '172.21.15.77:6837', 'nonce': 2003521417}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.72:6837/1417614203', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.72:6836', 'nonce': 1417614203}, {'type': 'v1', 'addr': '172.21.15.72:6837', 'nonce': 1417614203}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.77:6839/922723189', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6838', 'nonce': 922723189}, {'type': 'v1', 'addr': '172.21.15.77:6839', 'nonce': 922723189}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.72:6839/2976050650', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.72:6838', 'nonce': 2976050650}, {'type': 'v1', 'addr': '172.21.15.72:6839', 'nonce': 2976050650}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24494, 'qdb_cluster': [24472, 24494, 24467]} 2024-05-04T03:17:55.665 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-04T03:17:55.666 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph tell mds.h status 2024-05-04T03:17:55.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:55 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/1379598187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:17:55.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:55 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/1379598187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:17:55.915 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:17:56.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:56 smithi001 ceph-mon[29731]: pgmap v354: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-04T03:17:56.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:56 smithi077 ceph-mon[33531]: pgmap v354: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-04T03:17:56.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:56 smithi072 ceph-mon[33332]: pgmap v354: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-04T03:17:57.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.467+0000 7f1a0ba31700 1 -- 172.21.15.1:0/2886349026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a04103080 msgr2=0x7f1a04103490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:57.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.467+0000 7f1a0ba31700 1 --2- 172.21.15.1:0/2886349026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a04103080 0x7f1a04103490 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1a00009990 tx=0x7f1a00009ca0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.467+0000 7f1a0ba31700 1 -- 172.21.15.1:0/2886349026 shutdown_connections 2024-05-04T03:17:57.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.467+0000 7f1a0ba31700 1 --2- 172.21.15.1:0/2886349026 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a04110c30 0x7f1a04113050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.467+0000 7f1a0ba31700 1 --2- 172.21.15.1:0/2886349026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a04103080 0x7f1a04103490 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.467+0000 7f1a0ba31700 1 --2- 172.21.15.1:0/2886349026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a041039d0 0x7f1a041106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.467+0000 7f1a0ba31700 1 -- 172.21.15.1:0/2886349026 >> 172.21.15.1:0/2886349026 conn(0x7f1a040fedf0 msgr2=0x7f1a04101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:57.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.467+0000 7f1a0ba31700 1 -- 172.21.15.1:0/2886349026 shutdown_connections 2024-05-04T03:17:57.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.468+0000 7f1a0ba31700 1 -- 172.21.15.1:0/2886349026 wait complete. 2024-05-04T03:17:57.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.468+0000 7f1a0ba31700 1 Processor -- start 2024-05-04T03:17:57.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.468+0000 7f1a0ba31700 1 -- start start 2024-05-04T03:17:57.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.468+0000 7f1a0ba31700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a04103080 0x7f1a0419cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:57.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.468+0000 7f1a0ba31700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a041039d0 0x7f1a0419d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:57.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.468+0000 7f1a0ba31700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a04110c30 0x7f1a041a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:57.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.468+0000 7f1a0ba31700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1a041166a0 con 0x7f1a04103080 2024-05-04T03:17:57.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.469+0000 7f1a0ba31700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1a04116520 con 0x7f1a04110c30 2024-05-04T03:17:57.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.469+0000 7f1a0ba31700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1a04116820 con 0x7f1a041039d0 2024-05-04T03:17:57.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.469+0000 7f1a097cd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a04103080 0x7f1a0419cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:57.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.469+0000 7f1a097cd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a04103080 0x7f1a0419cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33900/0 (socket says 172.21.15.1:33900) 2024-05-04T03:17:57.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.469+0000 7f1a097cd700 1 -- 172.21.15.1:0/1442540059 learned_addr learned my addr 172.21.15.1:0/1442540059 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:57.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.469+0000 7f1a09fce700 1 --2- 172.21.15.1:0/1442540059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a04110c30 0x7f1a041a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:57.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.469+0000 7f1a08fcc700 1 --2- 172.21.15.1:0/1442540059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a041039d0 0x7f1a0419d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:57.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.469+0000 7f1a097cd700 1 -- 172.21.15.1:0/1442540059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a041039d0 msgr2=0x7f1a0419d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:57.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.469+0000 7f1a097cd700 1 --2- 172.21.15.1:0/1442540059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a041039d0 0x7f1a0419d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.469+0000 7f1a097cd700 1 -- 172.21.15.1:0/1442540059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a04110c30 msgr2=0x7f1a041a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:57.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.469+0000 7f1a097cd700 1 --2- 172.21.15.1:0/1442540059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a04110c30 0x7f1a041a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.469+0000 7f1a097cd700 1 -- 172.21.15.1:0/1442540059 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a041a1ec0 con 0x7f1a04103080 2024-05-04T03:17:57.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.469+0000 7f1a08fcc700 1 --2- 172.21.15.1:0/1442540059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a041039d0 0x7f1a0419d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:17:57.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.469+0000 7f1a097cd700 1 --2- 172.21.15.1:0/1442540059 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a04103080 0x7f1a0419cff0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1a00000c00 tx=0x7f1a000047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:57.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.469+0000 7f19fa7fc700 1 -- 172.21.15.1:0/1442540059 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a00027070 con 0x7f1a04103080 2024-05-04T03:17:57.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.469+0000 7f19fa7fc700 1 -- 172.21.15.1:0/1442540059 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a00003f20 con 0x7f1a04103080 2024-05-04T03:17:57.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.469+0000 7f1a0ba31700 1 -- 172.21.15.1:0/1442540059 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a041a2150 con 0x7f1a04103080 2024-05-04T03:17:57.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.469+0000 7f19fa7fc700 1 -- 172.21.15.1:0/1442540059 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a00022cf0 con 0x7f1a04103080 2024-05-04T03:17:57.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.470+0000 7f1a0ba31700 1 -- 172.21.15.1:0/1442540059 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a041a2630 con 0x7f1a04103080 2024-05-04T03:17:57.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.472+0000 7f19fa7fc700 1 -- 172.21.15.1:0/1442540059 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f1a0002a070 con 0x7f1a04103080 2024-05-04T03:17:57.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.473+0000 7f19fa7fc700 1 --2- 172.21.15.1:0/1442540059 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f19f00b2880 0x7f19f00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:57.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.473+0000 7f1a08fcc700 1 --2- 172.21.15.1:0/1442540059 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f19f00b2880 0x7f19f00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:57.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.473+0000 7f19fa7fc700 1 -- 172.21.15.1:0/1442540059 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f1a000e3010 con 0x7f1a04103080 2024-05-04T03:17:57.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.473+0000 7f1a1a445b80 1 Processor -- start 2024-05-04T03:17:57.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.473+0000 7f1a1a445b80 1 -- start start 2024-05-04T03:17:57.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.474+0000 7f1a08fcc700 1 --2- 172.21.15.1:0/1442540059 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f19f00b2880 0x7f19f00b4d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f19f4005fd0 tx=0x7f19f4005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:57.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.474+0000 7f1a1a445b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598c8a310d0 0x5598c8a314e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:57.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.474+0000 7f1a1a445b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598c8a2b1c0 0x5598c8a2b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:57.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.474+0000 7f1a1a445b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598c8a2bb10 0x5598c8a37db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:57.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.474+0000 7f1a1a445b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5598c894d8f0 con 0x5598c8a310d0 2024-05-04T03:17:57.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.474+0000 7f1a1a445b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5598c88b2d50 con 0x5598c8a2b1c0 2024-05-04T03:17:57.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.474+0000 7f1a1a445b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5598c89498b0 con 0x5598c8a2bb10 2024-05-04T03:17:57.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.474+0000 7f1a097cd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598c8a310d0 0x5598c8a314e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:57.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.474+0000 7f1a097cd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598c8a310d0 0x5598c8a314e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33902/0 (socket says 172.21.15.1:33902) 2024-05-04T03:17:57.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.474+0000 7f1a097cd700 1 -- 172.21.15.1:0/3224336780 learned_addr learned my addr 172.21.15.1:0/3224336780 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:57.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.474+0000 7f1a08fcc700 1 --2- 172.21.15.1:0/3224336780 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598c8a2b1c0 0x5598c8a2b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:57.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.474+0000 7f1a09fce700 1 --2- 172.21.15.1:0/3224336780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598c8a2bb10 0x5598c8a37db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:57.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.474+0000 7f1a097cd700 1 -- 172.21.15.1:0/3224336780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598c8a2bb10 msgr2=0x5598c8a37db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:57.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.474+0000 7f1a097cd700 1 --2- 172.21.15.1:0/3224336780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598c8a2bb10 0x5598c8a37db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.474+0000 7f1a097cd700 1 -- 172.21.15.1:0/3224336780 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598c8a2b1c0 msgr2=0x5598c8a2b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:57.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.474+0000 7f1a097cd700 1 --2- 172.21.15.1:0/3224336780 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598c8a2b1c0 0x5598c8a2b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.474+0000 7f1a097cd700 1 -- 172.21.15.1:0/3224336780 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598c889a880 con 0x5598c8a310d0 2024-05-04T03:17:57.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.474+0000 7f1a097cd700 1 --2- 172.21.15.1:0/3224336780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598c8a310d0 0x5598c8a314e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1a000e0b10 tx=0x7f1a00009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=366e9dc356b56025 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:57.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.474+0000 7f19ef7fe700 1 -- 172.21.15.1:0/3224336780 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a000aa780 con 0x5598c8a310d0 2024-05-04T03:17:57.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.475+0000 7f19ef7fe700 1 -- 172.21.15.1:0/3224336780 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a000aeb30 con 0x5598c8a310d0 2024-05-04T03:17:57.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.475+0000 7f19ef7fe700 1 -- 172.21.15.1:0/3224336780 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a000b3320 con 0x5598c8a310d0 2024-05-04T03:17:57.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.475+0000 7f1a1a445b80 1 -- 172.21.15.1:0/3224336780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598c8a310d0 msgr2=0x5598c8a314e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:57.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.475+0000 7f1a1a445b80 1 --2- 172.21.15.1:0/3224336780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598c8a310d0 0x5598c8a314e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1a000e0b10 tx=0x7f1a00009310 comp rx=0 tx=0).stop 2024-05-04T03:17:57.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.475+0000 7f1a1a445b80 1 -- 172.21.15.1:0/3224336780 shutdown_connections 2024-05-04T03:17:57.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.475+0000 7f1a1a445b80 1 --2- 172.21.15.1:0/3224336780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598c8a2bb10 0x5598c8a37db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.475+0000 7f1a1a445b80 1 --2- 172.21.15.1:0/3224336780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598c8a310d0 0x5598c8a314e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.475+0000 7f1a1a445b80 1 --2- 172.21.15.1:0/3224336780 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598c8a2b1c0 0x5598c8a2b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.475+0000 7f1a1a445b80 1 -- 172.21.15.1:0/3224336780 >> 172.21.15.1:0/3224336780 conn(0x5598c88addf0 msgr2=0x5598c8a26210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:57.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.476+0000 7f1a1a445b80 1 -- 172.21.15.1:0/3224336780 shutdown_connections 2024-05-04T03:17:57.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.476+0000 7f1a1a445b80 1 -- 172.21.15.1:0/3224336780 wait complete. 2024-05-04T03:17:57.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.476+0000 7f1a1a445b80 1 -- start start 2024-05-04T03:17:57.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.476+0000 7f1a1a445b80 1 Processor -- start 2024-05-04T03:17:57.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.476+0000 7f1a1a445b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598c8a2b1c0 0x5598c8a35330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:57.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.476+0000 7f1a1a445b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598c8a2bb10 0x5598c8a35870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:57.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.476+0000 7f1a1a445b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598c8a310d0 0x5598c8a35db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:57.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.476+0000 7f1a1a445b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5598c894d8f0 con 0x5598c8a2b1c0 2024-05-04T03:17:57.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.476+0000 7f1a1a445b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5598c8906e80 con 0x5598c8a2bb10 2024-05-04T03:17:57.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.476+0000 7f1a1a445b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5598c8a382f0 con 0x5598c8a310d0 2024-05-04T03:17:57.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.477+0000 7f1a1a445b80 10 client.0 _ll_register_callbacks cb 0x5598c88e5900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:17:57.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.477+0000 7f1a097cd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598c8a2b1c0 0x5598c8a35330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:57.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.477+0000 7f1a097cd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598c8a2b1c0 0x5598c8a35330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33906/0 (socket says 172.21.15.1:33906) 2024-05-04T03:17:57.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.477+0000 7f1a097cd700 1 -- 172.21.15.1:0/2637021269 learned_addr learned my addr 172.21.15.1:0/2637021269 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:57.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.477+0000 7f1a09fce700 1 --2- 172.21.15.1:0/2637021269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598c8a310d0 0x5598c8a35db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:57.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.477+0000 7f1a08fcc700 1 --2- 172.21.15.1:0/2637021269 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598c8a2bb10 0x5598c8a35870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:57.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.477+0000 7f1a097cd700 1 -- 172.21.15.1:0/2637021269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598c8a310d0 msgr2=0x5598c8a35db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:57.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.477+0000 7f1a097cd700 1 --2- 172.21.15.1:0/2637021269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598c8a310d0 0x5598c8a35db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.477+0000 7f1a097cd700 1 -- 172.21.15.1:0/2637021269 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598c8a2bb10 msgr2=0x5598c8a35870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:57.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.477+0000 7f1a097cd700 1 --2- 172.21.15.1:0/2637021269 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598c8a2bb10 0x5598c8a35870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.477+0000 7f1a097cd700 1 -- 172.21.15.1:0/2637021269 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598c8904710 con 0x5598c8a2b1c0 2024-05-04T03:17:57.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.477+0000 7f1a08fcc700 1 --2- 172.21.15.1:0/2637021269 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598c8a2bb10 0x5598c8a35870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:17:57.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.477+0000 7f1a097cd700 1 --2- 172.21.15.1:0/2637021269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598c8a2b1c0 0x5598c8a35330 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1a000aa710 tx=0x7f1a00022720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916620b8ca46812a server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:57.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.477+0000 7f19ecff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:17:57.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.477+0000 7f19ecff9700 1 -- 172.21.15.1:0/2637021269 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a000b30c0 con 0x5598c8a2b1c0 2024-05-04T03:17:57.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.477+0000 7f19ecff9700 1 -- 172.21.15.1:0/2637021269 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f1a000b3260 con 0x5598c8a2b1c0 2024-05-04T03:17:57.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.477+0000 7f1a1a445b80 1 -- 172.21.15.1:0/2637021269 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a000aeb30 con 0x5598c8a2b1c0 2024-05-04T03:17:57.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.478+0000 7f19ecff9700 1 -- 172.21.15.1:0/2637021269 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1a000920d0 con 0x5598c8a2b1c0 2024-05-04T03:17:57.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.478+0000 7f1a1a445b80 10 client.14931 fetch_fsmap learned FSMap version 25 2024-05-04T03:17:57.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.478+0000 7f1a1a445b80 1 -- 172.21.15.1:0/2637021269 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5598c8a42dc0 con 0x5598c8a2b1c0 2024-05-04T03:17:57.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.478+0000 7f19ecff9700 1 -- 172.21.15.1:0/2637021269 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a000aeb30 con 0x5598c8a2b1c0 2024-05-04T03:17:57.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.478+0000 7f19ecff9700 1 -- 172.21.15.1:0/2637021269 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f1a000a99e0 con 0x5598c8a2b1c0 2024-05-04T03:17:57.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.478+0000 7f1a1a445b80 10 client.14931 fetch_fsmap finished waiting for FSMap version 25 2024-05-04T03:17:57.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.478+0000 7f1a1a445b80 10 client.14931 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-04T03:17:57.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.478+0000 7f1a1a445b80 20 client.14931 populate_metadata read hostname 'smithi001' 2024-05-04T03:17:57.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.478+0000 7f1a1a445b80 1 --2- 172.21.15.1:0/2637021269 >> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] conn(0x5598c8a45240 0x5598c8a466b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:57.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.478+0000 7f1a1a445b80 4 client.14931 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:17:57.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.478+0000 7f1a1a445b80 1 -- 172.21.15.1:0/2637021269 --> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598c895b2d0 con 0x5598c8a45240 2024-05-04T03:17:57.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.478+0000 7f1a08fcc700 1 --2- 172.21.15.1:0/2637021269 >> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] conn(0x5598c8a45240 0x5598c8a466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:57.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.479+0000 7f1a08fcc700 1 --2- 172.21.15.1:0/2637021269 >> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] conn(0x5598c8a45240 0x5598c8a466b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b2d3d83118e055c9 server_cookie=9dca4cdaa1d0b5b4 in_seq=0 out_seq=0 2024-05-04T03:17:57.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.479+0000 7f19ecff9700 10 client.14931 ms_handle_connect on v2:172.21.15.72:6836/1417614203 2024-05-04T03:17:57.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.480+0000 7f19ecff9700 1 -- 172.21.15.1:0/2637021269 <== mds.2 v2:172.21.15.72:6836/1417614203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5598c895b2d0 con 0x5598c8a45240 2024-05-04T03:17:57.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.480+0000 7f19ecff9700 10 client.14931 handle_command_reply: tid=0 2024-05-04T03:17:57.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.480+0000 7f1a1a445b80 1 client.14931 shutdown 2024-05-04T03:17:57.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.480+0000 7f1a1a445b80 1 -- 172.21.15.1:0/2637021269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598c8a2b1c0 msgr2=0x5598c8a35330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:57.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.480+0000 7f1a1a445b80 1 --2- 172.21.15.1:0/2637021269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598c8a2b1c0 0x5598c8a35330 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1a000aa710 tx=0x7f1a00022720 comp rx=0 tx=0).stop 2024-05-04T03:17:57.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.480+0000 7f1a1a445b80 1 -- 172.21.15.1:0/2637021269 shutdown_connections 2024-05-04T03:17:57.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.480+0000 7f1a1a445b80 1 --2- 172.21.15.1:0/2637021269 >> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] conn(0x5598c8a45240 0x5598c8a466b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.480+0000 7f1a1a445b80 1 --2- 172.21.15.1:0/2637021269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598c8a310d0 0x5598c8a35db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.480+0000 7f1a1a445b80 1 --2- 172.21.15.1:0/2637021269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598c8a2b1c0 0x5598c8a35330 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.480+0000 7f1a1a445b80 1 --2- 172.21.15.1:0/2637021269 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598c8a2bb10 0x5598c8a35870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.480+0000 7f1a1a445b80 1 -- 172.21.15.1:0/2637021269 >> 172.21.15.1:0/2637021269 conn(0x5598c88addf0 msgr2=0x5598c8a2a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:57.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.480+0000 7f19ecff9700 0 client.14931 ms_handle_reset on v2:172.21.15.72:6836/1417614203 2024-05-04T03:17:57.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.480+0000 7f1a1a445b80 1 -- 172.21.15.1:0/2637021269 shutdown_connections 2024-05-04T03:17:57.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.480+0000 7f1a1a445b80 1 -- 172.21.15.1:0/2637021269 wait complete. 2024-05-04T03:17:57.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.480+0000 7f1a1a445b80 20 client.14931 trim_cache size 0 max 16384 2024-05-04T03:17:57.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.501+0000 7f1a1a445b80 1 Processor -- start 2024-05-04T03:17:57.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.501+0000 7f1a1a445b80 1 -- start start 2024-05-04T03:17:57.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.501+0000 7f1a1a445b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598c8a2b1c0 0x5598c8a2f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:57.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.501+0000 7f1a1a445b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598c8a2bb10 0x5598c8a205f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:57.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.501+0000 7f1a1a445b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598c8a310d0 0x5598c8a2a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:57.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.501+0000 7f1a1a445b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5598c8a449f0 con 0x5598c8a310d0 2024-05-04T03:17:57.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.501+0000 7f1a1a445b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5598c8a450c0 con 0x5598c8a2bb10 2024-05-04T03:17:57.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.501+0000 7f1a1a445b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5598c8a45790 con 0x5598c8a2b1c0 2024-05-04T03:17:57.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.501+0000 7f1a09fce700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598c8a310d0 0x5598c8a2a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:57.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.502+0000 7f1a09fce700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598c8a310d0 0x5598c8a2a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33916/0 (socket says 172.21.15.1:33916) 2024-05-04T03:17:57.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.502+0000 7f1a09fce700 1 -- 172.21.15.1:0/566550325 learned_addr learned my addr 172.21.15.1:0/566550325 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:57.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.502+0000 7f1a097cd700 1 --2- 172.21.15.1:0/566550325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598c8a2b1c0 0x5598c8a2f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:57.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.502+0000 7f1a08fcc700 1 --2- 172.21.15.1:0/566550325 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598c8a2bb10 0x5598c8a205f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:57.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.502+0000 7f1a09fce700 1 -- 172.21.15.1:0/566550325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598c8a2b1c0 msgr2=0x5598c8a2f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:57.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.502+0000 7f1a09fce700 1 --2- 172.21.15.1:0/566550325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598c8a2b1c0 0x5598c8a2f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.502+0000 7f1a09fce700 1 -- 172.21.15.1:0/566550325 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598c8a2bb10 msgr2=0x5598c8a205f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:57.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.502+0000 7f1a09fce700 1 --2- 172.21.15.1:0/566550325 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598c8a2bb10 0x5598c8a205f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.502+0000 7f1a09fce700 1 -- 172.21.15.1:0/566550325 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598c8a22760 con 0x5598c8a310d0 2024-05-04T03:17:57.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.502+0000 7f1a09fce700 1 --2- 172.21.15.1:0/566550325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598c8a310d0 0x5598c8a2a8c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f19fc009940 tx=0x7f19fc009100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ee0e40201d91ca server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:57.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.502+0000 7f19eeffd700 1 -- 172.21.15.1:0/566550325 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc01b030 con 0x5598c8a310d0 2024-05-04T03:17:57.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.502+0000 7f19eeffd700 1 -- 172.21.15.1:0/566550325 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19fc004bb0 con 0x5598c8a310d0 2024-05-04T03:17:57.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.502+0000 7f19eeffd700 1 -- 172.21.15.1:0/566550325 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc007a70 con 0x5598c8a310d0 2024-05-04T03:17:57.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.503+0000 7f1a1a445b80 1 -- 172.21.15.1:0/566550325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598c8a310d0 msgr2=0x5598c8a2a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:57.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.503+0000 7f1a1a445b80 1 --2- 172.21.15.1:0/566550325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598c8a310d0 0x5598c8a2a8c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f19fc009940 tx=0x7f19fc009100 comp rx=0 tx=0).stop 2024-05-04T03:17:57.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.503+0000 7f1a1a445b80 1 -- 172.21.15.1:0/566550325 shutdown_connections 2024-05-04T03:17:57.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.503+0000 7f1a1a445b80 1 --2- 172.21.15.1:0/566550325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598c8a310d0 0x5598c8a2a8c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.503+0000 7f1a1a445b80 1 --2- 172.21.15.1:0/566550325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598c8a2b1c0 0x5598c8a2f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.503+0000 7f1a1a445b80 1 --2- 172.21.15.1:0/566550325 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598c8a2bb10 0x5598c8a205f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.503+0000 7f1a1a445b80 1 -- 172.21.15.1:0/566550325 >> 172.21.15.1:0/566550325 conn(0x5598c88addf0 msgr2=0x5598c8a2a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:57.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.503+0000 7f1a1a445b80 1 -- 172.21.15.1:0/566550325 shutdown_connections 2024-05-04T03:17:57.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.503+0000 7f1a1a445b80 1 -- 172.21.15.1:0/566550325 wait complete. 2024-05-04T03:17:57.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.503+0000 7f1a1a445b80 1 -- start start 2024-05-04T03:17:57.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.504+0000 7f1a1a445b80 1 Processor -- start 2024-05-04T03:17:57.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.504+0000 7f1a1a445b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598c8a2b1c0 0x5598c8a32330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:57.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.504+0000 7f1a1a445b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598c8a2bb10 0x5598c8a32870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:57.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.504+0000 7f1a1a445b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598c8a310d0 0x5598c8a32db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:57.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.504+0000 7f1a1a445b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5598c8a46530 con 0x5598c8a2b1c0 2024-05-04T03:17:57.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.504+0000 7f1a1a445b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5598c8a45e60 con 0x5598c8a310d0 2024-05-04T03:17:57.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.504+0000 7f1a1a445b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5598c8a46c00 con 0x5598c8a2bb10 2024-05-04T03:17:57.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.504+0000 7f1a1a445b80 10 client.0 _ll_register_callbacks cb 0x5598c88e5900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:17:57.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.504+0000 7f1a097cd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598c8a2b1c0 0x5598c8a32330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:57.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.504+0000 7f1a097cd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598c8a2b1c0 0x5598c8a32330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33930/0 (socket says 172.21.15.1:33930) 2024-05-04T03:17:57.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.504+0000 7f1a097cd700 1 -- 172.21.15.1:0/2374516634 learned_addr learned my addr 172.21.15.1:0/2374516634 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:17:57.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.504+0000 7f1a08fcc700 1 --2- 172.21.15.1:0/2374516634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598c8a2bb10 0x5598c8a32870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:57.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.504+0000 7f1a097cd700 1 -- 172.21.15.1:0/2374516634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598c8a2bb10 msgr2=0x5598c8a32870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:57.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.504+0000 7f1a097cd700 1 --2- 172.21.15.1:0/2374516634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598c8a2bb10 0x5598c8a32870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.504+0000 7f1a097cd700 1 -- 172.21.15.1:0/2374516634 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598c8a310d0 msgr2=0x5598c8a32db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:17:57.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.504+0000 7f1a097cd700 1 --2- 172.21.15.1:0/2374516634 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598c8a310d0 0x5598c8a32db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.504+0000 7f1a097cd700 1 -- 172.21.15.1:0/2374516634 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598c8a278a0 con 0x5598c8a2b1c0 2024-05-04T03:17:57.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.505+0000 7f1a097cd700 1 --2- 172.21.15.1:0/2374516634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598c8a2b1c0 0x5598c8a32330 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1a000aa480 tx=0x7f1a00009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9a727c813b8097 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:17:57.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.505+0000 7f19ecff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:17:57.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.505+0000 7f19ecff9700 1 -- 172.21.15.1:0/2374516634 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a000aa550 con 0x5598c8a2b1c0 2024-05-04T03:17:57.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.505+0000 7f1a1a445b80 1 -- 172.21.15.1:0/2374516634 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19fc004bb0 con 0x5598c8a2b1c0 2024-05-04T03:17:57.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.505+0000 7f19ecff9700 1 -- 172.21.15.1:0/2374516634 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f1a000a6350 con 0x5598c8a2b1c0 2024-05-04T03:17:57.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.505+0000 7f19ecff9700 1 -- 172.21.15.1:0/2374516634 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1a000aaec0 con 0x5598c8a2b1c0 2024-05-04T03:17:57.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.505+0000 7f1a1a445b80 10 client.14937 fetch_fsmap learned FSMap version 25 2024-05-04T03:17:57.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.505+0000 7f1a1a445b80 1 -- 172.21.15.1:0/2374516634 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5598c8a278a0 con 0x5598c8a2b1c0 2024-05-04T03:17:57.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.505+0000 7f19ecff9700 1 -- 172.21.15.1:0/2374516634 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19fc004bb0 con 0x5598c8a2b1c0 2024-05-04T03:17:57.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.505+0000 7f19ecff9700 1 -- 172.21.15.1:0/2374516634 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f1a000cb400 con 0x5598c8a2b1c0 2024-05-04T03:17:57.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.505+0000 7f1a1a445b80 10 client.14937 fetch_fsmap finished waiting for FSMap version 25 2024-05-04T03:17:57.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.505+0000 7f1a1a445b80 10 client.14937 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-04T03:17:57.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.505+0000 7f1a1a445b80 20 client.14937 populate_metadata read hostname 'smithi001' 2024-05-04T03:17:57.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.505+0000 7f1a1a445b80 1 --2- 172.21.15.1:0/2374516634 >> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] conn(0x5598c8ab29e0 0x5598c8ab3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:17:57.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.506+0000 7f1a1a445b80 4 client.14937 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-04T03:17:57.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.506+0000 7f1a1a445b80 1 -- 172.21.15.1:0/2374516634 --> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] -- command(tid 0: {"prefix": "status"}) -- 0x5598c8955370 con 0x5598c8ab29e0 2024-05-04T03:17:57.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.506+0000 7f1a08fcc700 1 --2- 172.21.15.1:0/2374516634 >> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] conn(0x5598c8ab29e0 0x5598c8ab3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:17:57.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.506+0000 7f19ecff9700 10 client.14937 ms_handle_connect on v2:172.21.15.72:6836/1417614203 2024-05-04T03:17:57.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.506+0000 7f1a08fcc700 1 --2- 172.21.15.1:0/2374516634 >> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] conn(0x5598c8ab29e0 0x5598c8ab3e50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2984ea7b04696671 server_cookie=86684c8b39539697 in_seq=0 out_seq=0 2024-05-04T03:17:57.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.506+0000 7f19ecff9700 1 -- 172.21.15.1:0/2374516634 <== mds.2 v2:172.21.15.72:6836/1417614203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5598c8955370 con 0x5598c8ab29e0 2024-05-04T03:17:57.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.506+0000 7f19ecff9700 10 client.14937 handle_command_reply: tid=0 2024-05-04T03:17:57.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.506+0000 7f1a1a445b80 1 client.14937 shutdown 2024-05-04T03:17:57.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.506+0000 7f1a1a445b80 1 -- 172.21.15.1:0/2374516634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598c8a2b1c0 msgr2=0x5598c8a32330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:57.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.506+0000 7f1a1a445b80 1 --2- 172.21.15.1:0/2374516634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598c8a2b1c0 0x5598c8a32330 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1a000aa480 tx=0x7f1a00009ca0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.506+0000 7f1a1a445b80 1 -- 172.21.15.1:0/2374516634 shutdown_connections 2024-05-04T03:17:57.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.507+0000 7f1a1a445b80 1 --2- 172.21.15.1:0/2374516634 >> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] conn(0x5598c8ab29e0 0x5598c8ab3e50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.507+0000 7f1a1a445b80 1 --2- 172.21.15.1:0/2374516634 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598c8a310d0 0x5598c8a32db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.507+0000 7f1a1a445b80 1 --2- 172.21.15.1:0/2374516634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5598c8a2b1c0 0x5598c8a32330 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.507+0000 7f1a1a445b80 1 --2- 172.21.15.1:0/2374516634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5598c8a2bb10 0x5598c8a32870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.507+0000 7f19ecff9700 0 client.14937 ms_handle_reset on v2:172.21.15.72:6836/1417614203 2024-05-04T03:17:57.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.507+0000 7f1a1a445b80 1 -- 172.21.15.1:0/2374516634 >> 172.21.15.1:0/2374516634 conn(0x5598c88addf0 msgr2=0x5598c8a36630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:57.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.507+0000 7f1a1a445b80 1 -- 172.21.15.1:0/2374516634 shutdown_connections 2024-05-04T03:17:57.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.507+0000 7f1a1a445b80 1 -- 172.21.15.1:0/2374516634 wait complete. 2024-05-04T03:17:57.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.507+0000 7f1a1a445b80 20 client.14937 trim_cache size 0 max 16384 2024-05-04T03:17:57.516 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-04T03:17:57.517 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "4a572aa8-09c3-11ef-bc97-c7b262605968", 2024-05-04T03:17:57.517 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 2, 2024-05-04T03:17:57.517 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24467, 2024-05-04T03:17:57.517 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-05-04T03:17:57.517 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-05-04T03:17:57.517 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-05-04T03:17:57.517 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 114.273230361, 2024-05-04T03:17:57.517 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 25, 2024-05-04T03:17:57.517 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 83, 2024-05-04T03:17:57.517 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 81, 2024-05-04T03:17:57.517 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 155.94826306600001 2024-05-04T03:17:57.517 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-04T03:17:57.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.507+0000 7f1a0ba31700 1 -- 172.21.15.1:0/1442540059 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f19f00b2880 msgr2=0x7f19f00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:57.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.507+0000 7f1a0ba31700 1 --2- 172.21.15.1:0/1442540059 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f19f00b2880 0x7f19f00b4d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f19f4005fd0 tx=0x7f19f4005dc0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.507+0000 7f1a0ba31700 1 -- 172.21.15.1:0/1442540059 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a04103080 msgr2=0x7f1a0419cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:17:57.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.507+0000 7f1a0ba31700 1 --2- 172.21.15.1:0/1442540059 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a04103080 0x7f1a0419cff0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1a00000c00 tx=0x7f1a000047b0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.507+0000 7f1a0ba31700 1 -- 172.21.15.1:0/1442540059 shutdown_connections 2024-05-04T03:17:57.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.507+0000 7f1a0ba31700 1 --2- 172.21.15.1:0/1442540059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1a04110c30 0x7f1a041a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.507+0000 7f1a0ba31700 1 --2- 172.21.15.1:0/1442540059 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f19f00b2880 0x7f19f00b4d90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.507+0000 7f1a0ba31700 1 --2- 172.21.15.1:0/1442540059 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a04103080 0x7f1a0419cff0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.507+0000 7f1a0ba31700 1 --2- 172.21.15.1:0/1442540059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a041039d0 0x7f1a0419d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:17:57.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.507+0000 7f1a0ba31700 1 -- 172.21.15.1:0/1442540059 >> 172.21.15.1:0/1442540059 conn(0x7f1a040fedf0 msgr2=0x7f1a04112730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:17:57.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.507+0000 7f1a0ba31700 1 -- 172.21.15.1:0/1442540059 shutdown_connections 2024-05-04T03:17:57.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:17:57.507+0000 7f1a0ba31700 1 -- 172.21.15.1:0/1442540059 wait complete. 2024-05-04T03:17:58.245 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-04T03:17:58.494 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:17:58.600 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:17:58 smithi001 ceph-mon[29731]: pgmap v355: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-04T03:17:58.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:17:58 smithi077 ceph-mon[33531]: pgmap v355: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-04T03:17:58.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:17:58 smithi072 ceph-mon[33332]: pgmap v355: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-04T03:18:00.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.064+0000 7fcc76c35700 1 -- 172.21.15.1:0/1820175027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc70107df0 msgr2=0x7fcc70108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:00.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.064+0000 7fcc76c35700 1 --2- 172.21.15.1:0/1820175027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc70107df0 0x7fcc70108200 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c0099c0 tx=0x7fcc5c009cd0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.064+0000 7fcc76c35700 1 -- 172.21.15.1:0/1820175027 shutdown_connections 2024-05-04T03:18:00.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.064+0000 7fcc76c35700 1 --2- 172.21.15.1:0/1820175027 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc701034c0 0x7fcc701038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.064+0000 7fcc76c35700 1 --2- 172.21.15.1:0/1820175027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc70108740 0x7fcc701100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.064+0000 7fcc76c35700 1 --2- 172.21.15.1:0/1820175027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc70107df0 0x7fcc70108200 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.064+0000 7fcc76c35700 1 -- 172.21.15.1:0/1820175027 >> 172.21.15.1:0/1820175027 conn(0x7fcc700fedf0 msgr2=0x7fcc70101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:00.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.064+0000 7fcc76c35700 1 -- 172.21.15.1:0/1820175027 shutdown_connections 2024-05-04T03:18:00.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.064+0000 7fcc76c35700 1 -- 172.21.15.1:0/1820175027 wait complete. 2024-05-04T03:18:00.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.065+0000 7fcc76c35700 1 Processor -- start 2024-05-04T03:18:00.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.065+0000 7fcc76c35700 1 -- start start 2024-05-04T03:18:00.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.065+0000 7fcc76c35700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc701034c0 0x7fcc7019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:00.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.065+0000 7fcc76c35700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc70107df0 0x7fcc7019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:00.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.065+0000 7fcc76c35700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc70108740 0x7fcc7019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:00.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.065+0000 7fcc76c35700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcc701144c0 con 0x7fcc70108740 2024-05-04T03:18:00.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.065+0000 7fcc76c35700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fcc70114340 con 0x7fcc701034c0 2024-05-04T03:18:00.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.065+0000 7fcc76c35700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcc70114640 con 0x7fcc70107df0 2024-05-04T03:18:00.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.066+0000 7fcc751d2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc70108740 0x7fcc7019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:00.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.066+0000 7fcc751d2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc70108740 0x7fcc7019f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33958/0 (socket says 172.21.15.1:33958) 2024-05-04T03:18:00.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.066+0000 7fcc751d2700 1 -- 172.21.15.1:0/464882239 learned_addr learned my addr 172.21.15.1:0/464882239 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:00.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.066+0000 7fcc6ffff700 1 --2- 172.21.15.1:0/464882239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc70107df0 0x7fcc7019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:00.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.066+0000 7fcc749d1700 1 --2- 172.21.15.1:0/464882239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc701034c0 0x7fcc7019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:00.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.066+0000 7fcc751d2700 1 -- 172.21.15.1:0/464882239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc70107df0 msgr2=0x7fcc7019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:00.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.066+0000 7fcc751d2700 1 --2- 172.21.15.1:0/464882239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc70107df0 0x7fcc7019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.066+0000 7fcc751d2700 1 -- 172.21.15.1:0/464882239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc701034c0 msgr2=0x7fcc7019ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:00.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.066+0000 7fcc751d2700 1 --2- 172.21.15.1:0/464882239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc701034c0 0x7fcc7019ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.066+0000 7fcc751d2700 1 -- 172.21.15.1:0/464882239 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7019fce0 con 0x7fcc70108740 2024-05-04T03:18:00.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.066+0000 7fcc749d1700 1 --2- 172.21.15.1:0/464882239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc701034c0 0x7fcc7019ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:18:00.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.066+0000 7fcc751d2700 1 --2- 172.21.15.1:0/464882239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc70108740 0x7fcc7019f640 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcc6800ada0 tx=0x7fcc6800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:00.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.066+0000 7fcc6dffb700 1 -- 172.21.15.1:0/464882239 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc68017020 con 0x7fcc70108740 2024-05-04T03:18:00.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.066+0000 7fcc6dffb700 1 -- 172.21.15.1:0/464882239 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc6800ce30 con 0x7fcc70108740 2024-05-04T03:18:00.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.066+0000 7fcc6dffb700 1 -- 172.21.15.1:0/464882239 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc68005350 con 0x7fcc70108740 2024-05-04T03:18:00.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.066+0000 7fcc76c35700 1 -- 172.21.15.1:0/464882239 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc7019ffd0 con 0x7fcc70108740 2024-05-04T03:18:00.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.067+0000 7fcc76c35700 1 -- 172.21.15.1:0/464882239 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc701a78f0 con 0x7fcc70108740 2024-05-04T03:18:00.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.070+0000 7fcc6dffb700 1 -- 172.21.15.1:0/464882239 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fcc6800c980 con 0x7fcc70108740 2024-05-04T03:18:00.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.071+0000 7fcc6dffb700 1 --2- 172.21.15.1:0/464882239 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fcc600b2880 0x7fcc600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:00.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.071+0000 7fcc749d1700 1 --2- 172.21.15.1:0/464882239 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fcc600b2880 0x7fcc600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:00.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.071+0000 7fcc6dffb700 1 -- 172.21.15.1:0/464882239 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fcc680e1900 con 0x7fcc70108740 2024-05-04T03:18:00.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.071+0000 7fcc85649b80 1 Processor -- start 2024-05-04T03:18:00.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.071+0000 7fcc85649b80 1 -- start start 2024-05-04T03:18:00.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.071+0000 7fcc749d1700 1 --2- 172.21.15.1:0/464882239 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fcc600b2880 0x7fcc600b4d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcc64005950 tx=0x7fcc64005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:00.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.071+0000 7fcc85649b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b6492320d0 0x55b6492324e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:00.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.071+0000 7fcc85649b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b64922c1c0 0x55b64922c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:00.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.071+0000 7fcc85649b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b64922cb10 0x55b649238db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:00.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.071+0000 7fcc85649b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b64914e8f0 con 0x55b6492320d0 2024-05-04T03:18:00.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.071+0000 7fcc85649b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55b6490b3d50 con 0x55b64922c1c0 2024-05-04T03:18:00.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.071+0000 7fcc85649b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b64914a8b0 con 0x55b64922cb10 2024-05-04T03:18:00.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.072+0000 7fcc749d1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b6492320d0 0x55b6492324e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:00.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.072+0000 7fcc749d1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b6492320d0 0x55b6492324e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33972/0 (socket says 172.21.15.1:33972) 2024-05-04T03:18:00.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.072+0000 7fcc749d1700 1 -- 172.21.15.1:0/2316530740 learned_addr learned my addr 172.21.15.1:0/2316530740 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:00.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.072+0000 7fcc6ffff700 1 --2- 172.21.15.1:0/2316530740 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b64922c1c0 0x55b64922c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:00.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.072+0000 7fcc751d2700 1 --2- 172.21.15.1:0/2316530740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b64922cb10 0x55b649238db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:00.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.072+0000 7fcc749d1700 1 -- 172.21.15.1:0/2316530740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b64922cb10 msgr2=0x55b649238db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:00.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.072+0000 7fcc749d1700 1 --2- 172.21.15.1:0/2316530740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b64922cb10 0x55b649238db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.072+0000 7fcc749d1700 1 -- 172.21.15.1:0/2316530740 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b64922c1c0 msgr2=0x55b64922c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:00.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.072+0000 7fcc749d1700 1 --2- 172.21.15.1:0/2316530740 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b64922c1c0 0x55b64922c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.072+0000 7fcc749d1700 1 -- 172.21.15.1:0/2316530740 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b64909b880 con 0x55b6492320d0 2024-05-04T03:18:00.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.072+0000 7fcc749d1700 1 --2- 172.21.15.1:0/2316530740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b6492320d0 0x55b6492324e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fcc6400ed30 tx=0x7fcc64011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673ef95b91cf9919 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:00.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.072+0000 7fcc5affd700 1 -- 172.21.15.1:0/2316530740 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc64021070 con 0x55b6492320d0 2024-05-04T03:18:00.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.072+0000 7fcc5affd700 1 -- 172.21.15.1:0/2316530740 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc64004ce0 con 0x55b6492320d0 2024-05-04T03:18:00.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.072+0000 7fcc5affd700 1 -- 172.21.15.1:0/2316530740 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc64005650 con 0x55b6492320d0 2024-05-04T03:18:00.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.073+0000 7fcc85649b80 1 -- 172.21.15.1:0/2316530740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b6492320d0 msgr2=0x55b6492324e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:00.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.073+0000 7fcc85649b80 1 --2- 172.21.15.1:0/2316530740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b6492320d0 0x55b6492324e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fcc6400ed30 tx=0x7fcc64011000 comp rx=0 tx=0).stop 2024-05-04T03:18:00.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.073+0000 7fcc85649b80 1 -- 172.21.15.1:0/2316530740 shutdown_connections 2024-05-04T03:18:00.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.073+0000 7fcc85649b80 1 --2- 172.21.15.1:0/2316530740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b64922cb10 0x55b649238db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.073+0000 7fcc85649b80 1 --2- 172.21.15.1:0/2316530740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b6492320d0 0x55b6492324e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.073+0000 7fcc85649b80 1 --2- 172.21.15.1:0/2316530740 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b64922c1c0 0x55b64922c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.073+0000 7fcc85649b80 1 -- 172.21.15.1:0/2316530740 >> 172.21.15.1:0/2316530740 conn(0x55b6490aedf0 msgr2=0x55b649227210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:00.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.073+0000 7fcc85649b80 1 -- 172.21.15.1:0/2316530740 shutdown_connections 2024-05-04T03:18:00.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.073+0000 7fcc85649b80 1 -- 172.21.15.1:0/2316530740 wait complete. 2024-05-04T03:18:00.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.074+0000 7fcc85649b80 1 -- start start 2024-05-04T03:18:00.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.074+0000 7fcc85649b80 1 Processor -- start 2024-05-04T03:18:00.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.074+0000 7fcc85649b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b64922c1c0 0x55b6492408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:00.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.074+0000 7fcc85649b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b64922cb10 0x55b649240df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:00.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.074+0000 7fcc85649b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b6492320d0 0x55b64923acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:00.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.074+0000 7fcc85649b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b6492394a0 con 0x55b6492320d0 2024-05-04T03:18:00.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.074+0000 7fcc85649b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55b649107e80 con 0x55b64922c1c0 2024-05-04T03:18:00.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.074+0000 7fcc85649b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b649239620 con 0x55b64922cb10 2024-05-04T03:18:00.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.074+0000 7fcc85649b80 10 client.0 _ll_register_callbacks cb 0x55b6490e6900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:18:00.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.075+0000 7fcc751d2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b6492320d0 0x55b64923acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:00.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.075+0000 7fcc751d2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b6492320d0 0x55b64923acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33974/0 (socket says 172.21.15.1:33974) 2024-05-04T03:18:00.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.075+0000 7fcc751d2700 1 -- 172.21.15.1:0/3574773394 learned_addr learned my addr 172.21.15.1:0/3574773394 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:00.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.075+0000 7fcc749d1700 1 --2- 172.21.15.1:0/3574773394 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b64922c1c0 0x55b6492408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:00.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.075+0000 7fcc6ffff700 1 --2- 172.21.15.1:0/3574773394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b64922cb10 0x55b649240df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:00.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.075+0000 7fcc751d2700 1 -- 172.21.15.1:0/3574773394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b64922cb10 msgr2=0x55b649240df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:00.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.075+0000 7fcc751d2700 1 --2- 172.21.15.1:0/3574773394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b64922cb10 0x55b649240df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.075+0000 7fcc751d2700 1 -- 172.21.15.1:0/3574773394 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b64922c1c0 msgr2=0x55b6492408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:00.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.075+0000 7fcc751d2700 1 --2- 172.21.15.1:0/3574773394 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b64922c1c0 0x55b6492408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.075+0000 7fcc751d2700 1 -- 172.21.15.1:0/3574773394 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b649105710 con 0x55b6492320d0 2024-05-04T03:18:00.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.075+0000 7fcc6ffff700 1 --2- 172.21.15.1:0/3574773394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b64922cb10 0x55b649240df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:18:00.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.075+0000 7fcc749d1700 1 --2- 172.21.15.1:0/3574773394 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b64922c1c0 0x55b6492408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:18:00.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.075+0000 7fcc751d2700 1 --2- 172.21.15.1:0/3574773394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b6492320d0 0x55b64923acb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcc6800c730 tx=0x7fcc6800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e821355c17eb40d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:00.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.075+0000 7fcc4ffff700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:18:00.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.075+0000 7fcc4ffff700 1 -- 172.21.15.1:0/3574773394 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc680a96d0 con 0x55b6492320d0 2024-05-04T03:18:00.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.075+0000 7fcc4ffff700 1 -- 172.21.15.1:0/3574773394 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fcc680b1910 con 0x55b6492320d0 2024-05-04T03:18:00.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.075+0000 7fcc85649b80 1 -- 172.21.15.1:0/3574773394 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc64004ce0 con 0x55b6492320d0 2024-05-04T03:18:00.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.076+0000 7fcc4ffff700 1 -- 172.21.15.1:0/3574773394 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcc6806e030 con 0x55b6492320d0 2024-05-04T03:18:00.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.076+0000 7fcc85649b80 10 client.14949 fetch_fsmap learned FSMap version 25 2024-05-04T03:18:00.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.076+0000 7fcc85649b80 1 -- 172.21.15.1:0/3574773394 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b649248580 con 0x55b6492320d0 2024-05-04T03:18:00.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.076+0000 7fcc4ffff700 1 -- 172.21.15.1:0/3574773394 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc64004ce0 con 0x55b6492320d0 2024-05-04T03:18:00.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.076+0000 7fcc4ffff700 1 -- 172.21.15.1:0/3574773394 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fcc68012070 con 0x55b6492320d0 2024-05-04T03:18:00.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.076+0000 7fcc85649b80 10 client.14949 fetch_fsmap finished waiting for FSMap version 25 2024-05-04T03:18:00.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.076+0000 7fcc85649b80 10 client.14949 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-04T03:18:00.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.076+0000 7fcc85649b80 20 client.14949 populate_metadata read hostname 'smithi001' 2024-05-04T03:18:00.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.076+0000 7fcc85649b80 1 --2- 172.21.15.1:0/3574773394 >> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] conn(0x55b64924abe0 0x55b64924c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:00.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.076+0000 7fcc85649b80 4 client.14949 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:18:00.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.076+0000 7fcc85649b80 1 -- 172.21.15.1:0/3574773394 --> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b64915c2d0 con 0x55b64924abe0 2024-05-04T03:18:00.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.077+0000 7fcc749d1700 1 --2- 172.21.15.1:0/3574773394 >> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] conn(0x55b64924abe0 0x55b64924c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:00.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.077+0000 7fcc4ffff700 10 client.14949 ms_handle_connect on v2:172.21.15.77:6838/922723189 2024-05-04T03:18:00.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.077+0000 7fcc749d1700 1 --2- 172.21.15.1:0/3574773394 >> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] conn(0x55b64924abe0 0x55b64924c050 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=805c2b64c4db47c2 server_cookie=4f89e56f8e45b8ea in_seq=0 out_seq=0 2024-05-04T03:18:00.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.078+0000 7fcc4ffff700 1 -- 172.21.15.1:0/3574773394 <== mds.1 v2:172.21.15.77:6838/922723189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b64915c2d0 con 0x55b64924abe0 2024-05-04T03:18:00.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.078+0000 7fcc4ffff700 10 client.14949 handle_command_reply: tid=0 2024-05-04T03:18:00.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.078+0000 7fcc85649b80 1 client.14949 shutdown 2024-05-04T03:18:00.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.078+0000 7fcc85649b80 1 -- 172.21.15.1:0/3574773394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b6492320d0 msgr2=0x55b64923acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:00.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.078+0000 7fcc85649b80 1 --2- 172.21.15.1:0/3574773394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b6492320d0 0x55b64923acb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcc6800c730 tx=0x7fcc6800cdb0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.078+0000 7fcc85649b80 1 -- 172.21.15.1:0/3574773394 shutdown_connections 2024-05-04T03:18:00.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.078+0000 7fcc85649b80 1 --2- 172.21.15.1:0/3574773394 >> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] conn(0x55b64924abe0 0x55b64924c050 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.078+0000 7fcc85649b80 1 --2- 172.21.15.1:0/3574773394 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b64922c1c0 0x55b6492408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.078+0000 7fcc85649b80 1 --2- 172.21.15.1:0/3574773394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b6492320d0 0x55b64923acb0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.078+0000 7fcc85649b80 1 --2- 172.21.15.1:0/3574773394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b64922cb10 0x55b649240df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.078+0000 7fcc4ffff700 0 client.14949 ms_handle_reset on v2:172.21.15.77:6838/922723189 2024-05-04T03:18:00.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.078+0000 7fcc85649b80 1 -- 172.21.15.1:0/3574773394 >> 172.21.15.1:0/3574773394 conn(0x55b6490aedf0 msgr2=0x55b6492291a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:00.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.078+0000 7fcc85649b80 1 -- 172.21.15.1:0/3574773394 shutdown_connections 2024-05-04T03:18:00.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.078+0000 7fcc85649b80 1 -- 172.21.15.1:0/3574773394 wait complete. 2024-05-04T03:18:00.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.078+0000 7fcc85649b80 20 client.14949 trim_cache size 0 max 16384 2024-05-04T03:18:00.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.103+0000 7fcc85649b80 1 Processor -- start 2024-05-04T03:18:00.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.103+0000 7fcc85649b80 1 -- start start 2024-05-04T03:18:00.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.103+0000 7fcc85649b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b64922c1c0 0x55b64922b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:00.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.103+0000 7fcc85649b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b64922cb10 0x55b6492215f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:00.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.103+0000 7fcc85649b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b6492320d0 0x55b6492554b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:00.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.103+0000 7fcc85649b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b64924a390 con 0x55b64922cb10 2024-05-04T03:18:00.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.103+0000 7fcc85649b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55b64924aa60 con 0x55b6492320d0 2024-05-04T03:18:00.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.103+0000 7fcc85649b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b64924b130 con 0x55b64922c1c0 2024-05-04T03:18:00.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.103+0000 7fcc6ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b64922cb10 0x55b6492215f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:00.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.103+0000 7fcc749d1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b64922c1c0 0x55b64922b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:00.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.103+0000 7fcc6ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b64922cb10 0x55b6492215f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33980/0 (socket says 172.21.15.1:33980) 2024-05-04T03:18:00.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.103+0000 7fcc6ffff700 1 -- 172.21.15.1:0/1222192966 learned_addr learned my addr 172.21.15.1:0/1222192966 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:00.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.103+0000 7fcc751d2700 1 --2- 172.21.15.1:0/1222192966 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b6492320d0 0x55b6492554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:00.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.104+0000 7fcc6ffff700 1 -- 172.21.15.1:0/1222192966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b64922c1c0 msgr2=0x55b64922b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:00.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.104+0000 7fcc6ffff700 1 --2- 172.21.15.1:0/1222192966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b64922c1c0 0x55b64922b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.104+0000 7fcc6ffff700 1 -- 172.21.15.1:0/1222192966 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b6492320d0 msgr2=0x55b6492554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:00.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.104+0000 7fcc6ffff700 1 --2- 172.21.15.1:0/1222192966 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b6492320d0 0x55b6492554b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.104+0000 7fcc6ffff700 1 -- 172.21.15.1:0/1222192966 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b64922be00 con 0x55b64922cb10 2024-05-04T03:18:00.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.104+0000 7fcc751d2700 1 --2- 172.21.15.1:0/1222192966 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b6492320d0 0x55b6492554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:18:00.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.104+0000 7fcc6ffff700 1 --2- 172.21.15.1:0/1222192966 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b64922cb10 0x55b6492215f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcc70076b40 tx=0x7fcc5c0039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d677f7e4628b0208 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:00.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.104+0000 7fcc5a7fc700 1 -- 172.21.15.1:0/1222192966 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c027020 con 0x55b64922cb10 2024-05-04T03:18:00.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.104+0000 7fcc5a7fc700 1 -- 172.21.15.1:0/1222192966 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc5c003d60 con 0x55b64922cb10 2024-05-04T03:18:00.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.104+0000 7fcc5a7fc700 1 -- 172.21.15.1:0/1222192966 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c00f460 con 0x55b64922cb10 2024-05-04T03:18:00.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.105+0000 7fcc85649b80 1 -- 172.21.15.1:0/1222192966 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b64922cb10 msgr2=0x55b6492215f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:00.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.105+0000 7fcc85649b80 1 --2- 172.21.15.1:0/1222192966 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b64922cb10 0x55b6492215f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcc70076b40 tx=0x7fcc5c0039f0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.105+0000 7fcc85649b80 1 -- 172.21.15.1:0/1222192966 shutdown_connections 2024-05-04T03:18:00.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.105+0000 7fcc85649b80 1 --2- 172.21.15.1:0/1222192966 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b6492320d0 0x55b6492554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.105+0000 7fcc85649b80 1 --2- 172.21.15.1:0/1222192966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b64922c1c0 0x55b64922b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.105+0000 7fcc85649b80 1 --2- 172.21.15.1:0/1222192966 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b64922cb10 0x55b6492215f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.105+0000 7fcc85649b80 1 -- 172.21.15.1:0/1222192966 >> 172.21.15.1:0/1222192966 conn(0x55b6490aedf0 msgr2=0x55b64922b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:00.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.105+0000 7fcc85649b80 1 -- 172.21.15.1:0/1222192966 shutdown_connections 2024-05-04T03:18:00.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.105+0000 7fcc85649b80 1 -- 172.21.15.1:0/1222192966 wait complete. 2024-05-04T03:18:00.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.105+0000 7fcc85649b80 1 -- start start 2024-05-04T03:18:00.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.106+0000 7fcc85649b80 1 Processor -- start 2024-05-04T03:18:00.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.106+0000 7fcc85649b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b64922c1c0 0x55b649243080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:00.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.106+0000 7fcc85649b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b64922cb10 0x55b6492435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:00.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.106+0000 7fcc85649b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b6492320d0 0x55b6492b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:00.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.106+0000 7fcc85649b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55b64924bed0 con 0x55b64922cb10 2024-05-04T03:18:00.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.106+0000 7fcc85649b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55b64924b800 con 0x55b6492320d0 2024-05-04T03:18:00.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.106+0000 7fcc85649b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b64924c5a0 con 0x55b64922c1c0 2024-05-04T03:18:00.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.106+0000 7fcc85649b80 10 client.0 _ll_register_callbacks cb 0x55b6490e6900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:18:00.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.106+0000 7fcc6ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b64922cb10 0x55b6492435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:00.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.106+0000 7fcc749d1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b64922c1c0 0x55b649243080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:00.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.106+0000 7fcc6ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b64922cb10 0x55b6492435c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33984/0 (socket says 172.21.15.1:33984) 2024-05-04T03:18:00.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.106+0000 7fcc6ffff700 1 -- 172.21.15.1:0/1035309497 learned_addr learned my addr 172.21.15.1:0/1035309497 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:00.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.106+0000 7fcc751d2700 1 --2- 172.21.15.1:0/1035309497 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b6492320d0 0x55b6492b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:00.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.107+0000 7fcc6ffff700 1 -- 172.21.15.1:0/1035309497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b64922c1c0 msgr2=0x55b649243080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:00.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.107+0000 7fcc6ffff700 1 --2- 172.21.15.1:0/1035309497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b64922c1c0 0x55b649243080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.107+0000 7fcc6ffff700 1 -- 172.21.15.1:0/1035309497 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b6492320d0 msgr2=0x55b6492b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:00.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.107+0000 7fcc6ffff700 1 --2- 172.21.15.1:0/1035309497 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b6492320d0 0x55b6492b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.107+0000 7fcc6ffff700 1 -- 172.21.15.1:0/1035309497 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6492b5590 con 0x55b64922cb10 2024-05-04T03:18:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.107+0000 7fcc6ffff700 1 --2- 172.21.15.1:0/1035309497 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b64922cb10 0x55b6492435c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c006010 tx=0x7fcc5c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f453fd6b9b143f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.107+0000 7fcc4ffff700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:18:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.107+0000 7fcc4ffff700 1 -- 172.21.15.1:0/1035309497 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c027030 con 0x55b64922cb10 2024-05-04T03:18:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.107+0000 7fcc4ffff700 1 -- 172.21.15.1:0/1035309497 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fcc5c0052c0 con 0x55b64922cb10 2024-05-04T03:18:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.107+0000 7fcc85649b80 1 -- 172.21.15.1:0/1035309497 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc5c003d60 con 0x55b64922cb10 2024-05-04T03:18:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.107+0000 7fcc4ffff700 1 -- 172.21.15.1:0/1035309497 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcc5c0054c0 con 0x55b64922cb10 2024-05-04T03:18:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.107+0000 7fcc85649b80 10 client.14955 fetch_fsmap learned FSMap version 25 2024-05-04T03:18:00.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.107+0000 7fcc85649b80 1 -- 172.21.15.1:0/1035309497 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b6492b67d0 con 0x55b64922cb10 2024-05-04T03:18:00.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.107+0000 7fcc4ffff700 1 -- 172.21.15.1:0/1035309497 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc5c003d60 con 0x55b64922cb10 2024-05-04T03:18:00.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.108+0000 7fcc4ffff700 1 -- 172.21.15.1:0/1035309497 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fcc5c023440 con 0x55b64922cb10 2024-05-04T03:18:00.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.108+0000 7fcc85649b80 10 client.14955 fetch_fsmap finished waiting for FSMap version 25 2024-05-04T03:18:00.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.108+0000 7fcc85649b80 10 client.14955 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-04T03:18:00.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.108+0000 7fcc85649b80 20 client.14955 populate_metadata read hostname 'smithi001' 2024-05-04T03:18:00.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.108+0000 7fcc85649b80 1 --2- 172.21.15.1:0/1035309497 >> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] conn(0x55b6492b8750 0x55b6492b9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:00.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.108+0000 7fcc85649b80 4 client.14955 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-04T03:18:00.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.108+0000 7fcc85649b80 1 -- 172.21.15.1:0/1035309497 --> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] -- command(tid 0: {"prefix": "status"}) -- 0x55b649156370 con 0x55b6492b8750 2024-05-04T03:18:00.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.108+0000 7fcc749d1700 1 --2- 172.21.15.1:0/1035309497 >> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] conn(0x55b6492b8750 0x55b6492b9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:00.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.109+0000 7fcc749d1700 1 --2- 172.21.15.1:0/1035309497 >> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] conn(0x55b6492b8750 0x55b6492b9bc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c4a9773510ffee89 server_cookie=6247cdd62717aeea in_seq=0 out_seq=0 2024-05-04T03:18:00.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.109+0000 7fcc4ffff700 10 client.14955 ms_handle_connect on v2:172.21.15.77:6838/922723189 2024-05-04T03:18:00.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.109+0000 7fcc4ffff700 1 -- 172.21.15.1:0/1035309497 <== mds.1 v2:172.21.15.77:6838/922723189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b649156370 con 0x55b6492b8750 2024-05-04T03:18:00.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.109+0000 7fcc4ffff700 10 client.14955 handle_command_reply: tid=0 2024-05-04T03:18:00.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.109+0000 7fcc85649b80 1 client.14955 shutdown 2024-05-04T03:18:00.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.109+0000 7fcc85649b80 1 -- 172.21.15.1:0/1035309497 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b64922cb10 msgr2=0x55b6492435c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:00.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.109+0000 7fcc85649b80 1 --2- 172.21.15.1:0/1035309497 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b64922cb10 0x55b6492435c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c006010 tx=0x7fcc5c009310 comp rx=0 tx=0).stop 2024-05-04T03:18:00.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.109+0000 7fcc85649b80 1 -- 172.21.15.1:0/1035309497 shutdown_connections 2024-05-04T03:18:00.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.109+0000 7fcc85649b80 1 --2- 172.21.15.1:0/1035309497 >> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] conn(0x55b6492b8750 0x55b6492b9bc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.109+0000 7fcc4ffff700 0 client.14955 ms_handle_reset on v2:172.21.15.77:6838/922723189 2024-05-04T03:18:00.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.109+0000 7fcc85649b80 1 --2- 172.21.15.1:0/1035309497 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b6492320d0 0x55b6492b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.109+0000 7fcc85649b80 1 --2- 172.21.15.1:0/1035309497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b64922c1c0 0x55b649243080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.109+0000 7fcc85649b80 1 --2- 172.21.15.1:0/1035309497 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b64922cb10 0x55b6492435c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.109+0000 7fcc85649b80 1 -- 172.21.15.1:0/1035309497 >> 172.21.15.1:0/1035309497 conn(0x55b6490aedf0 msgr2=0x55b64922d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:00.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.109+0000 7fcc85649b80 1 -- 172.21.15.1:0/1035309497 shutdown_connections 2024-05-04T03:18:00.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.109+0000 7fcc85649b80 1 -- 172.21.15.1:0/1035309497 wait complete. 2024-05-04T03:18:00.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.109+0000 7fcc85649b80 20 client.14955 trim_cache size 0 max 16384 2024-05-04T03:18:00.119 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-04T03:18:00.119 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "4a572aa8-09c3-11ef-bc97-c7b262605968", 2024-05-04T03:18:00.119 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 1, 2024-05-04T03:18:00.119 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24472, 2024-05-04T03:18:00.119 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-05-04T03:18:00.119 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-05-04T03:18:00.119 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-05-04T03:18:00.120 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 117.88741716600001, 2024-05-04T03:18:00.120 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 25, 2024-05-04T03:18:00.120 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 83, 2024-05-04T03:18:00.120 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 81, 2024-05-04T03:18:00.120 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 155.850217474 2024-05-04T03:18:00.120 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-04T03:18:00.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.110+0000 7fcc76c35700 1 -- 172.21.15.1:0/464882239 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fcc600b2880 msgr2=0x7fcc600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:00.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.110+0000 7fcc76c35700 1 --2- 172.21.15.1:0/464882239 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fcc600b2880 0x7fcc600b4d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcc64005950 tx=0x7fcc64005f50 comp rx=0 tx=0).stop 2024-05-04T03:18:00.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.110+0000 7fcc76c35700 1 -- 172.21.15.1:0/464882239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc70108740 msgr2=0x7fcc7019f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:00.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.110+0000 7fcc76c35700 1 --2- 172.21.15.1:0/464882239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc70108740 0x7fcc7019f640 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcc6800ada0 tx=0x7fcc6800c330 comp rx=0 tx=0).stop 2024-05-04T03:18:00.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.110+0000 7fcc76c35700 1 -- 172.21.15.1:0/464882239 shutdown_connections 2024-05-04T03:18:00.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.110+0000 7fcc76c35700 1 --2- 172.21.15.1:0/464882239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc701034c0 0x7fcc7019ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.110+0000 7fcc76c35700 1 --2- 172.21.15.1:0/464882239 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fcc600b2880 0x7fcc600b4d90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.110+0000 7fcc76c35700 1 --2- 172.21.15.1:0/464882239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcc70108740 0x7fcc7019f640 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.110+0000 7fcc76c35700 1 --2- 172.21.15.1:0/464882239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc70107df0 0x7fcc7019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:00.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.110+0000 7fcc76c35700 1 -- 172.21.15.1:0/464882239 >> 172.21.15.1:0/464882239 conn(0x7fcc700fedf0 msgr2=0x7fcc70101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:00.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.110+0000 7fcc76c35700 1 -- 172.21.15.1:0/464882239 shutdown_connections 2024-05-04T03:18:00.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:00.110+0000 7fcc76c35700 1 -- 172.21.15.1:0/464882239 wait complete. 2024-05-04T03:18:00.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:00 smithi001 ceph-mon[29731]: pgmap v356: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-04T03:18:00.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:00 smithi077 ceph-mon[33531]: pgmap v356: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-04T03:18:00.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:00 smithi072 ceph-mon[33332]: pgmap v356: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-04T03:18:00.780 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-04T03:18:01.031 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:18:02.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:02 smithi001 ceph-mon[29731]: pgmap v357: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:18:02.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:02 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:18:02.635 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:02 smithi072 ceph-mon[33332]: pgmap v357: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:18:02.635 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:02 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:18:02.663 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:02 smithi077 ceph-mon[33531]: pgmap v357: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:18:02.663 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:02 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:18:02.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.690+0000 7f398a8c1700 1 -- 172.21.15.1:0/2880290046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f398410a3e0 msgr2=0x7f398410a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:02.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.690+0000 7f398a8c1700 1 --2- 172.21.15.1:0/2880290046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f398410a3e0 0x7f398410a7f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3974009990 tx=0x7f3974009ca0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.690+0000 7f398a8c1700 1 -- 172.21.15.1:0/2880290046 shutdown_connections 2024-05-04T03:18:02.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.690+0000 7f398a8c1700 1 --2- 172.21.15.1:0/2880290046 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3984103130 0x7f3984111120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.690+0000 7f398a8c1700 1 --2- 172.21.15.1:0/2880290046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f398410a3e0 0x7f398410a7f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.690+0000 7f398a8c1700 1 --2- 172.21.15.1:0/2880290046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39841027e0 0x7f3984102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.690+0000 7f398a8c1700 1 -- 172.21.15.1:0/2880290046 >> 172.21.15.1:0/2880290046 conn(0x7f39840fe6f0 msgr2=0x7f3984100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:02.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.690+0000 7f398a8c1700 1 -- 172.21.15.1:0/2880290046 shutdown_connections 2024-05-04T03:18:02.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.690+0000 7f398a8c1700 1 -- 172.21.15.1:0/2880290046 wait complete. 2024-05-04T03:18:02.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.691+0000 7f398a8c1700 1 Processor -- start 2024-05-04T03:18:02.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.691+0000 7f398a8c1700 1 -- start start 2024-05-04T03:18:02.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.691+0000 7f398a8c1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39841027e0 0x7f39841039c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:02.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.691+0000 7f398a8c1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3984103130 0x7f3984103f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:02.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.691+0000 7f398a8c1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f398410a3e0 0x7f3984104440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:02.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.691+0000 7f398a8c1700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3984114480 con 0x7f398410a3e0 2024-05-04T03:18:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.691+0000 7f398a8c1700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3984114300 con 0x7f3984103130 2024-05-04T03:18:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.691+0000 7f398a8c1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3984114600 con 0x7f39841027e0 2024-05-04T03:18:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.692+0000 7f3988e5e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f398410a3e0 0x7f3984104440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.692+0000 7f3988e5e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f398410a3e0 0x7f3984104440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41072/0 (socket says 172.21.15.1:41072) 2024-05-04T03:18:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.692+0000 7f3988e5e700 1 -- 172.21.15.1:0/1635698031 learned_addr learned my addr 172.21.15.1:0/1635698031 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.692+0000 7f39837fe700 1 --2- 172.21.15.1:0/1635698031 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3984103130 0x7f3984103f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:02.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.692+0000 7f3983fff700 1 --2- 172.21.15.1:0/1635698031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39841027e0 0x7f39841039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:02.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.692+0000 7f3983fff700 1 -- 172.21.15.1:0/1635698031 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3984103130 msgr2=0x7f3984103f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:02.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.692+0000 7f3983fff700 1 --2- 172.21.15.1:0/1635698031 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3984103130 0x7f3984103f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.692+0000 7f3983fff700 1 -- 172.21.15.1:0/1635698031 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f398410a3e0 msgr2=0x7f3984104440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:02.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.692+0000 7f3983fff700 1 --2- 172.21.15.1:0/1635698031 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f398410a3e0 0x7f3984104440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.692+0000 7f3983fff700 1 -- 172.21.15.1:0/1635698031 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39841a91b0 con 0x7f39841027e0 2024-05-04T03:18:02.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.692+0000 7f3988e5e700 1 --2- 172.21.15.1:0/1635698031 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f398410a3e0 0x7f3984104440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:18:02.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.692+0000 7f3983fff700 1 --2- 172.21.15.1:0/1635698031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39841027e0 0x7f39841039c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3974006010 tx=0x7f39740057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:02.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.693+0000 7f39817fa700 1 -- 172.21.15.1:0/1635698031 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3974027070 con 0x7f39841027e0 2024-05-04T03:18:02.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.693+0000 7f39817fa700 1 -- 172.21.15.1:0/1635698031 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3974022470 con 0x7f39841027e0 2024-05-04T03:18:02.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.693+0000 7f398a8c1700 1 -- 172.21.15.1:0/1635698031 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39841a9440 con 0x7f39841027e0 2024-05-04T03:18:02.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.693+0000 7f39817fa700 1 -- 172.21.15.1:0/1635698031 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3974017650 con 0x7f39841027e0 2024-05-04T03:18:02.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.693+0000 7f398a8c1700 1 -- 172.21.15.1:0/1635698031 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39841a98f0 con 0x7f39841027e0 2024-05-04T03:18:02.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.695+0000 7f39817fa700 1 -- 172.21.15.1:0/1635698031 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f397402a070 con 0x7f39841027e0 2024-05-04T03:18:02.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.696+0000 7f39817fa700 1 --2- 172.21.15.1:0/1635698031 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f396c0b2900 0x7f396c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:02.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.696+0000 7f39837fe700 1 --2- 172.21.15.1:0/1635698031 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f396c0b2900 0x7f396c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:02.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.696+0000 7f39817fa700 1 -- 172.21.15.1:0/1635698031 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f39740e2df0 con 0x7f39841027e0 2024-05-04T03:18:02.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.697+0000 7f39837fe700 1 --2- 172.21.15.1:0/1635698031 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f396c0b2900 0x7f396c0b4e10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f39841003e0 tx=0x7f39780073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:02.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.697+0000 7f39992d5b80 1 Processor -- start 2024-05-04T03:18:02.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.697+0000 7f39992d5b80 1 -- start start 2024-05-04T03:18:02.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.698+0000 7f39992d5b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d344d9f0d0 0x55d344d9f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:02.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.698+0000 7f39992d5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d344d991c0 0x55d344d995d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:02.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.698+0000 7f39992d5b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d344d99b10 0x55d344da5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:02.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.698+0000 7f39992d5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d344cbb8f0 con 0x55d344d991c0 2024-05-04T03:18:02.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.698+0000 7f39992d5b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55d344c20d50 con 0x55d344d9f0d0 2024-05-04T03:18:02.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.698+0000 7f39992d5b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d344cb78b0 con 0x55d344d99b10 2024-05-04T03:18:02.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.698+0000 7f39837fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d344d991c0 0x55d344d995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.698+0000 7f39837fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d344d991c0 0x55d344d995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41074/0 (socket says 172.21.15.1:41074) 2024-05-04T03:18:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.698+0000 7f39837fe700 1 -- 172.21.15.1:0/2504033593 learned_addr learned my addr 172.21.15.1:0/2504033593 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.698+0000 7f3983fff700 1 --2- 172.21.15.1:0/2504033593 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d344d9f0d0 0x55d344d9f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.698+0000 7f3988e5e700 1 --2- 172.21.15.1:0/2504033593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d344d99b10 0x55d344da5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.698+0000 7f3983fff700 1 -- 172.21.15.1:0/2504033593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d344d99b10 msgr2=0x55d344da5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.698+0000 7f3983fff700 1 --2- 172.21.15.1:0/2504033593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d344d99b10 0x55d344da5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.698+0000 7f3983fff700 1 -- 172.21.15.1:0/2504033593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d344d991c0 msgr2=0x55d344d995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.698+0000 7f3983fff700 1 --2- 172.21.15.1:0/2504033593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d344d991c0 0x55d344d995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.698+0000 7f3983fff700 1 -- 172.21.15.1:0/2504033593 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d344c08880 con 0x55d344d9f0d0 2024-05-04T03:18:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.698+0000 7f39837fe700 1 --2- 172.21.15.1:0/2504033593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d344d991c0 0x55d344d995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:18:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.698+0000 7f3988e5e700 1 --2- 172.21.15.1:0/2504033593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d344d99b10 0x55d344da5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:18:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.698+0000 7f3983fff700 1 --2- 172.21.15.1:0/2504033593 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d344d9f0d0 0x55d344d9f4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f39740e0a00 tx=0x7f39740a9da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3201f48ff4b71838 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.698+0000 7f396a7fc700 1 -- 172.21.15.1:0/2504033593 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39740aac60 con 0x55d344d9f0d0 2024-05-04T03:18:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.699+0000 7f396a7fc700 1 -- 172.21.15.1:0/2504033593 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39740aa560 con 0x55d344d9f0d0 2024-05-04T03:18:02.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.699+0000 7f396a7fc700 1 -- 172.21.15.1:0/2504033593 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39740aa7c0 con 0x55d344d9f0d0 2024-05-04T03:18:02.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.700+0000 7f39992d5b80 1 -- 172.21.15.1:0/2504033593 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d344d9f0d0 msgr2=0x55d344d9f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:02.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.700+0000 7f39992d5b80 1 --2- 172.21.15.1:0/2504033593 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d344d9f0d0 0x55d344d9f4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f39740e0a00 tx=0x7f39740a9da0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.700+0000 7f39992d5b80 1 -- 172.21.15.1:0/2504033593 shutdown_connections 2024-05-04T03:18:02.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.700+0000 7f39992d5b80 1 --2- 172.21.15.1:0/2504033593 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d344d9f0d0 0x55d344d9f4e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.700+0000 7f39992d5b80 1 --2- 172.21.15.1:0/2504033593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d344d99b10 0x55d344da5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.700+0000 7f39992d5b80 1 --2- 172.21.15.1:0/2504033593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d344d991c0 0x55d344d995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.700+0000 7f39992d5b80 1 -- 172.21.15.1:0/2504033593 >> 172.21.15.1:0/2504033593 conn(0x55d344c1bdf0 msgr2=0x55d344d94210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:02.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.700+0000 7f39992d5b80 1 -- 172.21.15.1:0/2504033593 shutdown_connections 2024-05-04T03:18:02.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.700+0000 7f39992d5b80 1 -- 172.21.15.1:0/2504033593 wait complete. 2024-05-04T03:18:02.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.700+0000 7f39992d5b80 1 -- start start 2024-05-04T03:18:02.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.701+0000 7f39992d5b80 1 Processor -- start 2024-05-04T03:18:02.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.701+0000 7f39992d5b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d344d991c0 0x55d344da33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:02.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.701+0000 7f39992d5b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d344d99b10 0x55d344da3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:02.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.701+0000 7f39992d5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d344d9f0d0 0x55d344da3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:02.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.701+0000 7f39992d5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d344da62f0 con 0x55d344d9f0d0 2024-05-04T03:18:02.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.701+0000 7f39992d5b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55d344c74e80 con 0x55d344d99b10 2024-05-04T03:18:02.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.701+0000 7f39992d5b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d344da6470 con 0x55d344d991c0 2024-05-04T03:18:02.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.701+0000 7f39992d5b80 10 client.0 _ll_register_callbacks cb 0x55d344c53900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:18:02.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.701+0000 7f3988e5e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d344d9f0d0 0x55d344da3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:02.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.701+0000 7f3983fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d344d991c0 0x55d344da33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:02.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.701+0000 7f3988e5e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d344d9f0d0 0x55d344da3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41090/0 (socket says 172.21.15.1:41090) 2024-05-04T03:18:02.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.701+0000 7f39837fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d344d99b10 0x55d344da3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:02.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.701+0000 7f3988e5e700 1 -- 172.21.15.1:0/3151948124 learned_addr learned my addr 172.21.15.1:0/3151948124 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:02.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.701+0000 7f3983fff700 1 -- 172.21.15.1:0/3151948124 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d344d99b10 msgr2=0x55d344da3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:02.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.701+0000 7f3983fff700 1 --2- 172.21.15.1:0/3151948124 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d344d99b10 0x55d344da3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.701+0000 7f3983fff700 1 -- 172.21.15.1:0/3151948124 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d344d9f0d0 msgr2=0x55d344da3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:02.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.701+0000 7f3983fff700 1 --2- 172.21.15.1:0/3151948124 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d344d9f0d0 0x55d344da3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.702+0000 7f3983fff700 1 -- 172.21.15.1:0/3151948124 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d344c72710 con 0x55d344d991c0 2024-05-04T03:18:02.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.702+0000 7f3983fff700 1 --2- 172.21.15.1:0/3151948124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d344d991c0 0x55d344da33c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f397400b680 tx=0x7f39740a6550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d50ab92511dadeb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:02.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.702+0000 7f39637fe700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-04T03:18:02.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.702+0000 7f39637fe700 1 -- 172.21.15.1:0/3151948124 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39740aac60 con 0x55d344d991c0 2024-05-04T03:18:02.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.702+0000 7f39992d5b80 1 -- 172.21.15.1:0/3151948124 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39740aa560 con 0x55d344d991c0 2024-05-04T03:18:02.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.702+0000 7f39637fe700 1 -- 172.21.15.1:0/3151948124 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f39740aa7c0 con 0x55d344d991c0 2024-05-04T03:18:02.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.702+0000 7f39637fe700 1 -- 172.21.15.1:0/3151948124 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f39740b3330 con 0x55d344d991c0 2024-05-04T03:18:02.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.702+0000 7f39992d5b80 10 client.24775 fetch_fsmap learned FSMap version 25 2024-05-04T03:18:02.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.702+0000 7f39992d5b80 1 -- 172.21.15.1:0/3151948124 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d344da62f0 con 0x55d344d991c0 2024-05-04T03:18:02.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.702+0000 7f39637fe700 1 -- 172.21.15.1:0/3151948124 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39740aa560 con 0x55d344d991c0 2024-05-04T03:18:02.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.703+0000 7f39637fe700 1 -- 172.21.15.1:0/3151948124 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f39740cb2f0 con 0x55d344d991c0 2024-05-04T03:18:02.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.703+0000 7f39992d5b80 10 client.24775 fetch_fsmap finished waiting for FSMap version 25 2024-05-04T03:18:02.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.703+0000 7f39992d5b80 10 client.24775 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-04T03:18:02.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.703+0000 7f39992d5b80 20 client.24775 populate_metadata read hostname 'smithi001' 2024-05-04T03:18:02.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.703+0000 7f39992d5b80 1 --2- 172.21.15.1:0/3151948124 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55d344db34a0 0x55d344db4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:02.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.703+0000 7f39992d5b80 4 client.24775 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:18:02.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.703+0000 7f39992d5b80 1 -- 172.21.15.1:0/3151948124 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d344cc92d0 con 0x55d344db34a0 2024-05-04T03:18:02.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.703+0000 7f39837fe700 1 --2- 172.21.15.1:0/3151948124 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55d344db34a0 0x55d344db4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:02.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.703+0000 7f39637fe700 10 client.24775 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:18:02.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.703+0000 7f39837fe700 1 --2- 172.21.15.1:0/3151948124 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55d344db34a0 0x55d344db4910 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=608fca60fe21ce55 server_cookie=6445a2e79e7f3c84 in_seq=0 out_seq=0 2024-05-04T03:18:02.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.704+0000 7f39637fe700 1 -- 172.21.15.1:0/3151948124 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d344cc92d0 con 0x55d344db34a0 2024-05-04T03:18:02.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.704+0000 7f39637fe700 10 client.24775 handle_command_reply: tid=0 2024-05-04T03:18:02.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.704+0000 7f39992d5b80 1 client.24775 shutdown 2024-05-04T03:18:02.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.705+0000 7f39992d5b80 1 -- 172.21.15.1:0/3151948124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d344d991c0 msgr2=0x55d344da33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:02.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.705+0000 7f39992d5b80 1 --2- 172.21.15.1:0/3151948124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d344d991c0 0x55d344da33c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f397400b680 tx=0x7f39740a6550 comp rx=0 tx=0).stop 2024-05-04T03:18:02.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.705+0000 7f39992d5b80 1 -- 172.21.15.1:0/3151948124 shutdown_connections 2024-05-04T03:18:02.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.705+0000 7f39992d5b80 1 --2- 172.21.15.1:0/3151948124 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55d344db34a0 0x55d344db4910 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.705+0000 7f39992d5b80 1 --2- 172.21.15.1:0/3151948124 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d344d9f0d0 0x55d344da3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.705+0000 7f39992d5b80 1 --2- 172.21.15.1:0/3151948124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d344d991c0 0x55d344da33c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.705+0000 7f39637fe700 0 client.24775 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:18:02.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.705+0000 7f39992d5b80 1 --2- 172.21.15.1:0/3151948124 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d344d99b10 0x55d344da3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.705+0000 7f39992d5b80 1 -- 172.21.15.1:0/3151948124 >> 172.21.15.1:0/3151948124 conn(0x55d344c1bdf0 msgr2=0x55d344d981b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:02.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.705+0000 7f39992d5b80 1 -- 172.21.15.1:0/3151948124 shutdown_connections 2024-05-04T03:18:02.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.705+0000 7f39992d5b80 1 -- 172.21.15.1:0/3151948124 wait complete. 2024-05-04T03:18:02.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.705+0000 7f39992d5b80 20 client.24775 trim_cache size 0 max 16384 2024-05-04T03:18:02.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.730+0000 7f39992d5b80 1 Processor -- start 2024-05-04T03:18:02.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.730+0000 7f39992d5b80 1 -- start start 2024-05-04T03:18:02.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.731+0000 7f39992d5b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d344d991c0 0x55d344d98be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:02.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.731+0000 7f39992d5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d344d99b10 0x55d344da3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:02.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.731+0000 7f39992d5b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d344d9f0d0 0x55d344daf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:02.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.731+0000 7f39992d5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d344db2c50 con 0x55d344d99b10 2024-05-04T03:18:02.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.731+0000 7f39992d5b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55d344db3320 con 0x55d344d991c0 2024-05-04T03:18:02.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.731+0000 7f39992d5b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d344db39f0 con 0x55d344d9f0d0 2024-05-04T03:18:02.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.731+0000 7f39837fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d344d99b10 0x55d344da3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:02.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.731+0000 7f39837fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d344d99b10 0x55d344da3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41098/0 (socket says 172.21.15.1:41098) 2024-05-04T03:18:02.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.731+0000 7f39837fe700 1 -- 172.21.15.1:0/1524255281 learned_addr learned my addr 172.21.15.1:0/1524255281 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:02.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.731+0000 7f3988e5e700 1 --2- 172.21.15.1:0/1524255281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d344d9f0d0 0x55d344daf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:02.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.731+0000 7f3983fff700 1 --2- 172.21.15.1:0/1524255281 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d344d991c0 0x55d344d98be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:02.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.731+0000 7f39837fe700 1 -- 172.21.15.1:0/1524255281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d344d9f0d0 msgr2=0x55d344daf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:02.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.731+0000 7f39837fe700 1 --2- 172.21.15.1:0/1524255281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d344d9f0d0 0x55d344daf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.731+0000 7f39837fe700 1 -- 172.21.15.1:0/1524255281 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d344d991c0 msgr2=0x55d344d98be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:02.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.731+0000 7f39837fe700 1 --2- 172.21.15.1:0/1524255281 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d344d991c0 0x55d344d98be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.731+0000 7f39837fe700 1 -- 172.21.15.1:0/1524255281 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d344da41c0 con 0x55d344d99b10 2024-05-04T03:18:02.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.731+0000 7f39837fe700 1 --2- 172.21.15.1:0/1524255281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d344d99b10 0x55d344da3c80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f39780079c0 tx=0x7f3978016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d82878c43182e484 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:02.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.732+0000 7f3969ffb700 1 -- 172.21.15.1:0/1524255281 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397802e070 con 0x55d344d99b10 2024-05-04T03:18:02.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.732+0000 7f3969ffb700 1 -- 172.21.15.1:0/1524255281 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3978015070 con 0x55d344d99b10 2024-05-04T03:18:02.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.732+0000 7f3969ffb700 1 -- 172.21.15.1:0/1524255281 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3978011440 con 0x55d344d99b10 2024-05-04T03:18:02.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.733+0000 7f39992d5b80 1 -- 172.21.15.1:0/1524255281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d344d99b10 msgr2=0x55d344da3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:02.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.733+0000 7f39992d5b80 1 --2- 172.21.15.1:0/1524255281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d344d99b10 0x55d344da3c80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f39780079c0 tx=0x7f3978016000 comp rx=0 tx=0).stop 2024-05-04T03:18:02.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.733+0000 7f39992d5b80 1 -- 172.21.15.1:0/1524255281 shutdown_connections 2024-05-04T03:18:02.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.733+0000 7f39992d5b80 1 --2- 172.21.15.1:0/1524255281 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d344d991c0 0x55d344d98be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.733+0000 7f39992d5b80 1 --2- 172.21.15.1:0/1524255281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d344d9f0d0 0x55d344daf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.733+0000 7f39992d5b80 1 --2- 172.21.15.1:0/1524255281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d344d99b10 0x55d344da3c80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.733+0000 7f39992d5b80 1 -- 172.21.15.1:0/1524255281 >> 172.21.15.1:0/1524255281 conn(0x55d344c1bdf0 msgr2=0x55d344da4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:02.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.733+0000 7f39992d5b80 1 -- 172.21.15.1:0/1524255281 shutdown_connections 2024-05-04T03:18:02.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.733+0000 7f39992d5b80 1 -- 172.21.15.1:0/1524255281 wait complete. 2024-05-04T03:18:02.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.733+0000 7f39992d5b80 1 -- start start 2024-05-04T03:18:02.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.733+0000 7f39992d5b80 1 Processor -- start 2024-05-04T03:18:02.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.734+0000 7f39992d5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d344d991c0 0x55d344da0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:02.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.734+0000 7f39992d5b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d344d99b10 0x55d344da0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:02.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.734+0000 7f39992d5b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d344d9f0d0 0x55d344da0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:02.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.734+0000 7f3983fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d344d991c0 0x55d344da0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:02.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.734+0000 7f39992d5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d344db4790 con 0x55d344d991c0 2024-05-04T03:18:02.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.734+0000 7f3983fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d344d991c0 0x55d344da0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41104/0 (socket says 172.21.15.1:41104) 2024-05-04T03:18:02.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.734+0000 7f3983fff700 1 -- 172.21.15.1:0/2044147995 learned_addr learned my addr 172.21.15.1:0/2044147995 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:02.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.734+0000 7f39992d5b80 1 -- 172.21.15.1:0/2044147995 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55d344db40c0 con 0x55d344d9f0d0 2024-05-04T03:18:02.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.734+0000 7f39992d5b80 1 -- 172.21.15.1:0/2044147995 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d344db4e60 con 0x55d344d99b10 2024-05-04T03:18:02.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.734+0000 7f39992d5b80 10 client.0 _ll_register_callbacks cb 0x55d344c53900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:18:02.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.734+0000 7f39837fe700 1 --2- 172.21.15.1:0/2044147995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d344d99b10 0x55d344da0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:02.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.734+0000 7f3988e5e700 1 --2- 172.21.15.1:0/2044147995 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d344d9f0d0 0x55d344da0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:02.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.734+0000 7f3983fff700 1 -- 172.21.15.1:0/2044147995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d344d99b10 msgr2=0x55d344da0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:02.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.734+0000 7f3983fff700 1 --2- 172.21.15.1:0/2044147995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d344d99b10 0x55d344da0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.734+0000 7f3983fff700 1 -- 172.21.15.1:0/2044147995 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d344d9f0d0 msgr2=0x55d344da0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:02.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.734+0000 7f3983fff700 1 --2- 172.21.15.1:0/2044147995 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d344d9f0d0 0x55d344da0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.734+0000 7f3983fff700 1 -- 172.21.15.1:0/2044147995 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d344dbc4c0 con 0x55d344d991c0 2024-05-04T03:18:02.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.735+0000 7f3983fff700 1 --2- 172.21.15.1:0/2044147995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d344d991c0 0x55d344da0420 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3974009750 tx=0x7f39740b3230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb02deff071d8c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:02.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.735+0000 7f39637fe700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:18:02.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.735+0000 7f39637fe700 1 -- 172.21.15.1:0/2044147995 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397400fad0 con 0x55d344d991c0 2024-05-04T03:18:02.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.735+0000 7f39637fe700 1 -- 172.21.15.1:0/2044147995 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f39740aa7c0 con 0x55d344d991c0 2024-05-04T03:18:02.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.735+0000 7f39992d5b80 1 -- 172.21.15.1:0/2044147995 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3978015070 con 0x55d344d991c0 2024-05-04T03:18:02.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.735+0000 7f39637fe700 1 -- 172.21.15.1:0/2044147995 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f39740a9cd0 con 0x55d344d991c0 2024-05-04T03:18:02.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.735+0000 7f39992d5b80 10 client.14973 fetch_fsmap learned FSMap version 25 2024-05-04T03:18:02.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.735+0000 7f39992d5b80 1 -- 172.21.15.1:0/2044147995 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d344dbc4c0 con 0x55d344d991c0 2024-05-04T03:18:02.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.735+0000 7f39637fe700 1 -- 172.21.15.1:0/2044147995 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3978015070 con 0x55d344d991c0 2024-05-04T03:18:02.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.735+0000 7f39637fe700 1 -- 172.21.15.1:0/2044147995 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f39740a5f20 con 0x55d344d991c0 2024-05-04T03:18:02.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.735+0000 7f39992d5b80 10 client.14973 fetch_fsmap finished waiting for FSMap version 25 2024-05-04T03:18:02.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.736+0000 7f39992d5b80 10 client.14973 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-04T03:18:02.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.736+0000 7f39992d5b80 20 client.14973 populate_metadata read hostname 'smithi001' 2024-05-04T03:18:02.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.736+0000 7f39992d5b80 1 --2- 172.21.15.1:0/2044147995 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55d344db34a0 0x55d344e22140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:02.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.736+0000 7f39992d5b80 4 client.14973 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-04T03:18:02.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.736+0000 7f39992d5b80 1 -- 172.21.15.1:0/2044147995 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "status"}) -- 0x55d344cc3370 con 0x55d344db34a0 2024-05-04T03:18:02.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.736+0000 7f39837fe700 1 --2- 172.21.15.1:0/2044147995 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55d344db34a0 0x55d344e22140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:02.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.736+0000 7f39837fe700 1 --2- 172.21.15.1:0/2044147995 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55d344db34a0 0x55d344e22140 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24876233d6636627 server_cookie=9866d800ae4726c in_seq=0 out_seq=0 2024-05-04T03:18:02.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.736+0000 7f39637fe700 10 client.14973 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:18:02.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.737+0000 7f39637fe700 1 -- 172.21.15.1:0/2044147995 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d344cc3370 con 0x55d344db34a0 2024-05-04T03:18:02.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.737+0000 7f39637fe700 10 client.14973 handle_command_reply: tid=0 2024-05-04T03:18:02.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.737+0000 7f39992d5b80 1 client.14973 shutdown 2024-05-04T03:18:02.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.737+0000 7f39992d5b80 1 -- 172.21.15.1:0/2044147995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d344d991c0 msgr2=0x55d344da0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:02.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.737+0000 7f39992d5b80 1 --2- 172.21.15.1:0/2044147995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d344d991c0 0x55d344da0420 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3974009750 tx=0x7f39740b3230 comp rx=0 tx=0).stop 2024-05-04T03:18:02.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.737+0000 7f39992d5b80 1 -- 172.21.15.1:0/2044147995 shutdown_connections 2024-05-04T03:18:02.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.737+0000 7f39992d5b80 1 --2- 172.21.15.1:0/2044147995 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55d344db34a0 0x55d344e22140 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.737+0000 7f39992d5b80 1 --2- 172.21.15.1:0/2044147995 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d344d9f0d0 0x55d344da0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.737+0000 7f39992d5b80 1 --2- 172.21.15.1:0/2044147995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d344d991c0 0x55d344da0420 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.737+0000 7f39992d5b80 1 --2- 172.21.15.1:0/2044147995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d344d99b10 0x55d344da0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.737+0000 7f39992d5b80 1 -- 172.21.15.1:0/2044147995 >> 172.21.15.1:0/2044147995 conn(0x55d344c1bdf0 msgr2=0x55d344d98940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:02.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.737+0000 7f39637fe700 0 client.14973 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:18:02.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.737+0000 7f39992d5b80 1 -- 172.21.15.1:0/2044147995 shutdown_connections 2024-05-04T03:18:02.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.737+0000 7f39992d5b80 1 -- 172.21.15.1:0/2044147995 wait complete. 2024-05-04T03:18:02.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.737+0000 7f39992d5b80 20 client.14973 trim_cache size 0 max 16384 2024-05-04T03:18:02.750 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-04T03:18:02.750 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "4a572aa8-09c3-11ef-bc97-c7b262605968", 2024-05-04T03:18:02.750 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-05-04T03:18:02.750 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24494, 2024-05-04T03:18:02.750 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-05-04T03:18:02.750 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-05-04T03:18:02.750 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-05-04T03:18:02.750 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 132.701052957, 2024-05-04T03:18:02.750 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 25, 2024-05-04T03:18:02.751 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 83, 2024-05-04T03:18:02.751 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 81, 2024-05-04T03:18:02.751 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 148.36589070400001 2024-05-04T03:18:02.751 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-04T03:18:02.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.738+0000 7f398a8c1700 1 -- 172.21.15.1:0/1635698031 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f396c0b2900 msgr2=0x7f396c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:02.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.738+0000 7f398a8c1700 1 --2- 172.21.15.1:0/1635698031 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f396c0b2900 0x7f396c0b4e10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f39841003e0 tx=0x7f39780073d0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.738+0000 7f398a8c1700 1 -- 172.21.15.1:0/1635698031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39841027e0 msgr2=0x7f39841039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:02.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.738+0000 7f398a8c1700 1 --2- 172.21.15.1:0/1635698031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39841027e0 0x7f39841039c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3974006010 tx=0x7f39740057c0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.738+0000 7f398a8c1700 1 -- 172.21.15.1:0/1635698031 shutdown_connections 2024-05-04T03:18:02.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.738+0000 7f398a8c1700 1 --2- 172.21.15.1:0/1635698031 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f396c0b2900 0x7f396c0b4e10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.738+0000 7f398a8c1700 1 --2- 172.21.15.1:0/1635698031 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f398410a3e0 0x7f3984104440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.738+0000 7f398a8c1700 1 --2- 172.21.15.1:0/1635698031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39841027e0 0x7f39841039c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.738+0000 7f398a8c1700 1 --2- 172.21.15.1:0/1635698031 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3984103130 0x7f3984103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:02.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.738+0000 7f398a8c1700 1 -- 172.21.15.1:0/1635698031 >> 172.21.15.1:0/1635698031 conn(0x7f39840fe6f0 msgr2=0x7f3984100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:02.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.738+0000 7f398a8c1700 1 -- 172.21.15.1:0/1635698031 shutdown_connections 2024-05-04T03:18:02.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:02.738+0000 7f398a8c1700 1 -- 172.21.15.1:0/1635698031 wait complete. 2024-05-04T03:18:03.376 INFO:tasks.cephadm:Setup complete, yielding 2024-05-04T03:18:03.376 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-04T03:18:03.386 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-05-04T03:18:03.386 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- bash -c 'ceph orch status' 2024-05-04T03:18:04.046 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:18:04.502 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:04 smithi001 ceph-mon[29731]: pgmap v358: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:18:04.502 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:04 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:18:04.502 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:04 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:18:04.502 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:04 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:18:04.502 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:04 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:18:04.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:04 smithi077 ceph-mon[33531]: pgmap v358: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:18:04.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:04 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:18:04.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:04 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:18:04.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:04 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:18:04.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:04 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:18:04.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:04 smithi072 ceph-mon[33332]: pgmap v358: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:18:04.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:04 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:18:04.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:04 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:18:04.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:04 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:18:04.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:04 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:18:05.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.697+0000 7f1b02e86700 1 -- 172.21.15.1:0/4007390766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1afc104370 msgr2=0x7f1afc104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:05.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.697+0000 7f1b02e86700 1 --2- 172.21.15.1:0/4007390766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1afc104370 0x7f1afc104780 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009990 tx=0x7f1ae8009ca0 comp rx=0 tx=0).stop 2024-05-04T03:18:05.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.697+0000 7f1b02e86700 1 -- 172.21.15.1:0/4007390766 shutdown_connections 2024-05-04T03:18:05.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.697+0000 7f1b02e86700 1 --2- 172.21.15.1:0/4007390766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1afc104d10 0x7f1afc10b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:05.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.697+0000 7f1b02e86700 1 --2- 172.21.15.1:0/4007390766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1afc103240 0x7f1afc103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:05.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.697+0000 7f1b02e86700 1 --2- 172.21.15.1:0/4007390766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1afc104370 0x7f1afc104780 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:05.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.697+0000 7f1b02e86700 1 -- 172.21.15.1:0/4007390766 >> 172.21.15.1:0/4007390766 conn(0x7f1afc0fe880 msgr2=0x7f1afc100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:05.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.697+0000 7f1b02e86700 1 -- 172.21.15.1:0/4007390766 shutdown_connections 2024-05-04T03:18:05.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.697+0000 7f1b02e86700 1 -- 172.21.15.1:0/4007390766 wait complete. 2024-05-04T03:18:05.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.698+0000 7f1b02e86700 1 Processor -- start 2024-05-04T03:18:05.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.698+0000 7f1b02e86700 1 -- start start 2024-05-04T03:18:05.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.699+0000 7f1b02e86700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1afc103240 0x7f1afc06c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:05.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.699+0000 7f1b02e86700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1afc104370 0x7f1afc19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:05.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.699+0000 7f1b02e86700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1afc104d10 0x7f1afc1a1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:05.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.699+0000 7f1b02e86700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc10eca0 con 0x7f1afc103240 2024-05-04T03:18:05.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.699+0000 7f1b02e86700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc10eb20 con 0x7f1afc104370 2024-05-04T03:18:05.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.699+0000 7f1b02e86700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc10ee20 con 0x7f1afc104d10 2024-05-04T03:18:05.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.699+0000 7f1b00c22700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1afc103240 0x7f1afc06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:05.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.699+0000 7f1b00c22700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1afc103240 0x7f1afc06c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41120/0 (socket says 172.21.15.1:41120) 2024-05-04T03:18:05.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.699+0000 7f1b00c22700 1 -- 172.21.15.1:0/1894960451 learned_addr learned my addr 172.21.15.1:0/1894960451 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:05.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.699+0000 7f1afbfff700 1 --2- 172.21.15.1:0/1894960451 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1afc104370 0x7f1afc19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:05.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.699+0000 7f1b01423700 1 --2- 172.21.15.1:0/1894960451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1afc104d10 0x7f1afc1a1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:05.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.699+0000 7f1b00c22700 1 -- 172.21.15.1:0/1894960451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1afc104d10 msgr2=0x7f1afc1a1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:05.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.699+0000 7f1b00c22700 1 --2- 172.21.15.1:0/1894960451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1afc104d10 0x7f1afc1a1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:05.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.699+0000 7f1b00c22700 1 -- 172.21.15.1:0/1894960451 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1afc104370 msgr2=0x7f1afc19f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:05.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.699+0000 7f1b00c22700 1 --2- 172.21.15.1:0/1894960451 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1afc104370 0x7f1afc19f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:05.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.699+0000 7f1b00c22700 1 -- 172.21.15.1:0/1894960451 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1afc1a2470 con 0x7f1afc103240 2024-05-04T03:18:05.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.699+0000 7f1b00c22700 1 --2- 172.21.15.1:0/1894960451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1afc103240 0x7f1afc06c090 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1af000d950 tx=0x7f1af000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:05.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.700+0000 7f1af9ffb700 1 -- 172.21.15.1:0/1894960451 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af0017070 con 0x7f1afc103240 2024-05-04T03:18:05.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.700+0000 7f1b02e86700 1 -- 172.21.15.1:0/1894960451 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1afc1a2760 con 0x7f1afc103240 2024-05-04T03:18:05.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.700+0000 7f1af9ffb700 1 -- 172.21.15.1:0/1894960451 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1af000f690 con 0x7f1afc103240 2024-05-04T03:18:05.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.700+0000 7f1af9ffb700 1 -- 172.21.15.1:0/1894960451 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af0003aa0 con 0x7f1afc103240 2024-05-04T03:18:05.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.700+0000 7f1b02e86700 1 -- 172.21.15.1:0/1894960451 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1afc1a2cf0 con 0x7f1afc103240 2024-05-04T03:18:05.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.701+0000 7f1b02e86700 1 -- 172.21.15.1:0/1894960451 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1afc0512c0 con 0x7f1afc103240 2024-05-04T03:18:05.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.702+0000 7f1af9ffb700 1 -- 172.21.15.1:0/1894960451 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f1af000f840 con 0x7f1afc103240 2024-05-04T03:18:05.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.703+0000 7f1af9ffb700 1 --2- 172.21.15.1:0/1894960451 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1aec0b2900 0x7f1aec0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:05.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.703+0000 7f1afbfff700 1 --2- 172.21.15.1:0/1894960451 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1aec0b2900 0x7f1aec0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:05.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.703+0000 7f1af9ffb700 1 -- 172.21.15.1:0/1894960451 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f1af00e1b40 con 0x7f1afc103240 2024-05-04T03:18:05.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.704+0000 7f1afbfff700 1 --2- 172.21.15.1:0/1894960451 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1aec0b2900 0x7f1aec0b4e10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800b5c0 tx=0x7f1ae801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:05.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.708+0000 7f1af9ffb700 1 -- 172.21.15.1:0/1894960451 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1af0020030 con 0x7f1afc103240 2024-05-04T03:18:05.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.953+0000 7f1b02e86700 1 -- 172.21.15.1:0/1894960451 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f1afc063c00 con 0x7f1aec0b2900 2024-05-04T03:18:05.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.954+0000 7f1af9ffb700 1 -- 172.21.15.1:0/1894960451 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f1afc063c00 con 0x7f1aec0b2900 2024-05-04T03:18:05.956 INFO:teuthology.orchestra.run.smithi001.stdout:Backend: cephadm 2024-05-04T03:18:05.956 INFO:teuthology.orchestra.run.smithi001.stdout:Available: Yes 2024-05-04T03:18:05.956 INFO:teuthology.orchestra.run.smithi001.stdout:Paused: No 2024-05-04T03:18:05.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.957+0000 7f1b02e86700 1 -- 172.21.15.1:0/1894960451 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1aec0b2900 msgr2=0x7f1aec0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:05.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.957+0000 7f1b02e86700 1 --2- 172.21.15.1:0/1894960451 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1aec0b2900 0x7f1aec0b4e10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800b5c0 tx=0x7f1ae801a040 comp rx=0 tx=0).stop 2024-05-04T03:18:05.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.957+0000 7f1b02e86700 1 -- 172.21.15.1:0/1894960451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1afc103240 msgr2=0x7f1afc06c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:05.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.957+0000 7f1b02e86700 1 --2- 172.21.15.1:0/1894960451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1afc103240 0x7f1afc06c090 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1af000d950 tx=0x7f1af000dc60 comp rx=0 tx=0).stop 2024-05-04T03:18:05.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.957+0000 7f1b02e86700 1 -- 172.21.15.1:0/1894960451 shutdown_connections 2024-05-04T03:18:05.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.957+0000 7f1b02e86700 1 --2- 172.21.15.1:0/1894960451 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1aec0b2900 0x7f1aec0b4e10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:05.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.957+0000 7f1b02e86700 1 --2- 172.21.15.1:0/1894960451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1afc104d10 0x7f1afc1a1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:05.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.957+0000 7f1b02e86700 1 --2- 172.21.15.1:0/1894960451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1afc103240 0x7f1afc06c090 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:05.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.957+0000 7f1b02e86700 1 --2- 172.21.15.1:0/1894960451 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1afc104370 0x7f1afc19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:05.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.957+0000 7f1b02e86700 1 -- 172.21.15.1:0/1894960451 >> 172.21.15.1:0/1894960451 conn(0x7f1afc0fe880 msgr2=0x7f1afc1075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:05.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.957+0000 7f1b02e86700 1 -- 172.21.15.1:0/1894960451 shutdown_connections 2024-05-04T03:18:05.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:05.957+0000 7f1b02e86700 1 -- 172.21.15.1:0/1894960451 wait complete. 2024-05-04T03:18:06.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:06 smithi001 ceph-mon[29731]: pgmap v359: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-04T03:18:06.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:06 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:18:06.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:06 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:18:06.667 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-04T03:18:06.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:06 smithi077 ceph-mon[33531]: pgmap v359: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-04T03:18:06.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:06 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:18:06.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:06 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:18:06.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:06 smithi072 ceph-mon[33332]: pgmap v359: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-04T03:18:06.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:06 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:18:06.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:06 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:18:06.918 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:18:07.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:07 smithi001 ceph-mon[29731]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:18:07.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:07 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:18:07.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:07 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:18:07.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:07 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:18:07.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:07 smithi077 ceph-mon[33531]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:18:07.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:07 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:18:07.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:07 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:18:07.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:07 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:18:07.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:07 smithi072 ceph-mon[33332]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:18:07.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:07 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:18:07.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:07 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:18:07.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:07 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:18:08.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.569+0000 7f9792092700 1 -- 172.21.15.1:0/1681002410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f978c105e30 msgr2=0x7f978c10c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:08.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.569+0000 7f9792092700 1 --2- 172.21.15.1:0/1681002410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f978c105e30 0x7f978c10c6e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f977c009990 tx=0x7f977c009ca0 comp rx=0 tx=0).stop 2024-05-04T03:18:08.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.569+0000 7f9792092700 1 -- 172.21.15.1:0/1681002410 shutdown_connections 2024-05-04T03:18:08.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.569+0000 7f9792092700 1 --2- 172.21.15.1:0/1681002410 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f978c104360 0x7f978c104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:08.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.569+0000 7f9792092700 1 --2- 172.21.15.1:0/1681002410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f978c105e30 0x7f978c10c6e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:08.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.569+0000 7f9792092700 1 --2- 172.21.15.1:0/1681002410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f978c105490 0x7f978c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:08.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.569+0000 7f9792092700 1 -- 172.21.15.1:0/1681002410 >> 172.21.15.1:0/1681002410 conn(0x7f978c0ff9c0 msgr2=0x7f978c101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:08.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.569+0000 7f9792092700 1 -- 172.21.15.1:0/1681002410 shutdown_connections 2024-05-04T03:18:08.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.569+0000 7f9792092700 1 -- 172.21.15.1:0/1681002410 wait complete. 2024-05-04T03:18:08.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.570+0000 7f9792092700 1 Processor -- start 2024-05-04T03:18:08.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.570+0000 7f9792092700 1 -- start start 2024-05-04T03:18:08.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.570+0000 7f9792092700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f978c104360 0x7f978c19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:08.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.570+0000 7f9792092700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f978c105490 0x7f978c19ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:08.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.570+0000 7f9792092700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f978c105e30 0x7f978c1a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:08.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.570+0000 7f9792092700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f978c10fde0 con 0x7f978c104360 2024-05-04T03:18:08.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.570+0000 7f9792092700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f978c10fc60 con 0x7f978c105e30 2024-05-04T03:18:08.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.570+0000 7f9792092700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f978c10ff60 con 0x7f978c105490 2024-05-04T03:18:08.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.570+0000 7f978b7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f978c104360 0x7f978c19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:08.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.570+0000 7f978b7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f978c104360 0x7f978c19a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41152/0 (socket says 172.21.15.1:41152) 2024-05-04T03:18:08.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.570+0000 7f978b7fe700 1 -- 172.21.15.1:0/1570296414 learned_addr learned my addr 172.21.15.1:0/1570296414 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:08.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.570+0000 7f978bfff700 1 --2- 172.21.15.1:0/1570296414 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f978c105e30 0x7f978c1a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:08.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.570+0000 7f978affd700 1 --2- 172.21.15.1:0/1570296414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f978c105490 0x7f978c19ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:08.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.571+0000 7f978b7fe700 1 -- 172.21.15.1:0/1570296414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f978c105490 msgr2=0x7f978c19ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:08.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.571+0000 7f978b7fe700 1 --2- 172.21.15.1:0/1570296414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f978c105490 0x7f978c19ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:08.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.571+0000 7f978b7fe700 1 -- 172.21.15.1:0/1570296414 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f978c105e30 msgr2=0x7f978c1a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:08.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.571+0000 7f978b7fe700 1 --2- 172.21.15.1:0/1570296414 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f978c105e30 0x7f978c1a1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:08.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.571+0000 7f978b7fe700 1 -- 172.21.15.1:0/1570296414 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978c1a23f0 con 0x7f978c104360 2024-05-04T03:18:08.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.571+0000 7f978b7fe700 1 --2- 172.21.15.1:0/1570296414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f978c104360 0x7f978c19a920 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9784007dd0 tx=0x7f978400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:08.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.571+0000 7f9788ff9700 1 -- 172.21.15.1:0/1570296414 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9784016020 con 0x7f978c104360 2024-05-04T03:18:08.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.571+0000 7f9792092700 1 -- 172.21.15.1:0/1570296414 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f978c1a26e0 con 0x7f978c104360 2024-05-04T03:18:08.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.571+0000 7f9788ff9700 1 -- 172.21.15.1:0/1570296414 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f978400ac90 con 0x7f978c104360 2024-05-04T03:18:08.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.571+0000 7f9788ff9700 1 -- 172.21.15.1:0/1570296414 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97840115e0 con 0x7f978c104360 2024-05-04T03:18:08.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.571+0000 7f9792092700 1 -- 172.21.15.1:0/1570296414 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f978c1a2c20 con 0x7f978c104360 2024-05-04T03:18:08.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.572+0000 7f9792092700 1 -- 172.21.15.1:0/1570296414 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f978c0512c0 con 0x7f978c104360 2024-05-04T03:18:08.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.574+0000 7f9788ff9700 1 -- 172.21.15.1:0/1570296414 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f9784019020 con 0x7f978c104360 2024-05-04T03:18:08.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.574+0000 7f9788ff9700 1 --2- 172.21.15.1:0/1570296414 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f97780b2880 0x7f97780b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:08.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.575+0000 7f978affd700 1 --2- 172.21.15.1:0/1570296414 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f97780b2880 0x7f97780b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:08.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.575+0000 7f9788ff9700 1 -- 172.21.15.1:0/1570296414 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f97840e2190 con 0x7f978c104360 2024-05-04T03:18:08.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.575+0000 7f978affd700 1 --2- 172.21.15.1:0/1570296414 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f97780b2880 0x7f97780b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9774005950 tx=0x7f977400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:08.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.578+0000 7f9788ff9700 1 -- 172.21.15.1:0/1570296414 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9784020030 con 0x7f978c104360 2024-05-04T03:18:08.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:08 smithi001 ceph-mon[29731]: pgmap v360: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-04T03:18:08.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:08 smithi077 ceph-mon[33531]: pgmap v360: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-04T03:18:08.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:08 smithi072 ceph-mon[33332]: pgmap v360: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-04T03:18:08.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.802+0000 7f9792092700 1 -- 172.21.15.1:0/1570296414 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f978c063c00 con 0x7f97780b2880 2024-05-04T03:18:08.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.815+0000 7f9788ff9700 1 -- 172.21.15.1:0/1570296414 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f978c063c00 con 0x7f97780b2880 2024-05-04T03:18:08.817 INFO:teuthology.orchestra.run.smithi001.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-04T03:18:08.817 INFO:teuthology.orchestra.run.smithi001.stdout:mds.a smithi001 running (2m) 2s ago 2m 84.6M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 70e62fe94017 2024-05-04T03:18:08.818 INFO:teuthology.orchestra.run.smithi001.stdout:mds.b smithi072 running (2m) 5s ago 2m 99.2M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 1e23e5da397c 2024-05-04T03:18:08.818 INFO:teuthology.orchestra.run.smithi001.stdout:mds.c smithi077 running (2m) 5s ago 2m 88.9M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 8c5f0f199e56 2024-05-04T03:18:08.818 INFO:teuthology.orchestra.run.smithi001.stdout:mds.d smithi001 running (2m) 2s ago 2m 85.0M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 efad207677c1 2024-05-04T03:18:08.818 INFO:teuthology.orchestra.run.smithi001.stdout:mds.e smithi072 running (2m) 5s ago 2m 90.1M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 f59a9e96a9bc 2024-05-04T03:18:08.818 INFO:teuthology.orchestra.run.smithi001.stdout:mds.f smithi077 running (2m) 5s ago 2m 87.2M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 f7ab088fc14c 2024-05-04T03:18:08.818 INFO:teuthology.orchestra.run.smithi001.stdout:mds.g smithi001 running (2m) 2s ago 2m 84.9M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 2a50cbfb89a0 2024-05-04T03:18:08.818 INFO:teuthology.orchestra.run.smithi001.stdout:mds.h smithi072 running (2m) 5s ago 2m 88.2M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 0cc596835e9b 2024-05-04T03:18:08.818 INFO:teuthology.orchestra.run.smithi001.stdout:mds.i smithi077 running (2m) 5s ago 2m 92.0M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 33378ccac362 2024-05-04T03:18:08.818 INFO:teuthology.orchestra.run.smithi001.stdout:mds.j smithi001 running (3m) 2s ago 3m 82.4M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 fe7d3006db2c 2024-05-04T03:18:08.818 INFO:teuthology.orchestra.run.smithi001.stdout:mds.k smithi072 running (3m) 5s ago 3m 82.6M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 040b9c4f8036 2024-05-04T03:18:08.818 INFO:teuthology.orchestra.run.smithi001.stdout:mds.l smithi077 running (3m) 5s ago 3m 81.5M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 f9f7deb8f457 2024-05-04T03:18:08.818 INFO:teuthology.orchestra.run.smithi001.stdout:mgr.x smithi001 *:9283,8765 running (10m) 2s ago 10m 596M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 5918334689e4 2024-05-04T03:18:08.818 INFO:teuthology.orchestra.run.smithi001.stdout:mgr.y smithi072 *:8443,8765 running (8m) 5s ago 8m 538M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 d698f7723d6e 2024-05-04T03:18:08.819 INFO:teuthology.orchestra.run.smithi001.stdout:mgr.z smithi077 *:8443,8765 running (8m) 5s ago 8m 536M - 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 70f83db8ce86 2024-05-04T03:18:08.819 INFO:teuthology.orchestra.run.smithi001.stdout:mon.a smithi001 running (10m) 2s ago 10m 120M 2048M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 f52e77ebe41e 2024-05-04T03:18:08.819 INFO:teuthology.orchestra.run.smithi001.stdout:mon.b smithi072 running (8m) 5s ago 8m 111M 2048M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 6bd75c9906c1 2024-05-04T03:18:08.819 INFO:teuthology.orchestra.run.smithi001.stdout:mon.c smithi077 running (8m) 5s ago 8m 109M 2048M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 2f48466bb790 2024-05-04T03:18:08.819 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0 smithi001 running (8m) 2s ago 8m 123M 4273M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 0fbc01c85d8b 2024-05-04T03:18:08.819 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1 smithi072 running (7m) 5s ago 7m 125M 4272M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 95e548501c95 2024-05-04T03:18:08.819 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2 smithi077 running (7m) 5s ago 7m 127M 4272M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 e3b02e68651d 2024-05-04T03:18:08.819 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3 smithi001 running (6m) 2s ago 6m 122M 4273M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 994ea795d932 2024-05-04T03:18:08.819 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4 smithi072 running (6m) 5s ago 6m 112M 4272M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 492d1105c52e 2024-05-04T03:18:08.819 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5 smithi077 running (6m) 5s ago 6m 125M 4272M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 309607a9285e 2024-05-04T03:18:08.819 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6 smithi001 running (5m) 2s ago 5m 120M 4273M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 e00742d9417e 2024-05-04T03:18:08.819 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7 smithi072 running (5m) 5s ago 5m 126M 4272M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 06eb9b19c757 2024-05-04T03:18:08.819 INFO:teuthology.orchestra.run.smithi001.stdout:osd.8 smithi077 running (4m) 5s ago 4m 121M 4272M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 d0855f59db56 2024-05-04T03:18:08.819 INFO:teuthology.orchestra.run.smithi001.stdout:osd.9 smithi001 running (4m) 2s ago 4m 112M 4273M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 a1fa31ce2687 2024-05-04T03:18:08.820 INFO:teuthology.orchestra.run.smithi001.stdout:osd.10 smithi072 running (3m) 5s ago 3m 115M 4272M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 96b02b0a9cd6 2024-05-04T03:18:08.820 INFO:teuthology.orchestra.run.smithi001.stdout:osd.11 smithi077 running (3m) 5s ago 3m 123M 4272M 19.0.0-3522-gbd4e00a3 afca8dfc9ff0 0973570496ae 2024-05-04T03:18:08.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.817+0000 7f9792092700 1 -- 172.21.15.1:0/1570296414 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f97780b2880 msgr2=0x7f97780b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:08.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.817+0000 7f9792092700 1 --2- 172.21.15.1:0/1570296414 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f97780b2880 0x7f97780b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9774005950 tx=0x7f977400a300 comp rx=0 tx=0).stop 2024-05-04T03:18:08.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.817+0000 7f9792092700 1 -- 172.21.15.1:0/1570296414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f978c104360 msgr2=0x7f978c19a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:08.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.817+0000 7f9792092700 1 --2- 172.21.15.1:0/1570296414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f978c104360 0x7f978c19a920 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9784007dd0 tx=0x7f978400a340 comp rx=0 tx=0).stop 2024-05-04T03:18:08.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.818+0000 7f9792092700 1 -- 172.21.15.1:0/1570296414 shutdown_connections 2024-05-04T03:18:08.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.818+0000 7f9792092700 1 --2- 172.21.15.1:0/1570296414 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f978c105e30 0x7f978c1a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:08.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.818+0000 7f9792092700 1 --2- 172.21.15.1:0/1570296414 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f97780b2880 0x7f97780b4d90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:08.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.818+0000 7f9792092700 1 --2- 172.21.15.1:0/1570296414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f978c104360 0x7f978c19a920 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:08.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.818+0000 7f9792092700 1 --2- 172.21.15.1:0/1570296414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f978c105490 0x7f978c19ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:08.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.818+0000 7f9792092700 1 -- 172.21.15.1:0/1570296414 >> 172.21.15.1:0/1570296414 conn(0x7f978c0ff9c0 msgr2=0x7f978c1086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:08.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.818+0000 7f9792092700 1 -- 172.21.15.1:0/1570296414 shutdown_connections 2024-05-04T03:18:08.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:08.818+0000 7f9792092700 1 -- 172.21.15.1:0/1570296414 wait complete. 2024-05-04T03:18:09.449 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-04T03:18:09.698 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:18:10.516 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:10 smithi001 ceph-mon[29731]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:18:10.516 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:10 smithi001 ceph-mon[29731]: pgmap v361: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-04T03:18:10.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:10 smithi077 ceph-mon[33531]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:18:10.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:10 smithi077 ceph-mon[33531]: pgmap v361: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-04T03:18:10.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:10 smithi072 ceph-mon[33332]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:18:10.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:10 smithi072 ceph-mon[33332]: pgmap v361: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-04T03:18:11.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.359+0000 7fdb3f913700 1 -- 172.21.15.1:0/388490055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb38104380 msgr2=0x7fdb38104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:11.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.359+0000 7fdb3f913700 1 --2- 172.21.15.1:0/388490055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb38104380 0x7fdb38104790 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdb34009990 tx=0x7fdb34009ca0 comp rx=0 tx=0).stop 2024-05-04T03:18:11.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.359+0000 7fdb3f913700 1 -- 172.21.15.1:0/388490055 shutdown_connections 2024-05-04T03:18:11.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.359+0000 7fdb3f913700 1 --2- 172.21.15.1:0/388490055 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb38103250 0x7fdb38103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:11.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.359+0000 7fdb3f913700 1 --2- 172.21.15.1:0/388490055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb38104d20 0x7fdb3810b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:11.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.359+0000 7fdb3f913700 1 --2- 172.21.15.1:0/388490055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb38104380 0x7fdb38104790 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:11.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.359+0000 7fdb3f913700 1 -- 172.21.15.1:0/388490055 >> 172.21.15.1:0/388490055 conn(0x7fdb380fe890 msgr2=0x7fdb38100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:11.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.359+0000 7fdb3f913700 1 -- 172.21.15.1:0/388490055 shutdown_connections 2024-05-04T03:18:11.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.359+0000 7fdb3f913700 1 -- 172.21.15.1:0/388490055 wait complete. 2024-05-04T03:18:11.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.360+0000 7fdb3f913700 1 Processor -- start 2024-05-04T03:18:11.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.360+0000 7fdb3f913700 1 -- start start 2024-05-04T03:18:11.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.360+0000 7fdb3f913700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb38103250 0x7fdb3819a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:11.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.360+0000 7fdb3f913700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb38104380 0x7fdb3819aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:11.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.360+0000 7fdb3f913700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb38104d20 0x7fdb381a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:11.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.360+0000 7fdb3f913700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3810fe50 con 0x7fdb38104380 2024-05-04T03:18:11.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.360+0000 7fdb3f913700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3810fcd0 con 0x7fdb38104d20 2024-05-04T03:18:11.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.360+0000 7fdb3f913700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3810ffd0 con 0x7fdb38103250 2024-05-04T03:18:11.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.360+0000 7fdb3ceae700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb38104380 0x7fdb3819aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:11.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.360+0000 7fdb3ceae700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb38104380 0x7fdb3819aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41176/0 (socket says 172.21.15.1:41176) 2024-05-04T03:18:11.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.360+0000 7fdb3ceae700 1 -- 172.21.15.1:0/1287130129 learned_addr learned my addr 172.21.15.1:0/1287130129 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:11.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.360+0000 7fdb3d6af700 1 --2- 172.21.15.1:0/1287130129 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb38103250 0x7fdb3819a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:11.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.361+0000 7fdb3deb0700 1 --2- 172.21.15.1:0/1287130129 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb38104d20 0x7fdb381a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:11.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.361+0000 7fdb3ceae700 1 -- 172.21.15.1:0/1287130129 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb38103250 msgr2=0x7fdb3819a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:11.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.361+0000 7fdb3ceae700 1 --2- 172.21.15.1:0/1287130129 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb38103250 0x7fdb3819a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:11.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.361+0000 7fdb3ceae700 1 -- 172.21.15.1:0/1287130129 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb38104d20 msgr2=0x7fdb381a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:11.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.361+0000 7fdb3ceae700 1 --2- 172.21.15.1:0/1287130129 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb38104d20 0x7fdb381a1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:11.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.361+0000 7fdb3ceae700 1 -- 172.21.15.1:0/1287130129 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb381a24d0 con 0x7fdb38104380 2024-05-04T03:18:11.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.361+0000 7fdb3ceae700 1 --2- 172.21.15.1:0/1287130129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb38104380 0x7fdb3819aeb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdb34003800 tx=0x7fdb340047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:11.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.361+0000 7fdb2e7fc700 1 -- 172.21.15.1:0/1287130129 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb34027070 con 0x7fdb38104380 2024-05-04T03:18:11.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.361+0000 7fdb3f913700 1 -- 172.21.15.1:0/1287130129 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb381a2760 con 0x7fdb38104380 2024-05-04T03:18:11.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.361+0000 7fdb2e7fc700 1 -- 172.21.15.1:0/1287130129 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb340041c0 con 0x7fdb38104380 2024-05-04T03:18:11.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.361+0000 7fdb2e7fc700 1 -- 172.21.15.1:0/1287130129 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3400f460 con 0x7fdb38104380 2024-05-04T03:18:11.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.361+0000 7fdb3f913700 1 -- 172.21.15.1:0/1287130129 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb381a2c30 con 0x7fdb38104380 2024-05-04T03:18:11.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.362+0000 7fdb3f913700 1 -- 172.21.15.1:0/1287130129 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb380512c0 con 0x7fdb38104380 2024-05-04T03:18:11.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.364+0000 7fdb2e7fc700 1 -- 172.21.15.1:0/1287130129 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fdb3402a070 con 0x7fdb38104380 2024-05-04T03:18:11.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.365+0000 7fdb2e7fc700 1 --2- 172.21.15.1:0/1287130129 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fdb240b2880 0x7fdb240b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:11.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.365+0000 7fdb3d6af700 1 --2- 172.21.15.1:0/1287130129 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fdb240b2880 0x7fdb240b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:11.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.365+0000 7fdb2e7fc700 1 -- 172.21.15.1:0/1287130129 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fdb340e3390 con 0x7fdb38104380 2024-05-04T03:18:11.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.365+0000 7fdb3d6af700 1 --2- 172.21.15.1:0/1287130129 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fdb240b2880 0x7fdb240b4d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdb38104280 tx=0x7fdb2800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:11.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.369+0000 7fdb2e7fc700 1 -- 172.21.15.1:0/1287130129 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdb34013070 con 0x7fdb38104380 2024-05-04T03:18:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.593+0000 7fdb3f913700 1 -- 172.21.15.1:0/1287130129 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fdb38063c00 con 0x7fdb240b2880 2024-05-04T03:18:11.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.600+0000 7fdb2e7fc700 1 -- 172.21.15.1:0/1287130129 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fdb38063c00 con 0x7fdb240b2880 2024-05-04T03:18:11.601 INFO:teuthology.orchestra.run.smithi001.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-04T03:18:11.602 INFO:teuthology.orchestra.run.smithi001.stdout:mds.all 12/12 8s ago 3m smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi072=b;smithi072=e;smithi072=h;smithi072=k;smithi077=c;smithi077=f;smithi077=i;smithi077=l;count:12 2024-05-04T03:18:11.602 INFO:teuthology.orchestra.run.smithi001.stdout:mgr 3/3 8s ago 8m smithi001=x;smithi072=y;smithi077=z;count:3 2024-05-04T03:18:11.602 INFO:teuthology.orchestra.run.smithi001.stdout:mon 3/3 8s ago 8m smithi001:172.21.15.1=a;smithi072:172.21.15.72=b;smithi077:172.21.15.77=c;count:3 2024-05-04T03:18:11.602 INFO:teuthology.orchestra.run.smithi001.stdout:osd 12 8s ago - 2024-05-04T03:18:11.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.602+0000 7fdb3f913700 1 -- 172.21.15.1:0/1287130129 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fdb240b2880 msgr2=0x7fdb240b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:11.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.602+0000 7fdb3f913700 1 --2- 172.21.15.1:0/1287130129 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fdb240b2880 0x7fdb240b4d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdb38104280 tx=0x7fdb2800a380 comp rx=0 tx=0).stop 2024-05-04T03:18:11.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.602+0000 7fdb3f913700 1 -- 172.21.15.1:0/1287130129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb38104380 msgr2=0x7fdb3819aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:11.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.602+0000 7fdb3f913700 1 --2- 172.21.15.1:0/1287130129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb38104380 0x7fdb3819aeb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdb34003800 tx=0x7fdb340047b0 comp rx=0 tx=0).stop 2024-05-04T03:18:11.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.602+0000 7fdb3f913700 1 -- 172.21.15.1:0/1287130129 shutdown_connections 2024-05-04T03:18:11.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.602+0000 7fdb3f913700 1 --2- 172.21.15.1:0/1287130129 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb38104d20 0x7fdb381a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:11.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.602+0000 7fdb3f913700 1 --2- 172.21.15.1:0/1287130129 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fdb240b2880 0x7fdb240b4d90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:11.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.602+0000 7fdb3f913700 1 --2- 172.21.15.1:0/1287130129 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb38103250 0x7fdb3819a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:11.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.602+0000 7fdb3f913700 1 --2- 172.21.15.1:0/1287130129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb38104380 0x7fdb3819aeb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:11.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.602+0000 7fdb3f913700 1 -- 172.21.15.1:0/1287130129 >> 172.21.15.1:0/1287130129 conn(0x7fdb380fe890 msgr2=0x7fdb381075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:11.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.602+0000 7fdb3f913700 1 -- 172.21.15.1:0/1287130129 shutdown_connections 2024-05-04T03:18:11.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:11.602+0000 7fdb3f913700 1 -- 172.21.15.1:0/1287130129 wait complete. 2024-05-04T03:18:12.209 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-04T03:18:12.458 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:18:12.653 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:12 smithi001 ceph-mon[29731]: pgmap v362: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:18:12.653 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:12 smithi001 ceph-mon[29731]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:18:12.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:12 smithi077 ceph-mon[33531]: pgmap v362: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:18:12.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:12 smithi077 ceph-mon[33531]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:18:12.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:12 smithi072 ceph-mon[33332]: pgmap v362: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:18:12.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:12 smithi072 ceph-mon[33332]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:18:14.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.185+0000 7fcbb7324700 1 -- 172.21.15.1:0/268440107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcbb0103ed0 msgr2=0x7fcbb01042e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:14.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.185+0000 7fcbb7324700 1 --2- 172.21.15.1:0/268440107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcbb0103ed0 0x7fcbb01042e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcba800b0a0 tx=0x7fcba800b3b0 comp rx=0 tx=0).stop 2024-05-04T03:18:14.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.185+0000 7fcbb7324700 1 -- 172.21.15.1:0/268440107 shutdown_connections 2024-05-04T03:18:14.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.185+0000 7fcbb7324700 1 --2- 172.21.15.1:0/268440107 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcbb0102c30 0x7fcbb0103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:14.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.185+0000 7fcbb7324700 1 --2- 172.21.15.1:0/268440107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcbb0103ed0 0x7fcbb01042e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:14.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.185+0000 7fcbb7324700 1 --2- 172.21.15.1:0/268440107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbb0103580 0x7fcbb0103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:14.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.185+0000 7fcbb7324700 1 -- 172.21.15.1:0/268440107 >> 172.21.15.1:0/268440107 conn(0x7fcbb00fe880 msgr2=0x7fcbb0100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:14.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.185+0000 7fcbb7324700 1 -- 172.21.15.1:0/268440107 shutdown_connections 2024-05-04T03:18:14.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.185+0000 7fcbb7324700 1 -- 172.21.15.1:0/268440107 wait complete. 2024-05-04T03:18:14.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.186+0000 7fcbb7324700 1 Processor -- start 2024-05-04T03:18:14.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.186+0000 7fcbb7324700 1 -- start start 2024-05-04T03:18:14.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.186+0000 7fcbb7324700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcbb0102c30 0x7fcbb019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:14.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.186+0000 7fcbb7324700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcbb0103580 0x7fcbb019cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:14.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.186+0000 7fcbb7324700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbb0103ed0 0x7fcbb01a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:14.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.186+0000 7fcbb7324700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb0111fa0 con 0x7fcbb0102c30 2024-05-04T03:18:14.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.186+0000 7fcbb7324700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb0111e20 con 0x7fcbb0103580 2024-05-04T03:18:14.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.186+0000 7fcbb7324700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb0112120 con 0x7fcbb0103ed0 2024-05-04T03:18:14.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.186+0000 7fcbb50c0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcbb0102c30 0x7fcbb019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:14.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.186+0000 7fcbb50c0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcbb0102c30 0x7fcbb019ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35244/0 (socket says 172.21.15.1:35244) 2024-05-04T03:18:14.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.186+0000 7fcbb50c0700 1 -- 172.21.15.1:0/3783388556 learned_addr learned my addr 172.21.15.1:0/3783388556 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:14.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.186+0000 7fcbb58c1700 1 --2- 172.21.15.1:0/3783388556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbb0103ed0 0x7fcbb01a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:14.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.186+0000 7fcbb48bf700 1 --2- 172.21.15.1:0/3783388556 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcbb0103580 0x7fcbb019cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:14.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.186+0000 7fcbb50c0700 1 -- 172.21.15.1:0/3783388556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbb0103ed0 msgr2=0x7fcbb01a3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:14.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.186+0000 7fcbb50c0700 1 --2- 172.21.15.1:0/3783388556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbb0103ed0 0x7fcbb01a3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:14.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.186+0000 7fcbb50c0700 1 -- 172.21.15.1:0/3783388556 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcbb0103580 msgr2=0x7fcbb019cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:14.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.186+0000 7fcbb50c0700 1 --2- 172.21.15.1:0/3783388556 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcbb0103580 0x7fcbb019cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:14.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.186+0000 7fcbb50c0700 1 -- 172.21.15.1:0/3783388556 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb01a4560 con 0x7fcbb0102c30 2024-05-04T03:18:14.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.187+0000 7fcbb48bf700 1 --2- 172.21.15.1:0/3783388556 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcbb0103580 0x7fcbb019cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:18:14.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.187+0000 7fcbb50c0700 1 --2- 172.21.15.1:0/3783388556 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcbb0102c30 0x7fcbb019ca90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00c9e0 tx=0x7fcbac00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:14.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.187+0000 7fcba67fc700 1 -- 172.21.15.1:0/3783388556 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac007aa0 con 0x7fcbb0102c30 2024-05-04T03:18:14.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.187+0000 7fcbb7324700 1 -- 172.21.15.1:0/3783388556 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb01a4850 con 0x7fcbb0102c30 2024-05-04T03:18:14.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.187+0000 7fcba67fc700 1 -- 172.21.15.1:0/3783388556 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbac007c40 con 0x7fcbb0102c30 2024-05-04T03:18:14.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.187+0000 7fcba67fc700 1 -- 172.21.15.1:0/3783388556 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac018660 con 0x7fcbb0102c30 2024-05-04T03:18:14.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.187+0000 7fcbb7324700 1 -- 172.21.15.1:0/3783388556 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb01a4d90 con 0x7fcbb0102c30 2024-05-04T03:18:14.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.188+0000 7fcbb7324700 1 -- 172.21.15.1:0/3783388556 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbb00512c0 con 0x7fcbb0102c30 2024-05-04T03:18:14.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.190+0000 7fcba67fc700 1 -- 172.21.15.1:0/3783388556 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fcbac018800 con 0x7fcbb0102c30 2024-05-04T03:18:14.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.190+0000 7fcba67fc700 1 --2- 172.21.15.1:0/3783388556 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fcb9c0b2880 0x7fcb9c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:14.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.191+0000 7fcbb48bf700 1 --2- 172.21.15.1:0/3783388556 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fcb9c0b2880 0x7fcb9c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:14.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.191+0000 7fcba67fc700 1 -- 172.21.15.1:0/3783388556 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fcbac0e23c0 con 0x7fcbb0102c30 2024-05-04T03:18:14.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.191+0000 7fcbb48bf700 1 --2- 172.21.15.1:0/3783388556 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fcb9c0b2880 0x7fcb9c0b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcba0005950 tx=0x7fcba00058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:14.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.194+0000 7fcba67fc700 1 -- 172.21.15.1:0/3783388556 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcbac014030 con 0x7fcbb0102c30 2024-05-04T03:18:14.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.416+0000 7fcbb7324700 1 -- 172.21.15.1:0/3783388556 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fcbb0063c00 con 0x7fcb9c0b2880 2024-05-04T03:18:14.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.418+0000 7fcba67fc700 1 -- 172.21.15.1:0/3783388556 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+187 (secure 0 0 0) 0x7fcbb0063c00 con 0x7fcb9c0b2880 2024-05-04T03:18:14.419 INFO:teuthology.orchestra.run.smithi001.stdout:HOST ADDR LABELS STATUS 2024-05-04T03:18:14.419 INFO:teuthology.orchestra.run.smithi001.stdout:smithi001 172.21.15.1 2024-05-04T03:18:14.420 INFO:teuthology.orchestra.run.smithi001.stdout:smithi072 172.21.15.72 2024-05-04T03:18:14.420 INFO:teuthology.orchestra.run.smithi001.stdout:smithi077 172.21.15.77 2024-05-04T03:18:14.420 INFO:teuthology.orchestra.run.smithi001.stdout:3 hosts in cluster 2024-05-04T03:18:14.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.420+0000 7fcbb7324700 1 -- 172.21.15.1:0/3783388556 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fcb9c0b2880 msgr2=0x7fcb9c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:14.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.420+0000 7fcbb7324700 1 --2- 172.21.15.1:0/3783388556 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fcb9c0b2880 0x7fcb9c0b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcba0005950 tx=0x7fcba00058e0 comp rx=0 tx=0).stop 2024-05-04T03:18:14.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.420+0000 7fcbb7324700 1 -- 172.21.15.1:0/3783388556 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcbb0102c30 msgr2=0x7fcbb019ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:14.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.420+0000 7fcbb7324700 1 --2- 172.21.15.1:0/3783388556 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcbb0102c30 0x7fcbb019ca90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00c9e0 tx=0x7fcbac00ccf0 comp rx=0 tx=0).stop 2024-05-04T03:18:14.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.420+0000 7fcbb7324700 1 -- 172.21.15.1:0/3783388556 shutdown_connections 2024-05-04T03:18:14.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.420+0000 7fcbb7324700 1 --2- 172.21.15.1:0/3783388556 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fcb9c0b2880 0x7fcb9c0b4d90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:14.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.420+0000 7fcbb7324700 1 --2- 172.21.15.1:0/3783388556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbb0103ed0 0x7fcbb01a3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:14.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.420+0000 7fcbb7324700 1 --2- 172.21.15.1:0/3783388556 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcbb0102c30 0x7fcbb019ca90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:14.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.420+0000 7fcbb7324700 1 --2- 172.21.15.1:0/3783388556 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcbb0103580 0x7fcbb019cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:14.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.420+0000 7fcbb7324700 1 -- 172.21.15.1:0/3783388556 >> 172.21.15.1:0/3783388556 conn(0x7fcbb00fe880 msgr2=0x7fcbb010b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:14.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.420+0000 7fcbb7324700 1 -- 172.21.15.1:0/3783388556 shutdown_connections 2024-05-04T03:18:14.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:14.420+0000 7fcbb7324700 1 -- 172.21.15.1:0/3783388556 wait complete. 2024-05-04T03:18:14.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:14 smithi077 ceph-mon[33531]: pgmap v363: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:18:14.720 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:14 smithi001 ceph-mon[29731]: pgmap v363: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:18:14.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:14 smithi072 ceph-mon[33332]: pgmap v363: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:18:15.035 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-04T03:18:15.286 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:18:15.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:15 smithi077 ceph-mon[33531]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:18:15.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:15 smithi001 ceph-mon[29731]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:18:15.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:15 smithi072 ceph-mon[33332]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:18:16.625 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:16 smithi001 ceph-mon[29731]: pgmap v364: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-04T03:18:16.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:16 smithi077 ceph-mon[33531]: pgmap v364: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-04T03:18:16.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:16 smithi072 ceph-mon[33332]: pgmap v364: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-04T03:18:16.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.867+0000 7f6eaebe7700 1 -- 172.21.15.1:0/4257194573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea8103b10 msgr2=0x7f6ea8103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:16.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.867+0000 7f6eaebe7700 1 --2- 172.21.15.1:0/4257194573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea8103b10 0x7f6ea8103f20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c009990 tx=0x7f6e9c009ca0 comp rx=0 tx=0).stop 2024-05-04T03:18:16.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.867+0000 7f6eaebe7700 1 -- 172.21.15.1:0/4257194573 shutdown_connections 2024-05-04T03:18:16.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.867+0000 7f6eaebe7700 1 --2- 172.21.15.1:0/4257194573 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ea81044b0 0x7f6ea810ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:16.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.867+0000 7f6eaebe7700 1 --2- 172.21.15.1:0/4257194573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ea81029e0 0x7f6ea8102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:16.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.867+0000 7f6eaebe7700 1 --2- 172.21.15.1:0/4257194573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea8103b10 0x7f6ea8103f20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:16.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.867+0000 7f6eaebe7700 1 -- 172.21.15.1:0/4257194573 >> 172.21.15.1:0/4257194573 conn(0x7f6ea80fe020 msgr2=0x7f6ea81004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:16.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.867+0000 7f6eaebe7700 1 -- 172.21.15.1:0/4257194573 shutdown_connections 2024-05-04T03:18:16.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.867+0000 7f6eaebe7700 1 -- 172.21.15.1:0/4257194573 wait complete. 2024-05-04T03:18:16.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.868+0000 7f6eaebe7700 1 Processor -- start 2024-05-04T03:18:16.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.868+0000 7f6eaebe7700 1 -- start start 2024-05-04T03:18:16.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.868+0000 7f6eaebe7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ea81029e0 0x7f6ea819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:16.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.868+0000 7f6eaebe7700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea8103b10 0x7f6ea819ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:16.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.868+0000 7f6eaebe7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ea81044b0 0x7f6ea81a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:16.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.868+0000 7f6eaebe7700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea810ed40 con 0x7f6ea8103b10 2024-05-04T03:18:16.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.868+0000 7f6eaebe7700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea810ebc0 con 0x7f6ea81044b0 2024-05-04T03:18:16.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.868+0000 7f6eaebe7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea810eec0 con 0x7f6ea81029e0 2024-05-04T03:18:16.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.868+0000 7f6ea7fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea8103b10 0x7f6ea819ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:16.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.868+0000 7f6ea7fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea8103b10 0x7f6ea819ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35272/0 (socket says 172.21.15.1:35272) 2024-05-04T03:18:16.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.868+0000 7f6ea7fff700 1 -- 172.21.15.1:0/3627904431 learned_addr learned my addr 172.21.15.1:0/3627904431 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:16.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.868+0000 7f6eac983700 1 --2- 172.21.15.1:0/3627904431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ea81029e0 0x7f6ea819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:16.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.868+0000 7f6ead184700 1 --2- 172.21.15.1:0/3627904431 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ea81044b0 0x7f6ea81a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:16.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.869+0000 7f6ea7fff700 1 -- 172.21.15.1:0/3627904431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ea81029e0 msgr2=0x7f6ea819a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:16.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.869+0000 7f6ea7fff700 1 --2- 172.21.15.1:0/3627904431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ea81029e0 0x7f6ea819a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:16.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.869+0000 7f6ea7fff700 1 -- 172.21.15.1:0/3627904431 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ea81044b0 msgr2=0x7f6ea81a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:16.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.869+0000 7f6ea7fff700 1 --2- 172.21.15.1:0/3627904431 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ea81044b0 0x7f6ea81a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:16.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.869+0000 7f6ea7fff700 1 -- 172.21.15.1:0/3627904431 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea81a24b0 con 0x7f6ea8103b10 2024-05-04T03:18:16.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.869+0000 7f6ea7fff700 1 --2- 172.21.15.1:0/3627904431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea8103b10 0x7f6ea819ae90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c006010 tx=0x7f6e9c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:16.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.869+0000 7f6ea5ffb700 1 -- 172.21.15.1:0/3627904431 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c027070 con 0x7f6ea8103b10 2024-05-04T03:18:16.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.869+0000 7f6eaebe7700 1 -- 172.21.15.1:0/3627904431 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea81a2740 con 0x7f6ea8103b10 2024-05-04T03:18:16.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.869+0000 7f6ea5ffb700 1 -- 172.21.15.1:0/3627904431 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e9c004000 con 0x7f6ea8103b10 2024-05-04T03:18:16.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.869+0000 7f6ea5ffb700 1 -- 172.21.15.1:0/3627904431 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c022d30 con 0x7f6ea8103b10 2024-05-04T03:18:16.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.869+0000 7f6eaebe7700 1 -- 172.21.15.1:0/3627904431 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea81a2c10 con 0x7f6ea8103b10 2024-05-04T03:18:16.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.870+0000 7f6eaebe7700 1 -- 172.21.15.1:0/3627904431 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ea80512c0 con 0x7f6ea8103b10 2024-05-04T03:18:16.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.872+0000 7f6ea5ffb700 1 -- 172.21.15.1:0/3627904431 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f6e9c02a070 con 0x7f6ea8103b10 2024-05-04T03:18:16.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.872+0000 7f6ea5ffb700 1 --2- 172.21.15.1:0/3627904431 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6e900b2880 0x7f6e900b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:16.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.873+0000 7f6eac983700 1 --2- 172.21.15.1:0/3627904431 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6e900b2880 0x7f6e900b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:16.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.873+0000 7f6ea5ffb700 1 -- 172.21.15.1:0/3627904431 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f6e9c0e32f0 con 0x7f6ea8103b10 2024-05-04T03:18:16.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.873+0000 7f6eac983700 1 --2- 172.21.15.1:0/3627904431 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6e900b2880 0x7f6e900b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8103a10 tx=0x7f6e9800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:16.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:16.877+0000 7f6ea5ffb700 1 -- 172.21.15.1:0/3627904431 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6e9c013070 con 0x7f6ea8103b10 2024-05-04T03:18:17.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:17.102+0000 7f6eaebe7700 1 -- 172.21.15.1:0/3627904431 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f6ea8100060 con 0x7f6e900b2880 2024-05-04T03:18:17.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:17.104+0000 7f6ea5ffb700 1 -- 172.21.15.1:0/3627904431 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f6ea8100060 con 0x7f6e900b2880 2024-05-04T03:18:17.105 INFO:teuthology.orchestra.run.smithi001.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-04T03:18:17.105 INFO:teuthology.orchestra.run.smithi001.stdout:smithi001 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513000D480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-05-04T03:18:17.106 INFO:teuthology.orchestra.run.smithi001.stdout:smithi072 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400V7400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-05-04T03:18:17.106 INFO:teuthology.orchestra.run.smithi001.stdout:smithi077 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400ZJ400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-05-04T03:18:17.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:17.106+0000 7f6eaebe7700 1 -- 172.21.15.1:0/3627904431 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6e900b2880 msgr2=0x7f6e900b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:17.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:17.106+0000 7f6eaebe7700 1 --2- 172.21.15.1:0/3627904431 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6e900b2880 0x7f6e900b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8103a10 tx=0x7f6e9800b410 comp rx=0 tx=0).stop 2024-05-04T03:18:17.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:17.106+0000 7f6eaebe7700 1 -- 172.21.15.1:0/3627904431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea8103b10 msgr2=0x7f6ea819ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:17.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:17.106+0000 7f6eaebe7700 1 --2- 172.21.15.1:0/3627904431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea8103b10 0x7f6ea819ae90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c006010 tx=0x7f6e9c0047b0 comp rx=0 tx=0).stop 2024-05-04T03:18:17.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:17.106+0000 7f6eaebe7700 1 -- 172.21.15.1:0/3627904431 shutdown_connections 2024-05-04T03:18:17.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:17.106+0000 7f6eaebe7700 1 --2- 172.21.15.1:0/3627904431 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ea81044b0 0x7f6ea81a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:17.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:17.106+0000 7f6eaebe7700 1 --2- 172.21.15.1:0/3627904431 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6e900b2880 0x7f6e900b4d90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:17.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:17.106+0000 7f6eaebe7700 1 --2- 172.21.15.1:0/3627904431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ea81029e0 0x7f6ea819a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:17.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:17.106+0000 7f6eaebe7700 1 --2- 172.21.15.1:0/3627904431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ea8103b10 0x7f6ea819ae90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:17.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:17.106+0000 7f6eaebe7700 1 -- 172.21.15.1:0/3627904431 >> 172.21.15.1:0/3627904431 conn(0x7f6ea80fe020 msgr2=0x7f6ea80ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:17.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:17.106+0000 7f6eaebe7700 1 -- 172.21.15.1:0/3627904431 shutdown_connections 2024-05-04T03:18:17.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:17.106+0000 7f6eaebe7700 1 -- 172.21.15.1:0/3627904431 wait complete. 2024-05-04T03:18:17.821 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-04T03:18:17.830 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-05-04T03:18:17.831 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-04T03:18:18.079 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:18:18.470 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:18 smithi001 ceph-mon[29731]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:18:18.471 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:18 smithi001 ceph-mon[29731]: pgmap v365: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-04T03:18:18.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:18 smithi077 ceph-mon[33531]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:18:18.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:18 smithi077 ceph-mon[33531]: pgmap v365: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-04T03:18:18.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:18 smithi072 ceph-mon[33332]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:18:18.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:18 smithi072 ceph-mon[33332]: pgmap v365: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-04T03:18:19.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.677+0000 7fecfbcf9700 1 -- 172.21.15.1:0/1163954941 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecf4104e30 msgr2=0x7fecf4107250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:19.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.677+0000 7fecfbcf9700 1 --2- 172.21.15.1:0/1163954941 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecf4104e30 0x7fecf4107250 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fecec009990 tx=0x7fecec009ca0 comp rx=0 tx=0).stop 2024-05-04T03:18:19.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.677+0000 7fecfbcf9700 1 -- 172.21.15.1:0/1163954941 shutdown_connections 2024-05-04T03:18:19.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.677+0000 7fecfbcf9700 1 --2- 172.21.15.1:0/1163954941 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecf410a0f0 0x7fecf410c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:19.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.677+0000 7fecfbcf9700 1 --2- 172.21.15.1:0/1163954941 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecf4104e30 0x7fecf4107250 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:19.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.677+0000 7fecfbcf9700 1 --2- 172.21.15.1:0/1163954941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecf4107790 0x7fecf4109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:19.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.677+0000 7fecfbcf9700 1 -- 172.21.15.1:0/1163954941 >> 172.21.15.1:0/1163954941 conn(0x7fecf40fe870 msgr2=0x7fecf4100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:19.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.677+0000 7fecfbcf9700 1 -- 172.21.15.1:0/1163954941 shutdown_connections 2024-05-04T03:18:19.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.677+0000 7fecfbcf9700 1 -- 172.21.15.1:0/1163954941 wait complete. 2024-05-04T03:18:19.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.678+0000 7fecfbcf9700 1 Processor -- start 2024-05-04T03:18:19.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.678+0000 7fecfbcf9700 1 -- start start 2024-05-04T03:18:19.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.678+0000 7fecfbcf9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecf4104e30 0x7fecf4104690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:19.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.678+0000 7fecfbcf9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecf4107790 0x7fecf4102d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:19.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.678+0000 7fecfbcf9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecf410a0f0 0x7fecf4103280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:19.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.678+0000 7fecfbcf9700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fecf410fd70 con 0x7fecf4107790 2024-05-04T03:18:19.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.678+0000 7fecfbcf9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fecf410fbf0 con 0x7fecf410a0f0 2024-05-04T03:18:19.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.678+0000 7fecfbcf9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fecf410fef0 con 0x7fecf4104e30 2024-05-04T03:18:19.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.678+0000 7fecf9294700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecf4107790 0x7fecf4102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:19.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.678+0000 7fecf9294700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecf4107790 0x7fecf4102d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35294/0 (socket says 172.21.15.1:35294) 2024-05-04T03:18:19.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.678+0000 7fecf9294700 1 -- 172.21.15.1:0/1366511245 learned_addr learned my addr 172.21.15.1:0/1366511245 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:19.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.678+0000 7fecf9a95700 1 --2- 172.21.15.1:0/1366511245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecf4104e30 0x7fecf4104690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:19.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.678+0000 7fecfa296700 1 --2- 172.21.15.1:0/1366511245 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecf410a0f0 0x7fecf4103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:19.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.679+0000 7fecf9294700 1 -- 172.21.15.1:0/1366511245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecf4104e30 msgr2=0x7fecf4104690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:19.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.679+0000 7fecf9294700 1 --2- 172.21.15.1:0/1366511245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecf4104e30 0x7fecf4104690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:19.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.679+0000 7fecf9294700 1 -- 172.21.15.1:0/1366511245 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecf410a0f0 msgr2=0x7fecf4103280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:19.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.679+0000 7fecf9294700 1 --2- 172.21.15.1:0/1366511245 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecf410a0f0 0x7fecf4103280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:19.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.679+0000 7fecf9294700 1 -- 172.21.15.1:0/1366511245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf41039e0 con 0x7fecf4107790 2024-05-04T03:18:19.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.679+0000 7fecf9294700 1 --2- 172.21.15.1:0/1366511245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecf4107790 0x7fecf4102d40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fece800c920 tx=0x7fece800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:19.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.679+0000 7fecf2ffd700 1 -- 172.21.15.1:0/1366511245 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece8004a30 con 0x7fecf4107790 2024-05-04T03:18:19.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.679+0000 7fecf2ffd700 1 -- 172.21.15.1:0/1366511245 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fece80054d0 con 0x7fecf4107790 2024-05-04T03:18:19.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.679+0000 7fecfbcf9700 1 -- 172.21.15.1:0/1366511245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf41a49f0 con 0x7fecf4107790 2024-05-04T03:18:19.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.679+0000 7fecf2ffd700 1 -- 172.21.15.1:0/1366511245 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fece8007c70 con 0x7fecf4107790 2024-05-04T03:18:19.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.679+0000 7fecfbcf9700 1 -- 172.21.15.1:0/1366511245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf41a4ed0 con 0x7fecf4107790 2024-05-04T03:18:19.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.680+0000 7fecfbcf9700 1 -- 172.21.15.1:0/1366511245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecf40512c0 con 0x7fecf4107790 2024-05-04T03:18:19.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.682+0000 7fecf2ffd700 1 -- 172.21.15.1:0/1366511245 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fece8004c60 con 0x7fecf4107790 2024-05-04T03:18:19.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.682+0000 7fecf2ffd700 1 --2- 172.21.15.1:0/1366511245 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fecdc0b2900 0x7fecdc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:19.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.683+0000 7fecf9a95700 1 --2- 172.21.15.1:0/1366511245 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fecdc0b2900 0x7fecdc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:19.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.683+0000 7fecf2ffd700 1 -- 172.21.15.1:0/1366511245 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fece80e2040 con 0x7fecf4107790 2024-05-04T03:18:19.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.683+0000 7fecf9a95700 1 --2- 172.21.15.1:0/1366511245 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fecdc0b2900 0x7fecdc0b4e10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fecec00b5c0 tx=0x7fecec01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:19.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.686+0000 7fecf2ffd700 1 -- 172.21.15.1:0/1366511245 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fece8014030 con 0x7fecf4107790 2024-05-04T03:18:19.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.947+0000 7fecfbcf9700 1 -- 172.21.15.1:0/1366511245 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fecf402e5f0 con 0x7fecf4107790 2024-05-04T03:18:19.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.947+0000 7fecf2ffd700 1 -- 172.21.15.1:0/1366511245 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3032 (secure 0 0 0) 0x7fece80a3e20 con 0x7fecf4107790 2024-05-04T03:18:19.950 INFO:teuthology.orchestra.run.smithi001.stdout:e25 2024-05-04T03:18:19.950 INFO:teuthology.orchestra.run.smithi001.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-04T03:18:19.950 INFO:teuthology.orchestra.run.smithi001.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-04T03:18:19.950 INFO:teuthology.orchestra.run.smithi001.stdout:legacy client fscid: 1 2024-05-04T03:18:19.950 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:18:19.951 INFO:teuthology.orchestra.run.smithi001.stdout:Filesystem 'cephfs' (1) 2024-05-04T03:18:19.951 INFO:teuthology.orchestra.run.smithi001.stdout:fs_name cephfs 2024-05-04T03:18:19.951 INFO:teuthology.orchestra.run.smithi001.stdout:epoch 25 2024-05-04T03:18:19.951 INFO:teuthology.orchestra.run.smithi001.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-04T03:18:19.951 INFO:teuthology.orchestra.run.smithi001.stdout:created 2024-05-04T03:15:49.823471+0000 2024-05-04T03:18:19.951 INFO:teuthology.orchestra.run.smithi001.stdout:modified 2024-05-04T03:16:07.918400+0000 2024-05-04T03:18:19.952 INFO:teuthology.orchestra.run.smithi001.stdout:tableserver 0 2024-05-04T03:18:19.952 INFO:teuthology.orchestra.run.smithi001.stdout:root 0 2024-05-04T03:18:19.952 INFO:teuthology.orchestra.run.smithi001.stdout:session_timeout 300 2024-05-04T03:18:19.952 INFO:teuthology.orchestra.run.smithi001.stdout:session_autoclose 300 2024-05-04T03:18:19.952 INFO:teuthology.orchestra.run.smithi001.stdout:max_file_size 1099511627776 2024-05-04T03:18:19.952 INFO:teuthology.orchestra.run.smithi001.stdout:max_xattr_size 65536 2024-05-04T03:18:19.952 INFO:teuthology.orchestra.run.smithi001.stdout:required_client_features {} 2024-05-04T03:18:19.953 INFO:teuthology.orchestra.run.smithi001.stdout:last_failure 0 2024-05-04T03:18:19.953 INFO:teuthology.orchestra.run.smithi001.stdout:last_failure_osd_epoch 0 2024-05-04T03:18:19.953 INFO:teuthology.orchestra.run.smithi001.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-04T03:18:19.953 INFO:teuthology.orchestra.run.smithi001.stdout:max_mds 3 2024-05-04T03:18:19.953 INFO:teuthology.orchestra.run.smithi001.stdout:in 0,1,2 2024-05-04T03:18:19.953 INFO:teuthology.orchestra.run.smithi001.stdout:up {0=24494,1=24472,2=24467} 2024-05-04T03:18:19.954 INFO:teuthology.orchestra.run.smithi001.stdout:failed 2024-05-04T03:18:19.954 INFO:teuthology.orchestra.run.smithi001.stdout:damaged 2024-05-04T03:18:19.954 INFO:teuthology.orchestra.run.smithi001.stdout:stopped 2024-05-04T03:18:19.954 INFO:teuthology.orchestra.run.smithi001.stdout:data_pools [3] 2024-05-04T03:18:19.954 INFO:teuthology.orchestra.run.smithi001.stdout:metadata_pool 2 2024-05-04T03:18:19.954 INFO:teuthology.orchestra.run.smithi001.stdout:inline_data disabled 2024-05-04T03:18:19.954 INFO:teuthology.orchestra.run.smithi001.stdout:balancer 2024-05-04T03:18:19.954 INFO:teuthology.orchestra.run.smithi001.stdout:bal_rank_mask -1 2024-05-04T03:18:19.954 INFO:teuthology.orchestra.run.smithi001.stdout:standby_count_wanted 0 2024-05-04T03:18:19.954 INFO:teuthology.orchestra.run.smithi001.stdout:qdb_cluster leader: 24494 members: 24472,24494,24467 2024-05-04T03:18:19.955 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.b{0:24494} state up:active seq 5 addr [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] compat {c=[1],r=[1],i=[1fff]}] 2024-05-04T03:18:19.955 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.c{0:24463} state up:standby-replay seq 1 addr [v2:172.21.15.77:6836/2003521417,v1:172.21.15.77:6837/2003521417] compat {c=[1],r=[1],i=[1fff]}] 2024-05-04T03:18:19.955 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.i{1:24472} state up:active seq 11 addr [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] compat {c=[1],r=[1],i=[1fff]}] 2024-05-04T03:18:19.955 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.f{1:24457} state up:standby-replay seq 1 addr [v2:172.21.15.77:6834/4279951305,v1:172.21.15.77:6835/4279951305] compat {c=[1],r=[1],i=[1fff]}] 2024-05-04T03:18:19.955 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.h{2:24467} state up:active seq 12 addr [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] compat {c=[1],r=[1],i=[1fff]}] 2024-05-04T03:18:19.955 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.e{2:24440} state up:standby-replay seq 1 addr [v2:172.21.15.72:6834/3930984976,v1:172.21.15.72:6835/3930984976] compat {c=[1],r=[1],i=[1fff]}] 2024-05-04T03:18:19.955 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:18:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:18:19.956 INFO:teuthology.orchestra.run.smithi001.stdout:Standby daemons: 2024-05-04T03:18:19.956 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:18:19.956 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.j{-1:14505} state up:standby seq 1 addr [v2:172.21.15.1:6834/2186681910,v1:172.21.15.1:6835/2186681910] compat {c=[1],r=[1],i=[1fff]}] 2024-05-04T03:18:19.956 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.d{-1:14517} state up:standby seq 1 addr [v2:172.21.15.1:6836/2747191047,v1:172.21.15.1:6837/2747191047] compat {c=[1],r=[1],i=[1fff]}] 2024-05-04T03:18:19.956 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.g{-1:14565} state up:standby seq 1 addr [v2:172.21.15.1:6838/4115120540,v1:172.21.15.1:6839/4115120540] compat {c=[1],r=[1],i=[1fff]}] 2024-05-04T03:18:19.956 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.a{-1:14577} state up:standby seq 1 addr [v2:172.21.15.1:6840/3405709780,v1:172.21.15.1:6841/3405709780] compat {c=[1],r=[1],i=[1fff]}] 2024-05-04T03:18:19.956 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.k{-1:24413} state up:standby seq 1 addr [v2:172.21.15.72:6832/3816098761,v1:172.21.15.72:6833/3816098761] compat {c=[1],r=[1],i=[1fff]}] 2024-05-04T03:18:19.956 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.l{-1:24424} state up:standby seq 1 addr [v2:172.21.15.77:6832/379026128,v1:172.21.15.77:6833/379026128] compat {c=[1],r=[1],i=[1fff]}] 2024-05-04T03:18:19.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.950+0000 7fecfbcf9700 1 -- 172.21.15.1:0/1366511245 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fecdc0b2900 msgr2=0x7fecdc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:19.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.950+0000 7fecfbcf9700 1 --2- 172.21.15.1:0/1366511245 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fecdc0b2900 0x7fecdc0b4e10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fecec00b5c0 tx=0x7fecec01a040 comp rx=0 tx=0).stop 2024-05-04T03:18:19.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.950+0000 7fecfbcf9700 1 -- 172.21.15.1:0/1366511245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecf4107790 msgr2=0x7fecf4102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:19.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.950+0000 7fecfbcf9700 1 --2- 172.21.15.1:0/1366511245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecf4107790 0x7fecf4102d40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fece800c920 tx=0x7fece800cc30 comp rx=0 tx=0).stop 2024-05-04T03:18:19.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.950+0000 7fecfbcf9700 1 -- 172.21.15.1:0/1366511245 shutdown_connections 2024-05-04T03:18:19.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.950+0000 7fecfbcf9700 1 --2- 172.21.15.1:0/1366511245 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fecf410a0f0 0x7fecf4103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:19.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.950+0000 7fecfbcf9700 1 --2- 172.21.15.1:0/1366511245 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fecdc0b2900 0x7fecdc0b4e10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:19.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.950+0000 7fecfbcf9700 1 --2- 172.21.15.1:0/1366511245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecf4104e30 0x7fecf4104690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:19.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.950+0000 7fecfbcf9700 1 --2- 172.21.15.1:0/1366511245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecf4107790 0x7fecf4102d40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:19.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.950+0000 7fecfbcf9700 1 -- 172.21.15.1:0/1366511245 >> 172.21.15.1:0/1366511245 conn(0x7fecf40fe870 msgr2=0x7fecf4105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:19.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.950+0000 7fecfbcf9700 1 -- 172.21.15.1:0/1366511245 shutdown_connections 2024-05-04T03:18:19.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:19.950+0000 7fecfbcf9700 1 -- 172.21.15.1:0/1366511245 wait complete. 2024-05-04T03:18:19.958 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-05-04T03:18:20.509 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:20 smithi001 ceph-mon[29731]: pgmap v366: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-04T03:18:20.509 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:20 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/1366511245' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-04T03:18:20.537 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-04T03:18:20.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:20 smithi077 ceph-mon[33531]: pgmap v366: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-04T03:18:20.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:20 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/1366511245' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-04T03:18:20.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:20 smithi072 ceph-mon[33332]: pgmap v366: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-04T03:18:20.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:20 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/1366511245' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-04T03:18:20.784 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:18:22.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.366+0000 7f93b693a700 1 -- 172.21.15.1:0/2610712011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93b0104d20 msgr2=0x7f93b010b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:22.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.366+0000 7f93b693a700 1 --2- 172.21.15.1:0/2610712011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93b0104d20 0x7f93b010b5d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f93a800b0a0 tx=0x7f93a800b3b0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.366+0000 7f93b693a700 1 -- 172.21.15.1:0/2610712011 shutdown_connections 2024-05-04T03:18:22.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.366+0000 7f93b693a700 1 --2- 172.21.15.1:0/2610712011 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93b0103250 0x7f93b0103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.366+0000 7f93b693a700 1 --2- 172.21.15.1:0/2610712011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93b0104d20 0x7f93b010b5d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.366+0000 7f93b693a700 1 --2- 172.21.15.1:0/2610712011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93b0104380 0x7f93b0104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.366+0000 7f93b693a700 1 -- 172.21.15.1:0/2610712011 >> 172.21.15.1:0/2610712011 conn(0x7f93b00fe890 msgr2=0x7f93b0100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:22.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.366+0000 7f93b693a700 1 -- 172.21.15.1:0/2610712011 shutdown_connections 2024-05-04T03:18:22.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.366+0000 7f93b693a700 1 -- 172.21.15.1:0/2610712011 wait complete. 2024-05-04T03:18:22.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.366+0000 7f93b693a700 1 Processor -- start 2024-05-04T03:18:22.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.366+0000 7f93b693a700 1 -- start start 2024-05-04T03:18:22.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.367+0000 7f93b693a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93b0103250 0x7f93b019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:22.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.367+0000 7f93b693a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93b0104380 0x7f93b019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:22.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.367+0000 7f93b693a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93b0104d20 0x7f93b01a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:22.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.367+0000 7f93b693a700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f93b010ecb0 con 0x7f93b0104380 2024-05-04T03:18:22.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.367+0000 7f93b693a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f93b010eb30 con 0x7f93b0103250 2024-05-04T03:18:22.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.367+0000 7f93b693a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f93b010ee30 con 0x7f93b0104d20 2024-05-04T03:18:22.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.367+0000 7f93af7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93b0104380 0x7f93b019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:22.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.367+0000 7f93af7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93b0104380 0x7f93b019ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36682/0 (socket says 172.21.15.1:36682) 2024-05-04T03:18:22.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.367+0000 7f93af7fe700 1 -- 172.21.15.1:0/2254574540 learned_addr learned my addr 172.21.15.1:0/2254574540 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:22.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.367+0000 7f93b4ed7700 1 --2- 172.21.15.1:0/2254574540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93b0104d20 0x7f93b01a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:22.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.367+0000 7f93affff700 1 --2- 172.21.15.1:0/2254574540 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93b0103250 0x7f93b019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:22.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.367+0000 7f93af7fe700 1 -- 172.21.15.1:0/2254574540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93b0104d20 msgr2=0x7f93b01a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:22.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.367+0000 7f93af7fe700 1 --2- 172.21.15.1:0/2254574540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93b0104d20 0x7f93b01a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.367+0000 7f93af7fe700 1 -- 172.21.15.1:0/2254574540 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93b0103250 msgr2=0x7f93b019a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:22.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.367+0000 7f93af7fe700 1 --2- 172.21.15.1:0/2254574540 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93b0103250 0x7f93b019a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.367+0000 7f93af7fe700 1 -- 172.21.15.1:0/2254574540 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b01055a0 con 0x7f93b0104380 2024-05-04T03:18:22.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.367+0000 7f93affff700 1 --2- 172.21.15.1:0/2254574540 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93b0103250 0x7f93b019a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:18:22.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.368+0000 7f93af7fe700 1 --2- 172.21.15.1:0/2254574540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93b0104380 0x7f93b019ae10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f939800d950 tx=0x7f939800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:22.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.368+0000 7f93ad7fa700 1 -- 172.21.15.1:0/2254574540 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9398016070 con 0x7f93b0104380 2024-05-04T03:18:22.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.368+0000 7f93ad7fa700 1 -- 172.21.15.1:0/2254574540 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9398009df0 con 0x7f93b0104380 2024-05-04T03:18:22.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.368+0000 7f93b693a700 1 -- 172.21.15.1:0/2254574540 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b0105890 con 0x7f93b0104380 2024-05-04T03:18:22.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.368+0000 7f93ad7fa700 1 -- 172.21.15.1:0/2254574540 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939800f3c0 con 0x7f93b0104380 2024-05-04T03:18:22.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.368+0000 7f93b693a700 1 -- 172.21.15.1:0/2254574540 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b0105dd0 con 0x7f93b0104380 2024-05-04T03:18:22.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.369+0000 7f93b693a700 1 -- 172.21.15.1:0/2254574540 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93b00512c0 con 0x7f93b0104380 2024-05-04T03:18:22.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.371+0000 7f93ad7fa700 1 -- 172.21.15.1:0/2254574540 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f9398019070 con 0x7f93b0104380 2024-05-04T03:18:22.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.371+0000 7f93ad7fa700 1 --2- 172.21.15.1:0/2254574540 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f939c0b2880 0x7f939c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:22.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.372+0000 7f93affff700 1 --2- 172.21.15.1:0/2254574540 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f939c0b2880 0x7f939c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:22.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.372+0000 7f93ad7fa700 1 -- 172.21.15.1:0/2254574540 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f93980e1e00 con 0x7f93b0104380 2024-05-04T03:18:22.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.372+0000 7f93affff700 1 --2- 172.21.15.1:0/2254574540 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f939c0b2880 0x7f939c0b4d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f93b0104280 tx=0x7f93a000a680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:22.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.379+0000 7f93ad7fa700 1 -- 172.21.15.1:0/2254574540 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f93980a9b40 con 0x7f93b0104380 2024-05-04T03:18:22.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.595+0000 7f93b693a700 1 -- 172.21.15.1:0/2254574540 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f93b0106280 con 0x7f93b0104380 2024-05-04T03:18:22.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.596+0000 7f93ad7fa700 1 -- 172.21.15.1:0/2254574540 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v83) ==== 56+0+5154 (secure 0 0 0) 0x7f93980a9440 con 0x7f93b0104380 2024-05-04T03:18:22.598 INFO:teuthology.orchestra.run.smithi001.stdout:epoch 83 2024-05-04T03:18:22.598 INFO:teuthology.orchestra.run.smithi001.stdout:fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:18:22.598 INFO:teuthology.orchestra.run.smithi001.stdout:created 2024-05-04T03:07:32.845547+0000 2024-05-04T03:18:22.599 INFO:teuthology.orchestra.run.smithi001.stdout:modified 2024-05-04T03:16:34.321580+0000 2024-05-04T03:18:22.599 INFO:teuthology.orchestra.run.smithi001.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-04T03:18:22.599 INFO:teuthology.orchestra.run.smithi001.stdout:crush_version 26 2024-05-04T03:18:22.599 INFO:teuthology.orchestra.run.smithi001.stdout:full_ratio 0.95 2024-05-04T03:18:22.599 INFO:teuthology.orchestra.run.smithi001.stdout:backfillfull_ratio 0.9 2024-05-04T03:18:22.599 INFO:teuthology.orchestra.run.smithi001.stdout:nearfull_ratio 0.85 2024-05-04T03:18:22.599 INFO:teuthology.orchestra.run.smithi001.stdout:require_min_compat_client luminous 2024-05-04T03:18:22.599 INFO:teuthology.orchestra.run.smithi001.stdout:min_compat_client luminous 2024-05-04T03:18:22.599 INFO:teuthology.orchestra.run.smithi001.stdout:require_osd_release squid 2024-05-04T03:18:22.599 INFO:teuthology.orchestra.run.smithi001.stdout:stretch_mode_enabled false 2024-05-04T03:18:22.599 INFO:teuthology.orchestra.run.smithi001.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-04T03:18:22.599 INFO:teuthology.orchestra.run.smithi001.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-04T03:18:22.599 INFO:teuthology.orchestra.run.smithi001.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-04T03:18:22.600 INFO:teuthology.orchestra.run.smithi001.stdout:max_osd 12 2024-05-04T03:18:22.600 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0 up in weight 1 up_from 8 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6802/3479011484,v1:172.21.15.1:6803/3479011484] [v2:172.21.15.1:6804/3479011484,v1:172.21.15.1:6805/3479011484] exists,up 100fe035-2454-4c95-90c7-759956ec36a7 2024-05-04T03:18:22.600 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1 up in weight 1 up_from 13 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.72:6800/3725706149,v1:172.21.15.72:6801/3725706149] [v2:172.21.15.72:6802/3725706149,v1:172.21.15.72:6803/3725706149] exists,up 5826ac76-ef6b-4040-8ad1-305db355af79 2024-05-04T03:18:22.600 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2 up in weight 1 up_from 18 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6800/515038343,v1:172.21.15.77:6801/515038343] [v2:172.21.15.77:6802/515038343,v1:172.21.15.77:6803/515038343] exists,up 923d76e5-b032-4190-9e35-546d27722a3c 2024-05-04T03:18:22.600 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3 up in weight 1 up_from 25 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6810/4050439365,v1:172.21.15.1:6811/4050439365] [v2:172.21.15.1:6812/4050439365,v1:172.21.15.1:6813/4050439365] exists,up 495cc0c5-6a99-481f-ad9e-38f05fb84356 2024-05-04T03:18:22.600 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4 up in weight 1 up_from 31 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.72:6808/2501732391,v1:172.21.15.72:6809/2501732391] [v2:172.21.15.72:6810/2501732391,v1:172.21.15.72:6811/2501732391] exists,up b86b8026-79c0-4056-9e67-a61f54b6ab5d 2024-05-04T03:18:22.600 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5 up in weight 1 up_from 37 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6808/3582852165,v1:172.21.15.77:6809/3582852165] [v2:172.21.15.77:6810/3582852165,v1:172.21.15.77:6811/3582852165] exists,up 80975cb9-39e0-46b8-a343-9ff69194a14e 2024-05-04T03:18:22.600 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6 up in weight 1 up_from 43 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6818/1799033649,v1:172.21.15.1:6819/1799033649] [v2:172.21.15.1:6820/1799033649,v1:172.21.15.1:6821/1799033649] exists,up aca3fac5-fbb5-42de-bf0e-e4a0db7381e2 2024-05-04T03:18:22.600 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7 up in weight 1 up_from 49 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.72:6816/3460835447,v1:172.21.15.72:6817/3460835447] [v2:172.21.15.72:6818/3460835447,v1:172.21.15.72:6819/3460835447] exists,up cbff6b8a-c463-4249-ae45-fdc6f99a5489 2024-05-04T03:18:22.600 INFO:teuthology.orchestra.run.smithi001.stdout:osd.8 up in weight 1 up_from 56 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6816/1124479323,v1:172.21.15.77:6817/1124479323] [v2:172.21.15.77:6818/1124479323,v1:172.21.15.77:6819/1124479323] exists,up e140bf13-4887-4d68-9373-3d4fc1c4584a 2024-05-04T03:18:22.600 INFO:teuthology.orchestra.run.smithi001.stdout:osd.9 up in weight 1 up_from 62 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6826/2681322272,v1:172.21.15.1:6827/2681322272] [v2:172.21.15.1:6828/2681322272,v1:172.21.15.1:6829/2681322272] exists,up f1e8a232-cb11-4b9d-8789-e1637fa5aaa8 2024-05-04T03:18:22.600 INFO:teuthology.orchestra.run.smithi001.stdout:osd.10 up in weight 1 up_from 67 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.72:6824/3293703569,v1:172.21.15.72:6825/3293703569] [v2:172.21.15.72:6826/3293703569,v1:172.21.15.72:6827/3293703569] exists,up 79b570af-3d1e-439d-a328-9e6bc7d67839 2024-05-04T03:18:22.600 INFO:teuthology.orchestra.run.smithi001.stdout:osd.11 up in weight 1 up_from 73 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6824/4155947548,v1:172.21.15.77:6825/4155947548] [v2:172.21.15.77:6826/4155947548,v1:172.21.15.77:6827/4155947548] exists,up e35bf83d-9ee4-4745-96c2-1c35cc8eefba 2024-05-04T03:18:22.600 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 2.8 [2,11] 2024-05-04T03:18:22.600 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 2.b [8,11] 2024-05-04T03:18:22.601 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.3 [4,9] 2024-05-04T03:18:22.601 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.13 [4,9] 2024-05-04T03:18:22.601 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.33 [5,9] 2024-05-04T03:18:22.601 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/2029897106 expires 2024-05-05T03:08:33.045884+0000 2024-05-04T03:18:22.601 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/402488450 expires 2024-05-05T03:08:33.045884+0000 2024-05-04T03:18:22.601 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:6801/128793132 expires 2024-05-05T03:08:33.045884+0000 2024-05-04T03:18:22.601 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/2233526606 expires 2024-05-05T03:08:06.135783+0000 2024-05-04T03:18:22.601 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/2715541855 expires 2024-05-05T03:08:33.045884+0000 2024-05-04T03:18:22.601 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/2415697325 expires 2024-05-05T03:08:06.135783+0000 2024-05-04T03:18:22.601 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/2305298106 expires 2024-05-05T03:08:06.135783+0000 2024-05-04T03:18:22.601 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:6800/128793132 expires 2024-05-05T03:08:33.045884+0000 2024-05-04T03:18:22.601 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:6800/4200529863 expires 2024-05-05T03:08:06.135783+0000 2024-05-04T03:18:22.601 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:6801/4200529863 expires 2024-05-05T03:08:06.135783+0000 2024-05-04T03:18:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.598+0000 7f93b693a700 1 -- 172.21.15.1:0/2254574540 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f939c0b2880 msgr2=0x7f939c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.598+0000 7f93b693a700 1 --2- 172.21.15.1:0/2254574540 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f939c0b2880 0x7f939c0b4d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f93b0104280 tx=0x7f93a000a680 comp rx=0 tx=0).stop 2024-05-04T03:18:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.598+0000 7f93b693a700 1 -- 172.21.15.1:0/2254574540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93b0104380 msgr2=0x7f93b019ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.598+0000 7f93b693a700 1 --2- 172.21.15.1:0/2254574540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93b0104380 0x7f93b019ae10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f939800d950 tx=0x7f939800dc60 comp rx=0 tx=0).stop 2024-05-04T03:18:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.599+0000 7f93b693a700 1 -- 172.21.15.1:0/2254574540 shutdown_connections 2024-05-04T03:18:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.599+0000 7f93b693a700 1 --2- 172.21.15.1:0/2254574540 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f93b0103250 0x7f93b019a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.599+0000 7f93b693a700 1 --2- 172.21.15.1:0/2254574540 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f939c0b2880 0x7f939c0b4d90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.599+0000 7f93b693a700 1 --2- 172.21.15.1:0/2254574540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93b0104d20 0x7f93b01a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.599+0000 7f93b693a700 1 --2- 172.21.15.1:0/2254574540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93b0104380 0x7f93b019ae10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.599+0000 7f93b693a700 1 -- 172.21.15.1:0/2254574540 >> 172.21.15.1:0/2254574540 conn(0x7f93b00fe890 msgr2=0x7f93b0100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.599+0000 7f93b693a700 1 -- 172.21.15.1:0/2254574540 shutdown_connections 2024-05-04T03:18:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:22.599+0000 7f93b693a700 1 -- 172.21.15.1:0/2254574540 wait complete. 2024-05-04T03:18:22.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:22 smithi077 ceph-mon[33531]: pgmap v367: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:18:22.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:22 smithi072 ceph-mon[33332]: pgmap v367: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:18:22.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:22 smithi001 ceph-mon[29731]: pgmap v367: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:18:23.311 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-04T03:18:23.322 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:18:23.575 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:18:23.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:23 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2254574540' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-04T03:18:23.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:23 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/2254574540' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-04T03:18:23.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:23 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/2254574540' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-04T03:18:24.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:24 smithi001 ceph-mon[29731]: pgmap v368: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:18:24.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:24 smithi077 ceph-mon[33531]: pgmap v368: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:18:24.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:24 smithi072 ceph-mon[33332]: pgmap v368: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:18:25.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.206+0000 7f5d273ad700 1 -- 172.21.15.1:0/1151417541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d20103c10 msgr2=0x7f5d20104020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:25.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.206+0000 7f5d273ad700 1 --2- 172.21.15.1:0/1151417541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d20103c10 0x7f5d20104020 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0099c0 tx=0x7f5d1c009cd0 comp rx=0 tx=0).stop 2024-05-04T03:18:25.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.206+0000 7f5d273ad700 1 -- 172.21.15.1:0/1151417541 shutdown_connections 2024-05-04T03:18:25.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.206+0000 7f5d273ad700 1 --2- 172.21.15.1:0/1151417541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d20108970 0x7f5d201102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:25.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.206+0000 7f5d273ad700 1 --2- 172.21.15.1:0/1151417541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d20103c10 0x7f5d20104020 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:25.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.206+0000 7f5d273ad700 1 --2- 172.21.15.1:0/1151417541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d20108020 0x7f5d20108430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:25.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.206+0000 7f5d273ad700 1 -- 172.21.15.1:0/1151417541 >> 172.21.15.1:0/1151417541 conn(0x7f5d200fe4e0 msgr2=0x7f5d20100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:25.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.206+0000 7f5d273ad700 1 -- 172.21.15.1:0/1151417541 shutdown_connections 2024-05-04T03:18:25.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.206+0000 7f5d273ad700 1 -- 172.21.15.1:0/1151417541 wait complete. 2024-05-04T03:18:25.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.207+0000 7f5d273ad700 1 Processor -- start 2024-05-04T03:18:25.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.207+0000 7f5d273ad700 1 -- start start 2024-05-04T03:18:25.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.208+0000 7f5d273ad700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d20103c10 0x7f5d2019be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:25.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.208+0000 7f5d273ad700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d20108020 0x7f5d2019c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:25.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.208+0000 7f5d273ad700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d20108970 0x7f5d201a6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:25.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.208+0000 7f5d273ad700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5d201135c0 con 0x7f5d20108020 2024-05-04T03:18:25.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.208+0000 7f5d273ad700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5d20113440 con 0x7f5d20108970 2024-05-04T03:18:25.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.208+0000 7f5d273ad700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5d20113740 con 0x7f5d20103c10 2024-05-04T03:18:25.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.208+0000 7f5d24948700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d20108020 0x7f5d2019c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:25.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.208+0000 7f5d24948700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d20108020 0x7f5d2019c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36708/0 (socket says 172.21.15.1:36708) 2024-05-04T03:18:25.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.208+0000 7f5d24948700 1 -- 172.21.15.1:0/1008980964 learned_addr learned my addr 172.21.15.1:0/1008980964 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:25.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.208+0000 7f5d2594a700 1 --2- 172.21.15.1:0/1008980964 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d20108970 0x7f5d201a6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:25.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.208+0000 7f5d25149700 1 --2- 172.21.15.1:0/1008980964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d20103c10 0x7f5d2019be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:25.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.209+0000 7f5d24948700 1 -- 172.21.15.1:0/1008980964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d20103c10 msgr2=0x7f5d2019be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:25.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.209+0000 7f5d24948700 1 --2- 172.21.15.1:0/1008980964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d20103c10 0x7f5d2019be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:25.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.209+0000 7f5d24948700 1 -- 172.21.15.1:0/1008980964 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d20108970 msgr2=0x7f5d201a6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:25.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.209+0000 7f5d24948700 1 --2- 172.21.15.1:0/1008980964 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d20108970 0x7f5d201a6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:25.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.209+0000 7f5d24948700 1 -- 172.21.15.1:0/1008980964 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2019cc30 con 0x7f5d20108020 2024-05-04T03:18:25.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.209+0000 7f5d25149700 1 --2- 172.21.15.1:0/1008980964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d20103c10 0x7f5d2019be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:18:25.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.209+0000 7f5d24948700 1 --2- 172.21.15.1:0/1008980964 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d20108020 0x7f5d2019c380 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000d950 tx=0x7f5d1000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:25.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.209+0000 7f5d167fc700 1 -- 172.21.15.1:0/1008980964 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d10016070 con 0x7f5d20108020 2024-05-04T03:18:25.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.209+0000 7f5d167fc700 1 -- 172.21.15.1:0/1008980964 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d1000de30 con 0x7f5d20108020 2024-05-04T03:18:25.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.209+0000 7f5d273ad700 1 -- 172.21.15.1:0/1008980964 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d201a7520 con 0x7f5d20108020 2024-05-04T03:18:25.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.209+0000 7f5d167fc700 1 -- 172.21.15.1:0/1008980964 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d10003ae0 con 0x7f5d20108020 2024-05-04T03:18:25.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.209+0000 7f5d273ad700 1 -- 172.21.15.1:0/1008980964 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d201a7a60 con 0x7f5d20108020 2024-05-04T03:18:25.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.210+0000 7f5d273ad700 1 -- 172.21.15.1:0/1008980964 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d200512c0 con 0x7f5d20108020 2024-05-04T03:18:25.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.212+0000 7f5d167fc700 1 -- 172.21.15.1:0/1008980964 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f5d1000bb60 con 0x7f5d20108020 2024-05-04T03:18:25.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.213+0000 7f5d167fc700 1 --2- 172.21.15.1:0/1008980964 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5d0c0b2900 0x7f5d0c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:25.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.213+0000 7f5d25149700 1 --2- 172.21.15.1:0/1008980964 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5d0c0b2900 0x7f5d0c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:25.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.213+0000 7f5d167fc700 1 -- 172.21.15.1:0/1008980964 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f5d100e20b0 con 0x7f5d20108020 2024-05-04T03:18:25.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.214+0000 7f5d25149700 1 --2- 172.21.15.1:0/1008980964 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5d0c0b2900 0x7f5d0c0b4e10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c003800 tx=0x7f5d1c005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:25.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.217+0000 7f5d167fc700 1 -- 172.21.15.1:0/1008980964 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5d100a9e90 con 0x7f5d20108020 2024-05-04T03:18:25.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.478+0000 7f5d273ad700 1 -- 172.21.15.1:0/1008980964 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d2006c120 con 0x7f5d20108020 2024-05-04T03:18:25.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.479+0000 7f5d167fc700 1 -- 172.21.15.1:0/1008980964 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11844 (secure 0 0 0) 0x7f5d100a9790 con 0x7f5d20108020 2024-05-04T03:18:25.480 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:18:25.481 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2186681910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2186681910},{"type":"v1","addr":"172.21.15.1:6835","nonce":2186681910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2747191047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2747191047},{"type":"v1","addr":"172.21.15.1:6837","nonce":2747191047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4115120540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4115120540},{"type":"v1","addr":"172.21.15.1:6839","nonce":4115120540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3405709780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3405709780},{"type":"v1","addr":"172.21.15.1:6841","nonce":3405709780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/3816098761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":3816098761},{"type":"v1","addr":"172.21.15.72:6833","nonce":3816098761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/379026128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":379026128},{"type":"v1","addr":"172.21.15.77:6833","nonce":379026128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:15:49.823471+0000","modified":"2024-05-04T03:16:07.918400+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24472,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/3930984976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3930984976},{"type":"v1","addr":"172.21.15.72:6835","nonce":3930984976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/4279951305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4279951305},{"type":"v1","addr":"172.21.15.77:6835","nonce":4279951305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2003521417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2003521417},{"type":"v1","addr":"172.21.15.77:6837","nonce":2003521417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/1417614203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1417614203},{"type":"v1","addr":"172.21.15.72:6837","nonce":1417614203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.77:6839/922723189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":922723189},{"type":"v1","addr":"172.21.15.77:6839","nonce":922723189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2976050650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2976050650},{"type":"v1","addr":"172.21.15.72:6839","nonce":2976050650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24472,24494,24467]},"id":1}]} 2024-05-04T03:18:25.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.481+0000 7f5d273ad700 1 -- 172.21.15.1:0/1008980964 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5d0c0b2900 msgr2=0x7f5d0c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:25.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.481+0000 7f5d273ad700 1 --2- 172.21.15.1:0/1008980964 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5d0c0b2900 0x7f5d0c0b4e10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c003800 tx=0x7f5d1c005e80 comp rx=0 tx=0).stop 2024-05-04T03:18:25.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.481+0000 7f5d273ad700 1 -- 172.21.15.1:0/1008980964 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d20108020 msgr2=0x7f5d2019c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:25.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.481+0000 7f5d273ad700 1 --2- 172.21.15.1:0/1008980964 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d20108020 0x7f5d2019c380 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000d950 tx=0x7f5d1000dd10 comp rx=0 tx=0).stop 2024-05-04T03:18:25.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.481+0000 7f5d273ad700 1 -- 172.21.15.1:0/1008980964 shutdown_connections 2024-05-04T03:18:25.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.481+0000 7f5d273ad700 1 --2- 172.21.15.1:0/1008980964 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d20108970 0x7f5d201a6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:25.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.481+0000 7f5d273ad700 1 --2- 172.21.15.1:0/1008980964 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5d0c0b2900 0x7f5d0c0b4e10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:25.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.481+0000 7f5d273ad700 1 --2- 172.21.15.1:0/1008980964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5d20103c10 0x7f5d2019be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:25.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.481+0000 7f5d273ad700 1 --2- 172.21.15.1:0/1008980964 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d20108020 0x7f5d2019c380 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:25.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.481+0000 7f5d273ad700 1 -- 172.21.15.1:0/1008980964 >> 172.21.15.1:0/1008980964 conn(0x7f5d200fe4e0 msgr2=0x7f5d201077c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:25.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.481+0000 7f5d273ad700 1 -- 172.21.15.1:0/1008980964 shutdown_connections 2024-05-04T03:18:25.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:25.481+0000 7f5d273ad700 1 -- 172.21.15.1:0/1008980964 wait complete. 2024-05-04T03:18:25.484 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-05-04T03:18:26.176 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:18:26.433 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:18:26.671 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:26 smithi001 ceph-mon[29731]: pgmap v369: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-04T03:18:26.672 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:26 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/1008980964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:18:26.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:26 smithi077 ceph-mon[33531]: pgmap v369: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-04T03:18:26.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:26 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/1008980964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:18:26.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:26 smithi072 ceph-mon[33332]: pgmap v369: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-04T03:18:26.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:26 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/1008980964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:18:28.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.103+0000 7f12c1006700 1 -- 172.21.15.1:0/3237901318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12bc076680 msgr2=0x7f12bc110660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:28.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.103+0000 7f12c1006700 1 --2- 172.21.15.1:0/3237901318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12bc076680 0x7f12bc110660 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f12b0009990 tx=0x7f12b0009ca0 comp rx=0 tx=0).stop 2024-05-04T03:18:28.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.103+0000 7f12c1006700 1 -- 172.21.15.1:0/3237901318 shutdown_connections 2024-05-04T03:18:28.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.103+0000 7f12c1006700 1 --2- 172.21.15.1:0/3237901318 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12bc075d30 0x7f12bc076140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:28.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.103+0000 7f12c1006700 1 --2- 172.21.15.1:0/3237901318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12bc110ba0 0x7f12bc113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:28.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.103+0000 7f12c1006700 1 --2- 172.21.15.1:0/3237901318 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12bc076680 0x7f12bc110660 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:28.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.103+0000 7f12c1006700 1 -- 172.21.15.1:0/3237901318 >> 172.21.15.1:0/3237901318 conn(0x7f12bc100040 msgr2=0x7f12bc102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:28.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.103+0000 7f12c1006700 1 -- 172.21.15.1:0/3237901318 shutdown_connections 2024-05-04T03:18:28.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.103+0000 7f12c1006700 1 -- 172.21.15.1:0/3237901318 wait complete. 2024-05-04T03:18:28.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.104+0000 7f12c1006700 1 Processor -- start 2024-05-04T03:18:28.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.104+0000 7f12c1006700 1 -- start start 2024-05-04T03:18:28.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.105+0000 7f12c1006700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12bc075d30 0x7f12bc19d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:28.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.105+0000 7f12c1006700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12bc076680 0x7f12bc19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:28.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.105+0000 7f12c1006700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12bc110ba0 0x7f12bc1a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:28.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.105+0000 7f12c1006700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f12bc116720 con 0x7f12bc076680 2024-05-04T03:18:28.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.105+0000 7f12c1006700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f12bc1165a0 con 0x7f12bc110ba0 2024-05-04T03:18:28.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.105+0000 7f12c1006700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f12bc1168a0 con 0x7f12bc075d30 2024-05-04T03:18:28.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.105+0000 7f12ba59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12bc076680 0x7f12bc19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:28.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.105+0000 7f12ba59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12bc076680 0x7f12bc19d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36728/0 (socket says 172.21.15.1:36728) 2024-05-04T03:18:28.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.105+0000 7f12ba59c700 1 -- 172.21.15.1:0/1289975078 learned_addr learned my addr 172.21.15.1:0/1289975078 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:28.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.105+0000 7f12bad9d700 1 --2- 172.21.15.1:0/1289975078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12bc075d30 0x7f12bc19d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:28.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.105+0000 7f12bb59e700 1 --2- 172.21.15.1:0/1289975078 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12bc110ba0 0x7f12bc1a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:28.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.105+0000 7f12ba59c700 1 -- 172.21.15.1:0/1289975078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12bc075d30 msgr2=0x7f12bc19d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:28.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.105+0000 7f12ba59c700 1 --2- 172.21.15.1:0/1289975078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12bc075d30 0x7f12bc19d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:28.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.105+0000 7f12ba59c700 1 -- 172.21.15.1:0/1289975078 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12bc110ba0 msgr2=0x7f12bc1a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:28.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.105+0000 7f12ba59c700 1 --2- 172.21.15.1:0/1289975078 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12bc110ba0 0x7f12bc1a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:28.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.105+0000 7f12ba59c700 1 -- 172.21.15.1:0/1289975078 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12bc1a1f80 con 0x7f12bc076680 2024-05-04T03:18:28.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.105+0000 7f12ba59c700 1 --2- 172.21.15.1:0/1289975078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12bc076680 0x7f12bc19d650 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f12b0000c00 tx=0x7f12b00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:28.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.106+0000 7f12a3fff700 1 -- 172.21.15.1:0/1289975078 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b0027070 con 0x7f12bc076680 2024-05-04T03:18:28.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.106+0000 7f12c1006700 1 -- 172.21.15.1:0/1289975078 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12bc1a2210 con 0x7f12bc076680 2024-05-04T03:18:28.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.106+0000 7f12a3fff700 1 -- 172.21.15.1:0/1289975078 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12b0003f20 con 0x7f12bc076680 2024-05-04T03:18:28.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.106+0000 7f12a3fff700 1 -- 172.21.15.1:0/1289975078 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b0022ca0 con 0x7f12bc076680 2024-05-04T03:18:28.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.106+0000 7f12c1006700 1 -- 172.21.15.1:0/1289975078 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12bc1a26f0 con 0x7f12bc076680 2024-05-04T03:18:28.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.107+0000 7f12c1006700 1 -- 172.21.15.1:0/1289975078 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12bc0512c0 con 0x7f12bc076680 2024-05-04T03:18:28.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.108+0000 7f12a3fff700 1 -- 172.21.15.1:0/1289975078 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f12b002a070 con 0x7f12bc076680 2024-05-04T03:18:28.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.109+0000 7f12a3fff700 1 --2- 172.21.15.1:0/1289975078 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f12a40b2880 0x7f12a40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:28.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.109+0000 7f12bad9d700 1 --2- 172.21.15.1:0/1289975078 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f12a40b2880 0x7f12a40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:28.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.109+0000 7f12a3fff700 1 -- 172.21.15.1:0/1289975078 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f12b00e33a0 con 0x7f12bc076680 2024-05-04T03:18:28.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.110+0000 7f12bad9d700 1 --2- 172.21.15.1:0/1289975078 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f12a40b2880 0x7f12a40b4d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f12ac005950 tx=0x7f12ac00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:28.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.113+0000 7f12a3fff700 1 -- 172.21.15.1:0/1289975078 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f12b0031080 con 0x7f12bc076680 2024-05-04T03:18:28.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.373+0000 7f12c1006700 1 -- 172.21.15.1:0/1289975078 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12bc19e410 con 0x7f12bc076680 2024-05-04T03:18:28.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.374+0000 7f12a3fff700 1 -- 172.21.15.1:0/1289975078 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11844 (secure 0 0 0) 0x7f12b00b3070 con 0x7f12bc076680 2024-05-04T03:18:28.376 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:18:28.376 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2186681910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2186681910},{"type":"v1","addr":"172.21.15.1:6835","nonce":2186681910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2747191047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2747191047},{"type":"v1","addr":"172.21.15.1:6837","nonce":2747191047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4115120540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4115120540},{"type":"v1","addr":"172.21.15.1:6839","nonce":4115120540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3405709780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3405709780},{"type":"v1","addr":"172.21.15.1:6841","nonce":3405709780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/3816098761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":3816098761},{"type":"v1","addr":"172.21.15.72:6833","nonce":3816098761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/379026128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":379026128},{"type":"v1","addr":"172.21.15.77:6833","nonce":379026128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:15:49.823471+0000","modified":"2024-05-04T03:16:07.918400+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24472,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/3930984976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3930984976},{"type":"v1","addr":"172.21.15.72:6835","nonce":3930984976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/4279951305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4279951305},{"type":"v1","addr":"172.21.15.77:6835","nonce":4279951305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2003521417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2003521417},{"type":"v1","addr":"172.21.15.77:6837","nonce":2003521417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/1417614203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1417614203},{"type":"v1","addr":"172.21.15.72:6837","nonce":1417614203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.77:6839/922723189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":922723189},{"type":"v1","addr":"172.21.15.77:6839","nonce":922723189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2976050650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2976050650},{"type":"v1","addr":"172.21.15.72:6839","nonce":2976050650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24472,24494,24467]},"id":1}]} 2024-05-04T03:18:28.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.376+0000 7f12c1006700 1 -- 172.21.15.1:0/1289975078 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f12a40b2880 msgr2=0x7f12a40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:28.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.376+0000 7f12c1006700 1 --2- 172.21.15.1:0/1289975078 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f12a40b2880 0x7f12a40b4d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f12ac005950 tx=0x7f12ac00b410 comp rx=0 tx=0).stop 2024-05-04T03:18:28.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.376+0000 7f12c1006700 1 -- 172.21.15.1:0/1289975078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12bc076680 msgr2=0x7f12bc19d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:28.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.376+0000 7f12c1006700 1 --2- 172.21.15.1:0/1289975078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12bc076680 0x7f12bc19d650 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f12b0000c00 tx=0x7f12b00047b0 comp rx=0 tx=0).stop 2024-05-04T03:18:28.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.376+0000 7f12c1006700 1 -- 172.21.15.1:0/1289975078 shutdown_connections 2024-05-04T03:18:28.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.376+0000 7f12c1006700 1 --2- 172.21.15.1:0/1289975078 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12bc110ba0 0x7f12bc1a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:28.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.377+0000 7f12c1006700 1 --2- 172.21.15.1:0/1289975078 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f12a40b2880 0x7f12a40b4d90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:28.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.377+0000 7f12c1006700 1 --2- 172.21.15.1:0/1289975078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12bc075d30 0x7f12bc19d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:28.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.377+0000 7f12c1006700 1 --2- 172.21.15.1:0/1289975078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12bc076680 0x7f12bc19d650 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:28.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.377+0000 7f12c1006700 1 -- 172.21.15.1:0/1289975078 >> 172.21.15.1:0/1289975078 conn(0x7f12bc100040 msgr2=0x7f12bc100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:28.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.377+0000 7f12c1006700 1 -- 172.21.15.1:0/1289975078 shutdown_connections 2024-05-04T03:18:28.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:28.377+0000 7f12c1006700 1 -- 172.21.15.1:0/1289975078 wait complete. 2024-05-04T03:18:28.379 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 25 2024-05-04T03:18:28.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:28 smithi001 ceph-mon[29731]: pgmap v370: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-04T03:18:28.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:28 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/1289975078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:18:28.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:28 smithi077 ceph-mon[33531]: pgmap v370: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-04T03:18:28.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:28 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/1289975078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:18:28.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:28 smithi072 ceph-mon[33332]: pgmap v370: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-04T03:18:28.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:28 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/1289975078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:18:29.178 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-04T03:18:29.434 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:18:30.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:30 smithi077 ceph-mon[33531]: pgmap v371: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-04T03:18:30.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:30 smithi001 ceph-mon[29731]: pgmap v371: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-04T03:18:30.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:30 smithi072 ceph-mon[33332]: pgmap v371: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-04T03:18:31.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.039+0000 7f2b64e0b700 1 -- 172.21.15.1:0/4067408398 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b60104330 msgr2=0x7f2b60110010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:31.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.039+0000 7f2b64e0b700 1 --2- 172.21.15.1:0/4067408398 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b60104330 0x7f2b60110010 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2b5800b0a0 tx=0x7f2b5800b3b0 comp rx=0 tx=0).stop 2024-05-04T03:18:31.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.039+0000 7f2b64e0b700 1 -- 172.21.15.1:0/4067408398 shutdown_connections 2024-05-04T03:18:31.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.039+0000 7f2b64e0b700 1 --2- 172.21.15.1:0/4067408398 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2b601092d0 0x7f2b601096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:31.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.039+0000 7f2b64e0b700 1 --2- 172.21.15.1:0/4067408398 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b60104330 0x7f2b60110010 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:31.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.039+0000 7f2b64e0b700 1 --2- 172.21.15.1:0/4067408398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b601039e0 0x7f2b60103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:31.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.039+0000 7f2b64e0b700 1 -- 172.21.15.1:0/4067408398 >> 172.21.15.1:0/4067408398 conn(0x7f2b600fe540 msgr2=0x7f2b60100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:31.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.039+0000 7f2b64e0b700 1 -- 172.21.15.1:0/4067408398 shutdown_connections 2024-05-04T03:18:31.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.039+0000 7f2b64e0b700 1 -- 172.21.15.1:0/4067408398 wait complete. 2024-05-04T03:18:31.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.040+0000 7f2b64e0b700 1 Processor -- start 2024-05-04T03:18:31.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.040+0000 7f2b64e0b700 1 -- start start 2024-05-04T03:18:31.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.041+0000 7f2b64e0b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b601039e0 0x7f2b6019bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:31.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.041+0000 7f2b64e0b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2b60104330 0x7f2b6019c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:31.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.041+0000 7f2b64e0b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b601092d0 0x7f2b6019c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:31.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.041+0000 7f2b64e0b700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2b60113370 con 0x7f2b601092d0 2024-05-04T03:18:31.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.041+0000 7f2b64e0b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2b601131f0 con 0x7f2b60104330 2024-05-04T03:18:31.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.041+0000 7f2b64e0b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2b601134f0 con 0x7f2b601039e0 2024-05-04T03:18:31.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.041+0000 7f2b5ed9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b601092d0 0x7f2b6019c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:31.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.041+0000 7f2b5ed9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b601092d0 0x7f2b6019c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36738/0 (socket says 172.21.15.1:36738) 2024-05-04T03:18:31.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.041+0000 7f2b5ed9d700 1 -- 172.21.15.1:0/2373487148 learned_addr learned my addr 172.21.15.1:0/2373487148 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:31.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.041+0000 7f2b5e59c700 1 --2- 172.21.15.1:0/2373487148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b601039e0 0x7f2b6019bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:31.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.041+0000 7f2b5dd9b700 1 --2- 172.21.15.1:0/2373487148 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2b60104330 0x7f2b6019c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:31.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.041+0000 7f2b5ed9d700 1 -- 172.21.15.1:0/2373487148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b601039e0 msgr2=0x7f2b6019bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:31.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.041+0000 7f2b5ed9d700 1 --2- 172.21.15.1:0/2373487148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b601039e0 0x7f2b6019bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:31.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.041+0000 7f2b5ed9d700 1 -- 172.21.15.1:0/2373487148 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2b60104330 msgr2=0x7f2b6019c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:31.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.041+0000 7f2b5ed9d700 1 --2- 172.21.15.1:0/2373487148 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2b60104330 0x7f2b6019c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:31.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.041+0000 7f2b5ed9d700 1 -- 172.21.15.1:0/2373487148 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b601a6f60 con 0x7f2b601092d0 2024-05-04T03:18:31.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.041+0000 7f2b5ed9d700 1 --2- 172.21.15.1:0/2373487148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b601092d0 0x7f2b6019c6b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2b5800b830 tx=0x7f2b58004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:31.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.041+0000 7f2b4f7fe700 1 -- 172.21.15.1:0/2373487148 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b58021070 con 0x7f2b601092d0 2024-05-04T03:18:31.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.041+0000 7f2b4f7fe700 1 -- 172.21.15.1:0/2373487148 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b58007e30 con 0x7f2b601092d0 2024-05-04T03:18:31.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.041+0000 7f2b64e0b700 1 -- 172.21.15.1:0/2373487148 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b601a71f0 con 0x7f2b601092d0 2024-05-04T03:18:31.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.042+0000 7f2b4f7fe700 1 -- 172.21.15.1:0/2373487148 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b58004700 con 0x7f2b601092d0 2024-05-04T03:18:31.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.042+0000 7f2b64e0b700 1 -- 172.21.15.1:0/2373487148 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b601a76a0 con 0x7f2b601092d0 2024-05-04T03:18:31.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.043+0000 7f2b64e0b700 1 -- 172.21.15.1:0/2373487148 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b600512c0 con 0x7f2b601092d0 2024-05-04T03:18:31.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.044+0000 7f2b4f7fe700 1 -- 172.21.15.1:0/2373487148 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f2b58007980 con 0x7f2b601092d0 2024-05-04T03:18:31.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.045+0000 7f2b4f7fe700 1 --2- 172.21.15.1:0/2373487148 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f2b480b2880 0x7f2b480b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:31.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.045+0000 7f2b5e59c700 1 --2- 172.21.15.1:0/2373487148 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f2b480b2880 0x7f2b480b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:31.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.045+0000 7f2b4f7fe700 1 -- 172.21.15.1:0/2373487148 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f2b5800f020 con 0x7f2b601092d0 2024-05-04T03:18:31.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.045+0000 7f2b5e59c700 1 --2- 172.21.15.1:0/2373487148 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f2b480b2880 0x7f2b480b4d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2b54005950 tx=0x7f2b5400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:31.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.050+0000 7f2b4f7fe700 1 -- 172.21.15.1:0/2373487148 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2b5802a020 con 0x7f2b601092d0 2024-05-04T03:18:31.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.263+0000 7f2b64e0b700 1 -- 172.21.15.1:0/2373487148 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b6002e5f0 con 0x7f2b601092d0 2024-05-04T03:18:31.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.265+0000 7f2b4f7fe700 1 -- 172.21.15.1:0/2373487148 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22380 (secure 0 0 0) 0x7f2b580ab510 con 0x7f2b601092d0 2024-05-04T03:18:31.268 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:18:31.268 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":83,"fsid":"4a572aa8-09c3-11ef-bc97-c7b262605968","created":"2024-05-04T03:07:32.845547+0000","modified":"2024-05-04T03:16:34.321580+0000","last_up_change":"2024-05-04T03:14:44.192571+0000","last_in_change":"2024-05-04T03:14:23.353342+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T03:11:01.139189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T03:15:43.151696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T03:15:46.753539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"100fe035-2454-4c95-90c7-759956ec36a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6803","nonce":3479011484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6805","nonce":3479011484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6809","nonce":3479011484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6807","nonce":3479011484}]},"public_addr":"172.21.15.1:6803/3479011484","cluster_addr":"172.21.15.1:6805/3479011484","heartbeat_back_addr":"172.21.15.1:6809/3479011484","heartbeat_front_addr":"172.21.15.1:6807/3479011484","state":["exists","up"]},{"osd":1,"uuid":"5826ac76-ef6b-4040-8ad1-305db355af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6801","nonce":3725706149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6803","nonce":3725706149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6807","nonce":3725706149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6805","nonce":3725706149}]},"public_addr":"172.21.15.72:6801/3725706149","cluster_addr":"172.21.15.72:6803/3725706149","heartbeat_back_addr":"172.21.15.72:6807/3725706149","heartbeat_front_addr":"172.21.15.72:6805/3725706149","state":["exists","up"]},{"osd":2,"uuid":"923d76e5-b032-4190-9e35-546d27722a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6801","nonce":515038343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6803","nonce":515038343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6807","nonce":515038343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6805","nonce":515038343}]},"public_addr":"172.21.15.77:6801/515038343","cluster_addr":"172.21.15.77:6803/515038343","heartbeat_back_addr":"172.21.15.77:6807/515038343","heartbeat_front_addr":"172.21.15.77:6805/515038343","state":["exists","up"]},{"osd":3,"uuid":"495cc0c5-6a99-481f-ad9e-38f05fb84356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6811","nonce":4050439365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6813","nonce":4050439365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6817","nonce":4050439365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6815","nonce":4050439365}]},"public_addr":"172.21.15.1:6811/4050439365","cluster_addr":"172.21.15.1:6813/4050439365","heartbeat_back_addr":"172.21.15.1:6817/4050439365","heartbeat_front_addr":"172.21.15.1:6815/4050439365","state":["exists","up"]},{"osd":4,"uuid":"b86b8026-79c0-4056-9e67-a61f54b6ab5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6809","nonce":2501732391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6811","nonce":2501732391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6815","nonce":2501732391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6813","nonce":2501732391}]},"public_addr":"172.21.15.72:6809/2501732391","cluster_addr":"172.21.15.72:6811/2501732391","heartbeat_back_addr":"172.21.15.72:6815/2501732391","heartbeat_front_addr":"172.21.15.72:6813/2501732391","state":["exists","up"]},{"osd":5,"uuid":"80975cb9-39e0-46b8-a343-9ff69194a14e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6809","nonce":3582852165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6811","nonce":3582852165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6815","nonce":3582852165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6813","nonce":3582852165}]},"public_addr":"172.21.15.77:6809/3582852165","cluster_addr":"172.21.15.77:6811/3582852165","heartbeat_back_addr":"172.21.15.77:6815/3582852165","heartbeat_front_addr":"172.21.15.77:6813/3582852165","state":["exists","up"]},{"osd":6,"uuid":"aca3fac5-fbb5-42de-bf0e-e4a0db7381e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6819","nonce":1799033649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6821","nonce":1799033649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6825","nonce":1799033649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6823","nonce":1799033649}]},"public_addr":"172.21.15.1:6819/1799033649","cluster_addr":"172.21.15.1:6821/1799033649","heartbeat_back_addr":"172.21.15.1:6825/1799033649","heartbeat_front_addr":"172.21.15.1:6823/1799033649","state":["exists","up"]},{"osd":7,"uuid":"cbff6b8a-c463-4249-ae45-fdc6f99a5489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6817","nonce":3460835447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6819","nonce":3460835447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6823","nonce":3460835447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6821","nonce":3460835447}]},"public_addr":"172.21.15.72:6817/3460835447","cluster_addr":"172.21.15.72:6819/3460835447","heartbeat_back_addr":"172.21.15.72:6823/3460835447","heartbeat_front_addr":"172.21.15.72:6821/3460835447","state":["exists","up"]},{"osd":8,"uuid":"e140bf13-4887-4d68-9373-3d4fc1c4584a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6817","nonce":1124479323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6819","nonce":1124479323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6823","nonce":1124479323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6821","nonce":1124479323}]},"public_addr":"172.21.15.77:6817/1124479323","cluster_addr":"172.21.15.77:6819/1124479323","heartbeat_back_addr":"172.21.15.77:6823/1124479323","heartbeat_front_addr":"172.21.15.77:6821/1124479323","state":["exists","up"]},{"osd":9,"uuid":"f1e8a232-cb11-4b9d-8789-e1637fa5aaa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6827","nonce":2681322272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6829","nonce":2681322272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6833","nonce":2681322272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6831","nonce":2681322272}]},"public_addr":"172.21.15.1:6827/2681322272","cluster_addr":"172.21.15.1:6829/2681322272","heartbeat_back_addr":"172.21.15.1:6833/2681322272","heartbeat_front_addr":"172.21.15.1:6831/2681322272","state":["exists","up"]},{"osd":10,"uuid":"79b570af-3d1e-439d-a328-9e6bc7d67839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6825","nonce":3293703569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6827","nonce":3293703569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6831","nonce":3293703569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6829","nonce":3293703569}]},"public_addr":"172.21.15.72:6825/3293703569","cluster_addr":"172.21.15.72:6827/3293703569","heartbeat_back_addr":"172.21.15.72:6831/3293703569","heartbeat_front_addr":"172.21.15.72:6829/3293703569","state":["exists","up"]},{"osd":11,"uuid":"e35bf83d-9ee4-4745-96c2-1c35cc8eefba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6825","nonce":4155947548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6827","nonce":4155947548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6831","nonce":4155947548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6829","nonce":4155947548}]},"public_addr":"172.21.15.77:6825/4155947548","cluster_addr":"172.21.15.77:6827/4155947548","heartbeat_back_addr":"172.21.15.77:6831/4155947548","heartbeat_front_addr":"172.21.15.77:6829/4155947548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:09.467126+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:33.861139+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:57.741962+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:11:21.746021+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:11:46.323252+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:12:10.640180+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:12:35.750591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:00.306549+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:25.012495+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:51.185229+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:14:16.403850+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:14:41.653316+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/2029897106":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/402488450":"2024-05-05T03:08:33.045884+0000","172.21.15.1:6801/128793132":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/2233526606":"2024-05-05T03:08:06.135783+0000","172.21.15.1:0/2715541855":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/2415697325":"2024-05-05T03:08:06.135783+0000","172.21.15.1:0/2305298106":"2024-05-05T03:08:06.135783+0000","172.21.15.1:6800/128793132":"2024-05-05T03:08:33.045884+0000","172.21.15.1:6800/4200529863":"2024-05-05T03:08:06.135783+0000","172.21.15.1:6801/4200529863":"2024-05-05T03:08:06.135783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T03:18:31.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.268+0000 7f2b64e0b700 1 -- 172.21.15.1:0/2373487148 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f2b480b2880 msgr2=0x7f2b480b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:31.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.268+0000 7f2b64e0b700 1 --2- 172.21.15.1:0/2373487148 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f2b480b2880 0x7f2b480b4d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2b54005950 tx=0x7f2b5400a300 comp rx=0 tx=0).stop 2024-05-04T03:18:31.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.268+0000 7f2b64e0b700 1 -- 172.21.15.1:0/2373487148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b601092d0 msgr2=0x7f2b6019c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:31.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.268+0000 7f2b64e0b700 1 --2- 172.21.15.1:0/2373487148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b601092d0 0x7f2b6019c6b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2b5800b830 tx=0x7f2b58004ce0 comp rx=0 tx=0).stop 2024-05-04T03:18:31.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.268+0000 7f2b64e0b700 1 -- 172.21.15.1:0/2373487148 shutdown_connections 2024-05-04T03:18:31.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.268+0000 7f2b64e0b700 1 --2- 172.21.15.1:0/2373487148 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f2b480b2880 0x7f2b480b4d90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:31.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.268+0000 7f2b64e0b700 1 --2- 172.21.15.1:0/2373487148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b601092d0 0x7f2b6019c6b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:31.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.268+0000 7f2b64e0b700 1 --2- 172.21.15.1:0/2373487148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b601039e0 0x7f2b6019bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:31.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.268+0000 7f2b64e0b700 1 --2- 172.21.15.1:0/2373487148 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2b60104330 0x7f2b6019c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:31.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.268+0000 7f2b64e0b700 1 -- 172.21.15.1:0/2373487148 >> 172.21.15.1:0/2373487148 conn(0x7f2b600fe540 msgr2=0x7f2b60100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:31.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.268+0000 7f2b64e0b700 1 -- 172.21.15.1:0/2373487148 shutdown_connections 2024-05-04T03:18:31.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:31.268+0000 7f2b64e0b700 1 -- 172.21.15.1:0/2373487148 wait complete. 2024-05-04T03:18:31.581 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:31 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2373487148' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:18:31.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:31 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/2373487148' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:18:31.913 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-04T03:15:49.823471+0000', 'modified': '2024-05-04T03:16:07.918400+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24494, 'mds_1': 24472, 'mds_2': 24467}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.72:6835/3930984976', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.72:6834', 'nonce': 3930984976}, {'type': 'v1', 'addr': '172.21.15.72:6835', 'nonce': 3930984976}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6835/4279951305', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6834', 'nonce': 4279951305}, {'type': 'v1', 'addr': '172.21.15.77:6835', 'nonce': 4279951305}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6837/2003521417', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6836', 'nonce': 2003521417}, {'type': 'v1', 'addr': '172.21.15.77:6837', 'nonce': 2003521417}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.72:6837/1417614203', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.72:6836', 'nonce': 1417614203}, {'type': 'v1', 'addr': '172.21.15.72:6837', 'nonce': 1417614203}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.77:6839/922723189', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6838', 'nonce': 922723189}, {'type': 'v1', 'addr': '172.21.15.77:6839', 'nonce': 922723189}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.72:6839/2976050650', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.72:6838', 'nonce': 2976050650}, {'type': 'v1', 'addr': '172.21.15.72:6839', 'nonce': 2976050650}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24494, 'qdb_cluster': [24472, 24494, 24467]} 2024-05-04T03:18:31.913 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-04T03:18:31.913 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph tell mds.h status 2024-05-04T03:18:31.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:31 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/2373487148' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:18:32.165 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:18:32.543 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:32 smithi001 ceph-mon[29731]: pgmap v372: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:18:32.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:32 smithi072 ceph-mon[33332]: pgmap v372: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:18:32.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:32 smithi077 ceph-mon[33531]: pgmap v372: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:18:33.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.863+0000 7f5dcf416700 1 -- 172.21.15.1:0/3997291488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5dc81045d0 msgr2=0x7f5dc81049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:33.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.863+0000 7f5dcf416700 1 --2- 172.21.15.1:0/3997291488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5dc81045d0 0x7f5dc81049e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5db800b0d0 tx=0x7f5db800b3e0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.863+0000 7f5dcf416700 1 -- 172.21.15.1:0/3997291488 shutdown_connections 2024-05-04T03:18:33.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.863+0000 7f5dcf416700 1 --2- 172.21.15.1:0/3997291488 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5dc8109850 0x7f5dc81111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.863+0000 7f5dcf416700 1 --2- 172.21.15.1:0/3997291488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5dc81045d0 0x7f5dc81049e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.863+0000 7f5dcf416700 1 --2- 172.21.15.1:0/3997291488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dc8108f00 0x7f5dc8109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.863+0000 7f5dcf416700 1 -- 172.21.15.1:0/3997291488 >> 172.21.15.1:0/3997291488 conn(0x7f5dc80ffea0 msgr2=0x7f5dc81022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:33.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.863+0000 7f5dcf416700 1 -- 172.21.15.1:0/3997291488 shutdown_connections 2024-05-04T03:18:33.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.863+0000 7f5dcf416700 1 -- 172.21.15.1:0/3997291488 wait complete. 2024-05-04T03:18:33.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.864+0000 7f5dcf416700 1 Processor -- start 2024-05-04T03:18:33.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.864+0000 7f5dcf416700 1 -- start start 2024-05-04T03:18:33.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.864+0000 7f5dcf416700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dc81045d0 0x7f5dc8076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:33.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.864+0000 7f5dcf416700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5dc8108f00 0x7f5dc8077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:33.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.864+0000 7f5dcf416700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5dc8109850 0x7f5dc8077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:33.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.864+0000 7f5dcf416700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc81144d0 con 0x7f5dc8108f00 2024-05-04T03:18:33.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.864+0000 7f5dcf416700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc8114350 con 0x7f5dc8109850 2024-05-04T03:18:33.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.864+0000 7f5dcf416700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc8114650 con 0x7f5dc81045d0 2024-05-04T03:18:33.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.865+0000 7f5dcc9b1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5dc8108f00 0x7f5dc8077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:33.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.865+0000 7f5dcc9b1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5dc8108f00 0x7f5dc8077110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43902/0 (socket says 172.21.15.1:43902) 2024-05-04T03:18:33.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.865+0000 7f5dcc9b1700 1 -- 172.21.15.1:0/780876522 learned_addr learned my addr 172.21.15.1:0/780876522 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:33.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.865+0000 7f5dcd9b3700 1 --2- 172.21.15.1:0/780876522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5dc8109850 0x7f5dc8077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:33.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.865+0000 7f5dcd1b2700 1 --2- 172.21.15.1:0/780876522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dc81045d0 0x7f5dc8076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:33.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.865+0000 7f5dcc9b1700 1 -- 172.21.15.1:0/780876522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dc81045d0 msgr2=0x7f5dc8076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:33.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.865+0000 7f5dcc9b1700 1 --2- 172.21.15.1:0/780876522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dc81045d0 0x7f5dc8076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.865+0000 7f5dcc9b1700 1 -- 172.21.15.1:0/780876522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5dc8109850 msgr2=0x7f5dc8077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:33.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.865+0000 7f5dcc9b1700 1 --2- 172.21.15.1:0/780876522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5dc8109850 0x7f5dc8077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.865+0000 7f5dcc9b1700 1 -- 172.21.15.1:0/780876522 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc81a9170 con 0x7f5dc8108f00 2024-05-04T03:18:33.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.865+0000 7f5dcd9b3700 1 --2- 172.21.15.1:0/780876522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5dc8109850 0x7f5dc8077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:18:33.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.865+0000 7f5dcc9b1700 1 --2- 172.21.15.1:0/780876522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5dc8108f00 0x7f5dc8077110 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00d980 tx=0x7f5dbc00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:33.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.865+0000 7f5dc67fc700 1 -- 172.21.15.1:0/780876522 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc017070 con 0x7f5dc8108f00 2024-05-04T03:18:33.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.865+0000 7f5dc67fc700 1 -- 172.21.15.1:0/780876522 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dbc010460 con 0x7f5dc8108f00 2024-05-04T03:18:33.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.865+0000 7f5dcf416700 1 -- 172.21.15.1:0/780876522 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc81a9460 con 0x7f5dc8108f00 2024-05-04T03:18:33.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.865+0000 7f5dc67fc700 1 -- 172.21.15.1:0/780876522 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc00de60 con 0x7f5dc8108f00 2024-05-04T03:18:33.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.865+0000 7f5dcf416700 1 -- 172.21.15.1:0/780876522 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc81a99a0 con 0x7f5dc8108f00 2024-05-04T03:18:33.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.868+0000 7f5dc67fc700 1 -- 172.21.15.1:0/780876522 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f5dbc00f620 con 0x7f5dc8108f00 2024-05-04T03:18:33.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.869+0000 7f5dc67fc700 1 --2- 172.21.15.1:0/780876522 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5db00b2880 0x7f5db00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:33.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.870+0000 7f5dcd1b2700 1 --2- 172.21.15.1:0/780876522 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5db00b2880 0x7f5db00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:33.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.870+0000 7f5dc67fc700 1 -- 172.21.15.1:0/780876522 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f5dbc0e1d80 con 0x7f5dc8108f00 2024-05-04T03:18:33.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.870+0000 7f5ddde2ab80 1 Processor -- start 2024-05-04T03:18:33.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.870+0000 7f5ddde2ab80 1 -- start start 2024-05-04T03:18:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.870+0000 7f5dcd1b2700 1 --2- 172.21.15.1:0/780876522 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5db00b2880 0x7f5db00b4d90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009fd0 tx=0x7f5db800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.870+0000 7f5ddde2ab80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5650c6de4da0 0x5650c6de51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.870+0000 7f5ddde2ab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c6ddeef0 0x5650c6ddf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.870+0000 7f5ddde2ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650c6ddf840 0x5650c6deb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.870+0000 7f5ddde2ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5650c6d01810 con 0x5650c6ddf840 2024-05-04T03:18:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.870+0000 7f5ddde2ab80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5650c6ca0770 con 0x5650c6de4da0 2024-05-04T03:18:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.870+0000 7f5ddde2ab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5650c6cfd7d0 con 0x5650c6ddeef0 2024-05-04T03:18:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.870+0000 7f5dcd9b3700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650c6ddf840 0x5650c6deb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.870+0000 7f5dcd9b3700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650c6ddf840 0x5650c6deb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43908/0 (socket says 172.21.15.1:43908) 2024-05-04T03:18:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.870+0000 7f5dcd9b3700 1 -- 172.21.15.1:0/771451906 learned_addr learned my addr 172.21.15.1:0/771451906 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.870+0000 7f5dcc9b1700 1 --2- 172.21.15.1:0/771451906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c6ddeef0 0x5650c6ddf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.870+0000 7f5dcd1b2700 1 --2- 172.21.15.1:0/771451906 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5650c6de4da0 0x5650c6de51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.871+0000 7f5dcd9b3700 1 -- 172.21.15.1:0/771451906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c6ddeef0 msgr2=0x5650c6ddf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.871+0000 7f5dcd9b3700 1 --2- 172.21.15.1:0/771451906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c6ddeef0 0x5650c6ddf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.871+0000 7f5dcd9b3700 1 -- 172.21.15.1:0/771451906 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5650c6de4da0 msgr2=0x5650c6de51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.871+0000 7f5dcd9b3700 1 --2- 172.21.15.1:0/771451906 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5650c6de4da0 0x5650c6de51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.871+0000 7f5dcd9b3700 1 -- 172.21.15.1:0/771451906 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650c6c4e880 con 0x5650c6ddf840 2024-05-04T03:18:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.871+0000 7f5dcd9b3700 1 --2- 172.21.15.1:0/771451906 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650c6ddf840 0x5650c6deb9f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000cc70 tx=0x7f5dc000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cfe8f868509ac8c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.871+0000 7f5daf7fe700 1 -- 172.21.15.1:0/771451906 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc000ed70 con 0x5650c6ddf840 2024-05-04T03:18:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.871+0000 7f5daf7fe700 1 -- 172.21.15.1:0/771451906 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dc0004ce0 con 0x5650c6ddf840 2024-05-04T03:18:33.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.871+0000 7f5daf7fe700 1 -- 172.21.15.1:0/771451906 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0010450 con 0x5650c6ddf840 2024-05-04T03:18:33.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.872+0000 7f5ddde2ab80 1 -- 172.21.15.1:0/771451906 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650c6ddf840 msgr2=0x5650c6deb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:33.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.872+0000 7f5ddde2ab80 1 --2- 172.21.15.1:0/771451906 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650c6ddf840 0x5650c6deb9f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000cc70 tx=0x7f5dc000cf80 comp rx=0 tx=0).stop 2024-05-04T03:18:33.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.872+0000 7f5ddde2ab80 1 -- 172.21.15.1:0/771451906 shutdown_connections 2024-05-04T03:18:33.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.872+0000 7f5ddde2ab80 1 --2- 172.21.15.1:0/771451906 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5650c6de4da0 0x5650c6de51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.872+0000 7f5ddde2ab80 1 --2- 172.21.15.1:0/771451906 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650c6ddf840 0x5650c6deb9f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.872+0000 7f5ddde2ab80 1 --2- 172.21.15.1:0/771451906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c6ddeef0 0x5650c6ddf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.872+0000 7f5ddde2ab80 1 -- 172.21.15.1:0/771451906 >> 172.21.15.1:0/771451906 conn(0x5650c6c61df0 msgr2=0x5650c6dd9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:33.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.872+0000 7f5ddde2ab80 1 -- 172.21.15.1:0/771451906 shutdown_connections 2024-05-04T03:18:33.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.872+0000 7f5ddde2ab80 1 -- 172.21.15.1:0/771451906 wait complete. 2024-05-04T03:18:33.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.873+0000 7f5ddde2ab80 1 -- start start 2024-05-04T03:18:33.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.873+0000 7f5ddde2ab80 1 Processor -- start 2024-05-04T03:18:33.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.873+0000 7f5ddde2ab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c6ddeef0 0x5650c6de8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:33.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.873+0000 7f5ddde2ab80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5650c6ddf840 0x5650c6de94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:33.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.873+0000 7f5ddde2ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650c6de4da0 0x5650c6de99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:33.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.873+0000 7f5ddde2ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5650c6dec170 con 0x5650c6de4da0 2024-05-04T03:18:33.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.873+0000 7f5ddde2ab80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5650c6c67be0 con 0x5650c6ddf840 2024-05-04T03:18:33.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.873+0000 7f5ddde2ab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5650c6dec2f0 con 0x5650c6ddeef0 2024-05-04T03:18:33.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.873+0000 7f5ddde2ab80 10 client.0 _ll_register_callbacks cb 0x5650c6c998d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:18:33.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.874+0000 7f5dcd9b3700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650c6de4da0 0x5650c6de99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:33.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.874+0000 7f5dcd9b3700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650c6de4da0 0x5650c6de99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43914/0 (socket says 172.21.15.1:43914) 2024-05-04T03:18:33.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.874+0000 7f5dcd9b3700 1 -- 172.21.15.1:0/3606578407 learned_addr learned my addr 172.21.15.1:0/3606578407 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:33.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.874+0000 7f5dcd1b2700 1 --2- 172.21.15.1:0/3606578407 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c6ddeef0 0x5650c6de8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:33.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.874+0000 7f5dcc9b1700 1 --2- 172.21.15.1:0/3606578407 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5650c6ddf840 0x5650c6de94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:33.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.874+0000 7f5dcd9b3700 1 -- 172.21.15.1:0/3606578407 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c6ddeef0 msgr2=0x5650c6de8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:33.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.874+0000 7f5dcd9b3700 1 --2- 172.21.15.1:0/3606578407 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c6ddeef0 0x5650c6de8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.874+0000 7f5dcd9b3700 1 -- 172.21.15.1:0/3606578407 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5650c6ddf840 msgr2=0x5650c6de94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:33.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.874+0000 7f5dcd9b3700 1 --2- 172.21.15.1:0/3606578407 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5650c6ddf840 0x5650c6de94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.874+0000 7f5dcd9b3700 1 -- 172.21.15.1:0/3606578407 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650c6c9d640 con 0x5650c6de4da0 2024-05-04T03:18:33.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.874+0000 7f5dcd9b3700 1 --2- 172.21.15.1:0/3606578407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650c6de4da0 0x5650c6de99e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00057e0 tx=0x7f5dc0005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8100a8d5a0985e15 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:33.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.874+0000 7f5dacff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:18:33.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.874+0000 7f5dacff9700 1 -- 172.21.15.1:0/3606578407 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc000eb70 con 0x5650c6de4da0 2024-05-04T03:18:33.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.875+0000 7f5dacff9700 1 -- 172.21.15.1:0/3606578407 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f5dc000ed10 con 0x5650c6de4da0 2024-05-04T03:18:33.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.875+0000 7f5ddde2ab80 1 -- 172.21.15.1:0/3606578407 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dc0004ce0 con 0x5650c6de4da0 2024-05-04T03:18:33.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.875+0000 7f5dacff9700 1 -- 172.21.15.1:0/3606578407 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5dc002b030 con 0x5650c6de4da0 2024-05-04T03:18:33.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.875+0000 7f5ddde2ab80 10 client.15045 fetch_fsmap learned FSMap version 25 2024-05-04T03:18:33.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.875+0000 7f5ddde2ab80 1 -- 172.21.15.1:0/3606578407 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5650c6df6e40 con 0x5650c6de4da0 2024-05-04T03:18:33.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.875+0000 7f5dacff9700 1 -- 172.21.15.1:0/3606578407 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dc0004ce0 con 0x5650c6de4da0 2024-05-04T03:18:33.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.875+0000 7f5dacff9700 1 -- 172.21.15.1:0/3606578407 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f5dc0003680 con 0x5650c6de4da0 2024-05-04T03:18:33.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.875+0000 7f5ddde2ab80 10 client.15045 fetch_fsmap finished waiting for FSMap version 25 2024-05-04T03:18:33.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.875+0000 7f5ddde2ab80 10 client.15045 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-04T03:18:33.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.875+0000 7f5ddde2ab80 20 client.15045 populate_metadata read hostname 'smithi001' 2024-05-04T03:18:33.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.875+0000 7f5ddde2ab80 1 --2- 172.21.15.1:0/3606578407 >> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] conn(0x5650c6df92c0 0x5650c6dfa730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:33.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.875+0000 7f5ddde2ab80 4 client.15045 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:18:33.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.875+0000 7f5ddde2ab80 1 -- 172.21.15.1:0/3606578407 --> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650c6d0f1f0 con 0x5650c6df92c0 2024-05-04T03:18:33.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.876+0000 7f5dcd1b2700 1 --2- 172.21.15.1:0/3606578407 >> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] conn(0x5650c6df92c0 0x5650c6dfa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:33.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.876+0000 7f5dacff9700 10 client.15045 ms_handle_connect on v2:172.21.15.72:6836/1417614203 2024-05-04T03:18:33.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.876+0000 7f5dcd1b2700 1 --2- 172.21.15.1:0/3606578407 >> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] conn(0x5650c6df92c0 0x5650c6dfa730 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=28fcf860cc39dabc server_cookie=91453aceeb1d4d64 in_seq=0 out_seq=0 2024-05-04T03:18:33.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.877+0000 7f5dacff9700 1 -- 172.21.15.1:0/3606578407 <== mds.2 v2:172.21.15.72:6836/1417614203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5650c6d0f1f0 con 0x5650c6df92c0 2024-05-04T03:18:33.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.877+0000 7f5dacff9700 10 client.15045 handle_command_reply: tid=0 2024-05-04T03:18:33.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.877+0000 7f5ddde2ab80 1 client.15045 shutdown 2024-05-04T03:18:33.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.877+0000 7f5ddde2ab80 1 -- 172.21.15.1:0/3606578407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650c6de4da0 msgr2=0x5650c6de99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.877+0000 7f5ddde2ab80 1 --2- 172.21.15.1:0/3606578407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650c6de4da0 0x5650c6de99e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00057e0 tx=0x7f5dc0005810 comp rx=0 tx=0).stop 2024-05-04T03:18:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.877+0000 7f5ddde2ab80 1 -- 172.21.15.1:0/3606578407 shutdown_connections 2024-05-04T03:18:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.877+0000 7f5ddde2ab80 1 --2- 172.21.15.1:0/3606578407 >> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] conn(0x5650c6df92c0 0x5650c6dfa730 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.877+0000 7f5ddde2ab80 1 --2- 172.21.15.1:0/3606578407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650c6de4da0 0x5650c6de99e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.877+0000 7f5ddde2ab80 1 --2- 172.21.15.1:0/3606578407 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c6ddeef0 0x5650c6de8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.877+0000 7f5ddde2ab80 1 --2- 172.21.15.1:0/3606578407 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5650c6ddf840 0x5650c6de94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.877+0000 7f5ddde2ab80 1 -- 172.21.15.1:0/3606578407 >> 172.21.15.1:0/3606578407 conn(0x5650c6c61df0 msgr2=0x5650c6dddee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.877+0000 7f5dacff9700 0 client.15045 ms_handle_reset on v2:172.21.15.72:6836/1417614203 2024-05-04T03:18:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.877+0000 7f5ddde2ab80 1 -- 172.21.15.1:0/3606578407 shutdown_connections 2024-05-04T03:18:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.877+0000 7f5ddde2ab80 1 -- 172.21.15.1:0/3606578407 wait complete. 2024-05-04T03:18:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.878+0000 7f5ddde2ab80 20 client.15045 trim_cache size 0 max 16384 2024-05-04T03:18:33.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.899+0000 7f5ddde2ab80 1 Processor -- start 2024-05-04T03:18:33.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.899+0000 7f5ddde2ab80 1 -- start start 2024-05-04T03:18:33.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.899+0000 7f5ddde2ab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c6ddeef0 0x5650c6dd44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:33.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.899+0000 7f5ddde2ab80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5650c6ddf840 0x5650c6de9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:33.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.899+0000 7f5ddde2ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650c6de4da0 0x5650c6de7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:33.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.899+0000 7f5ddde2ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5650c6df8a70 con 0x5650c6de4da0 2024-05-04T03:18:33.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.899+0000 7f5ddde2ab80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5650c6df9140 con 0x5650c6ddf840 2024-05-04T03:18:33.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.899+0000 7f5ddde2ab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5650c6df9810 con 0x5650c6ddeef0 2024-05-04T03:18:33.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.899+0000 7f5dcd9b3700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650c6de4da0 0x5650c6de7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:33.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.899+0000 7f5dcd9b3700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650c6de4da0 0x5650c6de7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43920/0 (socket says 172.21.15.1:43920) 2024-05-04T03:18:33.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.899+0000 7f5dcd9b3700 1 -- 172.21.15.1:0/2620925188 learned_addr learned my addr 172.21.15.1:0/2620925188 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:33.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.899+0000 7f5dcc9b1700 1 --2- 172.21.15.1:0/2620925188 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5650c6ddf840 0x5650c6de9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:33.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.899+0000 7f5dcd1b2700 1 --2- 172.21.15.1:0/2620925188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c6ddeef0 0x5650c6dd44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:33.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.900+0000 7f5dcd9b3700 1 -- 172.21.15.1:0/2620925188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c6ddeef0 msgr2=0x5650c6dd44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:33.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.900+0000 7f5dcd9b3700 1 --2- 172.21.15.1:0/2620925188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c6ddeef0 0x5650c6dd44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.900+0000 7f5dcd9b3700 1 -- 172.21.15.1:0/2620925188 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5650c6ddf840 msgr2=0x5650c6de9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:33.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.900+0000 7f5dcd9b3700 1 --2- 172.21.15.1:0/2620925188 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5650c6ddf840 0x5650c6de9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.900+0000 7f5dcd9b3700 1 -- 172.21.15.1:0/2620925188 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650c6dd6470 con 0x5650c6de4da0 2024-05-04T03:18:33.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.900+0000 7f5dcc9b1700 1 --2- 172.21.15.1:0/2620925188 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5650c6ddf840 0x5650c6de9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:18:33.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.900+0000 7f5dcd1b2700 1 --2- 172.21.15.1:0/2620925188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c6ddeef0 0x5650c6dd44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:18:33.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.900+0000 7f5dcd9b3700 1 --2- 172.21.15.1:0/2620925188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650c6de4da0 0x5650c6de7850 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00059d0 tx=0x7f5dc0005410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c476efc1a736447 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:33.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.900+0000 7f5daeffd700 1 -- 172.21.15.1:0/2620925188 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0003dd0 con 0x5650c6de4da0 2024-05-04T03:18:33.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.900+0000 7f5daeffd700 1 -- 172.21.15.1:0/2620925188 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dc0034070 con 0x5650c6de4da0 2024-05-04T03:18:33.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.900+0000 7f5daeffd700 1 -- 172.21.15.1:0/2620925188 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc001a950 con 0x5650c6de4da0 2024-05-04T03:18:33.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.901+0000 7f5ddde2ab80 1 -- 172.21.15.1:0/2620925188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650c6de4da0 msgr2=0x5650c6de7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:33.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.901+0000 7f5ddde2ab80 1 --2- 172.21.15.1:0/2620925188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650c6de4da0 0x5650c6de7850 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00059d0 tx=0x7f5dc0005410 comp rx=0 tx=0).stop 2024-05-04T03:18:33.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.901+0000 7f5ddde2ab80 1 -- 172.21.15.1:0/2620925188 shutdown_connections 2024-05-04T03:18:33.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.901+0000 7f5ddde2ab80 1 --2- 172.21.15.1:0/2620925188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650c6de4da0 0x5650c6de7850 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.901+0000 7f5ddde2ab80 1 --2- 172.21.15.1:0/2620925188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c6ddeef0 0x5650c6dd44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.901+0000 7f5ddde2ab80 1 --2- 172.21.15.1:0/2620925188 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5650c6ddf840 0x5650c6de9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.901+0000 7f5ddde2ab80 1 -- 172.21.15.1:0/2620925188 >> 172.21.15.1:0/2620925188 conn(0x5650c6c61df0 msgr2=0x5650c6de3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:33.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.901+0000 7f5ddde2ab80 1 -- 172.21.15.1:0/2620925188 shutdown_connections 2024-05-04T03:18:33.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.901+0000 7f5ddde2ab80 1 -- 172.21.15.1:0/2620925188 wait complete. 2024-05-04T03:18:33.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.901+0000 7f5ddde2ab80 1 -- start start 2024-05-04T03:18:33.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.901+0000 7f5ddde2ab80 1 Processor -- start 2024-05-04T03:18:33.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.902+0000 7f5ddde2ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650c6ddeef0 0x5650c6e03bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:33.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.902+0000 7f5ddde2ab80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5650c6ddf840 0x5650c6e040f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:33.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.902+0000 7f5ddde2ab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c6de4da0 0x5650c6dfeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:33.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.902+0000 7f5ddde2ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5650c6dfa5b0 con 0x5650c6ddeef0 2024-05-04T03:18:33.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.902+0000 7f5ddde2ab80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5650c6df9ee0 con 0x5650c6ddf840 2024-05-04T03:18:33.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.902+0000 7f5ddde2ab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5650c6dfac80 con 0x5650c6de4da0 2024-05-04T03:18:33.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.902+0000 7f5ddde2ab80 10 client.0 _ll_register_callbacks cb 0x5650c6c998d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:18:33.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.902+0000 7f5dcd1b2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650c6ddeef0 0x5650c6e03bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.902+0000 7f5dcd1b2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650c6ddeef0 0x5650c6e03bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43932/0 (socket says 172.21.15.1:43932) 2024-05-04T03:18:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.902+0000 7f5dcd1b2700 1 -- 172.21.15.1:0/3327189890 learned_addr learned my addr 172.21.15.1:0/3327189890 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.902+0000 7f5dcc9b1700 1 --2- 172.21.15.1:0/3327189890 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5650c6ddf840 0x5650c6e040f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.902+0000 7f5dcd9b3700 1 --2- 172.21.15.1:0/3327189890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c6de4da0 0x5650c6dfeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.902+0000 7f5dcd1b2700 1 -- 172.21.15.1:0/3327189890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c6de4da0 msgr2=0x5650c6dfeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.902+0000 7f5dcd1b2700 1 --2- 172.21.15.1:0/3327189890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c6de4da0 0x5650c6dfeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.902+0000 7f5dcd1b2700 1 -- 172.21.15.1:0/3327189890 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5650c6ddf840 msgr2=0x5650c6e040f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.902+0000 7f5dcd1b2700 1 --2- 172.21.15.1:0/3327189890 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5650c6ddf840 0x5650c6e040f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.902+0000 7f5dcd1b2700 1 -- 172.21.15.1:0/3327189890 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650c6dff0a0 con 0x5650c6ddeef0 2024-05-04T03:18:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.902+0000 7f5dcc9b1700 1 --2- 172.21.15.1:0/3327189890 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5650c6ddf840 0x5650c6e040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:18:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.902+0000 7f5dcd9b3700 1 --2- 172.21.15.1:0/3327189890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c6de4da0 0x5650c6dfeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:18:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.902+0000 7f5dcd1b2700 1 --2- 172.21.15.1:0/3327189890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650c6ddeef0 0x5650c6e03bb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5db8000f80 tx=0x7f5db800b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6679b643e7b2770 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.902+0000 7f5dacff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:18:33.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.903+0000 7f5dacff9700 1 -- 172.21.15.1:0/3327189890 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db8007d20 con 0x5650c6ddeef0 2024-05-04T03:18:33.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.903+0000 7f5dacff9700 1 -- 172.21.15.1:0/3327189890 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f5db80322d0 con 0x5650c6ddeef0 2024-05-04T03:18:33.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.903+0000 7f5ddde2ab80 1 -- 172.21.15.1:0/3327189890 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dc0034070 con 0x5650c6ddeef0 2024-05-04T03:18:33.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.903+0000 7f5dacff9700 1 -- 172.21.15.1:0/3327189890 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5db8042050 con 0x5650c6ddeef0 2024-05-04T03:18:33.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.903+0000 7f5ddde2ab80 10 client.15051 fetch_fsmap learned FSMap version 25 2024-05-04T03:18:33.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.903+0000 7f5ddde2ab80 1 -- 172.21.15.1:0/3327189890 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5650c6e64df0 con 0x5650c6ddeef0 2024-05-04T03:18:33.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.903+0000 7f5dacff9700 1 -- 172.21.15.1:0/3327189890 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dc0034070 con 0x5650c6ddeef0 2024-05-04T03:18:33.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.903+0000 7f5dacff9700 1 -- 172.21.15.1:0/3327189890 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f5db8032ce0 con 0x5650c6ddeef0 2024-05-04T03:18:33.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.903+0000 7f5ddde2ab80 10 client.15051 fetch_fsmap finished waiting for FSMap version 25 2024-05-04T03:18:33.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.903+0000 7f5ddde2ab80 10 client.15051 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-04T03:18:33.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.903+0000 7f5ddde2ab80 20 client.15051 populate_metadata read hostname 'smithi001' 2024-05-04T03:18:33.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.903+0000 7f5ddde2ab80 1 --2- 172.21.15.1:0/3327189890 >> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] conn(0x5650c6e672e0 0x5650c6e68750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:33.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.903+0000 7f5ddde2ab80 4 client.15051 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-04T03:18:33.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.903+0000 7f5ddde2ab80 1 -- 172.21.15.1:0/3327189890 --> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] -- command(tid 0: {"prefix": "status"}) -- 0x5650c6d09290 con 0x5650c6e672e0 2024-05-04T03:18:33.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.903+0000 7f5dcc9b1700 1 --2- 172.21.15.1:0/3327189890 >> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] conn(0x5650c6e672e0 0x5650c6e68750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:33.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.904+0000 7f5dcc9b1700 1 --2- 172.21.15.1:0/3327189890 >> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] conn(0x5650c6e672e0 0x5650c6e68750 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=af350d0fcb07b789 server_cookie=51722baff75d0b04 in_seq=0 out_seq=0 2024-05-04T03:18:33.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.904+0000 7f5dacff9700 10 client.15051 ms_handle_connect on v2:172.21.15.72:6836/1417614203 2024-05-04T03:18:33.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.904+0000 7f5dacff9700 1 -- 172.21.15.1:0/3327189890 <== mds.2 v2:172.21.15.72:6836/1417614203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x5650c6d09290 con 0x5650c6e672e0 2024-05-04T03:18:33.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.904+0000 7f5dacff9700 10 client.15051 handle_command_reply: tid=0 2024-05-04T03:18:33.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.904+0000 7f5ddde2ab80 1 client.15051 shutdown 2024-05-04T03:18:33.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.904+0000 7f5ddde2ab80 1 -- 172.21.15.1:0/3327189890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650c6ddeef0 msgr2=0x5650c6e03bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:33.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.904+0000 7f5ddde2ab80 1 --2- 172.21.15.1:0/3327189890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650c6ddeef0 0x5650c6e03bb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5db8000f80 tx=0x7f5db800b3e0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.904+0000 7f5ddde2ab80 1 -- 172.21.15.1:0/3327189890 shutdown_connections 2024-05-04T03:18:33.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.904+0000 7f5ddde2ab80 1 --2- 172.21.15.1:0/3327189890 >> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] conn(0x5650c6e672e0 0x5650c6e68750 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.904+0000 7f5ddde2ab80 1 --2- 172.21.15.1:0/3327189890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5650c6de4da0 0x5650c6dfeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.904+0000 7f5ddde2ab80 1 --2- 172.21.15.1:0/3327189890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650c6ddeef0 0x5650c6e03bb0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.904+0000 7f5dacff9700 0 client.15051 ms_handle_reset on v2:172.21.15.72:6836/1417614203 2024-05-04T03:18:33.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.904+0000 7f5ddde2ab80 1 --2- 172.21.15.1:0/3327189890 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5650c6ddf840 0x5650c6e040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.904+0000 7f5ddde2ab80 1 -- 172.21.15.1:0/3327189890 >> 172.21.15.1:0/3327189890 conn(0x5650c6c61df0 msgr2=0x5650c6de7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:33.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.904+0000 7f5ddde2ab80 1 -- 172.21.15.1:0/3327189890 shutdown_connections 2024-05-04T03:18:33.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.904+0000 7f5ddde2ab80 1 -- 172.21.15.1:0/3327189890 wait complete. 2024-05-04T03:18:33.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.905+0000 7f5ddde2ab80 20 client.15051 trim_cache size 0 max 16384 2024-05-04T03:18:33.914 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-04T03:18:33.914 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "4a572aa8-09c3-11ef-bc97-c7b262605968", 2024-05-04T03:18:33.914 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 2, 2024-05-04T03:18:33.915 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24467, 2024-05-04T03:18:33.915 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-05-04T03:18:33.915 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-05-04T03:18:33.915 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-05-04T03:18:33.915 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 150.67100323, 2024-05-04T03:18:33.915 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 25, 2024-05-04T03:18:33.915 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 83, 2024-05-04T03:18:33.915 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 81, 2024-05-04T03:18:33.915 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 192.346036204 2024-05-04T03:18:33.915 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-04T03:18:33.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.905+0000 7f5dcf416700 1 -- 172.21.15.1:0/780876522 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5db00b2880 msgr2=0x7f5db00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:33.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.905+0000 7f5dcf416700 1 --2- 172.21.15.1:0/780876522 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5db00b2880 0x7f5db00b4d90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009fd0 tx=0x7f5db800b040 comp rx=0 tx=0).stop 2024-05-04T03:18:33.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.905+0000 7f5dcf416700 1 -- 172.21.15.1:0/780876522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5dc8108f00 msgr2=0x7f5dc8077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:33.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.905+0000 7f5dcf416700 1 --2- 172.21.15.1:0/780876522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5dc8108f00 0x7f5dc8077110 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00d980 tx=0x7f5dbc00dd40 comp rx=0 tx=0).stop 2024-05-04T03:18:33.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.905+0000 7f5dcf416700 1 -- 172.21.15.1:0/780876522 shutdown_connections 2024-05-04T03:18:33.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.905+0000 7f5dcf416700 1 --2- 172.21.15.1:0/780876522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5dc8109850 0x7f5dc8077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.905+0000 7f5dcf416700 1 --2- 172.21.15.1:0/780876522 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5db00b2880 0x7f5db00b4d90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.905+0000 7f5dcf416700 1 --2- 172.21.15.1:0/780876522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dc81045d0 0x7f5dc8076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.905+0000 7f5dcf416700 1 --2- 172.21.15.1:0/780876522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5dc8108f00 0x7f5dc8077110 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:33.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.905+0000 7f5dcf416700 1 -- 172.21.15.1:0/780876522 >> 172.21.15.1:0/780876522 conn(0x7f5dc80ffea0 msgr2=0x7f5dc81022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:33.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.905+0000 7f5dcf416700 1 -- 172.21.15.1:0/780876522 shutdown_connections 2024-05-04T03:18:33.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:33.905+0000 7f5dcf416700 1 -- 172.21.15.1:0/780876522 wait complete. 2024-05-04T03:18:34.557 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-04T03:18:34.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:34 smithi072 ceph-mon[33332]: pgmap v373: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:18:34.804 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:18:34.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:34 smithi001 ceph-mon[29731]: pgmap v373: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:18:34.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:34 smithi077 ceph-mon[33531]: pgmap v373: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:18:36.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.418+0000 7ff25e969700 1 -- 172.21.15.1:0/3438781429 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff258107df0 msgr2=0x7ff258108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:36.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.418+0000 7ff25e969700 1 --2- 172.21.15.1:0/3438781429 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff258107df0 0x7ff258108200 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff24c0099c0 tx=0x7ff24c009cd0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.419+0000 7ff25e969700 1 -- 172.21.15.1:0/3438781429 shutdown_connections 2024-05-04T03:18:36.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.419+0000 7ff25e969700 1 --2- 172.21.15.1:0/3438781429 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2581034c0 0x7ff2581038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.419+0000 7ff25e969700 1 --2- 172.21.15.1:0/3438781429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff258108740 0x7ff2581100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.419+0000 7ff25e969700 1 --2- 172.21.15.1:0/3438781429 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff258107df0 0x7ff258108200 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.419+0000 7ff25e969700 1 -- 172.21.15.1:0/3438781429 >> 172.21.15.1:0/3438781429 conn(0x7ff2580fedc0 msgr2=0x7ff2581011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:36.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.419+0000 7ff25e969700 1 -- 172.21.15.1:0/3438781429 shutdown_connections 2024-05-04T03:18:36.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.419+0000 7ff25e969700 1 -- 172.21.15.1:0/3438781429 wait complete. 2024-05-04T03:18:36.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.419+0000 7ff25e969700 1 Processor -- start 2024-05-04T03:18:36.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.419+0000 7ff25e969700 1 -- start start 2024-05-04T03:18:36.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.420+0000 7ff25e969700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2581034c0 0x7ff258199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:36.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.420+0000 7ff25e969700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff258107df0 0x7ff25819a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:36.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.420+0000 7ff25e969700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff258108740 0x7ff25819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:36.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.420+0000 7ff25e969700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff2581133c0 con 0x7ff2581034c0 2024-05-04T03:18:36.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.420+0000 7ff25e969700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff258113240 con 0x7ff258108740 2024-05-04T03:18:36.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.420+0000 7ff25e969700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff258113540 con 0x7ff258107df0 2024-05-04T03:18:36.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.420+0000 7ff257fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2581034c0 0x7ff258199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:36.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.420+0000 7ff257fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2581034c0 0x7ff258199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43944/0 (socket says 172.21.15.1:43944) 2024-05-04T03:18:36.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.420+0000 7ff257fff700 1 -- 172.21.15.1:0/705672847 learned_addr learned my addr 172.21.15.1:0/705672847 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:36.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.420+0000 7ff2577fe700 1 --2- 172.21.15.1:0/705672847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff258107df0 0x7ff25819a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:36.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.420+0000 7ff25cf06700 1 --2- 172.21.15.1:0/705672847 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff258108740 0x7ff25819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:36.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.420+0000 7ff257fff700 1 -- 172.21.15.1:0/705672847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff258107df0 msgr2=0x7ff25819a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:36.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.420+0000 7ff257fff700 1 --2- 172.21.15.1:0/705672847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff258107df0 0x7ff25819a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.420+0000 7ff257fff700 1 -- 172.21.15.1:0/705672847 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff258108740 msgr2=0x7ff25819e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:36.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.420+0000 7ff257fff700 1 --2- 172.21.15.1:0/705672847 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff258108740 0x7ff25819e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.420+0000 7ff257fff700 1 -- 172.21.15.1:0/705672847 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff25819ec40 con 0x7ff2581034c0 2024-05-04T03:18:36.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.420+0000 7ff25cf06700 1 --2- 172.21.15.1:0/705672847 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff258108740 0x7ff25819e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:18:36.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.421+0000 7ff257fff700 1 --2- 172.21.15.1:0/705672847 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2581034c0 0x7ff258199d70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff24800d950 tx=0x7ff24800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:36.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.421+0000 7ff2557fa700 1 -- 172.21.15.1:0/705672847 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff248016070 con 0x7ff2581034c0 2024-05-04T03:18:36.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.421+0000 7ff2557fa700 1 -- 172.21.15.1:0/705672847 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff248009df0 con 0x7ff2581034c0 2024-05-04T03:18:36.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.421+0000 7ff25e969700 1 -- 172.21.15.1:0/705672847 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff25819ef30 con 0x7ff2581034c0 2024-05-04T03:18:36.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.421+0000 7ff2557fa700 1 -- 172.21.15.1:0/705672847 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24800f3c0 con 0x7ff2581034c0 2024-05-04T03:18:36.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.421+0000 7ff25e969700 1 -- 172.21.15.1:0/705672847 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2581a6850 con 0x7ff2581034c0 2024-05-04T03:18:36.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.424+0000 7ff2557fa700 1 -- 172.21.15.1:0/705672847 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7ff248019070 con 0x7ff2581034c0 2024-05-04T03:18:36.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.424+0000 7ff2557fa700 1 --2- 172.21.15.1:0/705672847 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ff2400b2880 0x7ff2400b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:36.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.424+0000 7ff2577fe700 1 --2- 172.21.15.1:0/705672847 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ff2400b2880 0x7ff2400b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:36.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.424+0000 7ff2557fa700 1 -- 172.21.15.1:0/705672847 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7ff2480e1d70 con 0x7ff2581034c0 2024-05-04T03:18:36.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.425+0000 7ff2577fe700 1 --2- 172.21.15.1:0/705672847 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ff2400b2880 0x7ff2400b4d90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff24c006010 tx=0x7ff24c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:36.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.425+0000 7ff26d37db80 1 Processor -- start 2024-05-04T03:18:36.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.425+0000 7ff26d37db80 1 -- start start 2024-05-04T03:18:36.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.425+0000 7ff26d37db80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd12967ef0 0x55bd12968300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:36.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.425+0000 7ff26d37db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd12968840 0x55bd12976570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:36.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.425+0000 7ff26d37db80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bd12976ab0 0x55bd12978ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:36.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.425+0000 7ff26d37db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bd1288a810 con 0x55bd12968840 2024-05-04T03:18:36.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.425+0000 7ff26d37db80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55bd12829770 con 0x55bd12967ef0 2024-05-04T03:18:36.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.425+0000 7ff26d37db80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bd128867d0 con 0x55bd12976ab0 2024-05-04T03:18:36.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.425+0000 7ff2577fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd12968840 0x55bd12976570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:36.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.426+0000 7ff2577fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd12968840 0x55bd12976570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43948/0 (socket says 172.21.15.1:43948) 2024-05-04T03:18:36.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.426+0000 7ff2577fe700 1 -- 172.21.15.1:0/3493037784 learned_addr learned my addr 172.21.15.1:0/3493037784 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:36.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.426+0000 7ff25cf06700 1 --2- 172.21.15.1:0/3493037784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bd12976ab0 0x55bd12978ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:36.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.426+0000 7ff257fff700 1 --2- 172.21.15.1:0/3493037784 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd12967ef0 0x55bd12968300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:36.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.426+0000 7ff2577fe700 1 -- 172.21.15.1:0/3493037784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bd12976ab0 msgr2=0x55bd12978ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:36.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.426+0000 7ff2577fe700 1 --2- 172.21.15.1:0/3493037784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bd12976ab0 0x55bd12978ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.426+0000 7ff2577fe700 1 -- 172.21.15.1:0/3493037784 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd12967ef0 msgr2=0x55bd12968300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:36.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.426+0000 7ff2577fe700 1 --2- 172.21.15.1:0/3493037784 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd12967ef0 0x55bd12968300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.426+0000 7ff2577fe700 1 -- 172.21.15.1:0/3493037784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd127d7880 con 0x55bd12968840 2024-05-04T03:18:36.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.426+0000 7ff2577fe700 1 --2- 172.21.15.1:0/3493037784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd12968840 0x55bd12976570 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff24c0099c0 tx=0x7ff24c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f041fee93b481d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:36.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.426+0000 7ff23e7fc700 1 -- 172.21.15.1:0/3493037784 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c005430 con 0x55bd12968840 2024-05-04T03:18:36.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.426+0000 7ff23e7fc700 1 -- 172.21.15.1:0/3493037784 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff24c0055d0 con 0x55bd12968840 2024-05-04T03:18:36.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.426+0000 7ff23e7fc700 1 -- 172.21.15.1:0/3493037784 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c00f9c0 con 0x55bd12968840 2024-05-04T03:18:36.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.427+0000 7ff26d37db80 1 -- 172.21.15.1:0/3493037784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd12968840 msgr2=0x55bd12976570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:36.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.427+0000 7ff26d37db80 1 --2- 172.21.15.1:0/3493037784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd12968840 0x55bd12976570 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff24c0099c0 tx=0x7ff24c01e000 comp rx=0 tx=0).stop 2024-05-04T03:18:36.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.427+0000 7ff26d37db80 1 -- 172.21.15.1:0/3493037784 shutdown_connections 2024-05-04T03:18:36.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.427+0000 7ff26d37db80 1 --2- 172.21.15.1:0/3493037784 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd12967ef0 0x55bd12968300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.427+0000 7ff26d37db80 1 --2- 172.21.15.1:0/3493037784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bd12976ab0 0x55bd12978ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.427+0000 7ff26d37db80 1 --2- 172.21.15.1:0/3493037784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd12968840 0x55bd12976570 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.427+0000 7ff26d37db80 1 -- 172.21.15.1:0/3493037784 >> 172.21.15.1:0/3493037784 conn(0x55bd127eadf0 msgr2=0x55bd12962bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:36.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.427+0000 7ff26d37db80 1 -- 172.21.15.1:0/3493037784 shutdown_connections 2024-05-04T03:18:36.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.427+0000 7ff26d37db80 1 -- 172.21.15.1:0/3493037784 wait complete. 2024-05-04T03:18:36.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.428+0000 7ff26d37db80 1 -- start start 2024-05-04T03:18:36.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.428+0000 7ff26d37db80 1 Processor -- start 2024-05-04T03:18:36.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.428+0000 7ff26d37db80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd12967ef0 0x55bd12973a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:36.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.428+0000 7ff26d37db80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bd12968840 0x55bd12973fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:36.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.428+0000 7ff26d37db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd12976ab0 0x55bd12974500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:36.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.428+0000 7ff26d37db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bd12979410 con 0x55bd12976ab0 2024-05-04T03:18:36.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.428+0000 7ff26d37db80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55bd127f0be0 con 0x55bd12967ef0 2024-05-04T03:18:36.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.428+0000 7ff26d37db80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bd12979590 con 0x55bd12968840 2024-05-04T03:18:36.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.428+0000 7ff26d37db80 10 client.0 _ll_register_callbacks cb 0x55bd128228d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:18:36.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.428+0000 7ff25cf06700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd12976ab0 0x55bd12974500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:36.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.428+0000 7ff25cf06700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd12976ab0 0x55bd12974500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43964/0 (socket says 172.21.15.1:43964) 2024-05-04T03:18:36.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.428+0000 7ff25cf06700 1 -- 172.21.15.1:0/4143586081 learned_addr learned my addr 172.21.15.1:0/4143586081 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:36.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.429+0000 7ff2577fe700 1 --2- 172.21.15.1:0/4143586081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bd12968840 0x55bd12973fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:36.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.429+0000 7ff257fff700 1 --2- 172.21.15.1:0/4143586081 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd12967ef0 0x55bd12973a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:36.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.429+0000 7ff25cf06700 1 -- 172.21.15.1:0/4143586081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bd12968840 msgr2=0x55bd12973fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:36.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.429+0000 7ff25cf06700 1 --2- 172.21.15.1:0/4143586081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bd12968840 0x55bd12973fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.429+0000 7ff25cf06700 1 -- 172.21.15.1:0/4143586081 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd12967ef0 msgr2=0x55bd12973a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:36.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.429+0000 7ff25cf06700 1 --2- 172.21.15.1:0/4143586081 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd12967ef0 0x55bd12973a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.429+0000 7ff25cf06700 1 -- 172.21.15.1:0/4143586081 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd12826640 con 0x55bd12976ab0 2024-05-04T03:18:36.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.429+0000 7ff25cf06700 1 --2- 172.21.15.1:0/4143586081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd12976ab0 0x55bd12974500 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff25000ce90 tx=0x7ff25000cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a7da8c8f5ee455 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:36.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.429+0000 7ff2377fe700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:18:36.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.429+0000 7ff2377fe700 1 -- 172.21.15.1:0/4143586081 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff250007920 con 0x55bd12976ab0 2024-05-04T03:18:36.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.429+0000 7ff2377fe700 1 -- 172.21.15.1:0/4143586081 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7ff250007ac0 con 0x55bd12976ab0 2024-05-04T03:18:36.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.429+0000 7ff26d37db80 1 -- 172.21.15.1:0/4143586081 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff24c0055d0 con 0x55bd12976ab0 2024-05-04T03:18:36.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.429+0000 7ff2377fe700 1 -- 172.21.15.1:0/4143586081 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff250007cc0 con 0x55bd12976ab0 2024-05-04T03:18:36.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.429+0000 7ff26d37db80 10 client.15063 fetch_fsmap learned FSMap version 25 2024-05-04T03:18:36.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.429+0000 7ff26d37db80 1 -- 172.21.15.1:0/4143586081 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bd12984090 con 0x55bd12976ab0 2024-05-04T03:18:36.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.430+0000 7ff2377fe700 1 -- 172.21.15.1:0/4143586081 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff24c0055d0 con 0x55bd12976ab0 2024-05-04T03:18:36.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.430+0000 7ff2377fe700 1 -- 172.21.15.1:0/4143586081 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff25001f400 con 0x55bd12976ab0 2024-05-04T03:18:36.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.430+0000 7ff26d37db80 10 client.15063 fetch_fsmap finished waiting for FSMap version 25 2024-05-04T03:18:36.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.430+0000 7ff26d37db80 10 client.15063 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-04T03:18:36.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.430+0000 7ff26d37db80 20 client.15063 populate_metadata read hostname 'smithi001' 2024-05-04T03:18:36.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.430+0000 7ff26d37db80 1 --2- 172.21.15.1:0/4143586081 >> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] conn(0x55bd12986710 0x55bd12987b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:36.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.430+0000 7ff26d37db80 4 client.15063 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:18:36.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.430+0000 7ff26d37db80 1 -- 172.21.15.1:0/4143586081 --> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd128981f0 con 0x55bd12986710 2024-05-04T03:18:36.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.430+0000 7ff257fff700 1 --2- 172.21.15.1:0/4143586081 >> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] conn(0x55bd12986710 0x55bd12987b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:36.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.431+0000 7ff2377fe700 10 client.15063 ms_handle_connect on v2:172.21.15.77:6838/922723189 2024-05-04T03:18:36.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.431+0000 7ff257fff700 1 --2- 172.21.15.1:0/4143586081 >> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] conn(0x55bd12986710 0x55bd12987b80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3cd59c09e498fa97 server_cookie=ed31543f4e629e4b in_seq=0 out_seq=0 2024-05-04T03:18:36.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.431+0000 7ff2377fe700 1 -- 172.21.15.1:0/4143586081 <== mds.1 v2:172.21.15.77:6838/922723189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bd128981f0 con 0x55bd12986710 2024-05-04T03:18:36.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.431+0000 7ff2377fe700 10 client.15063 handle_command_reply: tid=0 2024-05-04T03:18:36.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.432+0000 7ff26d37db80 1 client.15063 shutdown 2024-05-04T03:18:36.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.432+0000 7ff26d37db80 1 -- 172.21.15.1:0/4143586081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd12976ab0 msgr2=0x55bd12974500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:36.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.432+0000 7ff26d37db80 1 --2- 172.21.15.1:0/4143586081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd12976ab0 0x55bd12974500 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff25000ce90 tx=0x7ff25000cec0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.432+0000 7ff26d37db80 1 -- 172.21.15.1:0/4143586081 shutdown_connections 2024-05-04T03:18:36.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.432+0000 7ff26d37db80 1 --2- 172.21.15.1:0/4143586081 >> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] conn(0x55bd12986710 0x55bd12987b80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.432+0000 7ff2377fe700 0 client.15063 ms_handle_reset on v2:172.21.15.77:6838/922723189 2024-05-04T03:18:36.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.432+0000 7ff26d37db80 1 --2- 172.21.15.1:0/4143586081 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd12967ef0 0x55bd12973a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.432+0000 7ff26d37db80 1 --2- 172.21.15.1:0/4143586081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd12976ab0 0x55bd12974500 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.432+0000 7ff26d37db80 1 --2- 172.21.15.1:0/4143586081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bd12968840 0x55bd12973fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.432+0000 7ff26d37db80 1 -- 172.21.15.1:0/4143586081 >> 172.21.15.1:0/4143586081 conn(0x55bd127eadf0 msgr2=0x55bd12964ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:36.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.432+0000 7ff26d37db80 1 -- 172.21.15.1:0/4143586081 shutdown_connections 2024-05-04T03:18:36.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.432+0000 7ff26d37db80 1 -- 172.21.15.1:0/4143586081 wait complete. 2024-05-04T03:18:36.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.432+0000 7ff26d37db80 20 client.15063 trim_cache size 0 max 16384 2024-05-04T03:18:36.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.456+0000 7ff26d37db80 1 Processor -- start 2024-05-04T03:18:36.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.456+0000 7ff26d37db80 1 -- start start 2024-05-04T03:18:36.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.456+0000 7ff26d37db80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bd127eadf0 0x55bd12982830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:36.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.456+0000 7ff26d37db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd12967ef0 0x55bd12974340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:36.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.456+0000 7ff26d37db80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd12968840 0x55bd12970ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:36.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.456+0000 7ff26d37db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bd12985ec0 con 0x55bd12967ef0 2024-05-04T03:18:36.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.456+0000 7ff26d37db80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55bd12986590 con 0x55bd12968840 2024-05-04T03:18:36.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.456+0000 7ff26d37db80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bd12986c60 con 0x55bd127eadf0 2024-05-04T03:18:36.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.456+0000 7ff2577fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd12967ef0 0x55bd12974340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:36.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.456+0000 7ff2577fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd12967ef0 0x55bd12974340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43976/0 (socket says 172.21.15.1:43976) 2024-05-04T03:18:36.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.456+0000 7ff2577fe700 1 -- 172.21.15.1:0/999012623 learned_addr learned my addr 172.21.15.1:0/999012623 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:36.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.457+0000 7ff25cf06700 1 --2- 172.21.15.1:0/999012623 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd12968840 0x55bd12970ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:36.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.457+0000 7ff257fff700 1 --2- 172.21.15.1:0/999012623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bd127eadf0 0x55bd12982830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:36.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.457+0000 7ff2577fe700 1 -- 172.21.15.1:0/999012623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bd127eadf0 msgr2=0x55bd12982830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:36.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.457+0000 7ff2577fe700 1 --2- 172.21.15.1:0/999012623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bd127eadf0 0x55bd12982830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.457+0000 7ff2577fe700 1 -- 172.21.15.1:0/999012623 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd12968840 msgr2=0x55bd12970ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:36.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.457+0000 7ff2577fe700 1 --2- 172.21.15.1:0/999012623 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd12968840 0x55bd12970ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.457+0000 7ff2577fe700 1 -- 172.21.15.1:0/999012623 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd1295f470 con 0x55bd12967ef0 2024-05-04T03:18:36.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.457+0000 7ff25cf06700 1 --2- 172.21.15.1:0/999012623 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd12968840 0x55bd12970ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:18:36.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.457+0000 7ff2577fe700 1 --2- 172.21.15.1:0/999012623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd12967ef0 0x55bd12974340 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff24c0098e0 tx=0x7ff24c003fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65918383525dbd4b server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:36.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.457+0000 7ff23dffb700 1 -- 172.21.15.1:0/999012623 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c00f460 con 0x55bd12967ef0 2024-05-04T03:18:36.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.457+0000 7ff23dffb700 1 -- 172.21.15.1:0/999012623 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff24c00b730 con 0x55bd12967ef0 2024-05-04T03:18:36.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.457+0000 7ff23dffb700 1 -- 172.21.15.1:0/999012623 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c038450 con 0x55bd12967ef0 2024-05-04T03:18:36.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.458+0000 7ff26d37db80 1 -- 172.21.15.1:0/999012623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd12967ef0 msgr2=0x55bd12974340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:36.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.458+0000 7ff26d37db80 1 --2- 172.21.15.1:0/999012623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd12967ef0 0x55bd12974340 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff24c0098e0 tx=0x7ff24c003fa0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.458+0000 7ff26d37db80 1 -- 172.21.15.1:0/999012623 shutdown_connections 2024-05-04T03:18:36.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.458+0000 7ff26d37db80 1 --2- 172.21.15.1:0/999012623 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd12968840 0x55bd12970ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.458+0000 7ff26d37db80 1 --2- 172.21.15.1:0/999012623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bd127eadf0 0x55bd12982830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.458+0000 7ff26d37db80 1 --2- 172.21.15.1:0/999012623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd12967ef0 0x55bd12974340 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.458+0000 7ff26d37db80 1 -- 172.21.15.1:0/999012623 >> 172.21.15.1:0/999012623 conn(0x55bd12963a30 msgr2=0x55bd12985550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:36.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.458+0000 7ff26d37db80 1 -- 172.21.15.1:0/999012623 shutdown_connections 2024-05-04T03:18:36.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.458+0000 7ff26d37db80 1 -- 172.21.15.1:0/999012623 wait complete. 2024-05-04T03:18:36.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.459+0000 7ff26d37db80 1 -- start start 2024-05-04T03:18:36.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.459+0000 7ff26d37db80 1 Processor -- start 2024-05-04T03:18:36.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.459+0000 7ff26d37db80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bd127eadf0 0x55bd12970950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:36.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.459+0000 7ff26d37db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd12967ef0 0x55bd12970e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:36.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.459+0000 7ff26d37db80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd12968840 0x55bd129713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:36.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.459+0000 7ff26d37db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55bd12987a00 con 0x55bd12967ef0 2024-05-04T03:18:36.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.459+0000 7ff26d37db80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55bd12987330 con 0x55bd12968840 2024-05-04T03:18:36.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.459+0000 7ff26d37db80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bd129880d0 con 0x55bd127eadf0 2024-05-04T03:18:36.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.459+0000 7ff26d37db80 10 client.0 _ll_register_callbacks cb 0x55bd128228d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:18:36.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.459+0000 7ff2577fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd12967ef0 0x55bd12970e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:36.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.459+0000 7ff2577fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd12967ef0 0x55bd12970e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43982/0 (socket says 172.21.15.1:43982) 2024-05-04T03:18:36.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.459+0000 7ff2577fe700 1 -- 172.21.15.1:0/789435652 learned_addr learned my addr 172.21.15.1:0/789435652 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:36.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.459+0000 7ff257fff700 1 --2- 172.21.15.1:0/789435652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bd127eadf0 0x55bd12970950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:36.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.459+0000 7ff25cf06700 1 --2- 172.21.15.1:0/789435652 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd12968840 0x55bd129713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:36.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.460+0000 7ff2577fe700 1 -- 172.21.15.1:0/789435652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bd127eadf0 msgr2=0x55bd12970950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:36.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.460+0000 7ff2577fe700 1 --2- 172.21.15.1:0/789435652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bd127eadf0 0x55bd12970950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.460+0000 7ff2577fe700 1 -- 172.21.15.1:0/789435652 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd12968840 msgr2=0x55bd129713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:36.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.460+0000 7ff2577fe700 1 --2- 172.21.15.1:0/789435652 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd12968840 0x55bd129713d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.460+0000 7ff2577fe700 1 -- 172.21.15.1:0/789435652 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd1298f470 con 0x55bd12967ef0 2024-05-04T03:18:36.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.460+0000 7ff2577fe700 1 --2- 172.21.15.1:0/789435652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd12967ef0 0x55bd12970e90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009750 tx=0x7ff24c00f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90cb4779e08afc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:36.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.460+0000 7ff2377fe700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:18:36.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.460+0000 7ff2377fe700 1 -- 172.21.15.1:0/789435652 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24c00bde0 con 0x55bd12967ef0 2024-05-04T03:18:36.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.460+0000 7ff2377fe700 1 -- 172.21.15.1:0/789435652 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7ff24c043680 con 0x55bd12967ef0 2024-05-04T03:18:36.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.460+0000 7ff26d37db80 1 -- 172.21.15.1:0/789435652 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff24c00b730 con 0x55bd12967ef0 2024-05-04T03:18:36.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.460+0000 7ff2377fe700 1 -- 172.21.15.1:0/789435652 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff24c0438c0 con 0x55bd12967ef0 2024-05-04T03:18:36.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.460+0000 7ff26d37db80 10 client.15069 fetch_fsmap learned FSMap version 25 2024-05-04T03:18:36.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.460+0000 7ff26d37db80 1 -- 172.21.15.1:0/789435652 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bd129f1e10 con 0x55bd12967ef0 2024-05-04T03:18:36.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.460+0000 7ff2377fe700 1 -- 172.21.15.1:0/789435652 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff24c00b730 con 0x55bd12967ef0 2024-05-04T03:18:36.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.461+0000 7ff2377fe700 1 -- 172.21.15.1:0/789435652 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff24c022070 con 0x55bd12967ef0 2024-05-04T03:18:36.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.461+0000 7ff26d37db80 10 client.15069 fetch_fsmap finished waiting for FSMap version 25 2024-05-04T03:18:36.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.461+0000 7ff26d37db80 10 client.15069 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-04T03:18:36.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.461+0000 7ff26d37db80 20 client.15069 populate_metadata read hostname 'smithi001' 2024-05-04T03:18:36.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.461+0000 7ff26d37db80 1 --2- 172.21.15.1:0/789435652 >> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] conn(0x55bd12976ab0 0x55bd129f4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:36.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.461+0000 7ff26d37db80 4 client.15069 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-04T03:18:36.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.461+0000 7ff26d37db80 1 -- 172.21.15.1:0/789435652 --> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] -- command(tid 0: {"prefix": "status"}) -- 0x55bd12892290 con 0x55bd12976ab0 2024-05-04T03:18:36.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.461+0000 7ff257fff700 1 --2- 172.21.15.1:0/789435652 >> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] conn(0x55bd12976ab0 0x55bd129f4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:36.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.461+0000 7ff2377fe700 10 client.15069 ms_handle_connect on v2:172.21.15.77:6838/922723189 2024-05-04T03:18:36.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.461+0000 7ff257fff700 1 --2- 172.21.15.1:0/789435652 >> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] conn(0x55bd12976ab0 0x55bd129f4fc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=175742b7e65dcac8 server_cookie=43fb2682ab4f358d in_seq=0 out_seq=0 2024-05-04T03:18:36.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.462+0000 7ff2377fe700 1 -- 172.21.15.1:0/789435652 <== mds.1 v2:172.21.15.77:6838/922723189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55bd12892290 con 0x55bd12976ab0 2024-05-04T03:18:36.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.462+0000 7ff2377fe700 10 client.15069 handle_command_reply: tid=0 2024-05-04T03:18:36.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.462+0000 7ff26d37db80 1 client.15069 shutdown 2024-05-04T03:18:36.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.462+0000 7ff26d37db80 1 -- 172.21.15.1:0/789435652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd12967ef0 msgr2=0x55bd12970e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:36.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.462+0000 7ff26d37db80 1 --2- 172.21.15.1:0/789435652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd12967ef0 0x55bd12970e90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009750 tx=0x7ff24c00f710 comp rx=0 tx=0).stop 2024-05-04T03:18:36.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.462+0000 7ff26d37db80 1 -- 172.21.15.1:0/789435652 shutdown_connections 2024-05-04T03:18:36.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.462+0000 7ff26d37db80 1 --2- 172.21.15.1:0/789435652 >> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] conn(0x55bd12976ab0 0x55bd129f4fc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.462+0000 7ff26d37db80 1 --2- 172.21.15.1:0/789435652 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55bd12968840 0x55bd129713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.462+0000 7ff26d37db80 1 --2- 172.21.15.1:0/789435652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bd127eadf0 0x55bd12970950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.462+0000 7ff2377fe700 0 client.15069 ms_handle_reset on v2:172.21.15.77:6838/922723189 2024-05-04T03:18:36.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.462+0000 7ff26d37db80 1 --2- 172.21.15.1:0/789435652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd12967ef0 0x55bd12970e90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.462+0000 7ff26d37db80 1 -- 172.21.15.1:0/789435652 >> 172.21.15.1:0/789435652 conn(0x55bd12963a30 msgr2=0x55bd1295ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:36.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.462+0000 7ff26d37db80 1 -- 172.21.15.1:0/789435652 shutdown_connections 2024-05-04T03:18:36.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.462+0000 7ff26d37db80 1 -- 172.21.15.1:0/789435652 wait complete. 2024-05-04T03:18:36.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.462+0000 7ff26d37db80 20 client.15069 trim_cache size 0 max 16384 2024-05-04T03:18:36.469 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-04T03:18:36.469 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "4a572aa8-09c3-11ef-bc97-c7b262605968", 2024-05-04T03:18:36.469 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 1, 2024-05-04T03:18:36.469 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24472, 2024-05-04T03:18:36.469 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-05-04T03:18:36.470 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-05-04T03:18:36.470 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-05-04T03:18:36.470 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 154.240360788, 2024-05-04T03:18:36.470 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 25, 2024-05-04T03:18:36.470 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 83, 2024-05-04T03:18:36.470 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 81, 2024-05-04T03:18:36.470 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 192.20316141199999 2024-05-04T03:18:36.470 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-04T03:18:36.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.463+0000 7ff25e969700 1 -- 172.21.15.1:0/705672847 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ff2400b2880 msgr2=0x7ff2400b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:36.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.463+0000 7ff25e969700 1 --2- 172.21.15.1:0/705672847 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ff2400b2880 0x7ff2400b4d90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff24c006010 tx=0x7ff24c000bc0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.463+0000 7ff25e969700 1 -- 172.21.15.1:0/705672847 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2581034c0 msgr2=0x7ff258199d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:36.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.463+0000 7ff25e969700 1 --2- 172.21.15.1:0/705672847 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2581034c0 0x7ff258199d70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff24800d950 tx=0x7ff24800dc60 comp rx=0 tx=0).stop 2024-05-04T03:18:36.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.463+0000 7ff25e969700 1 -- 172.21.15.1:0/705672847 shutdown_connections 2024-05-04T03:18:36.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.463+0000 7ff25e969700 1 --2- 172.21.15.1:0/705672847 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff258108740 0x7ff25819e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.463+0000 7ff25e969700 1 --2- 172.21.15.1:0/705672847 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ff2400b2880 0x7ff2400b4d90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.463+0000 7ff25e969700 1 --2- 172.21.15.1:0/705672847 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2581034c0 0x7ff258199d70 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.463+0000 7ff25e969700 1 --2- 172.21.15.1:0/705672847 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff258107df0 0x7ff25819a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:36.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.463+0000 7ff25e969700 1 -- 172.21.15.1:0/705672847 >> 172.21.15.1:0/705672847 conn(0x7ff2580fedc0 msgr2=0x7ff2581096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:36.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.463+0000 7ff25e969700 1 -- 172.21.15.1:0/705672847 shutdown_connections 2024-05-04T03:18:36.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:36.463+0000 7ff25e969700 1 -- 172.21.15.1:0/705672847 wait complete. 2024-05-04T03:18:36.717 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:36 smithi001 ceph-mon[29731]: pgmap v374: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-04T03:18:36.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:36 smithi072 ceph-mon[33332]: pgmap v374: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-04T03:18:36.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:36 smithi077 ceph-mon[33531]: pgmap v374: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-05-04T03:18:37.087 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-04T03:18:37.335 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:18:38.731 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:38 smithi001 ceph-mon[29731]: pgmap v375: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-04T03:18:38.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:38 smithi072 ceph-mon[33332]: pgmap v375: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-04T03:18:38.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:38 smithi077 ceph-mon[33531]: pgmap v375: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-04T03:18:39.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.152+0000 7fd68f324700 1 -- 172.21.15.1:0/2144651836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd688104610 msgr2=0x7fd688104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:39.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.152+0000 7fd68f324700 1 --2- 172.21.15.1:0/2144651836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd688104610 0x7fd688104a20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd68000b0d0 tx=0x7fd68000b3e0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.152+0000 7fd68f324700 1 -- 172.21.15.1:0/2144651836 shutdown_connections 2024-05-04T03:18:39.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.152+0000 7fd68f324700 1 --2- 172.21.15.1:0/2144651836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd688109890 0x7fd6881111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.152+0000 7fd68f324700 1 --2- 172.21.15.1:0/2144651836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd688104610 0x7fd688104a20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.152+0000 7fd68f324700 1 --2- 172.21.15.1:0/2144651836 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd688108f40 0x7fd688109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.152+0000 7fd68f324700 1 -- 172.21.15.1:0/2144651836 >> 172.21.15.1:0/2144651836 conn(0x7fd68807af70 msgr2=0x7fd68807b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:39.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.153+0000 7fd68f324700 1 -- 172.21.15.1:0/2144651836 shutdown_connections 2024-05-04T03:18:39.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.153+0000 7fd68f324700 1 -- 172.21.15.1:0/2144651836 wait complete. 2024-05-04T03:18:39.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.153+0000 7fd68f324700 1 Processor -- start 2024-05-04T03:18:39.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.153+0000 7fd68f324700 1 -- start start 2024-05-04T03:18:39.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.154+0000 7fd68f324700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd688104610 0x7fd688075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:39.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.154+0000 7fd68f324700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd688108f40 0x7fd688076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:39.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.154+0000 7fd68f324700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd688109890 0x7fd688076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:39.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.154+0000 7fd68f324700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881144c0 con 0x7fd688108f40 2024-05-04T03:18:39.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.154+0000 7fd68f324700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd688114340 con 0x7fd688104610 2024-05-04T03:18:39.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.154+0000 7fd68f324700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd688114640 con 0x7fd688109890 2024-05-04T03:18:39.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.154+0000 7fd68c8bf700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd688108f40 0x7fd688076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:39.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.154+0000 7fd68c8bf700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd688108f40 0x7fd688076000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44008/0 (socket says 172.21.15.1:44008) 2024-05-04T03:18:39.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.154+0000 7fd68c8bf700 1 -- 172.21.15.1:0/483808776 learned_addr learned my addr 172.21.15.1:0/483808776 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:39.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.154+0000 7fd68d0c0700 1 --2- 172.21.15.1:0/483808776 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd688104610 0x7fd688075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:39.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.154+0000 7fd68d8c1700 1 --2- 172.21.15.1:0/483808776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd688109890 0x7fd688076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:39.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.154+0000 7fd68c8bf700 1 -- 172.21.15.1:0/483808776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd688109890 msgr2=0x7fd688076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:39.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.154+0000 7fd68c8bf700 1 --2- 172.21.15.1:0/483808776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd688109890 0x7fd688076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.154+0000 7fd68c8bf700 1 -- 172.21.15.1:0/483808776 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd688104610 msgr2=0x7fd688075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:39.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.154+0000 7fd68c8bf700 1 --2- 172.21.15.1:0/483808776 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd688104610 0x7fd688075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.154+0000 7fd68c8bf700 1 -- 172.21.15.1:0/483808776 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6881a9180 con 0x7fd688108f40 2024-05-04T03:18:39.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.154+0000 7fd68c8bf700 1 --2- 172.21.15.1:0/483808776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd688108f40 0x7fd688076000 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00d980 tx=0x7fd67c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:39.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.154+0000 7fd6867fc700 1 -- 172.21.15.1:0/483808776 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c017070 con 0x7fd688108f40 2024-05-04T03:18:39.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.154+0000 7fd68f324700 1 -- 172.21.15.1:0/483808776 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6881a9470 con 0x7fd688108f40 2024-05-04T03:18:39.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.154+0000 7fd6867fc700 1 -- 172.21.15.1:0/483808776 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd67c009de0 con 0x7fd688108f40 2024-05-04T03:18:39.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.155+0000 7fd6867fc700 1 -- 172.21.15.1:0/483808776 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c00f3c0 con 0x7fd688108f40 2024-05-04T03:18:39.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.155+0000 7fd68f324700 1 -- 172.21.15.1:0/483808776 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6881a99b0 con 0x7fd688108f40 2024-05-04T03:18:39.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.157+0000 7fd6867fc700 1 -- 172.21.15.1:0/483808776 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fd67c00f560 con 0x7fd688108f40 2024-05-04T03:18:39.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.158+0000 7fd6867fc700 1 --2- 172.21.15.1:0/483808776 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd6700b2880 0x7fd6700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:39.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.158+0000 7fd68d0c0700 1 --2- 172.21.15.1:0/483808776 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd6700b2880 0x7fd6700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:39.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.158+0000 7fd6867fc700 1 -- 172.21.15.1:0/483808776 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fd67c0e2000 con 0x7fd688108f40 2024-05-04T03:18:39.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.158+0000 7fd69dd38b80 1 Processor -- start 2024-05-04T03:18:39.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.158+0000 7fd69dd38b80 1 -- start start 2024-05-04T03:18:39.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.159+0000 7fd69dd38b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef19dc30d0 0x55ef19dc34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:39.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.159+0000 7fd69dd38b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef19dbd1c0 0x55ef19dbd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:39.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.159+0000 7fd69dd38b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19dbdb10 0x55ef19dc9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:39.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.159+0000 7fd69dd38b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ef19cdf8f0 con 0x55ef19dbdb10 2024-05-04T03:18:39.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.159+0000 7fd68d0c0700 1 --2- 172.21.15.1:0/483808776 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd6700b2880 0x7fd6700b4d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd680009900 tx=0x7fd68000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:39.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.159+0000 7fd69dd38b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ef19c44d50 con 0x55ef19dc30d0 2024-05-04T03:18:39.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.159+0000 7fd69dd38b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ef19cdb8b0 con 0x55ef19dbd1c0 2024-05-04T03:18:39.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.159+0000 7fd68d8c1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19dbdb10 0x55ef19dc9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:39.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.159+0000 7fd68d8c1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19dbdb10 0x55ef19dc9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44014/0 (socket says 172.21.15.1:44014) 2024-05-04T03:18:39.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.159+0000 7fd68d8c1700 1 -- 172.21.15.1:0/229645222 learned_addr learned my addr 172.21.15.1:0/229645222 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:39.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.159+0000 7fd68d0c0700 1 --2- 172.21.15.1:0/229645222 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef19dc30d0 0x55ef19dc34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:39.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.159+0000 7fd68c8bf700 1 --2- 172.21.15.1:0/229645222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef19dbd1c0 0x55ef19dbd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:39.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.159+0000 7fd68d8c1700 1 -- 172.21.15.1:0/229645222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef19dbd1c0 msgr2=0x55ef19dbd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:39.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.159+0000 7fd68d8c1700 1 --2- 172.21.15.1:0/229645222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef19dbd1c0 0x55ef19dbd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.159+0000 7fd68d8c1700 1 -- 172.21.15.1:0/229645222 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef19dc30d0 msgr2=0x55ef19dc34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:39.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.159+0000 7fd68d8c1700 1 --2- 172.21.15.1:0/229645222 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef19dc30d0 0x55ef19dc34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.159+0000 7fd68d8c1700 1 -- 172.21.15.1:0/229645222 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef19c2c880 con 0x55ef19dbdb10 2024-05-04T03:18:39.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.159+0000 7fd68d0c0700 1 --2- 172.21.15.1:0/229645222 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef19dc30d0 0x55ef19dc34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:18:39.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.159+0000 7fd68c8bf700 1 --2- 172.21.15.1:0/229645222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef19dbd1c0 0x55ef19dbd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:18:39.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.159+0000 7fd68d8c1700 1 --2- 172.21.15.1:0/229645222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19dbdb10 0x55ef19dc9db0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd67800cc00 tx=0x7fd67800cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6872c6696cb70a61 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:39.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.160+0000 7fd66f7fe700 1 -- 172.21.15.1:0/229645222 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67800eda0 con 0x55ef19dbdb10 2024-05-04T03:18:39.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.160+0000 7fd66f7fe700 1 -- 172.21.15.1:0/229645222 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd678004ce0 con 0x55ef19dbdb10 2024-05-04T03:18:39.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.160+0000 7fd66f7fe700 1 -- 172.21.15.1:0/229645222 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678010690 con 0x55ef19dbdb10 2024-05-04T03:18:39.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.161+0000 7fd69dd38b80 1 -- 172.21.15.1:0/229645222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19dbdb10 msgr2=0x55ef19dc9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:39.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.161+0000 7fd69dd38b80 1 --2- 172.21.15.1:0/229645222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19dbdb10 0x55ef19dc9db0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd67800cc00 tx=0x7fd67800cf10 comp rx=0 tx=0).stop 2024-05-04T03:18:39.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.161+0000 7fd69dd38b80 1 -- 172.21.15.1:0/229645222 shutdown_connections 2024-05-04T03:18:39.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.161+0000 7fd69dd38b80 1 --2- 172.21.15.1:0/229645222 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef19dc30d0 0x55ef19dc34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.161+0000 7fd69dd38b80 1 --2- 172.21.15.1:0/229645222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19dbdb10 0x55ef19dc9db0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.161+0000 7fd69dd38b80 1 --2- 172.21.15.1:0/229645222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef19dbd1c0 0x55ef19dbd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.161+0000 7fd69dd38b80 1 -- 172.21.15.1:0/229645222 >> 172.21.15.1:0/229645222 conn(0x55ef19c3fdf0 msgr2=0x55ef19db8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:39.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.161+0000 7fd69dd38b80 1 -- 172.21.15.1:0/229645222 shutdown_connections 2024-05-04T03:18:39.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.161+0000 7fd69dd38b80 1 -- 172.21.15.1:0/229645222 wait complete. 2024-05-04T03:18:39.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.161+0000 7fd69dd38b80 1 -- start start 2024-05-04T03:18:39.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.161+0000 7fd69dd38b80 1 Processor -- start 2024-05-04T03:18:39.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.162+0000 7fd69dd38b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef19dbd1c0 0x55ef19dc7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:39.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.162+0000 7fd69dd38b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19dbdb10 0x55ef19dc7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:39.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.162+0000 7fd69dd38b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef19dc30d0 0x55ef19dc7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:39.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.162+0000 7fd69dd38b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ef19dca4a0 con 0x55ef19dbdb10 2024-05-04T03:18:39.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.162+0000 7fd69dd38b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ef19c98e80 con 0x55ef19dc30d0 2024-05-04T03:18:39.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.162+0000 7fd69dd38b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ef19dca620 con 0x55ef19dbd1c0 2024-05-04T03:18:39.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.162+0000 7fd69dd38b80 10 client.0 _ll_register_callbacks cb 0x55ef19c77900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:18:39.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.162+0000 7fd68c8bf700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19dbdb10 0x55ef19dc7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:39.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.162+0000 7fd68c8bf700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19dbdb10 0x55ef19dc7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44030/0 (socket says 172.21.15.1:44030) 2024-05-04T03:18:39.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.162+0000 7fd68c8bf700 1 -- 172.21.15.1:0/3062801312 learned_addr learned my addr 172.21.15.1:0/3062801312 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:39.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.162+0000 7fd68d8c1700 1 --2- 172.21.15.1:0/3062801312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef19dc30d0 0x55ef19dc7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:39.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.162+0000 7fd68d0c0700 1 --2- 172.21.15.1:0/3062801312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef19dbd1c0 0x55ef19dc7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:39.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.162+0000 7fd68c8bf700 1 -- 172.21.15.1:0/3062801312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef19dbd1c0 msgr2=0x55ef19dc7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:39.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.162+0000 7fd68c8bf700 1 --2- 172.21.15.1:0/3062801312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef19dbd1c0 0x55ef19dc7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.162+0000 7fd68c8bf700 1 -- 172.21.15.1:0/3062801312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef19dc30d0 msgr2=0x55ef19dc7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:39.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.162+0000 7fd68c8bf700 1 --2- 172.21.15.1:0/3062801312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef19dc30d0 0x55ef19dc7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.162+0000 7fd68c8bf700 1 -- 172.21.15.1:0/3062801312 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef19c96710 con 0x55ef19dbdb10 2024-05-04T03:18:39.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.162+0000 7fd68c8bf700 1 --2- 172.21.15.1:0/3062801312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19dbdb10 0x55ef19dc7760 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0a5760 tx=0x7fd67c0a9de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10246ba24d9b96af server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:39.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.162+0000 7fd66cff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:18:39.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.162+0000 7fd66cff9700 1 -- 172.21.15.1:0/3062801312 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c020040 con 0x55ef19dbdb10 2024-05-04T03:18:39.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.163+0000 7fd66cff9700 1 -- 172.21.15.1:0/3062801312 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fd67c0a5cd0 con 0x55ef19dbdb10 2024-05-04T03:18:39.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.163+0000 7fd69dd38b80 1 -- 172.21.15.1:0/3062801312 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd678004ce0 con 0x55ef19dbdb10 2024-05-04T03:18:39.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.163+0000 7fd66cff9700 1 -- 172.21.15.1:0/3062801312 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd67c0ada90 con 0x55ef19dbdb10 2024-05-04T03:18:39.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.163+0000 7fd69dd38b80 10 client.15081 fetch_fsmap learned FSMap version 25 2024-05-04T03:18:39.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.163+0000 7fd69dd38b80 1 -- 172.21.15.1:0/3062801312 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ef19dd5030 con 0x55ef19dbdb10 2024-05-04T03:18:39.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.163+0000 7fd66cff9700 1 -- 172.21.15.1:0/3062801312 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd678004ce0 con 0x55ef19dbdb10 2024-05-04T03:18:39.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.163+0000 7fd66cff9700 1 -- 172.21.15.1:0/3062801312 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fd67c0b20b0 con 0x55ef19dbdb10 2024-05-04T03:18:39.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.163+0000 7fd69dd38b80 10 client.15081 fetch_fsmap finished waiting for FSMap version 25 2024-05-04T03:18:39.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.163+0000 7fd69dd38b80 10 client.15081 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-04T03:18:39.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.163+0000 7fd69dd38b80 20 client.15081 populate_metadata read hostname 'smithi001' 2024-05-04T03:18:39.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.163+0000 7fd69dd38b80 1 --2- 172.21.15.1:0/3062801312 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55ef19dd75c0 0x55ef19dd8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:39.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.163+0000 7fd69dd38b80 4 client.15081 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:18:39.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.163+0000 7fd69dd38b80 1 -- 172.21.15.1:0/3062801312 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef19ced2d0 con 0x55ef19dd75c0 2024-05-04T03:18:39.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.164+0000 7fd68d0c0700 1 --2- 172.21.15.1:0/3062801312 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55ef19dd75c0 0x55ef19dd8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.164+0000 7fd68d0c0700 1 --2- 172.21.15.1:0/3062801312 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55ef19dd75c0 0x55ef19dd8a30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=922130b43986d31d server_cookie=cb36140469e377bc in_seq=0 out_seq=0 2024-05-04T03:18:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.164+0000 7fd66cff9700 10 client.15081 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:18:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.165+0000 7fd66cff9700 1 -- 172.21.15.1:0/3062801312 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ef19ced2d0 con 0x55ef19dd75c0 2024-05-04T03:18:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.165+0000 7fd66cff9700 10 client.15081 handle_command_reply: tid=0 2024-05-04T03:18:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.165+0000 7fd69dd38b80 1 client.15081 shutdown 2024-05-04T03:18:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.165+0000 7fd69dd38b80 1 -- 172.21.15.1:0/3062801312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19dbdb10 msgr2=0x55ef19dc7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.165+0000 7fd69dd38b80 1 --2- 172.21.15.1:0/3062801312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19dbdb10 0x55ef19dc7760 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0a5760 tx=0x7fd67c0a9de0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.165+0000 7fd69dd38b80 1 -- 172.21.15.1:0/3062801312 shutdown_connections 2024-05-04T03:18:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.165+0000 7fd69dd38b80 1 --2- 172.21.15.1:0/3062801312 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55ef19dd75c0 0x55ef19dd8a30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.165+0000 7fd69dd38b80 1 --2- 172.21.15.1:0/3062801312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef19dc30d0 0x55ef19dc7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.165+0000 7fd69dd38b80 1 --2- 172.21.15.1:0/3062801312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef19dbd1c0 0x55ef19dc7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.165+0000 7fd69dd38b80 1 --2- 172.21.15.1:0/3062801312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19dbdb10 0x55ef19dc7760 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.165+0000 7fd69dd38b80 1 -- 172.21.15.1:0/3062801312 >> 172.21.15.1:0/3062801312 conn(0x55ef19c3fdf0 msgr2=0x55ef19dbc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.165+0000 7fd66cff9700 0 client.15081 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:18:39.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.165+0000 7fd69dd38b80 1 -- 172.21.15.1:0/3062801312 shutdown_connections 2024-05-04T03:18:39.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.165+0000 7fd69dd38b80 1 -- 172.21.15.1:0/3062801312 wait complete. 2024-05-04T03:18:39.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.165+0000 7fd69dd38b80 20 client.15081 trim_cache size 0 max 16384 2024-05-04T03:18:39.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.187+0000 7fd69dd38b80 1 Processor -- start 2024-05-04T03:18:39.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.187+0000 7fd69dd38b80 1 -- start start 2024-05-04T03:18:39.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.187+0000 7fd69dd38b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef19dbd1c0 0x55ef19dbcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:39.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.187+0000 7fd69dd38b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef19dbdb10 0x55ef19dc19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:39.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.187+0000 7fd69dd38b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19dc30d0 0x55ef19db25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:39.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.187+0000 7fd69dd38b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ef19dd6d70 con 0x55ef19dc30d0 2024-05-04T03:18:39.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.187+0000 7fd69dd38b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ef19dd7440 con 0x55ef19dbdb10 2024-05-04T03:18:39.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.187+0000 7fd69dd38b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ef19dd7b10 con 0x55ef19dbd1c0 2024-05-04T03:18:39.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.187+0000 7fd68d8c1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19dc30d0 0x55ef19db25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:39.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.187+0000 7fd68d8c1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19dc30d0 0x55ef19db25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44040/0 (socket says 172.21.15.1:44040) 2024-05-04T03:18:39.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.187+0000 7fd68d8c1700 1 -- 172.21.15.1:0/3520796339 learned_addr learned my addr 172.21.15.1:0/3520796339 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:39.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.187+0000 7fd68d0c0700 1 --2- 172.21.15.1:0/3520796339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef19dbd1c0 0x55ef19dbcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:39.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.187+0000 7fd68c8bf700 1 --2- 172.21.15.1:0/3520796339 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef19dbdb10 0x55ef19dc19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:39.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.187+0000 7fd68d8c1700 1 -- 172.21.15.1:0/3520796339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef19dbd1c0 msgr2=0x55ef19dbcbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:39.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.187+0000 7fd68d8c1700 1 --2- 172.21.15.1:0/3520796339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef19dbd1c0 0x55ef19dbcbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.187+0000 7fd68d8c1700 1 -- 172.21.15.1:0/3520796339 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef19dbdb10 msgr2=0x55ef19dc19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:39.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.187+0000 7fd68d8c1700 1 --2- 172.21.15.1:0/3520796339 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef19dbdb10 0x55ef19dc19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.187+0000 7fd68d8c1700 1 -- 172.21.15.1:0/3520796339 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef19dc82e0 con 0x55ef19dc30d0 2024-05-04T03:18:39.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.187+0000 7fd68d0c0700 1 --2- 172.21.15.1:0/3520796339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef19dbd1c0 0x55ef19dbcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:18:39.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.187+0000 7fd68d8c1700 1 --2- 172.21.15.1:0/3520796339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19dc30d0 0x55ef19db25f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd678005790 tx=0x7fd6780054f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46dbad27e51cd4d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:39.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.188+0000 7fd66effd700 1 -- 172.21.15.1:0/3520796339 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6780052f0 con 0x55ef19dc30d0 2024-05-04T03:18:39.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.188+0000 7fd66effd700 1 -- 172.21.15.1:0/3520796339 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd67800eba0 con 0x55ef19dc30d0 2024-05-04T03:18:39.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.188+0000 7fd66effd700 1 -- 172.21.15.1:0/3520796339 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67801a6c0 con 0x55ef19dc30d0 2024-05-04T03:18:39.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.188+0000 7fd69dd38b80 1 -- 172.21.15.1:0/3520796339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19dc30d0 msgr2=0x55ef19db25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:39.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.188+0000 7fd69dd38b80 1 --2- 172.21.15.1:0/3520796339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19dc30d0 0x55ef19db25f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd678005790 tx=0x7fd6780054f0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.189+0000 7fd69dd38b80 1 -- 172.21.15.1:0/3520796339 shutdown_connections 2024-05-04T03:18:39.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.189+0000 7fd69dd38b80 1 --2- 172.21.15.1:0/3520796339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19dc30d0 0x55ef19db25f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.189+0000 7fd69dd38b80 1 --2- 172.21.15.1:0/3520796339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef19dbd1c0 0x55ef19dbcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.189+0000 7fd69dd38b80 1 --2- 172.21.15.1:0/3520796339 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef19dbdb10 0x55ef19dc19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.189+0000 7fd69dd38b80 1 -- 172.21.15.1:0/3520796339 >> 172.21.15.1:0/3520796339 conn(0x55ef19c3fdf0 msgr2=0x55ef19db81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:39.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.189+0000 7fd69dd38b80 1 -- 172.21.15.1:0/3520796339 shutdown_connections 2024-05-04T03:18:39.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.189+0000 7fd69dd38b80 1 -- 172.21.15.1:0/3520796339 wait complete. 2024-05-04T03:18:39.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.189+0000 7fd69dd38b80 1 -- start start 2024-05-04T03:18:39.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.189+0000 7fd69dd38b80 1 Processor -- start 2024-05-04T03:18:39.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.189+0000 7fd69dd38b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef19dbd1c0 0x55ef19de2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:39.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.190+0000 7fd69dd38b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19dbdb10 0x55ef19ddce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.190+0000 7fd69dd38b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef19dc30d0 0x55ef19ddd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.190+0000 7fd69dd38b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ef19dd88b0 con 0x55ef19dbdb10 2024-05-04T03:18:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.190+0000 7fd69dd38b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ef19dd81e0 con 0x55ef19dbd1c0 2024-05-04T03:18:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.190+0000 7fd69dd38b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ef19dd8f80 con 0x55ef19dc30d0 2024-05-04T03:18:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.190+0000 7fd69dd38b80 10 client.0 _ll_register_callbacks cb 0x55ef19c77900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:18:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.190+0000 7fd68c8bf700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19dbdb10 0x55ef19ddce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.190+0000 7fd68c8bf700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19dbdb10 0x55ef19ddce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44042/0 (socket says 172.21.15.1:44042) 2024-05-04T03:18:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.190+0000 7fd68c8bf700 1 -- 172.21.15.1:0/3052878900 learned_addr learned my addr 172.21.15.1:0/3052878900 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.190+0000 7fd68d0c0700 1 --2- 172.21.15.1:0/3052878900 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef19dbd1c0 0x55ef19de2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.190+0000 7fd68d8c1700 1 --2- 172.21.15.1:0/3052878900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef19dc30d0 0x55ef19ddd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.190+0000 7fd68c8bf700 1 -- 172.21.15.1:0/3052878900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef19dc30d0 msgr2=0x55ef19ddd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.190+0000 7fd68c8bf700 1 --2- 172.21.15.1:0/3052878900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef19dc30d0 0x55ef19ddd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.190+0000 7fd68c8bf700 1 -- 172.21.15.1:0/3052878900 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef19dbd1c0 msgr2=0x55ef19de2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:39.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.190+0000 7fd68c8bf700 1 --2- 172.21.15.1:0/3052878900 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef19dbd1c0 0x55ef19de2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.190+0000 7fd68c8bf700 1 -- 172.21.15.1:0/3052878900 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef19ddd8e0 con 0x55ef19dbdb10 2024-05-04T03:18:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.190+0000 7fd68d8c1700 1 --2- 172.21.15.1:0/3052878900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef19dc30d0 0x55ef19ddd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:18:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.190+0000 7fd68c8bf700 1 --2- 172.21.15.1:0/3052878900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19dbdb10 0x55ef19ddce60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009520 tx=0x7fd67c005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd77ba2888e25f56 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.190+0000 7fd66cff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:18:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.190+0000 7fd66cff9700 1 -- 172.21.15.1:0/3052878900 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c020050 con 0x55ef19dbdb10 2024-05-04T03:18:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.191+0000 7fd66cff9700 1 -- 172.21.15.1:0/3052878900 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fd67c0a5cd0 con 0x55ef19dbdb10 2024-05-04T03:18:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.191+0000 7fd69dd38b80 1 -- 172.21.15.1:0/3052878900 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd67800eba0 con 0x55ef19dbdb10 2024-05-04T03:18:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.191+0000 7fd66cff9700 1 -- 172.21.15.1:0/3052878900 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd67c0a94b0 con 0x55ef19dbdb10 2024-05-04T03:18:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.191+0000 7fd69dd38b80 10 client.15087 fetch_fsmap learned FSMap version 25 2024-05-04T03:18:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.191+0000 7fd69dd38b80 1 -- 172.21.15.1:0/3052878900 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ef19e435e0 con 0x55ef19dbdb10 2024-05-04T03:18:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.191+0000 7fd66cff9700 1 -- 172.21.15.1:0/3052878900 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd67800eba0 con 0x55ef19dbdb10 2024-05-04T03:18:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.191+0000 7fd66cff9700 1 -- 172.21.15.1:0/3052878900 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fd67c0bec50 con 0x55ef19dbdb10 2024-05-04T03:18:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.191+0000 7fd69dd38b80 10 client.15087 fetch_fsmap finished waiting for FSMap version 25 2024-05-04T03:18:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.191+0000 7fd69dd38b80 10 client.15087 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-04T03:18:39.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.191+0000 7fd69dd38b80 20 client.15087 populate_metadata read hostname 'smithi001' 2024-05-04T03:18:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.191+0000 7fd69dd38b80 1 --2- 172.21.15.1:0/3052878900 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55ef19e457f0 0x55ef19e46c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.191+0000 7fd69dd38b80 4 client.15087 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-04T03:18:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.191+0000 7fd69dd38b80 1 -- 172.21.15.1:0/3052878900 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "status"}) -- 0x55ef19ce7370 con 0x55ef19e457f0 2024-05-04T03:18:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.192+0000 7fd68d0c0700 1 --2- 172.21.15.1:0/3052878900 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55ef19e457f0 0x55ef19e46c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.192+0000 7fd66cff9700 10 client.15087 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:18:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.192+0000 7fd68d0c0700 1 --2- 172.21.15.1:0/3052878900 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55ef19e457f0 0x55ef19e46c60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1b9d5b53816ccef server_cookie=12a406ced2e6399c in_seq=0 out_seq=0 2024-05-04T03:18:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.192+0000 7fd66cff9700 1 -- 172.21.15.1:0/3052878900 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ef19ce7370 con 0x55ef19e457f0 2024-05-04T03:18:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.192+0000 7fd66cff9700 10 client.15087 handle_command_reply: tid=0 2024-05-04T03:18:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.192+0000 7fd69dd38b80 1 client.15087 shutdown 2024-05-04T03:18:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.192+0000 7fd69dd38b80 1 -- 172.21.15.1:0/3052878900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19dbdb10 msgr2=0x55ef19ddce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.192+0000 7fd69dd38b80 1 --2- 172.21.15.1:0/3052878900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19dbdb10 0x55ef19ddce60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009520 tx=0x7fd67c005c40 comp rx=0 tx=0).stop 2024-05-04T03:18:39.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.192+0000 7fd69dd38b80 1 -- 172.21.15.1:0/3052878900 shutdown_connections 2024-05-04T03:18:39.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.192+0000 7fd69dd38b80 1 --2- 172.21.15.1:0/3052878900 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55ef19e457f0 0x55ef19e46c60 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.192+0000 7fd69dd38b80 1 --2- 172.21.15.1:0/3052878900 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ef19dbd1c0 0x55ef19de2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.192+0000 7fd69dd38b80 1 --2- 172.21.15.1:0/3052878900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef19dc30d0 0x55ef19ddd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.192+0000 7fd69dd38b80 1 --2- 172.21.15.1:0/3052878900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ef19dbdb10 0x55ef19ddce60 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.192+0000 7fd69dd38b80 1 -- 172.21.15.1:0/3052878900 >> 172.21.15.1:0/3052878900 conn(0x55ef19c3fdf0 msgr2=0x55ef19db81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:39.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.192+0000 7fd66cff9700 0 client.15087 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:18:39.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.193+0000 7fd69dd38b80 1 -- 172.21.15.1:0/3052878900 shutdown_connections 2024-05-04T03:18:39.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.193+0000 7fd69dd38b80 1 -- 172.21.15.1:0/3052878900 wait complete. 2024-05-04T03:18:39.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.193+0000 7fd69dd38b80 20 client.15087 trim_cache size 0 max 16384 2024-05-04T03:18:39.199 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-04T03:18:39.199 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "4a572aa8-09c3-11ef-bc97-c7b262605968", 2024-05-04T03:18:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-05-04T03:18:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24494, 2024-05-04T03:18:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-05-04T03:18:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-05-04T03:18:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-05-04T03:18:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 169.15674865899999, 2024-05-04T03:18:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 25, 2024-05-04T03:18:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 83, 2024-05-04T03:18:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 81, 2024-05-04T03:18:39.200 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 184.821585973 2024-05-04T03:18:39.200 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-04T03:18:39.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.193+0000 7fd68f324700 1 -- 172.21.15.1:0/483808776 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd6700b2880 msgr2=0x7fd6700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:39.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.193+0000 7fd68f324700 1 --2- 172.21.15.1:0/483808776 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd6700b2880 0x7fd6700b4d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd680009900 tx=0x7fd68000b040 comp rx=0 tx=0).stop 2024-05-04T03:18:39.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.193+0000 7fd68f324700 1 -- 172.21.15.1:0/483808776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd688108f40 msgr2=0x7fd688076000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:39.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.193+0000 7fd68f324700 1 --2- 172.21.15.1:0/483808776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd688108f40 0x7fd688076000 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00d980 tx=0x7fd67c00dc90 comp rx=0 tx=0).stop 2024-05-04T03:18:39.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.193+0000 7fd68f324700 1 -- 172.21.15.1:0/483808776 shutdown_connections 2024-05-04T03:18:39.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.193+0000 7fd68f324700 1 --2- 172.21.15.1:0/483808776 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd688104610 0x7fd688075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.193+0000 7fd68f324700 1 --2- 172.21.15.1:0/483808776 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd6700b2880 0x7fd6700b4d90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.193+0000 7fd68f324700 1 --2- 172.21.15.1:0/483808776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd688109890 0x7fd688076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.193+0000 7fd68f324700 1 --2- 172.21.15.1:0/483808776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd688108f40 0x7fd688076000 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:39.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.193+0000 7fd68f324700 1 -- 172.21.15.1:0/483808776 >> 172.21.15.1:0/483808776 conn(0x7fd68807af70 msgr2=0x7fd6881019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:39.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.193+0000 7fd68f324700 1 -- 172.21.15.1:0/483808776 shutdown_connections 2024-05-04T03:18:39.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:39.193+0000 7fd68f324700 1 -- 172.21.15.1:0/483808776 wait complete. 2024-05-04T03:18:39.895 INFO:teuthology.run_tasks:Running task sequential... 2024-05-04T03:18:39.911 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-04T03:18:39.911 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-04T03:18:39.912 INFO:teuthology.task.print:Enabling mgr modules 2024-05-04T03:18:39.912 INFO:teuthology.run_tasks:Running task kclient... 2024-05-04T03:18:39.945 INFO:tasks.kclient:Mounting kernel clients... 2024-05-04T03:18:39.946 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-04T03:18:39.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:18:39.946 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-05-04T03:18:39.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:18:39.963 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link delete ceph-brx 2024-05-04T03:18:40.031 INFO:teuthology.orchestra.run.smithi001.stderr:Cannot find device "ceph-brx" 2024-05-04T03:18:40.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T03:18:40.033 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-05-04T03:18:40.033 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-04T03:18:40.033 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-04T03:18:40.034 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-04T03:18:40.322 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:18:40.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:40 smithi001 ceph-mon[29731]: pgmap v376: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-04T03:18:40.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:40 smithi072 ceph-mon[33332]: pgmap v376: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-04T03:18:40.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:40 smithi077 ceph-mon[33531]: pgmap v376: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-04T03:18:42.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.044+0000 7fc0d408c700 1 -- 172.21.15.1:0/3441847732 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0cc107e10 msgr2=0x7fc0cc108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:42.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.044+0000 7fc0d408c700 1 --2- 172.21.15.1:0/3441847732 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0cc107e10 0x7fc0cc108220 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc0099c0 tx=0x7fc0bc009cd0 comp rx=0 tx=0).stop 2024-05-04T03:18:42.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.044+0000 7fc0d408c700 1 -- 172.21.15.1:0/3441847732 shutdown_connections 2024-05-04T03:18:42.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.044+0000 7fc0d408c700 1 --2- 172.21.15.1:0/3441847732 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0cc1034e0 0x7fc0cc1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:42.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.044+0000 7fc0d408c700 1 --2- 172.21.15.1:0/3441847732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0cc108760 0x7fc0cc1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:42.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.044+0000 7fc0d408c700 1 --2- 172.21.15.1:0/3441847732 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0cc107e10 0x7fc0cc108220 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:42.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.044+0000 7fc0d408c700 1 -- 172.21.15.1:0/3441847732 >> 172.21.15.1:0/3441847732 conn(0x7fc0cc0fef30 msgr2=0x7fc0cc101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:42.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.044+0000 7fc0d408c700 1 -- 172.21.15.1:0/3441847732 shutdown_connections 2024-05-04T03:18:42.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.044+0000 7fc0d408c700 1 -- 172.21.15.1:0/3441847732 wait complete. 2024-05-04T03:18:42.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.045+0000 7fc0d408c700 1 Processor -- start 2024-05-04T03:18:42.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.045+0000 7fc0d408c700 1 -- start start 2024-05-04T03:18:42.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.045+0000 7fc0d408c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0cc1034e0 0x7fc0cc0ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:42.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.045+0000 7fc0d408c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0cc107e10 0x7fc0cc0ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:42.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.045+0000 7fc0d408c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0cc108760 0x7fc0cc0ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:42.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.045+0000 7fc0d408c700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc0cc113400 con 0x7fc0cc108760 2024-05-04T03:18:42.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.045+0000 7fc0d408c700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc0cc113280 con 0x7fc0cc107e10 2024-05-04T03:18:42.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.045+0000 7fc0d408c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc0cc113580 con 0x7fc0cc1034e0 2024-05-04T03:18:42.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.045+0000 7fc0d2629700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0cc108760 0x7fc0cc0ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:42.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.045+0000 7fc0d2629700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0cc108760 0x7fc0cc0ffe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41038/0 (socket says 172.21.15.1:41038) 2024-05-04T03:18:42.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.045+0000 7fc0d2629700 1 -- 172.21.15.1:0/991456887 learned_addr learned my addr 172.21.15.1:0/991456887 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:42.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.045+0000 7fc0d1e28700 1 --2- 172.21.15.1:0/991456887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0cc1034e0 0x7fc0cc0ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:42.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.045+0000 7fc0d1627700 1 --2- 172.21.15.1:0/991456887 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0cc107e10 0x7fc0cc0ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:42.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.045+0000 7fc0d2629700 1 -- 172.21.15.1:0/991456887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0cc1034e0 msgr2=0x7fc0cc0ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:42.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.045+0000 7fc0d2629700 1 --2- 172.21.15.1:0/991456887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0cc1034e0 0x7fc0cc0ff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:42.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.045+0000 7fc0d2629700 1 -- 172.21.15.1:0/991456887 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0cc107e10 msgr2=0x7fc0cc0ff950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:42.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.045+0000 7fc0d2629700 1 --2- 172.21.15.1:0/991456887 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0cc107e10 0x7fc0cc0ff950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:42.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.045+0000 7fc0d2629700 1 -- 172.21.15.1:0/991456887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0cc1a9260 con 0x7fc0cc108760 2024-05-04T03:18:42.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.045+0000 7fc0d1627700 1 --2- 172.21.15.1:0/991456887 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0cc107e10 0x7fc0cc0ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:18:42.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.046+0000 7fc0d2629700 1 --2- 172.21.15.1:0/991456887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0cc108760 0x7fc0cc0ffe90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400ada0 tx=0x7fc0c400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:42.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.046+0000 7fc0c2ffd700 1 -- 172.21.15.1:0/991456887 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4017020 con 0x7fc0cc108760 2024-05-04T03:18:42.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.046+0000 7fc0c2ffd700 1 -- 172.21.15.1:0/991456887 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0c400ce30 con 0x7fc0cc108760 2024-05-04T03:18:42.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.046+0000 7fc0d408c700 1 -- 172.21.15.1:0/991456887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0cc1a9550 con 0x7fc0cc108760 2024-05-04T03:18:42.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.046+0000 7fc0c2ffd700 1 -- 172.21.15.1:0/991456887 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4005350 con 0x7fc0cc108760 2024-05-04T03:18:42.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.046+0000 7fc0d408c700 1 -- 172.21.15.1:0/991456887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0cc1a9ab0 con 0x7fc0cc108760 2024-05-04T03:18:42.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.047+0000 7fc0d408c700 1 -- 172.21.15.1:0/991456887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0cc0512c0 con 0x7fc0cc108760 2024-05-04T03:18:42.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.049+0000 7fc0c2ffd700 1 -- 172.21.15.1:0/991456887 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fc0c400c980 con 0x7fc0cc108760 2024-05-04T03:18:42.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.049+0000 7fc0c2ffd700 1 --2- 172.21.15.1:0/991456887 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc0b80b2900 0x7fc0b80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:42.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.049+0000 7fc0d1e28700 1 --2- 172.21.15.1:0/991456887 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc0b80b2900 0x7fc0b80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:42.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.050+0000 7fc0c2ffd700 1 -- 172.21.15.1:0/991456887 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fc0c40e1d90 con 0x7fc0cc108760 2024-05-04T03:18:42.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.050+0000 7fc0d1e28700 1 --2- 172.21.15.1:0/991456887 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc0b80b2900 0x7fc0b80b4e10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8009a10 tx=0x7fc0c8009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:42.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.053+0000 7fc0c2ffd700 1 -- 172.21.15.1:0/991456887 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc0c4021020 con 0x7fc0cc108760 2024-05-04T03:18:42.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.314+0000 7fc0d408c700 1 -- 172.21.15.1:0/991456887 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc0cc02e5f0 con 0x7fc0cc108760 2024-05-04T03:18:42.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.315+0000 7fc0c2ffd700 1 -- 172.21.15.1:0/991456887 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7fc0c40a9c00 con 0x7fc0cc108760 2024-05-04T03:18:42.317 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-04T03:18:42.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.317+0000 7fc0d408c700 1 -- 172.21.15.1:0/991456887 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc0b80b2900 msgr2=0x7fc0b80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:42.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.317+0000 7fc0d408c700 1 --2- 172.21.15.1:0/991456887 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc0b80b2900 0x7fc0b80b4e10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8009a10 tx=0x7fc0c8009190 comp rx=0 tx=0).stop 2024-05-04T03:18:42.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.317+0000 7fc0d408c700 1 -- 172.21.15.1:0/991456887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0cc108760 msgr2=0x7fc0cc0ffe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:42.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.317+0000 7fc0d408c700 1 --2- 172.21.15.1:0/991456887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0cc108760 0x7fc0cc0ffe90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400ada0 tx=0x7fc0c400c330 comp rx=0 tx=0).stop 2024-05-04T03:18:42.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.317+0000 7fc0d408c700 1 -- 172.21.15.1:0/991456887 shutdown_connections 2024-05-04T03:18:42.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.317+0000 7fc0d408c700 1 --2- 172.21.15.1:0/991456887 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc0b80b2900 0x7fc0b80b4e10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:42.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.317+0000 7fc0d408c700 1 --2- 172.21.15.1:0/991456887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc0cc108760 0x7fc0cc0ffe90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:42.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.317+0000 7fc0d408c700 1 --2- 172.21.15.1:0/991456887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc0cc1034e0 0x7fc0cc0ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:42.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.317+0000 7fc0d408c700 1 --2- 172.21.15.1:0/991456887 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc0cc107e10 0x7fc0cc0ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:42.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.317+0000 7fc0d408c700 1 -- 172.21.15.1:0/991456887 >> 172.21.15.1:0/991456887 conn(0x7fc0cc0fef30 msgr2=0x7fc0cc06c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:42.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.317+0000 7fc0d408c700 1 -- 172.21.15.1:0/991456887 shutdown_connections 2024-05-04T03:18:42.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:42.317+0000 7fc0d408c700 1 -- 172.21.15.1:0/991456887 wait complete. 2024-05-04T03:18:42.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:42 smithi001 ceph-mon[29731]: pgmap v377: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:18:42.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:42 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/991456887' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-04T03:18:42.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:42 smithi072 ceph-mon[33332]: pgmap v377: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:18:42.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:42 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/991456887' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-04T03:18:42.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:42 smithi077 ceph-mon[33531]: pgmap v377: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:18:42.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:42 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/991456887' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-04T03:18:43.019 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-04T03:18:43.020 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-04T03:18:43.020 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-05-04T03:18:43.020 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-04T03:18:43.020 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-04T03:18:43.020 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-04T03:18:43.020 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-04T03:18:43.020 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-04T03:18:43.020 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-04T03:18:43.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:18:43.020 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-05-04T03:18:43.037 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-04T03:18:43.037 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-04T03:18:43.038 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-05-04T03:18:43.038 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-04T03:18:43.038 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-05-04T03:18:43.038 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-04T03:18:43.038 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-04T03:18:43.038 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-05-04T03:18:43.038 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-05-04T03:18:43.038 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-04T03:18:43.038 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-05-04T03:18:43.038 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-05-04T03:18:43.038 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-04T03:18:43.038 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-05-04T03:18:43.039 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-05-04T03:18:43.039 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-04T03:18:43.039 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 41947sec preferred_lft 41947sec 2024-05-04T03:18:43.039 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link noprefixroute 2024-05-04T03:18:43.039 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-05-04T03:18:43.039 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-04T03:18:43.039 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-05-04T03:18:43.039 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-05-04T03:18:43.039 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-04T03:18:43.040 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T03:18:43.040 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-04T03:18:43.040 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add name ceph-brx type bridge 2024-05-04T03:18:43.040 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip addr flush dev ceph-brx 2024-05-04T03:18:43.040 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set ceph-brx up 2024-05-04T03:18:43.040 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-04T03:18:43.040 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-04T03:18:43.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T03:18:43.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T03:18:43.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:18:43.232 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-04T03:18:43.258 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-05-04T03:18:43.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:18:43.259 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-05-04T03:18:43.330 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-05-04T03:18:43.331 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-04T03:18:43.331 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-04T03:18:43.331 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-04T03:18:43.331 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-04T03:18:43.331 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T03:18:43.331 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-04T03:18:43.332 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-04T03:18:43.332 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-04T03:18:43.332 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-04T03:18:43.332 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-04T03:18:43.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T03:18:43.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T03:18:43.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:18:43.611 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-05-04T03:18:43.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:18:43.627 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list-id 2024-05-04T03:18:43.683 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T03:18:43.683 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-04T03:18:43.683 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-04T03:18:43.683 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-04T03:18:43.683 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-04T03:18:43.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T03:18:43.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T03:18:43.801 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-04T03:18:43.801 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T03:18:43.801 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-04T03:18:43.801 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-04T03:18:43.801 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-04T03:18:43.801 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-04T03:18:43.801 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-04T03:18:43.802 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-04T03:18:43.802 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-04T03:18:43.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T03:18:43.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T03:18:43.954 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-04T03:18:43.954 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-05-04T03:18:43.954 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2024-05-04T03:18:43.954 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set dev brx.0 master ceph-brx 2024-05-04T03:18:43.954 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-05-04T03:18:44.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T03:18:44.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T03:18:44.031 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-04T03:18:44.031 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-04T03:18:44.047 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-04T03:18:44.047 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-04T03:18:44.047 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-04T03:18:44.103 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-04T03:18:44.103 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-04T03:18:44.103 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/fca5699f-7058-441f-8333-bcf3e9a0f362 2024-05-04T03:18:44.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:18:44.103 DEBUG:teuthology.orchestra.run.smithi001:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/fca5699f-7058-441f-8333-bcf3e9a0f362 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-05-04T03:18:44.211 INFO:teuthology.orchestra.run.smithi001.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-05-04T03:18:44.211 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-05-04T03:18:44.321 INFO:teuthology.orchestra.run.smithi001.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-05-04T03:18:44.322 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-05-04T03:18:44.322 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: resolved to: "172.21.15.1:6789,172.21.15.72:6789,172.21.15.77:6789" 2024-05-04T03:18:44.322 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: trying mount with new device syntax: 0@4a572aa8-09c3-11ef-bc97-c7b262605968.cephfs=/volumes/qa/sv_1/fca5699f-7058-441f-8333-bcf3e9a0f362 2024-05-04T03:18:44.322 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mon_addr=172.21.15.1:6789/172.21.15.72:6789/172.21.15.77:6789" will pass to kernel 2024-05-04T03:18:44.323 INFO:teuthology.orchestra.run.smithi001.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-04T03:18:44.323 INFO:teuthology.orchestra.run.smithi001.stdout: You just mounted a file system that supports labels which does not 2024-05-04T03:18:44.323 INFO:teuthology.orchestra.run.smithi001.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-04T03:18:44.323 INFO:teuthology.orchestra.run.smithi001.stdout: applications will generate AVC messages and not be allowed access to 2024-05-04T03:18:44.323 INFO:teuthology.orchestra.run.smithi001.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-04T03:18:44.325 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-04T03:18:44.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:18:44.325 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-04T03:18:44.355 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-04T03:18:44.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-04T03:18:44.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T03:18:44.455 INFO:teuthology.orchestra.run.smithi001.stdout:client24866 2024-05-04T03:18:44.455 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-04T03:18:44.455 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-04T03:18:44.472 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-04T03:18:44.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-04T03:18:44.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T03:18:44.569 INFO:teuthology.orchestra.run.smithi001.stdout:client24866 2024-05-04T03:18:44.569 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-04T03:18:44.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-04T03:18:44.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T03:18:44.626 INFO:teuthology.orchestra.run.smithi001.stdout:client24866 2024-05-04T03:18:44.627 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968.client24866/status') 2024-05-04T03:18:44.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-04T03:18:44.686 INFO:teuthology.orchestra.run.smithi001.stdout:instance: client.24866 (3)172.21.15.1:0/276877218 2024-05-04T03:18:44.686 INFO:teuthology.orchestra.run.smithi001.stdout:blocklisted: false 2024-05-04T03:18:44.686 INFO:teuthology.orchestra.run.smithi001.stderr:0+1 records in 2024-05-04T03:18:44.686 INFO:teuthology.orchestra.run.smithi001.stderr:0+1 records out 2024-05-04T03:18:44.686 INFO:teuthology.orchestra.run.smithi001.stderr:69 bytes copied, 2.4375e-05 s, 2.8 MB/s 2024-05-04T03:18:44.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-04T03:18:44.693 INFO:teuthology.run_tasks:Running task exec... 2024-05-04T03:18:44.705 INFO:teuthology.task.exec:Executing custom commands... 2024-05-04T03:18:44.705 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-05-04T03:18:44.705 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-05-04T03:18:44.727 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:44 smithi001 ceph-mon[29731]: pgmap v378: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:18:44.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:44 smithi072 ceph-mon[33332]: pgmap v378: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:18:44.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.816+0000 7fb2047f2640 1 -- 172.21.15.1:0/524557863 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1fc108a70 msgr2=0x7fb1fc10ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:44.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.816+0000 7fb2047f2640 1 --2- 172.21.15.1:0/524557863 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1fc108a70 0x7fb1fc10ae90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4009ca0 tx=0x7fb1f402a8c0 comp rx=0 tx=0).stop 2024-05-04T03:18:44.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.816+0000 7fb2047f2640 1 -- 172.21.15.1:0/524557863 shutdown_connections 2024-05-04T03:18:44.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.816+0000 7fb2047f2640 1 --2- 172.21.15.1:0/524557863 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb1fc103d70 0x7fb1fc104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:44.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.816+0000 7fb2047f2640 1 --2- 172.21.15.1:0/524557863 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1fc108a70 0x7fb1fc10ae90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:44.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.816+0000 7fb2047f2640 1 --2- 172.21.15.1:0/524557863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1fc102b30 0x7fb1fc103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:44.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.816+0000 7fb2047f2640 1 -- 172.21.15.1:0/524557863 >> 172.21.15.1:0/524557863 conn(0x7fb1fc0fdd20 msgr2=0x7fb1fc1001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:44.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.816+0000 7fb2047f2640 1 -- 172.21.15.1:0/524557863 shutdown_connections 2024-05-04T03:18:44.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.816+0000 7fb2047f2640 1 -- 172.21.15.1:0/524557863 wait complete. 2024-05-04T03:18:44.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.816+0000 7fb2047f2640 1 Processor -- start 2024-05-04T03:18:44.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.817+0000 7fb2047f2640 1 -- start start 2024-05-04T03:18:44.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.817+0000 7fb2047f2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1fc102b30 0x7fb1fc19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:44.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.817+0000 7fb2047f2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1fc103d70 0x7fb1fc19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:44.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.817+0000 7fb2047f2640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb1fc108a70 0x7fb1fc1a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:44.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.817+0000 7fb2047f2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc10f750 con 0x7fb1fc102b30 2024-05-04T03:18:44.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.817+0000 7fb2047f2640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc10f5d0 con 0x7fb1fc108a70 2024-05-04T03:18:44.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.817+0000 7fb2047f2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb1fc10f8d0 con 0x7fb1fc103d70 2024-05-04T03:18:44.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.817+0000 7fb20258e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1fc102b30 0x7fb1fc19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:44.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.817+0000 7fb20258e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1fc102b30 0x7fb1fc19a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41050/0 (socket says 172.21.15.1:41050) 2024-05-04T03:18:44.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.817+0000 7fb20258e640 1 -- 172.21.15.1:0/2704076866 learned_addr learned my addr 172.21.15.1:0/2704076866 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:44.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.817+0000 7fb201d8d640 1 --2- 172.21.15.1:0/2704076866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1fc103d70 0x7fb1fc19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:44.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.817+0000 7fb202d8f640 1 --2- 172.21.15.1:0/2704076866 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb1fc108a70 0x7fb1fc1a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:44.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.817+0000 7fb20258e640 1 -- 172.21.15.1:0/2704076866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1fc103d70 msgr2=0x7fb1fc19a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:44.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.817+0000 7fb20258e640 1 --2- 172.21.15.1:0/2704076866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1fc103d70 0x7fb1fc19a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:44.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.817+0000 7fb20258e640 1 -- 172.21.15.1:0/2704076866 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb1fc108a70 msgr2=0x7fb1fc1a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:44.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.817+0000 7fb20258e640 1 --2- 172.21.15.1:0/2704076866 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb1fc108a70 0x7fb1fc1a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:44.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.817+0000 7fb20258e640 1 -- 172.21.15.1:0/2704076866 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1fc1a1e30 con 0x7fb1fc102b30 2024-05-04T03:18:44.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.817+0000 7fb201d8d640 1 --2- 172.21.15.1:0/2704076866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1fc103d70 0x7fb1fc19a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:18:44.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.817+0000 7fb20258e640 1 --2- 172.21.15.1:0/2704076866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1fc102b30 0x7fb1fc19a2f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb1f800e9f0 tx=0x7fb1f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:44.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.818+0000 7fb1f37fe640 1 -- 172.21.15.1:0/2704076866 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f800ccc0 con 0x7fb1fc102b30 2024-05-04T03:18:44.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.818+0000 7fb1f37fe640 1 -- 172.21.15.1:0/2704076866 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1f80035f0 con 0x7fb1fc102b30 2024-05-04T03:18:44.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.818+0000 7fb1f37fe640 1 -- 172.21.15.1:0/2704076866 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f800ce60 con 0x7fb1fc102b30 2024-05-04T03:18:44.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.818+0000 7fb2047f2640 1 -- 172.21.15.1:0/2704076866 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1fc1a2120 con 0x7fb1fc102b30 2024-05-04T03:18:44.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.818+0000 7fb2047f2640 1 -- 172.21.15.1:0/2704076866 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1fc1a2670 con 0x7fb1fc102b30 2024-05-04T03:18:44.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.819+0000 7fb2047f2640 1 -- 172.21.15.1:0/2704076866 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1fc107170 con 0x7fb1fc102b30 2024-05-04T03:18:44.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.820+0000 7fb1f37fe640 1 -- 172.21.15.1:0/2704076866 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fb1f8003aa0 con 0x7fb1fc102b30 2024-05-04T03:18:44.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.821+0000 7fb1f37fe640 1 --2- 172.21.15.1:0/2704076866 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fb1e80b2920 0x7fb1e80b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:44.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.821+0000 7fb1f37fe640 1 -- 172.21.15.1:0/2704076866 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fb1f80e2680 con 0x7fb1fc102b30 2024-05-04T03:18:44.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.821+0000 7fb201d8d640 1 --2- 172.21.15.1:0/2704076866 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fb1e80b2920 0x7fb1e80b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:44.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.822+0000 7fb201d8d640 1 --2- 172.21.15.1:0/2704076866 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fb1e80b2920 0x7fb1e80b4e30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec005fd0 tx=0x7fb1ec00b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:44.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:44.825+0000 7fb1f37fe640 1 -- 172.21.15.1:0/2704076866 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb1f80aa460 con 0x7fb1fc102b30 2024-05-04T03:18:44.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:44 smithi077 ceph-mon[33531]: pgmap v378: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:18:45.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.056+0000 7fb2047f2640 1 -- 172.21.15.1:0/2704076866 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fb1fc02dc30 con 0x7fb1fc102b30 2024-05-04T03:18:45.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.453+0000 7fb1f37fe640 1 -- 172.21.15.1:0/2704076866 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7fb1f80a9d60 con 0x7fb1fc102b30 2024-05-04T03:18:45.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.455+0000 7fb2047f2640 1 -- 172.21.15.1:0/2704076866 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fb1e80b2920 msgr2=0x7fb1e80b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:45.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.455+0000 7fb2047f2640 1 --2- 172.21.15.1:0/2704076866 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fb1e80b2920 0x7fb1e80b4e30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec005fd0 tx=0x7fb1ec00b5f0 comp rx=0 tx=0).stop 2024-05-04T03:18:45.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.456+0000 7fb2047f2640 1 -- 172.21.15.1:0/2704076866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1fc102b30 msgr2=0x7fb1fc19a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:45.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.456+0000 7fb2047f2640 1 --2- 172.21.15.1:0/2704076866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1fc102b30 0x7fb1fc19a2f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb1f800e9f0 tx=0x7fb1f800eeb0 comp rx=0 tx=0).stop 2024-05-04T03:18:45.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.456+0000 7fb2047f2640 1 -- 172.21.15.1:0/2704076866 shutdown_connections 2024-05-04T03:18:45.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.456+0000 7fb2047f2640 1 --2- 172.21.15.1:0/2704076866 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fb1e80b2920 0x7fb1e80b4e30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:45.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.456+0000 7fb2047f2640 1 --2- 172.21.15.1:0/2704076866 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb1fc108a70 0x7fb1fc1a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:45.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.456+0000 7fb2047f2640 1 --2- 172.21.15.1:0/2704076866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1fc103d70 0x7fb1fc19a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:45.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.456+0000 7fb2047f2640 1 --2- 172.21.15.1:0/2704076866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1fc102b30 0x7fb1fc19a2f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:45.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.456+0000 7fb2047f2640 1 -- 172.21.15.1:0/2704076866 >> 172.21.15.1:0/2704076866 conn(0x7fb1fc0fdd20 msgr2=0x7fb1fc101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:45.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.456+0000 7fb2047f2640 1 -- 172.21.15.1:0/2704076866 shutdown_connections 2024-05-04T03:18:45.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.456+0000 7fb2047f2640 1 -- 172.21.15.1:0/2704076866 wait complete. 2024-05-04T03:18:45.472 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-05-04T03:18:45.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.632+0000 7f858e654640 1 -- 172.21.15.1:0/3605747870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8588106d70 msgr2=0x7f8588109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:45.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.632+0000 7f858e654640 1 --2- 172.21.15.1:0/3605747870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8588106d70 0x7f8588109250 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f857c009860 tx=0x7f857c02a340 comp rx=0 tx=0).stop 2024-05-04T03:18:45.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.632+0000 7f858e654640 1 -- 172.21.15.1:0/3605747870 shutdown_connections 2024-05-04T03:18:45.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.632+0000 7f858e654640 1 --2- 172.21.15.1:0/3605747870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8588106d70 0x7f8588109250 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:45.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.632+0000 7f858e654640 1 --2- 172.21.15.1:0/3605747870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85881098e0 0x7f858810bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:45.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.632+0000 7f858e654640 1 --2- 172.21.15.1:0/3605747870 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8588104360 0x7f8588106780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:45.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.632+0000 7f858e654640 1 -- 172.21.15.1:0/3605747870 >> 172.21.15.1:0/3605747870 conn(0x7f85880fdd40 msgr2=0x7f85881001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:45.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.632+0000 7f858e654640 1 -- 172.21.15.1:0/3605747870 shutdown_connections 2024-05-04T03:18:45.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.632+0000 7f858e654640 1 -- 172.21.15.1:0/3605747870 wait complete. 2024-05-04T03:18:45.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.633+0000 7f858e654640 1 Processor -- start 2024-05-04T03:18:45.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.633+0000 7f858e654640 1 -- start start 2024-05-04T03:18:45.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.633+0000 7f858e654640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8588104360 0x7f858819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:45.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.633+0000 7f858e654640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8588106d70 0x7f858819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:45.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.633+0000 7f858e654640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85881098e0 0x7f85881a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:45.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.633+0000 7f858e654640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f858810f750 con 0x7f8588106d70 2024-05-04T03:18:45.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.633+0000 7f858e654640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f858810f5d0 con 0x7f8588104360 2024-05-04T03:18:45.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.633+0000 7f858e654640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f858810f8d0 con 0x7f85881098e0 2024-05-04T03:18:45.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.633+0000 7f85877fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8588106d70 0x7f858819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:45.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.634+0000 7f85877fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8588106d70 0x7f858819a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41080/0 (socket says 172.21.15.1:41080) 2024-05-04T03:18:45.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.634+0000 7f85877fe640 1 -- 172.21.15.1:0/4242825609 learned_addr learned my addr 172.21.15.1:0/4242825609 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:45.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.634+0000 7f8587fff640 1 --2- 172.21.15.1:0/4242825609 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8588104360 0x7f858819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:45.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.634+0000 7f858cbf1640 1 --2- 172.21.15.1:0/4242825609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85881098e0 0x7f85881a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:45.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.634+0000 7f8587fff640 1 -- 172.21.15.1:0/4242825609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85881098e0 msgr2=0x7f85881a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:45.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.634+0000 7f8587fff640 1 --2- 172.21.15.1:0/4242825609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85881098e0 0x7f85881a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:45.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.634+0000 7f8587fff640 1 -- 172.21.15.1:0/4242825609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8588106d70 msgr2=0x7f858819a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:45.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.634+0000 7f8587fff640 1 --2- 172.21.15.1:0/4242825609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8588106d70 0x7f858819a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:45.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.634+0000 7f8587fff640 1 -- 172.21.15.1:0/4242825609 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85881a1e30 con 0x7f8588104360 2024-05-04T03:18:45.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.634+0000 7f85877fe640 1 --2- 172.21.15.1:0/4242825609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8588106d70 0x7f858819a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:18:45.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.634+0000 7f8587fff640 1 --2- 172.21.15.1:0/4242825609 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8588104360 0x7f858819a2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f858806b040 tx=0x7f857800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:45.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.634+0000 7f85857fa640 1 -- 172.21.15.1:0/4242825609 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857800ccd0 con 0x7f8588104360 2024-05-04T03:18:45.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.634+0000 7f858e654640 1 -- 172.21.15.1:0/4242825609 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85881a2120 con 0x7f8588104360 2024-05-04T03:18:45.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.634+0000 7f85857fa640 1 -- 172.21.15.1:0/4242825609 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85780035f0 con 0x7f8588104360 2024-05-04T03:18:45.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.634+0000 7f85857fa640 1 -- 172.21.15.1:0/4242825609 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8578010690 con 0x7f8588104360 2024-05-04T03:18:45.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.634+0000 7f858e654640 1 -- 172.21.15.1:0/4242825609 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85881a2670 con 0x7f8588104360 2024-05-04T03:18:45.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.636+0000 7f858e654640 1 -- 172.21.15.1:0/4242825609 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8588050870 con 0x7f8588104360 2024-05-04T03:18:45.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.637+0000 7f85857fa640 1 -- 172.21.15.1:0/4242825609 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f8578004020 con 0x7f8588104360 2024-05-04T03:18:45.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.637+0000 7f85857fa640 1 --2- 172.21.15.1:0/4242825609 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f85700b28a0 0x7f85700b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:45.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.637+0000 7f85857fa640 1 -- 172.21.15.1:0/4242825609 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f85780e2390 con 0x7f8588104360 2024-05-04T03:18:45.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.637+0000 7f85877fe640 1 --2- 172.21.15.1:0/4242825609 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f85700b28a0 0x7f85700b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:45.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.638+0000 7f85877fe640 1 --2- 172.21.15.1:0/4242825609 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f85700b28a0 0x7f85700b4db0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f857c0303f0 tx=0x7f857c030470 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:45.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.642+0000 7f85857fa640 1 -- 172.21.15.1:0/4242825609 <== mon.2 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8578009510 con 0x7f8588104360 2024-05-04T03:18:45.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:45 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/2704076866' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-04T03:18:45.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.861+0000 7f858e654640 1 -- 172.21.15.1:0/4242825609 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f8588063110 con 0x7f85700b28a0 2024-05-04T03:18:45.869 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:45 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2704076866' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-04T03:18:45.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.878+0000 7f85857fa640 1 -- 172.21.15.1:0/4242825609 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f8588063110 con 0x7f85700b28a0 2024-05-04T03:18:45.881 INFO:teuthology.orchestra.run.smithi001.stdout:{} 2024-05-04T03:18:45.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.880+0000 7f858e654640 1 -- 172.21.15.1:0/4242825609 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f85700b28a0 msgr2=0x7f85700b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:45.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.880+0000 7f858e654640 1 --2- 172.21.15.1:0/4242825609 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f85700b28a0 0x7f85700b4db0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f857c0303f0 tx=0x7f857c030470 comp rx=0 tx=0).stop 2024-05-04T03:18:45.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.880+0000 7f858e654640 1 -- 172.21.15.1:0/4242825609 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8588104360 msgr2=0x7f858819a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:45.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.880+0000 7f858e654640 1 --2- 172.21.15.1:0/4242825609 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8588104360 0x7f858819a2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f858806b040 tx=0x7f857800ee50 comp rx=0 tx=0).stop 2024-05-04T03:18:45.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.880+0000 7f858e654640 1 -- 172.21.15.1:0/4242825609 shutdown_connections 2024-05-04T03:18:45.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.880+0000 7f858e654640 1 --2- 172.21.15.1:0/4242825609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85881098e0 0x7f85881a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:45.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.880+0000 7f858e654640 1 --2- 172.21.15.1:0/4242825609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8588106d70 0x7f858819a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:45.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.880+0000 7f858e654640 1 --2- 172.21.15.1:0/4242825609 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f85700b28a0 0x7f85700b4db0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:45.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.880+0000 7f858e654640 1 --2- 172.21.15.1:0/4242825609 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8588104360 0x7f858819a2f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:45.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.880+0000 7f858e654640 1 -- 172.21.15.1:0/4242825609 >> 172.21.15.1:0/4242825609 conn(0x7f85880fdd40 msgr2=0x7f8588104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:45.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.880+0000 7f858e654640 1 -- 172.21.15.1:0/4242825609 shutdown_connections 2024-05-04T03:18:45.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:45.880+0000 7f858e654640 1 -- 172.21.15.1:0/4242825609 wait complete. 2024-05-04T03:18:45.895 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-04T03:18:45.906 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-04T03:18:45.956 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-05-04T03:18:45.956 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-04T03:18:45.957 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:45 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/2704076866' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-04T03:18:45.959 INFO:teuthology.run_tasks:Running task quiescer... 2024-05-04T03:18:45.996 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-04T03:18:45.996 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-04T03:18:46.129 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:18:45 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:18:45.866+0000 7f3fca2e8700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-04T03:18:46.129 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:18:45 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:18:45.866+0000 7f3fca2e8700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-04T03:18:46.129 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:18:45 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:18:45.866+0000 7f3fca2e8700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-04T03:18:46.129 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:18:45 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:18:45.866+0000 7f3fca2e8700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-04T03:18:46.129 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:18:45 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:18:45.866+0000 7f3fca2e8700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-04T03:18:46.245 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:18:46.618 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:46 smithi001 ceph-mon[29731]: pgmap v379: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-04T03:18:46.618 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:46 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2704076866' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-04T03:18:46.619 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:46 smithi001 ceph-mon[29731]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:18:46.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:46 smithi072 ceph-mon[33332]: pgmap v379: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-04T03:18:46.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:46 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/2704076866' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-04T03:18:46.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:46 smithi072 ceph-mon[33332]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:18:46.829 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:46 smithi077 ceph-mon[33531]: pgmap v379: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-04T03:18:46.830 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:46 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/2704076866' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-04T03:18:46.830 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:46 smithi077 ceph-mon[33531]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:18:47.603 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:47 smithi001 ceph-mon[29731]: from='client.24878 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:18:47.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:47 smithi072 ceph-mon[33332]: from='client.24878 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:18:47.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.876+0000 7f002465d700 1 -- 172.21.15.1:0/3130258554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f001c1045d0 msgr2=0x7f001c1049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:47.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.876+0000 7f002465d700 1 --2- 172.21.15.1:0/3130258554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f001c1045d0 0x7f001c1049e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f00180099c0 tx=0x7f0018009cd0 comp rx=0 tx=0).stop 2024-05-04T03:18:47.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.876+0000 7f002465d700 1 -- 172.21.15.1:0/3130258554 shutdown_connections 2024-05-04T03:18:47.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.876+0000 7f002465d700 1 --2- 172.21.15.1:0/3130258554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f001c109850 0x7f001c1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:47.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.877+0000 7f002465d700 1 --2- 172.21.15.1:0/3130258554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f001c1045d0 0x7f001c1049e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:47.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.877+0000 7f002465d700 1 --2- 172.21.15.1:0/3130258554 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f001c108f00 0x7f001c109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:47.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.877+0000 7f002465d700 1 -- 172.21.15.1:0/3130258554 >> 172.21.15.1:0/3130258554 conn(0x7f001c0ffea0 msgr2=0x7f001c1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:47.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.877+0000 7f002465d700 1 -- 172.21.15.1:0/3130258554 shutdown_connections 2024-05-04T03:18:47.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.877+0000 7f002465d700 1 -- 172.21.15.1:0/3130258554 wait complete. 2024-05-04T03:18:47.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.877+0000 7f002465d700 1 Processor -- start 2024-05-04T03:18:47.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.877+0000 7f002465d700 1 -- start start 2024-05-04T03:18:47.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.878+0000 7f002465d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f001c1045d0 0x7f001c19ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:47.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.878+0000 7f002465d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f001c108f00 0x7f001c19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:47.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.878+0000 7f002465d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f001c109850 0x7f001c19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:47.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.878+0000 7f002465d700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f001c1144a0 con 0x7f001c108f00 2024-05-04T03:18:47.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.878+0000 7f002465d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f001c114320 con 0x7f001c1045d0 2024-05-04T03:18:47.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.878+0000 7f002465d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f001c114620 con 0x7f001c109850 2024-05-04T03:18:47.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.878+0000 7f0021bf8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f001c108f00 0x7f001c19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:47.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.878+0000 7f0021bf8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f001c108f00 0x7f001c19b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41128/0 (socket says 172.21.15.1:41128) 2024-05-04T03:18:47.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.878+0000 7f0021bf8700 1 -- 172.21.15.1:0/730041188 learned_addr learned my addr 172.21.15.1:0/730041188 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:47.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.878+0000 7f00223f9700 1 --2- 172.21.15.1:0/730041188 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f001c1045d0 0x7f001c19ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:47.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.878+0000 7f0022bfa700 1 --2- 172.21.15.1:0/730041188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f001c109850 0x7f001c19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:47.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.878+0000 7f0021bf8700 1 -- 172.21.15.1:0/730041188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f001c109850 msgr2=0x7f001c19f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:47.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.878+0000 7f0021bf8700 1 --2- 172.21.15.1:0/730041188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f001c109850 0x7f001c19f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:47.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.878+0000 7f0021bf8700 1 -- 172.21.15.1:0/730041188 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f001c1045d0 msgr2=0x7f001c19ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:47.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.878+0000 7f0021bf8700 1 --2- 172.21.15.1:0/730041188 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f001c1045d0 0x7f001c19ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:47.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.878+0000 7f0021bf8700 1 -- 172.21.15.1:0/730041188 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f001c19fc60 con 0x7f001c108f00 2024-05-04T03:18:47.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.878+0000 7f0021bf8700 1 --2- 172.21.15.1:0/730041188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f001c108f00 0x7f001c19b430 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f000c00c9e0 tx=0x7f000c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:47.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.879+0000 7f00137fe700 1 -- 172.21.15.1:0/730041188 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c007aa0 con 0x7f001c108f00 2024-05-04T03:18:47.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.879+0000 7f002465d700 1 -- 172.21.15.1:0/730041188 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f001c19ff50 con 0x7f001c108f00 2024-05-04T03:18:47.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.879+0000 7f00137fe700 1 -- 172.21.15.1:0/730041188 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f000c007c40 con 0x7f001c108f00 2024-05-04T03:18:47.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.879+0000 7f00137fe700 1 -- 172.21.15.1:0/730041188 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c00fe60 con 0x7f001c108f00 2024-05-04T03:18:47.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.879+0000 7f002465d700 1 -- 172.21.15.1:0/730041188 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f001c1a0490 con 0x7f001c108f00 2024-05-04T03:18:47.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.880+0000 7f002465d700 1 -- 172.21.15.1:0/730041188 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f001c0512c0 con 0x7f001c108f00 2024-05-04T03:18:47.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.881+0000 7f00137fe700 1 -- 172.21.15.1:0/730041188 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7f000c00f450 con 0x7f001c108f00 2024-05-04T03:18:47.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.882+0000 7f00137fe700 1 --2- 172.21.15.1:0/730041188 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f00080b2910 0x7f00080b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:47.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.882+0000 7f00223f9700 1 --2- 172.21.15.1:0/730041188 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f00080b2910 0x7f00080b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:47.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.882+0000 7f00137fe700 1 -- 172.21.15.1:0/730041188 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f000c0e2550 con 0x7f001c108f00 2024-05-04T03:18:47.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.882+0000 7f00223f9700 1 --2- 172.21.15.1:0/730041188 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f00080b2910 0x7f00080b4e20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0018009990 tx=0x7f0018000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:47.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:47.886+0000 7f00137fe700 1 -- 172.21.15.1:0/730041188 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f000c014030 con 0x7f001c108f00 2024-05-04T03:18:47.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:47 smithi077 ceph-mon[33531]: from='client.24878 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-04T03:18:48.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:48.098+0000 7f002465d700 1 -- 172.21.15.1:0/730041188 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f001c063c00 con 0x7f00080b2910 2024-05-04T03:18:48.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:48.106+0000 7f00137fe700 1 -- 172.21.15.1:0/730041188 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258823 (secure 0 0 0) 0x7f0018067320 con 0x7f00080b2910 2024-05-04T03:18:48.108 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:18:48.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:48.109+0000 7f002465d700 1 -- 172.21.15.1:0/730041188 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f00080b2910 msgr2=0x7f00080b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:48.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:48.109+0000 7f002465d700 1 --2- 172.21.15.1:0/730041188 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f00080b2910 0x7f00080b4e20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0018009990 tx=0x7f0018000bc0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:48.109+0000 7f002465d700 1 -- 172.21.15.1:0/730041188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f001c108f00 msgr2=0x7f001c19b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:48.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:48.109+0000 7f002465d700 1 --2- 172.21.15.1:0/730041188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f001c108f00 0x7f001c19b430 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f000c00c9e0 tx=0x7f000c00ccf0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:48.109+0000 7f002465d700 1 -- 172.21.15.1:0/730041188 shutdown_connections 2024-05-04T03:18:48.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:48.109+0000 7f002465d700 1 --2- 172.21.15.1:0/730041188 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f001c1045d0 0x7f001c19ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:48.109+0000 7f002465d700 1 --2- 172.21.15.1:0/730041188 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f00080b2910 0x7f00080b4e20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:48.109+0000 7f002465d700 1 --2- 172.21.15.1:0/730041188 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f001c109850 0x7f001c19f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:48.109+0000 7f002465d700 1 --2- 172.21.15.1:0/730041188 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f001c108f00 0x7f001c19b430 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:48.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:48.109+0000 7f002465d700 1 -- 172.21.15.1:0/730041188 >> 172.21.15.1:0/730041188 conn(0x7f001c0ffea0 msgr2=0x7f001c1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:48.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:48.109+0000 7f002465d700 1 -- 172.21.15.1:0/730041188 shutdown_connections 2024-05-04T03:18:48.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:48.109+0000 7f002465d700 1 -- 172.21.15.1:0/730041188 wait complete. 2024-05-04T03:18:48.116 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-04T03:18:48.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:48 smithi072 ceph-mon[33332]: pgmap v380: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-04T03:18:48.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:48 smithi072 ceph-mon[33332]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-04T03:18:48.774 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:48 smithi001 ceph-mon[29731]: pgmap v380: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-04T03:18:48.774 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:48 smithi001 ceph-mon[29731]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-04T03:18:48.806 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":380,"stamp":"2024-05-04T03:18:47.185888+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661433,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1054,"num_read_kb":626,"num_write":300,"num_write_kb":2276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":743,"ondisk_log_size":743,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":411784,"kb_used_data":12712,"kb_used_omap":18,"kb_used_meta":398957,"kb_avail":1124530040,"statfs":{"total":1151940427776,"available":1151518760960,"internally_reserved":0,"allocated":13017088,"data_stored":7267748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":408532281},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003469"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.173150+0000","last_change":"2024-05-04T03:15:48.049361+0000","last_active":"2024-05-04T03:17:33.173150+0000","last_peered":"2024-05-04T03:17:33.173150+0000","last_clean":"2024-05-04T03:17:33.173150+0000","last_became_active":"2024-05-04T03:15:48.048876+0000","last_became_peered":"2024-05-04T03:15:48.048876+0000","last_unstale":"2024-05-04T03:17:33.173150+0000","last_undegraded":"2024-05-04T03:17:33.173150+0000","last_fullsized":"2024-05-04T03:17:33.173150+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:50:52.568516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165921+0000","last_change":"2024-05-04T03:15:48.055554+0000","last_active":"2024-05-04T03:17:33.165921+0000","last_peered":"2024-05-04T03:17:33.165921+0000","last_clean":"2024-05-04T03:17:33.165921+0000","last_became_active":"2024-05-04T03:15:48.055155+0000","last_became_peered":"2024-05-04T03:15:48.055155+0000","last_unstale":"2024-05-04T03:17:33.165921+0000","last_undegraded":"2024-05-04T03:17:33.165921+0000","last_fullsized":"2024-05-04T03:17:33.165921+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:38:07.824031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.167129+0000","last_change":"2024-05-04T03:15:48.047974+0000","last_active":"2024-05-04T03:17:33.167129+0000","last_peered":"2024-05-04T03:17:33.167129+0000","last_clean":"2024-05-04T03:17:33.167129+0000","last_became_active":"2024-05-04T03:15:48.044299+0000","last_became_peered":"2024-05-04T03:15:48.044299+0000","last_unstale":"2024-05-04T03:17:33.167129+0000","last_undegraded":"2024-05-04T03:17:33.167129+0000","last_fullsized":"2024-05-04T03:17:33.167129+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:49:08.014987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.156171+0000","last_change":"2024-05-04T03:15:48.050899+0000","last_active":"2024-05-04T03:17:33.156171+0000","last_peered":"2024-05-04T03:17:33.156171+0000","last_clean":"2024-05-04T03:17:33.156171+0000","last_became_active":"2024-05-04T03:15:48.048812+0000","last_became_peered":"2024-05-04T03:15:48.048812+0000","last_unstale":"2024-05-04T03:17:33.156171+0000","last_undegraded":"2024-05-04T03:17:33.156171+0000","last_fullsized":"2024-05-04T03:17:33.156171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:21:01.829349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165098+0000","last_change":"2024-05-04T03:15:48.044403+0000","last_active":"2024-05-04T03:17:33.165098+0000","last_peered":"2024-05-04T03:17:33.165098+0000","last_clean":"2024-05-04T03:17:33.165098+0000","last_became_active":"2024-05-04T03:15:48.044021+0000","last_became_peered":"2024-05-04T03:15:48.044021+0000","last_unstale":"2024-05-04T03:17:33.165098+0000","last_undegraded":"2024-05-04T03:17:33.165098+0000","last_fullsized":"2024-05-04T03:17:33.165098+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:50:54.762070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.167642+0000","last_change":"2024-05-04T03:15:48.053233+0000","last_active":"2024-05-04T03:17:33.167642+0000","last_peered":"2024-05-04T03:17:33.167642+0000","last_clean":"2024-05-04T03:17:33.167642+0000","last_became_active":"2024-05-04T03:15:48.052809+0000","last_became_peered":"2024-05-04T03:15:48.052809+0000","last_unstale":"2024-05-04T03:17:33.167642+0000","last_undegraded":"2024-05-04T03:17:33.167642+0000","last_fullsized":"2024-05-04T03:17:33.167642+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:27:20.082694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.156722+0000","last_change":"2024-05-04T03:15:48.045108+0000","last_active":"2024-05-04T03:17:33.156722+0000","last_peered":"2024-05-04T03:17:33.156722+0000","last_clean":"2024-05-04T03:17:33.156722+0000","last_became_active":"2024-05-04T03:15:48.044719+0000","last_became_peered":"2024-05-04T03:15:48.044719+0000","last_unstale":"2024-05-04T03:17:33.156722+0000","last_undegraded":"2024-05-04T03:17:33.156722+0000","last_fullsized":"2024-05-04T03:17:33.156722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:55:02.586294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164990+0000","last_change":"2024-05-04T03:15:48.043559+0000","last_active":"2024-05-04T03:17:33.164990+0000","last_peered":"2024-05-04T03:17:33.164990+0000","last_clean":"2024-05-04T03:17:33.164990+0000","last_became_active":"2024-05-04T03:15:48.043138+0000","last_became_peered":"2024-05-04T03:15:48.043138+0000","last_unstale":"2024-05-04T03:17:33.164990+0000","last_undegraded":"2024-05-04T03:17:33.164990+0000","last_fullsized":"2024-05-04T03:17:33.164990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:05:25.411916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.174068+0000","last_change":"2024-05-04T03:15:48.039632+0000","last_active":"2024-05-04T03:17:33.174068+0000","last_peered":"2024-05-04T03:17:33.174068+0000","last_clean":"2024-05-04T03:17:33.174068+0000","last_became_active":"2024-05-04T03:15:48.039231+0000","last_became_peered":"2024-05-04T03:15:48.039231+0000","last_unstale":"2024-05-04T03:17:33.174068+0000","last_undegraded":"2024-05-04T03:17:33.174068+0000","last_fullsized":"2024-05-04T03:17:33.174068+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:33:12.853172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.168249+0000","last_change":"2024-05-04T03:15:48.055138+0000","last_active":"2024-05-04T03:17:33.168249+0000","last_peered":"2024-05-04T03:17:33.168249+0000","last_clean":"2024-05-04T03:17:33.168249+0000","last_became_active":"2024-05-04T03:15:48.054791+0000","last_became_peered":"2024-05-04T03:15:48.054791+0000","last_unstale":"2024-05-04T03:17:33.168249+0000","last_undegraded":"2024-05-04T03:17:33.168249+0000","last_fullsized":"2024-05-04T03:17:33.168249+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:33:36.822549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165773+0000","last_change":"2024-05-04T03:15:48.050125+0000","last_active":"2024-05-04T03:17:33.165773+0000","last_peered":"2024-05-04T03:17:33.165773+0000","last_clean":"2024-05-04T03:17:33.165773+0000","last_became_active":"2024-05-04T03:15:48.049390+0000","last_became_peered":"2024-05-04T03:15:48.049390+0000","last_unstale":"2024-05-04T03:17:33.165773+0000","last_undegraded":"2024-05-04T03:17:33.165773+0000","last_fullsized":"2024-05-04T03:17:33.165773+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:27:05.740383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.163474+0000","last_change":"2024-05-04T03:15:48.050700+0000","last_active":"2024-05-04T03:17:33.163474+0000","last_peered":"2024-05-04T03:17:33.163474+0000","last_clean":"2024-05-04T03:17:33.163474+0000","last_became_active":"2024-05-04T03:15:48.049288+0000","last_became_peered":"2024-05-04T03:15:48.049288+0000","last_unstale":"2024-05-04T03:17:33.163474+0000","last_undegraded":"2024-05-04T03:17:33.163474+0000","last_fullsized":"2024-05-04T03:17:33.163474+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:40:07.072006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.168343+0000","last_change":"2024-05-04T03:16:34.337123+0000","last_active":"2024-05-04T03:17:33.168343+0000","last_peered":"2024-05-04T03:17:33.168343+0000","last_clean":"2024-05-04T03:17:33.168343+0000","last_became_active":"2024-05-04T03:16:34.336657+0000","last_became_peered":"2024-05-04T03:16:34.336657+0000","last_unstale":"2024-05-04T03:17:33.168343+0000","last_undegraded":"2024-05-04T03:17:33.168343+0000","last_fullsized":"2024-05-04T03:17:33.168343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:58:15.939941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166400+0000","last_change":"2024-05-04T03:15:48.047258+0000","last_active":"2024-05-04T03:17:33.166400+0000","last_peered":"2024-05-04T03:17:33.166400+0000","last_clean":"2024-05-04T03:17:33.166400+0000","last_became_active":"2024-05-04T03:15:48.046897+0000","last_became_peered":"2024-05-04T03:15:48.046897+0000","last_unstale":"2024-05-04T03:17:33.166400+0000","last_undegraded":"2024-05-04T03:17:33.166400+0000","last_fullsized":"2024-05-04T03:17:33.166400+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:14:39.217475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166318+0000","last_change":"2024-05-04T03:15:48.051240+0000","last_active":"2024-05-04T03:17:33.166318+0000","last_peered":"2024-05-04T03:17:33.166318+0000","last_clean":"2024-05-04T03:17:33.166318+0000","last_became_active":"2024-05-04T03:15:48.050081+0000","last_became_peered":"2024-05-04T03:15:48.050081+0000","last_unstale":"2024-05-04T03:17:33.166318+0000","last_undegraded":"2024-05-04T03:17:33.166318+0000","last_fullsized":"2024-05-04T03:17:33.166318+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:39:06.773121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.171066+0000","last_change":"2024-05-04T03:15:48.050936+0000","last_active":"2024-05-04T03:17:33.171066+0000","last_peered":"2024-05-04T03:17:33.171066+0000","last_clean":"2024-05-04T03:17:33.171066+0000","last_became_active":"2024-05-04T03:15:48.050567+0000","last_became_peered":"2024-05-04T03:15:48.050567+0000","last_unstale":"2024-05-04T03:17:33.171066+0000","last_undegraded":"2024-05-04T03:17:33.171066+0000","last_fullsized":"2024-05-04T03:17:33.171066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:17:43.469889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.173267+0000","last_change":"2024-05-04T03:15:48.047200+0000","last_active":"2024-05-04T03:17:33.173267+0000","last_peered":"2024-05-04T03:17:33.173267+0000","last_clean":"2024-05-04T03:17:33.173267+0000","last_became_active":"2024-05-04T03:15:48.046782+0000","last_became_peered":"2024-05-04T03:15:48.046782+0000","last_unstale":"2024-05-04T03:17:33.173267+0000","last_undegraded":"2024-05-04T03:17:33.173267+0000","last_fullsized":"2024-05-04T03:17:33.173267+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:32:56.773916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.154784+0000","last_change":"2024-05-04T03:15:48.042114+0000","last_active":"2024-05-04T03:17:33.154784+0000","last_peered":"2024-05-04T03:17:33.154784+0000","last_clean":"2024-05-04T03:17:33.154784+0000","last_became_active":"2024-05-04T03:15:48.040921+0000","last_became_peered":"2024-05-04T03:15:48.040921+0000","last_unstale":"2024-05-04T03:17:33.154784+0000","last_undegraded":"2024-05-04T03:17:33.154784+0000","last_fullsized":"2024-05-04T03:17:33.154784+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:16:20.130996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164439+0000","last_change":"2024-05-04T03:15:48.051060+0000","last_active":"2024-05-04T03:17:33.164439+0000","last_peered":"2024-05-04T03:17:33.164439+0000","last_clean":"2024-05-04T03:17:33.164439+0000","last_became_active":"2024-05-04T03:15:48.050642+0000","last_became_peered":"2024-05-04T03:15:48.050642+0000","last_unstale":"2024-05-04T03:17:33.164439+0000","last_undegraded":"2024-05-04T03:17:33.164439+0000","last_fullsized":"2024-05-04T03:17:33.164439+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:23:31.193072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.329355+0000","last_change":"2024-05-04T03:15:45.013477+0000","last_active":"2024-05-04T03:16:34.329355+0000","last_peered":"2024-05-04T03:16:34.329355+0000","last_clean":"2024-05-04T03:16:34.329355+0000","last_became_active":"2024-05-04T03:15:45.013161+0000","last_became_peered":"2024-05-04T03:15:45.013161+0000","last_unstale":"2024-05-04T03:16:34.329355+0000","last_undegraded":"2024-05-04T03:16:34.329355+0000","last_fullsized":"2024-05-04T03:16:34.329355+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:08:14.087090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164546+0000","last_change":"2024-05-04T03:15:48.045810+0000","last_active":"2024-05-04T03:17:33.164546+0000","last_peered":"2024-05-04T03:17:33.164546+0000","last_clean":"2024-05-04T03:17:33.164546+0000","last_became_active":"2024-05-04T03:15:48.045449+0000","last_became_peered":"2024-05-04T03:15:48.045449+0000","last_unstale":"2024-05-04T03:17:33.164546+0000","last_undegraded":"2024-05-04T03:17:33.164546+0000","last_fullsized":"2024-05-04T03:17:33.164546+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:48:05.084208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.417032+0000","last_change":"2024-05-04T03:15:45.204196+0000","last_active":"2024-05-04T03:16:34.417032+0000","last_peered":"2024-05-04T03:16:34.417032+0000","last_clean":"2024-05-04T03:16:34.417032+0000","last_became_active":"2024-05-04T03:15:45.203821+0000","last_became_peered":"2024-05-04T03:15:45.203821+0000","last_unstale":"2024-05-04T03:16:34.417032+0000","last_undegraded":"2024-05-04T03:16:34.417032+0000","last_fullsized":"2024-05-04T03:16:34.417032+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:17:05.074450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.167259+0000","last_change":"2024-05-04T03:15:48.045131+0000","last_active":"2024-05-04T03:17:33.167259+0000","last_peered":"2024-05-04T03:17:33.167259+0000","last_clean":"2024-05-04T03:17:33.167259+0000","last_became_active":"2024-05-04T03:15:48.044793+0000","last_became_peered":"2024-05-04T03:15:48.044793+0000","last_unstale":"2024-05-04T03:17:33.167259+0000","last_undegraded":"2024-05-04T03:17:33.167259+0000","last_fullsized":"2024-05-04T03:17:33.167259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:03:37.769324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334752+0000","last_change":"2024-05-04T03:15:45.013644+0000","last_active":"2024-05-04T03:16:34.334752+0000","last_peered":"2024-05-04T03:16:34.334752+0000","last_clean":"2024-05-04T03:16:34.334752+0000","last_became_active":"2024-05-04T03:15:45.013282+0000","last_became_peered":"2024-05-04T03:15:45.013282+0000","last_unstale":"2024-05-04T03:16:34.334752+0000","last_undegraded":"2024-05-04T03:16:34.334752+0000","last_fullsized":"2024-05-04T03:16:34.334752+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:21:21.204433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.174582+0000","last_change":"2024-05-04T03:15:48.043046+0000","last_active":"2024-05-04T03:17:33.174582+0000","last_peered":"2024-05-04T03:17:33.174582+0000","last_clean":"2024-05-04T03:17:33.174582+0000","last_became_active":"2024-05-04T03:15:48.042621+0000","last_became_peered":"2024-05-04T03:15:48.042621+0000","last_unstale":"2024-05-04T03:17:33.174582+0000","last_undegraded":"2024-05-04T03:17:33.174582+0000","last_fullsized":"2024-05-04T03:17:33.174582+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:03:58.916423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334940+0000","last_change":"2024-05-04T03:15:45.010618+0000","last_active":"2024-05-04T03:16:34.334940+0000","last_peered":"2024-05-04T03:16:34.334940+0000","last_clean":"2024-05-04T03:16:34.334940+0000","last_became_active":"2024-05-04T03:15:45.009239+0000","last_became_peered":"2024-05-04T03:15:45.009239+0000","last_unstale":"2024-05-04T03:16:34.334940+0000","last_undegraded":"2024-05-04T03:16:34.334940+0000","last_fullsized":"2024-05-04T03:16:34.334940+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:49:46.777571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.145981+0000","last_change":"2024-05-04T03:15:48.049334+0000","last_active":"2024-05-04T03:17:33.145981+0000","last_peered":"2024-05-04T03:17:33.145981+0000","last_clean":"2024-05-04T03:17:33.145981+0000","last_became_active":"2024-05-04T03:15:48.048984+0000","last_became_peered":"2024-05-04T03:15:48.048984+0000","last_unstale":"2024-05-04T03:17:33.145981+0000","last_undegraded":"2024-05-04T03:17:33.145981+0000","last_fullsized":"2024-05-04T03:17:33.145981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:28:25.366106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":194,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:18:43.505278+0000","last_change":"2024-05-04T03:15:45.015277+0000","last_active":"2024-05-04T03:18:43.505278+0000","last_peered":"2024-05-04T03:18:43.505278+0000","last_clean":"2024-05-04T03:18:43.505278+0000","last_became_active":"2024-05-04T03:15:45.014542+0000","last_became_peered":"2024-05-04T03:15:45.014542+0000","last_unstale":"2024-05-04T03:18:43.505278+0000","last_undegraded":"2024-05-04T03:18:43.505278+0000","last_fullsized":"2024-05-04T03:18:43.505278+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:22:15.594866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":161,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.147597+0000","last_change":"2024-05-04T03:15:48.045029+0000","last_active":"2024-05-04T03:17:33.147597+0000","last_peered":"2024-05-04T03:17:33.147597+0000","last_clean":"2024-05-04T03:17:33.147597+0000","last_became_active":"2024-05-04T03:15:48.041262+0000","last_became_peered":"2024-05-04T03:15:48.041262+0000","last_unstale":"2024-05-04T03:17:33.147597+0000","last_undegraded":"2024-05-04T03:17:33.147597+0000","last_fullsized":"2024-05-04T03:17:33.147597+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:45:56.882957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334215+0000","last_change":"2024-05-04T03:15:45.007530+0000","last_active":"2024-05-04T03:16:34.334215+0000","last_peered":"2024-05-04T03:16:34.334215+0000","last_clean":"2024-05-04T03:16:34.334215+0000","last_became_active":"2024-05-04T03:15:45.007054+0000","last_became_peered":"2024-05-04T03:15:45.007054+0000","last_unstale":"2024-05-04T03:16:34.334215+0000","last_undegraded":"2024-05-04T03:16:34.334215+0000","last_fullsized":"2024-05-04T03:16:34.334215+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:09:13.845679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.142714+0000","last_change":"2024-05-04T03:15:48.050956+0000","last_active":"2024-05-04T03:17:33.142714+0000","last_peered":"2024-05-04T03:17:33.142714+0000","last_clean":"2024-05-04T03:17:33.142714+0000","last_became_active":"2024-05-04T03:15:48.050597+0000","last_became_peered":"2024-05-04T03:15:48.050597+0000","last_unstale":"2024-05-04T03:17:33.142714+0000","last_undegraded":"2024-05-04T03:17:33.142714+0000","last_fullsized":"2024-05-04T03:17:33.142714+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:27:34.127889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.331600+0000","last_change":"2024-05-04T03:15:45.007027+0000","last_active":"2024-05-04T03:16:34.331600+0000","last_peered":"2024-05-04T03:16:34.331600+0000","last_clean":"2024-05-04T03:16:34.331600+0000","last_became_active":"2024-05-04T03:15:45.006659+0000","last_became_peered":"2024-05-04T03:15:45.006659+0000","last_unstale":"2024-05-04T03:16:34.331600+0000","last_undegraded":"2024-05-04T03:16:34.331600+0000","last_fullsized":"2024-05-04T03:16:34.331600+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:06:11.827093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164377+0000","last_change":"2024-05-04T03:15:48.046196+0000","last_active":"2024-05-04T03:17:33.164377+0000","last_peered":"2024-05-04T03:17:33.164377+0000","last_clean":"2024-05-04T03:17:33.164377+0000","last_became_active":"2024-05-04T03:15:48.044316+0000","last_became_peered":"2024-05-04T03:15:48.044316+0000","last_unstale":"2024-05-04T03:17:33.164377+0000","last_undegraded":"2024-05-04T03:17:33.164377+0000","last_fullsized":"2024-05-04T03:17:33.164377+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:04:39.301705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.416682+0000","last_change":"2024-05-04T03:15:45.015360+0000","last_active":"2024-05-04T03:16:34.416682+0000","last_peered":"2024-05-04T03:16:34.416682+0000","last_clean":"2024-05-04T03:16:34.416682+0000","last_became_active":"2024-05-04T03:15:45.014696+0000","last_became_peered":"2024-05-04T03:15:45.014696+0000","last_unstale":"2024-05-04T03:16:34.416682+0000","last_undegraded":"2024-05-04T03:16:34.416682+0000","last_fullsized":"2024-05-04T03:16:34.416682+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:17:28.838651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.163052+0000","last_change":"2024-05-04T03:15:48.039823+0000","last_active":"2024-05-04T03:17:33.163052+0000","last_peered":"2024-05-04T03:17:33.163052+0000","last_clean":"2024-05-04T03:17:33.163052+0000","last_became_active":"2024-05-04T03:15:48.039447+0000","last_became_peered":"2024-05-04T03:15:48.039447+0000","last_unstale":"2024-05-04T03:17:33.163052+0000","last_undegraded":"2024-05-04T03:17:33.163052+0000","last_fullsized":"2024-05-04T03:17:33.163052+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:45:55.136946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":188,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:18:41.501396+0000","last_change":"2024-05-04T03:15:45.004662+0000","last_active":"2024-05-04T03:18:41.501396+0000","last_peered":"2024-05-04T03:18:41.501396+0000","last_clean":"2024-05-04T03:18:41.501396+0000","last_became_active":"2024-05-04T03:15:45.004265+0000","last_became_peered":"2024-05-04T03:15:45.004265+0000","last_unstale":"2024-05-04T03:18:41.501396+0000","last_undegraded":"2024-05-04T03:18:41.501396+0000","last_fullsized":"2024-05-04T03:18:41.501396+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:14:03.065346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":158,"num_read_kb":158,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.173995+0000","last_change":"2024-05-04T03:15:48.042093+0000","last_active":"2024-05-04T03:17:33.173995+0000","last_peered":"2024-05-04T03:17:33.173995+0000","last_clean":"2024-05-04T03:17:33.173995+0000","last_became_active":"2024-05-04T03:15:48.041646+0000","last_became_peered":"2024-05-04T03:15:48.041646+0000","last_unstale":"2024-05-04T03:17:33.173995+0000","last_undegraded":"2024-05-04T03:17:33.173995+0000","last_fullsized":"2024-05-04T03:17:33.173995+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:06:32.083493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.335070+0000","last_change":"2024-05-04T03:15:45.016731+0000","last_active":"2024-05-04T03:16:34.335070+0000","last_peered":"2024-05-04T03:16:34.335070+0000","last_clean":"2024-05-04T03:16:34.335070+0000","last_became_active":"2024-05-04T03:15:45.016313+0000","last_became_peered":"2024-05-04T03:15:45.016313+0000","last_unstale":"2024-05-04T03:16:34.335070+0000","last_undegraded":"2024-05-04T03:16:34.335070+0000","last_fullsized":"2024-05-04T03:16:34.335070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:42:40.830986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166326+0000","last_change":"2024-05-04T03:15:48.053649+0000","last_active":"2024-05-04T03:17:33.166326+0000","last_peered":"2024-05-04T03:17:33.166326+0000","last_clean":"2024-05-04T03:17:33.166326+0000","last_became_active":"2024-05-04T03:15:48.052407+0000","last_became_peered":"2024-05-04T03:15:48.052407+0000","last_unstale":"2024-05-04T03:17:33.166326+0000","last_undegraded":"2024-05-04T03:17:33.166326+0000","last_fullsized":"2024-05-04T03:17:33.166326+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:37:28.359272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.417301+0000","last_change":"2024-05-04T03:15:45.205180+0000","last_active":"2024-05-04T03:16:34.417301+0000","last_peered":"2024-05-04T03:16:34.417301+0000","last_clean":"2024-05-04T03:16:34.417301+0000","last_became_active":"2024-05-04T03:15:45.204828+0000","last_became_peered":"2024-05-04T03:15:45.204828+0000","last_unstale":"2024-05-04T03:16:34.417301+0000","last_undegraded":"2024-05-04T03:16:34.417301+0000","last_fullsized":"2024-05-04T03:16:34.417301+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:27:34.063540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.171291+0000","last_change":"2024-05-04T03:15:48.047154+0000","last_active":"2024-05-04T03:17:33.171291+0000","last_peered":"2024-05-04T03:17:33.171291+0000","last_clean":"2024-05-04T03:17:33.171291+0000","last_became_active":"2024-05-04T03:15:48.046794+0000","last_became_peered":"2024-05-04T03:15:48.046794+0000","last_unstale":"2024-05-04T03:17:33.171291+0000","last_undegraded":"2024-05-04T03:17:33.171291+0000","last_fullsized":"2024-05-04T03:17:33.171291+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:21:42.723252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'35","reported_seq":388,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:18:42.503141+0000","last_change":"2024-05-04T03:15:45.008930+0000","last_active":"2024-05-04T03:18:42.503141+0000","last_peered":"2024-05-04T03:18:42.503141+0000","last_clean":"2024-05-04T03:18:42.503141+0000","last_became_active":"2024-05-04T03:15:45.008459+0000","last_became_peered":"2024-05-04T03:15:45.008459+0000","last_unstale":"2024-05-04T03:18:42.503141+0000","last_undegraded":"2024-05-04T03:18:42.503141+0000","last_fullsized":"2024-05-04T03:18:42.503141+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:07:52.609156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66381,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":326,"num_read_kb":228,"num_write":36,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.143904+0000","last_change":"2024-05-04T03:15:48.051848+0000","last_active":"2024-05-04T03:17:33.143904+0000","last_peered":"2024-05-04T03:17:33.143904+0000","last_clean":"2024-05-04T03:17:33.143904+0000","last_became_active":"2024-05-04T03:15:48.051171+0000","last_became_peered":"2024-05-04T03:15:48.051171+0000","last_unstale":"2024-05-04T03:17:33.143904+0000","last_undegraded":"2024-05-04T03:17:33.143904+0000","last_fullsized":"2024-05-04T03:17:33.143904+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:20:32.722845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.330397+0000","last_change":"2024-05-04T03:15:45.007970+0000","last_active":"2024-05-04T03:16:34.330397+0000","last_peered":"2024-05-04T03:16:34.330397+0000","last_clean":"2024-05-04T03:16:34.330397+0000","last_became_active":"2024-05-04T03:15:45.007605+0000","last_became_peered":"2024-05-04T03:15:45.007605+0000","last_unstale":"2024-05-04T03:16:34.330397+0000","last_undegraded":"2024-05-04T03:16:34.330397+0000","last_fullsized":"2024-05-04T03:16:34.330397+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:52:08.466858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.169073+0000","last_change":"2024-05-04T03:15:48.050647+0000","last_active":"2024-05-04T03:17:33.169073+0000","last_peered":"2024-05-04T03:17:33.169073+0000","last_clean":"2024-05-04T03:17:33.169073+0000","last_became_active":"2024-05-04T03:15:48.050296+0000","last_became_peered":"2024-05-04T03:15:48.050296+0000","last_unstale":"2024-05-04T03:17:33.169073+0000","last_undegraded":"2024-05-04T03:17:33.169073+0000","last_fullsized":"2024-05-04T03:17:33.169073+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:14:23.902094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.416945+0000","last_change":"2024-05-04T03:15:45.014936+0000","last_active":"2024-05-04T03:16:34.416945+0000","last_peered":"2024-05-04T03:16:34.416945+0000","last_clean":"2024-05-04T03:16:34.416945+0000","last_became_active":"2024-05-04T03:15:45.014474+0000","last_became_peered":"2024-05-04T03:15:45.014474+0000","last_unstale":"2024-05-04T03:16:34.416945+0000","last_undegraded":"2024-05-04T03:16:34.416945+0000","last_fullsized":"2024-05-04T03:16:34.416945+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:26:37.972523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.169574+0000","last_change":"2024-05-04T03:16:34.336836+0000","last_active":"2024-05-04T03:17:33.169574+0000","last_peered":"2024-05-04T03:17:33.169574+0000","last_clean":"2024-05-04T03:17:33.169574+0000","last_became_active":"2024-05-04T03:16:34.336008+0000","last_became_peered":"2024-05-04T03:16:34.336008+0000","last_unstale":"2024-05-04T03:17:33.169574+0000","last_undegraded":"2024-05-04T03:17:33.169574+0000","last_fullsized":"2024-05-04T03:17:33.169574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:51:52.803138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'5","reported_seq":195,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:18:42.502794+0000","last_change":"2024-05-04T03:15:45.013177+0000","last_active":"2024-05-04T03:18:42.502794+0000","last_peered":"2024-05-04T03:18:42.502794+0000","last_clean":"2024-05-04T03:18:42.502794+0000","last_became_active":"2024-05-04T03:15:45.012764+0000","last_became_peered":"2024-05-04T03:15:45.012764+0000","last_unstale":"2024-05-04T03:18:42.502794+0000","last_undegraded":"2024-05-04T03:18:42.502794+0000","last_fullsized":"2024-05-04T03:18:42.502794+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:07:43.578443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":164,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.159654+0000","last_change":"2024-05-04T03:15:48.046453+0000","last_active":"2024-05-04T03:17:33.159654+0000","last_peered":"2024-05-04T03:17:33.159654+0000","last_clean":"2024-05-04T03:17:33.159654+0000","last_became_active":"2024-05-04T03:15:48.045990+0000","last_became_peered":"2024-05-04T03:15:48.045990+0000","last_unstale":"2024-05-04T03:17:33.159654+0000","last_undegraded":"2024-05-04T03:17:33.159654+0000","last_fullsized":"2024-05-04T03:17:33.159654+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:54:57.285122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334414+0000","last_change":"2024-05-04T03:15:45.205539+0000","last_active":"2024-05-04T03:16:34.334414+0000","last_peered":"2024-05-04T03:16:34.334414+0000","last_clean":"2024-05-04T03:16:34.334414+0000","last_became_active":"2024-05-04T03:15:45.205174+0000","last_became_peered":"2024-05-04T03:15:45.205174+0000","last_unstale":"2024-05-04T03:16:34.334414+0000","last_undegraded":"2024-05-04T03:16:34.334414+0000","last_fullsized":"2024-05-04T03:16:34.334414+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:03:53.398368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.141832+0000","last_change":"2024-05-04T03:15:48.051824+0000","last_active":"2024-05-04T03:17:33.141832+0000","last_peered":"2024-05-04T03:17:33.141832+0000","last_clean":"2024-05-04T03:17:33.141832+0000","last_became_active":"2024-05-04T03:15:48.051483+0000","last_became_peered":"2024-05-04T03:15:48.051483+0000","last_unstale":"2024-05-04T03:17:33.141832+0000","last_undegraded":"2024-05-04T03:17:33.141832+0000","last_fullsized":"2024-05-04T03:17:33.141832+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:57:56.644486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334847+0000","last_change":"2024-05-04T03:15:45.011062+0000","last_active":"2024-05-04T03:16:34.334847+0000","last_peered":"2024-05-04T03:16:34.334847+0000","last_clean":"2024-05-04T03:16:34.334847+0000","last_became_active":"2024-05-04T03:15:45.010730+0000","last_became_peered":"2024-05-04T03:15:45.010730+0000","last_unstale":"2024-05-04T03:16:34.334847+0000","last_undegraded":"2024-05-04T03:16:34.334847+0000","last_fullsized":"2024-05-04T03:16:34.334847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:45:55.979067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.167833+0000","last_change":"2024-05-04T03:15:48.045965+0000","last_active":"2024-05-04T03:17:33.167833+0000","last_peered":"2024-05-04T03:17:33.167833+0000","last_clean":"2024-05-04T03:17:33.167833+0000","last_became_active":"2024-05-04T03:15:48.045615+0000","last_became_peered":"2024-05-04T03:15:48.045615+0000","last_unstale":"2024-05-04T03:17:33.167833+0000","last_undegraded":"2024-05-04T03:17:33.167833+0000","last_fullsized":"2024-05-04T03:17:33.167833+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:14:23.497484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.329258+0000","last_change":"2024-05-04T03:15:45.204223+0000","last_active":"2024-05-04T03:16:34.329258+0000","last_peered":"2024-05-04T03:16:34.329258+0000","last_clean":"2024-05-04T03:16:34.329258+0000","last_became_active":"2024-05-04T03:15:45.203774+0000","last_became_peered":"2024-05-04T03:15:45.203774+0000","last_unstale":"2024-05-04T03:16:34.329258+0000","last_undegraded":"2024-05-04T03:16:34.329258+0000","last_fullsized":"2024-05-04T03:16:34.329258+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:02:34.456055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.167144+0000","last_change":"2024-05-04T03:15:48.045574+0000","last_active":"2024-05-04T03:17:33.167144+0000","last_peered":"2024-05-04T03:17:33.167144+0000","last_clean":"2024-05-04T03:17:33.167144+0000","last_became_active":"2024-05-04T03:15:48.045237+0000","last_became_peered":"2024-05-04T03:15:48.045237+0000","last_unstale":"2024-05-04T03:17:33.167144+0000","last_undegraded":"2024-05-04T03:17:33.167144+0000","last_fullsized":"2024-05-04T03:17:33.167144+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:53:46.346199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.333946+0000","last_change":"2024-05-04T03:15:45.013895+0000","last_active":"2024-05-04T03:16:34.333946+0000","last_peered":"2024-05-04T03:16:34.333946+0000","last_clean":"2024-05-04T03:16:34.333946+0000","last_became_active":"2024-05-04T03:15:45.013306+0000","last_became_peered":"2024-05-04T03:15:45.013306+0000","last_unstale":"2024-05-04T03:16:34.333946+0000","last_undegraded":"2024-05-04T03:16:34.333946+0000","last_fullsized":"2024-05-04T03:16:34.333946+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:39:18.616432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.158996+0000","last_change":"2024-05-04T03:16:34.336997+0000","last_active":"2024-05-04T03:17:33.158996+0000","last_peered":"2024-05-04T03:17:33.158996+0000","last_clean":"2024-05-04T03:17:33.158996+0000","last_became_active":"2024-05-04T03:16:34.336505+0000","last_became_peered":"2024-05-04T03:16:34.336505+0000","last_unstale":"2024-05-04T03:17:33.158996+0000","last_undegraded":"2024-05-04T03:17:33.158996+0000","last_fullsized":"2024-05-04T03:17:33.158996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:23:38.979216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.149672+0000","last_change":"2024-05-04T03:15:48.050122+0000","last_active":"2024-05-04T03:17:33.149672+0000","last_peered":"2024-05-04T03:17:33.149672+0000","last_clean":"2024-05-04T03:17:33.149672+0000","last_became_active":"2024-05-04T03:15:48.049794+0000","last_became_peered":"2024-05-04T03:15:48.049794+0000","last_unstale":"2024-05-04T03:17:33.149672+0000","last_undegraded":"2024-05-04T03:17:33.149672+0000","last_fullsized":"2024-05-04T03:17:33.149672+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:29:11.066068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":190,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:18:41.500735+0000","last_change":"2024-05-04T03:15:45.205143+0000","last_active":"2024-05-04T03:18:41.500735+0000","last_peered":"2024-05-04T03:18:41.500735+0000","last_clean":"2024-05-04T03:18:41.500735+0000","last_became_active":"2024-05-04T03:15:45.204727+0000","last_became_peered":"2024-05-04T03:15:45.204727+0000","last_unstale":"2024-05-04T03:18:41.500735+0000","last_undegraded":"2024-05-04T03:18:41.500735+0000","last_fullsized":"2024-05-04T03:18:41.500735+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:15:05.799556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":158,"num_read_kb":158,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.163750+0000","last_change":"2024-05-04T03:15:48.042914+0000","last_active":"2024-05-04T03:17:33.163750+0000","last_peered":"2024-05-04T03:17:33.163750+0000","last_clean":"2024-05-04T03:17:33.163750+0000","last_became_active":"2024-05-04T03:15:48.041936+0000","last_became_peered":"2024-05-04T03:15:48.041936+0000","last_unstale":"2024-05-04T03:17:33.163750+0000","last_undegraded":"2024-05-04T03:17:33.163750+0000","last_fullsized":"2024-05-04T03:17:33.163750+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:11:48.807602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.331287+0000","last_change":"2024-05-04T03:15:45.011045+0000","last_active":"2024-05-04T03:16:34.331287+0000","last_peered":"2024-05-04T03:16:34.331287+0000","last_clean":"2024-05-04T03:16:34.331287+0000","last_became_active":"2024-05-04T03:15:45.010608+0000","last_became_peered":"2024-05-04T03:15:45.010608+0000","last_unstale":"2024-05-04T03:16:34.331287+0000","last_undegraded":"2024-05-04T03:16:34.331287+0000","last_fullsized":"2024-05-04T03:16:34.331287+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:00:36.243588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":102,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.333935+0000","last_change":"2024-05-04T03:14:19.895070+0000","last_active":"2024-05-04T03:16:34.333935+0000","last_peered":"2024-05-04T03:16:34.333935+0000","last_clean":"2024-05-04T03:16:34.333935+0000","last_became_active":"2024-05-04T03:14:19.889987+0000","last_became_peered":"2024-05-04T03:14:19.889987+0000","last_unstale":"2024-05-04T03:16:34.333935+0000","last_undegraded":"2024-05-04T03:16:34.333935+0000","last_fullsized":"2024-05-04T03:16:34.333935+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:11:01.712629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:11:01.712629+0000","last_clean_scrub_stamp":"2024-05-04T03:11:01.712629+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:03:02.769719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166064+0000","last_change":"2024-05-04T03:15:48.044928+0000","last_active":"2024-05-04T03:17:33.166064+0000","last_peered":"2024-05-04T03:17:33.166064+0000","last_clean":"2024-05-04T03:17:33.166064+0000","last_became_active":"2024-05-04T03:15:48.044532+0000","last_became_peered":"2024-05-04T03:15:48.044532+0000","last_unstale":"2024-05-04T03:17:33.166064+0000","last_undegraded":"2024-05-04T03:17:33.166064+0000","last_fullsized":"2024-05-04T03:17:33.166064+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:27:00.825692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334103+0000","last_change":"2024-05-04T03:15:45.014469+0000","last_active":"2024-05-04T03:16:34.334103+0000","last_peered":"2024-05-04T03:16:34.334103+0000","last_clean":"2024-05-04T03:16:34.334103+0000","last_became_active":"2024-05-04T03:15:45.013763+0000","last_became_peered":"2024-05-04T03:15:45.013763+0000","last_unstale":"2024-05-04T03:16:34.334103+0000","last_undegraded":"2024-05-04T03:16:34.334103+0000","last_fullsized":"2024-05-04T03:16:34.334103+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:19:36.379930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334623+0000","last_change":"2024-05-04T03:15:45.204536+0000","last_active":"2024-05-04T03:16:34.334623+0000","last_peered":"2024-05-04T03:16:34.334623+0000","last_clean":"2024-05-04T03:16:34.334623+0000","last_became_active":"2024-05-04T03:15:45.204072+0000","last_became_peered":"2024-05-04T03:15:45.204072+0000","last_unstale":"2024-05-04T03:16:34.334623+0000","last_undegraded":"2024-05-04T03:16:34.334623+0000","last_fullsized":"2024-05-04T03:16:34.334623+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:49:12.726220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.162922+0000","last_change":"2024-05-04T03:15:48.044982+0000","last_active":"2024-05-04T03:17:33.162922+0000","last_peered":"2024-05-04T03:17:33.162922+0000","last_clean":"2024-05-04T03:17:33.162922+0000","last_became_active":"2024-05-04T03:15:48.043996+0000","last_became_peered":"2024-05-04T03:15:48.043996+0000","last_unstale":"2024-05-04T03:17:33.162922+0000","last_undegraded":"2024-05-04T03:17:33.162922+0000","last_fullsized":"2024-05-04T03:17:33.162922+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:09:39.770023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'11","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:34.057552+0000","last_change":"2024-05-04T03:15:45.008153+0000","last_active":"2024-05-04T03:17:34.057552+0000","last_peered":"2024-05-04T03:17:34.057552+0000","last_clean":"2024-05-04T03:17:34.057552+0000","last_became_active":"2024-05-04T03:15:45.007698+0000","last_became_peered":"2024-05-04T03:15:45.007698+0000","last_unstale":"2024-05-04T03:17:34.057552+0000","last_undegraded":"2024-05-04T03:17:34.057552+0000","last_fullsized":"2024-05-04T03:17:34.057552+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:56:52.322049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.168750+0000","last_change":"2024-05-04T03:15:48.055767+0000","last_active":"2024-05-04T03:17:33.168750+0000","last_peered":"2024-05-04T03:17:33.168750+0000","last_clean":"2024-05-04T03:17:33.168750+0000","last_became_active":"2024-05-04T03:15:48.055252+0000","last_became_peered":"2024-05-04T03:15:48.055252+0000","last_unstale":"2024-05-04T03:17:33.168750+0000","last_undegraded":"2024-05-04T03:17:33.168750+0000","last_fullsized":"2024-05-04T03:17:33.168750+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:28:11.736803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.331539+0000","last_change":"2024-05-04T03:15:45.012838+0000","last_active":"2024-05-04T03:16:34.331539+0000","last_peered":"2024-05-04T03:16:34.331539+0000","last_clean":"2024-05-04T03:16:34.331539+0000","last_became_active":"2024-05-04T03:15:45.012429+0000","last_became_peered":"2024-05-04T03:15:45.012429+0000","last_unstale":"2024-05-04T03:16:34.331539+0000","last_undegraded":"2024-05-04T03:16:34.331539+0000","last_fullsized":"2024-05-04T03:16:34.331539+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:22:22.602551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165546+0000","last_change":"2024-05-04T03:15:48.056993+0000","last_active":"2024-05-04T03:17:33.165546+0000","last_peered":"2024-05-04T03:17:33.165546+0000","last_clean":"2024-05-04T03:17:33.165546+0000","last_became_active":"2024-05-04T03:15:48.055938+0000","last_became_peered":"2024-05-04T03:15:48.055938+0000","last_unstale":"2024-05-04T03:17:33.165546+0000","last_undegraded":"2024-05-04T03:17:33.165546+0000","last_fullsized":"2024-05-04T03:17:33.165546+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:23:59.521692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.340536+0000","last_change":"2024-05-04T03:16:34.340048+0000","last_active":"2024-05-04T03:16:34.340536+0000","last_peered":"2024-05-04T03:16:34.340536+0000","last_clean":"2024-05-04T03:16:34.340536+0000","last_became_active":"2024-05-04T03:16:34.336386+0000","last_became_peered":"2024-05-04T03:16:34.336386+0000","last_unstale":"2024-05-04T03:16:34.340536+0000","last_undegraded":"2024-05-04T03:16:34.340536+0000","last_fullsized":"2024-05-04T03:16:34.340536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:45:43.734097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.157539+0000","last_change":"2024-05-04T03:15:48.043614+0000","last_active":"2024-05-04T03:17:33.157539+0000","last_peered":"2024-05-04T03:17:33.157539+0000","last_clean":"2024-05-04T03:17:33.157539+0000","last_became_active":"2024-05-04T03:15:48.042390+0000","last_became_peered":"2024-05-04T03:15:48.042390+0000","last_unstale":"2024-05-04T03:17:33.157539+0000","last_undegraded":"2024-05-04T03:17:33.157539+0000","last_fullsized":"2024-05-04T03:17:33.157539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:56:22.791368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.416585+0000","last_change":"2024-05-04T03:15:45.005570+0000","last_active":"2024-05-04T03:16:34.416585+0000","last_peered":"2024-05-04T03:16:34.416585+0000","last_clean":"2024-05-04T03:16:34.416585+0000","last_became_active":"2024-05-04T03:15:45.005105+0000","last_became_peered":"2024-05-04T03:15:45.005105+0000","last_unstale":"2024-05-04T03:16:34.416585+0000","last_undegraded":"2024-05-04T03:16:34.416585+0000","last_fullsized":"2024-05-04T03:16:34.416585+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:40:17.275618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166899+0000","last_change":"2024-05-04T03:15:48.045081+0000","last_active":"2024-05-04T03:17:33.166899+0000","last_peered":"2024-05-04T03:17:33.166899+0000","last_clean":"2024-05-04T03:17:33.166899+0000","last_became_active":"2024-05-04T03:15:48.044745+0000","last_became_peered":"2024-05-04T03:15:48.044745+0000","last_unstale":"2024-05-04T03:17:33.166899+0000","last_undegraded":"2024-05-04T03:17:33.166899+0000","last_fullsized":"2024-05-04T03:17:33.166899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:40:53.486222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:56.748549+0000","last_change":"2024-05-04T03:15:45.008926+0000","last_active":"2024-05-04T03:17:56.748549+0000","last_peered":"2024-05-04T03:17:56.748549+0000","last_clean":"2024-05-04T03:17:56.748549+0000","last_became_active":"2024-05-04T03:15:45.008412+0000","last_became_peered":"2024-05-04T03:15:45.008412+0000","last_unstale":"2024-05-04T03:17:56.748549+0000","last_undegraded":"2024-05-04T03:17:56.748549+0000","last_fullsized":"2024-05-04T03:17:56.748549+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:34:33.107866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164466+0000","last_change":"2024-05-04T03:15:48.054580+0000","last_active":"2024-05-04T03:17:33.164466+0000","last_peered":"2024-05-04T03:17:33.164466+0000","last_clean":"2024-05-04T03:17:33.164466+0000","last_became_active":"2024-05-04T03:15:48.054242+0000","last_became_peered":"2024-05-04T03:15:48.054242+0000","last_unstale":"2024-05-04T03:17:33.164466+0000","last_undegraded":"2024-05-04T03:17:33.164466+0000","last_fullsized":"2024-05-04T03:17:33.164466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:16:19.426571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.341316+0000","last_change":"2024-05-04T03:16:34.340794+0000","last_active":"2024-05-04T03:16:34.341316+0000","last_peered":"2024-05-04T03:16:34.341316+0000","last_clean":"2024-05-04T03:16:34.341316+0000","last_became_active":"2024-05-04T03:16:34.336038+0000","last_became_peered":"2024-05-04T03:16:34.336038+0000","last_unstale":"2024-05-04T03:16:34.341316+0000","last_undegraded":"2024-05-04T03:16:34.341316+0000","last_fullsized":"2024-05-04T03:16:34.341316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:45:19.323938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.172190+0000","last_change":"2024-05-04T03:15:48.045284+0000","last_active":"2024-05-04T03:17:33.172190+0000","last_peered":"2024-05-04T03:17:33.172190+0000","last_clean":"2024-05-04T03:17:33.172190+0000","last_became_active":"2024-05-04T03:15:48.044673+0000","last_became_peered":"2024-05-04T03:15:48.044673+0000","last_unstale":"2024-05-04T03:17:33.172190+0000","last_undegraded":"2024-05-04T03:17:33.172190+0000","last_fullsized":"2024-05-04T03:17:33.172190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:51:52.389748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.330835+0000","last_change":"2024-05-04T03:15:45.205648+0000","last_active":"2024-05-04T03:16:34.330835+0000","last_peered":"2024-05-04T03:16:34.330835+0000","last_clean":"2024-05-04T03:16:34.330835+0000","last_became_active":"2024-05-04T03:15:45.204169+0000","last_became_peered":"2024-05-04T03:15:45.204169+0000","last_unstale":"2024-05-04T03:16:34.330835+0000","last_undegraded":"2024-05-04T03:16:34.330835+0000","last_fullsized":"2024-05-04T03:16:34.330835+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:20:48.859276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.171852+0000","last_change":"2024-05-04T03:15:48.049808+0000","last_active":"2024-05-04T03:17:33.171852+0000","last_peered":"2024-05-04T03:17:33.171852+0000","last_clean":"2024-05-04T03:17:33.171852+0000","last_became_active":"2024-05-04T03:15:48.048998+0000","last_became_peered":"2024-05-04T03:15:48.048998+0000","last_unstale":"2024-05-04T03:17:33.171852+0000","last_undegraded":"2024-05-04T03:17:33.171852+0000","last_fullsized":"2024-05-04T03:17:33.171852+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:54:46.786406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.417121+0000","last_change":"2024-05-04T03:15:45.008874+0000","last_active":"2024-05-04T03:16:34.417121+0000","last_peered":"2024-05-04T03:16:34.417121+0000","last_clean":"2024-05-04T03:16:34.417121+0000","last_became_active":"2024-05-04T03:15:45.008530+0000","last_became_peered":"2024-05-04T03:15:45.008530+0000","last_unstale":"2024-05-04T03:16:34.417121+0000","last_undegraded":"2024-05-04T03:16:34.417121+0000","last_fullsized":"2024-05-04T03:16:34.417121+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:31:41.697386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.168872+0000","last_change":"2024-05-04T03:15:48.046042+0000","last_active":"2024-05-04T03:17:33.168872+0000","last_peered":"2024-05-04T03:17:33.168872+0000","last_clean":"2024-05-04T03:17:33.168872+0000","last_became_active":"2024-05-04T03:15:48.045387+0000","last_became_peered":"2024-05-04T03:15:48.045387+0000","last_unstale":"2024-05-04T03:17:33.168872+0000","last_undegraded":"2024-05-04T03:17:33.168872+0000","last_fullsized":"2024-05-04T03:17:33.168872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:43:50.782623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.331686+0000","last_change":"2024-05-04T03:15:45.009126+0000","last_active":"2024-05-04T03:16:34.331686+0000","last_peered":"2024-05-04T03:16:34.331686+0000","last_clean":"2024-05-04T03:16:34.331686+0000","last_became_active":"2024-05-04T03:15:45.008802+0000","last_became_peered":"2024-05-04T03:15:45.008802+0000","last_unstale":"2024-05-04T03:16:34.331686+0000","last_undegraded":"2024-05-04T03:16:34.331686+0000","last_fullsized":"2024-05-04T03:16:34.331686+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:25:30.867870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.169092+0000","last_change":"2024-05-04T03:15:48.050478+0000","last_active":"2024-05-04T03:17:33.169092+0000","last_peered":"2024-05-04T03:17:33.169092+0000","last_clean":"2024-05-04T03:17:33.169092+0000","last_became_active":"2024-05-04T03:15:48.045756+0000","last_became_peered":"2024-05-04T03:15:48.045756+0000","last_unstale":"2024-05-04T03:17:33.169092+0000","last_undegraded":"2024-05-04T03:17:33.169092+0000","last_fullsized":"2024-05-04T03:17:33.169092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:05:11.098381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165891+0000","last_change":"2024-05-04T03:15:48.042778+0000","last_active":"2024-05-04T03:17:33.165891+0000","last_peered":"2024-05-04T03:17:33.165891+0000","last_clean":"2024-05-04T03:17:33.165891+0000","last_became_active":"2024-05-04T03:15:48.042349+0000","last_became_peered":"2024-05-04T03:15:48.042349+0000","last_unstale":"2024-05-04T03:17:33.165891+0000","last_undegraded":"2024-05-04T03:17:33.165891+0000","last_fullsized":"2024-05-04T03:17:33.165891+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:40:12.260026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.161439+0000","last_change":"2024-05-04T03:15:48.048544+0000","last_active":"2024-05-04T03:17:33.161439+0000","last_peered":"2024-05-04T03:17:33.161439+0000","last_clean":"2024-05-04T03:17:33.161439+0000","last_became_active":"2024-05-04T03:15:48.047873+0000","last_became_peered":"2024-05-04T03:15:48.047873+0000","last_unstale":"2024-05-04T03:17:33.161439+0000","last_undegraded":"2024-05-04T03:17:33.161439+0000","last_fullsized":"2024-05-04T03:17:33.161439+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:48:02.290401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166682+0000","last_change":"2024-05-04T03:15:48.053467+0000","last_active":"2024-05-04T03:17:33.166682+0000","last_peered":"2024-05-04T03:17:33.166682+0000","last_clean":"2024-05-04T03:17:33.166682+0000","last_became_active":"2024-05-04T03:15:48.050796+0000","last_became_peered":"2024-05-04T03:15:48.050796+0000","last_unstale":"2024-05-04T03:17:33.166682+0000","last_undegraded":"2024-05-04T03:17:33.166682+0000","last_fullsized":"2024-05-04T03:17:33.166682+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:11:15.009467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.159030+0000","last_change":"2024-05-04T03:15:48.054616+0000","last_active":"2024-05-04T03:17:33.159030+0000","last_peered":"2024-05-04T03:17:33.159030+0000","last_clean":"2024-05-04T03:17:33.159030+0000","last_became_active":"2024-05-04T03:15:48.054142+0000","last_became_peered":"2024-05-04T03:15:48.054142+0000","last_unstale":"2024-05-04T03:17:33.159030+0000","last_undegraded":"2024-05-04T03:17:33.159030+0000","last_fullsized":"2024-05-04T03:17:33.159030+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:18:24.067666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166465+0000","last_change":"2024-05-04T03:15:48.046969+0000","last_active":"2024-05-04T03:17:33.166465+0000","last_peered":"2024-05-04T03:17:33.166465+0000","last_clean":"2024-05-04T03:17:33.166465+0000","last_became_active":"2024-05-04T03:15:48.046637+0000","last_became_peered":"2024-05-04T03:15:48.046637+0000","last_unstale":"2024-05-04T03:17:33.166465+0000","last_undegraded":"2024-05-04T03:17:33.166465+0000","last_fullsized":"2024-05-04T03:17:33.166465+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:03:01.763037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164220+0000","last_change":"2024-05-04T03:15:48.041753+0000","last_active":"2024-05-04T03:17:33.164220+0000","last_peered":"2024-05-04T03:17:33.164220+0000","last_clean":"2024-05-04T03:17:33.164220+0000","last_became_active":"2024-05-04T03:15:48.041236+0000","last_became_peered":"2024-05-04T03:15:48.041236+0000","last_unstale":"2024-05-04T03:17:33.164220+0000","last_undegraded":"2024-05-04T03:17:33.164220+0000","last_fullsized":"2024-05-04T03:17:33.164220+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:41:02.928257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.162074+0000","last_change":"2024-05-04T03:15:48.044192+0000","last_active":"2024-05-04T03:17:33.162074+0000","last_peered":"2024-05-04T03:17:33.162074+0000","last_clean":"2024-05-04T03:17:33.162074+0000","last_became_active":"2024-05-04T03:15:48.043849+0000","last_became_peered":"2024-05-04T03:15:48.043849+0000","last_unstale":"2024-05-04T03:17:33.162074+0000","last_undegraded":"2024-05-04T03:17:33.162074+0000","last_fullsized":"2024-05-04T03:17:33.162074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:55:22.000788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164261+0000","last_change":"2024-05-04T03:15:48.045675+0000","last_active":"2024-05-04T03:17:33.164261+0000","last_peered":"2024-05-04T03:17:33.164261+0000","last_clean":"2024-05-04T03:17:33.164261+0000","last_became_active":"2024-05-04T03:15:48.045321+0000","last_became_peered":"2024-05-04T03:15:48.045321+0000","last_unstale":"2024-05-04T03:17:33.164261+0000","last_undegraded":"2024-05-04T03:17:33.164261+0000","last_fullsized":"2024-05-04T03:17:33.164261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:12:06.060729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.143245+0000","last_change":"2024-05-04T03:15:48.040872+0000","last_active":"2024-05-04T03:17:33.143245+0000","last_peered":"2024-05-04T03:17:33.143245+0000","last_clean":"2024-05-04T03:17:33.143245+0000","last_became_active":"2024-05-04T03:15:48.040485+0000","last_became_peered":"2024-05-04T03:15:48.040485+0000","last_unstale":"2024-05-04T03:17:33.143245+0000","last_undegraded":"2024-05-04T03:17:33.143245+0000","last_fullsized":"2024-05-04T03:17:33.143245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:58:01.789270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.161920+0000","last_change":"2024-05-04T03:15:48.049080+0000","last_active":"2024-05-04T03:17:33.161920+0000","last_peered":"2024-05-04T03:17:33.161920+0000","last_clean":"2024-05-04T03:17:33.161920+0000","last_became_active":"2024-05-04T03:15:48.044624+0000","last_became_peered":"2024-05-04T03:15:48.044624+0000","last_unstale":"2024-05-04T03:17:33.161920+0000","last_undegraded":"2024-05-04T03:17:33.161920+0000","last_fullsized":"2024-05-04T03:17:33.161920+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:47:08.647401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.171410+0000","last_change":"2024-05-04T03:15:48.049394+0000","last_active":"2024-05-04T03:17:33.171410+0000","last_peered":"2024-05-04T03:17:33.171410+0000","last_clean":"2024-05-04T03:17:33.171410+0000","last_became_active":"2024-05-04T03:15:48.048980+0000","last_became_peered":"2024-05-04T03:15:48.048980+0000","last_unstale":"2024-05-04T03:17:33.171410+0000","last_undegraded":"2024-05-04T03:17:33.171410+0000","last_fullsized":"2024-05-04T03:17:33.171410+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:43:24.914665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.171163+0000","last_change":"2024-05-04T03:15:48.051672+0000","last_active":"2024-05-04T03:17:33.171163+0000","last_peered":"2024-05-04T03:17:33.171163+0000","last_clean":"2024-05-04T03:17:33.171163+0000","last_became_active":"2024-05-04T03:15:48.051314+0000","last_became_peered":"2024-05-04T03:15:48.051314+0000","last_unstale":"2024-05-04T03:17:33.171163+0000","last_undegraded":"2024-05-04T03:17:33.171163+0000","last_fullsized":"2024-05-04T03:17:33.171163+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:19:22.808102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.167760+0000","last_change":"2024-05-04T03:15:48.053045+0000","last_active":"2024-05-04T03:17:33.167760+0000","last_peered":"2024-05-04T03:17:33.167760+0000","last_clean":"2024-05-04T03:17:33.167760+0000","last_became_active":"2024-05-04T03:15:48.052659+0000","last_became_peered":"2024-05-04T03:15:48.052659+0000","last_unstale":"2024-05-04T03:17:33.167760+0000","last_undegraded":"2024-05-04T03:17:33.167760+0000","last_fullsized":"2024-05-04T03:17:33.167760+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:53:25.624540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70831,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":978,"num_read_kb":562,"num_write":175,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":446464,"data_stored":217198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612660,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":440503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808888,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1605632,"data_stored":1098897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972413,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":440739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168644,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":440659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397576,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32652,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712500,"statfs":{"total":95995035648,"available":95961600000,"internally_reserved":0,"allocated":1511424,"data_stored":1032730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593805,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40380,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704772,"statfs":{"total":95995035648,"available":95953686528,"internally_reserved":0,"allocated":905216,"data_stored":440128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":974848,"data_stored":506302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40400,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704752,"statfs":{"total":95995035648,"available":95953666048,"internally_reserved":0,"allocated":925696,"data_stored":441399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":1003520,"data_stored":512973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182491,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40396,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704756,"statfs":{"total":95995035648,"available":95953670144,"internally_reserved":0,"allocated":921600,"data_stored":440439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986263,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":441097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790035,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36824,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708328,"statfs":{"total":95995035648,"available":95957327872,"internally_reserved":0,"allocated":1523712,"data_stored":1031882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":73052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":68491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-04T03:18:48.810 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-04T03:18:48.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:48 smithi077 ceph-mon[33531]: pgmap v380: 97 pgs: 97 active+clean; 646 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-04T03:18:48.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:48 smithi077 ceph-mon[33531]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-04T03:18:49.065 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:18:49.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:49 smithi072 ceph-mon[33332]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:18:49.756 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:49 smithi001 ceph-mon[29731]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:18:49.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:49 smithi077 ceph-mon[33531]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:18:50.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.582+0000 7f6de9584700 1 -- 172.21.15.1:0/1599679566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6de41045d0 msgr2=0x7f6de41049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:50.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.582+0000 7f6de9584700 1 --2- 172.21.15.1:0/1599679566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6de41045d0 0x7f6de41049e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40099c0 tx=0x7f6dd4009cd0 comp rx=0 tx=0).stop 2024-05-04T03:18:50.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.582+0000 7f6de9584700 1 -- 172.21.15.1:0/1599679566 shutdown_connections 2024-05-04T03:18:50.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.582+0000 7f6de9584700 1 --2- 172.21.15.1:0/1599679566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6de4109850 0x7f6de41111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:50.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.582+0000 7f6de9584700 1 --2- 172.21.15.1:0/1599679566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6de41045d0 0x7f6de41049e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:50.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.582+0000 7f6de9584700 1 --2- 172.21.15.1:0/1599679566 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6de4108f00 0x7f6de4109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:50.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.582+0000 7f6de9584700 1 -- 172.21.15.1:0/1599679566 >> 172.21.15.1:0/1599679566 conn(0x7f6de40ffea0 msgr2=0x7f6de41022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:50.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.582+0000 7f6de9584700 1 -- 172.21.15.1:0/1599679566 shutdown_connections 2024-05-04T03:18:50.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.582+0000 7f6de9584700 1 -- 172.21.15.1:0/1599679566 wait complete. 2024-05-04T03:18:50.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.583+0000 7f6de9584700 1 Processor -- start 2024-05-04T03:18:50.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.583+0000 7f6de9584700 1 -- start start 2024-05-04T03:18:50.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.583+0000 7f6de9584700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6de41045d0 0x7f6de419ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:50.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.583+0000 7f6de9584700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6de4108f00 0x7f6de419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:50.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.583+0000 7f6de9584700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6de4109850 0x7f6de419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:50.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.583+0000 7f6de9584700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6de4114480 con 0x7f6de4109850 2024-05-04T03:18:50.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.583+0000 7f6de9584700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6de4114300 con 0x7f6de41045d0 2024-05-04T03:18:50.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.583+0000 7f6de9584700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6de4114600 con 0x7f6de4108f00 2024-05-04T03:18:50.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.583+0000 7f6de37fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6de4109850 0x7f6de419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:50.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.583+0000 7f6de37fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6de4109850 0x7f6de419f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41138/0 (socket says 172.21.15.1:41138) 2024-05-04T03:18:50.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.583+0000 7f6de37fe700 1 -- 172.21.15.1:0/639448501 learned_addr learned my addr 172.21.15.1:0/639448501 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:50.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.583+0000 7f6de2ffd700 1 --2- 172.21.15.1:0/639448501 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6de41045d0 0x7f6de419ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:50.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.584+0000 7f6de27fc700 1 --2- 172.21.15.1:0/639448501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6de4108f00 0x7f6de419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:50.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.584+0000 7f6de37fe700 1 -- 172.21.15.1:0/639448501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6de4108f00 msgr2=0x7f6de419b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:50.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.584+0000 7f6de37fe700 1 --2- 172.21.15.1:0/639448501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6de4108f00 0x7f6de419b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:50.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.584+0000 7f6de37fe700 1 -- 172.21.15.1:0/639448501 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6de41045d0 msgr2=0x7f6de419ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:50.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.584+0000 7f6de37fe700 1 --2- 172.21.15.1:0/639448501 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6de41045d0 0x7f6de419ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:50.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.584+0000 7f6de37fe700 1 -- 172.21.15.1:0/639448501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6de419fd40 con 0x7f6de4109850 2024-05-04T03:18:50.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.584+0000 7f6de2ffd700 1 --2- 172.21.15.1:0/639448501 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6de41045d0 0x7f6de419ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:18:50.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.584+0000 7f6de37fe700 1 --2- 172.21.15.1:0/639448501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6de4109850 0x7f6de419f6a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc007ee0 tx=0x7f6ddc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:50.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.584+0000 7f6dcbfff700 1 -- 172.21.15.1:0/639448501 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc00dbd0 con 0x7f6de4109850 2024-05-04T03:18:50.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.584+0000 7f6de9584700 1 -- 172.21.15.1:0/639448501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6de41a0030 con 0x7f6de4109850 2024-05-04T03:18:50.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.584+0000 7f6dcbfff700 1 -- 172.21.15.1:0/639448501 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ddc00dd70 con 0x7f6de4109850 2024-05-04T03:18:50.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.584+0000 7f6dcbfff700 1 -- 172.21.15.1:0/639448501 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc005990 con 0x7f6de4109850 2024-05-04T03:18:50.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.584+0000 7f6de9584700 1 -- 172.21.15.1:0/639448501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6de41a7950 con 0x7f6de4109850 2024-05-04T03:18:50.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.585+0000 7f6de9584700 1 -- 172.21.15.1:0/639448501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6de40512c0 con 0x7f6de4109850 2024-05-04T03:18:50.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.587+0000 7f6dcbfff700 1 -- 172.21.15.1:0/639448501 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7f6ddc01a020 con 0x7f6de4109850 2024-05-04T03:18:50.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.587+0000 7f6dcbfff700 1 --2- 172.21.15.1:0/639448501 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6dcc0b2910 0x7f6dcc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:50.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.588+0000 7f6dcbfff700 1 -- 172.21.15.1:0/639448501 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f6ddc0e2040 con 0x7f6de4109850 2024-05-04T03:18:50.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.588+0000 7f6de2ffd700 1 --2- 172.21.15.1:0/639448501 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6dcc0b2910 0x7f6dcc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:50.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.588+0000 7f6de2ffd700 1 --2- 172.21.15.1:0/639448501 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6dcc0b2910 0x7f6dcc0b4e20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009990 tx=0x7f6dd4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:50.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.591+0000 7f6dcbfff700 1 -- 172.21.15.1:0/639448501 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6ddc010070 con 0x7f6de4109850 2024-05-04T03:18:50.594 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:50 smithi001 ceph-mon[29731]: pgmap v381: 97 pgs: 97 active+clean; 651 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-05-04T03:18:50.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:50 smithi072 ceph-mon[33332]: pgmap v381: 97 pgs: 97 active+clean; 651 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-05-04T03:18:50.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.805+0000 7f6de9584700 1 -- 172.21.15.1:0/639448501 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6de419c1d0 con 0x7f6dcc0b2910 2024-05-04T03:18:50.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.812+0000 7f6dcbfff700 1 -- 172.21.15.1:0/639448501 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258827 (secure 0 0 0) 0x7f6dd4067320 con 0x7f6dcc0b2910 2024-05-04T03:18:50.820 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:18:50.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.815+0000 7f6de9584700 1 -- 172.21.15.1:0/639448501 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6dcc0b2910 msgr2=0x7f6dcc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:50.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.815+0000 7f6de9584700 1 --2- 172.21.15.1:0/639448501 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6dcc0b2910 0x7f6dcc0b4e20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009990 tx=0x7f6dd4000bc0 comp rx=0 tx=0).stop 2024-05-04T03:18:50.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.815+0000 7f6de9584700 1 -- 172.21.15.1:0/639448501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6de4109850 msgr2=0x7f6de419f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:50.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.815+0000 7f6de9584700 1 --2- 172.21.15.1:0/639448501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6de4109850 0x7f6de419f6a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc007ee0 tx=0x7f6ddc00d3b0 comp rx=0 tx=0).stop 2024-05-04T03:18:50.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.815+0000 7f6de9584700 1 -- 172.21.15.1:0/639448501 shutdown_connections 2024-05-04T03:18:50.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.815+0000 7f6de9584700 1 --2- 172.21.15.1:0/639448501 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6de41045d0 0x7f6de419ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:50.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.815+0000 7f6de9584700 1 --2- 172.21.15.1:0/639448501 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6dcc0b2910 0x7f6dcc0b4e20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:50.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.815+0000 7f6de9584700 1 --2- 172.21.15.1:0/639448501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6de4109850 0x7f6de419f6a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:50.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.815+0000 7f6de9584700 1 --2- 172.21.15.1:0/639448501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6de4108f00 0x7f6de419b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:50.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.815+0000 7f6de9584700 1 -- 172.21.15.1:0/639448501 >> 172.21.15.1:0/639448501 conn(0x7f6de40ffea0 msgr2=0x7f6de410a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:50.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.815+0000 7f6de9584700 1 -- 172.21.15.1:0/639448501 shutdown_connections 2024-05-04T03:18:50.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:50.815+0000 7f6de9584700 1 -- 172.21.15.1:0/639448501 wait complete. 2024-05-04T03:18:50.823 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-04T03:18:50.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:50 smithi077 ceph-mon[33531]: pgmap v381: 97 pgs: 97 active+clean; 651 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-05-04T03:18:51.590 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2024-05-04T03:18:49.186361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666837,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1082,"num_read_kb":648,"num_write":313,"num_write_kb":2290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":753,"ondisk_log_size":753,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":411792,"kb_used_data":12720,"kb_used_omap":18,"kb_used_meta":398957,"kb_avail":1124530032,"statfs":{"total":1151940427776,"available":1151518752768,"internally_reserved":0,"allocated":13025280,"data_stored":7279255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":408532281},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5712,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":52,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003228"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.173150+0000","last_change":"2024-05-04T03:15:48.049361+0000","last_active":"2024-05-04T03:17:33.173150+0000","last_peered":"2024-05-04T03:17:33.173150+0000","last_clean":"2024-05-04T03:17:33.173150+0000","last_became_active":"2024-05-04T03:15:48.048876+0000","last_became_peered":"2024-05-04T03:15:48.048876+0000","last_unstale":"2024-05-04T03:17:33.173150+0000","last_undegraded":"2024-05-04T03:17:33.173150+0000","last_fullsized":"2024-05-04T03:17:33.173150+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:50:52.568516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165921+0000","last_change":"2024-05-04T03:15:48.055554+0000","last_active":"2024-05-04T03:17:33.165921+0000","last_peered":"2024-05-04T03:17:33.165921+0000","last_clean":"2024-05-04T03:17:33.165921+0000","last_became_active":"2024-05-04T03:15:48.055155+0000","last_became_peered":"2024-05-04T03:15:48.055155+0000","last_unstale":"2024-05-04T03:17:33.165921+0000","last_undegraded":"2024-05-04T03:17:33.165921+0000","last_fullsized":"2024-05-04T03:17:33.165921+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:38:07.824031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.167129+0000","last_change":"2024-05-04T03:15:48.047974+0000","last_active":"2024-05-04T03:17:33.167129+0000","last_peered":"2024-05-04T03:17:33.167129+0000","last_clean":"2024-05-04T03:17:33.167129+0000","last_became_active":"2024-05-04T03:15:48.044299+0000","last_became_peered":"2024-05-04T03:15:48.044299+0000","last_unstale":"2024-05-04T03:17:33.167129+0000","last_undegraded":"2024-05-04T03:17:33.167129+0000","last_fullsized":"2024-05-04T03:17:33.167129+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:49:08.014987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.156171+0000","last_change":"2024-05-04T03:15:48.050899+0000","last_active":"2024-05-04T03:17:33.156171+0000","last_peered":"2024-05-04T03:17:33.156171+0000","last_clean":"2024-05-04T03:17:33.156171+0000","last_became_active":"2024-05-04T03:15:48.048812+0000","last_became_peered":"2024-05-04T03:15:48.048812+0000","last_unstale":"2024-05-04T03:17:33.156171+0000","last_undegraded":"2024-05-04T03:17:33.156171+0000","last_fullsized":"2024-05-04T03:17:33.156171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:21:01.829349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165098+0000","last_change":"2024-05-04T03:15:48.044403+0000","last_active":"2024-05-04T03:17:33.165098+0000","last_peered":"2024-05-04T03:17:33.165098+0000","last_clean":"2024-05-04T03:17:33.165098+0000","last_became_active":"2024-05-04T03:15:48.044021+0000","last_became_peered":"2024-05-04T03:15:48.044021+0000","last_unstale":"2024-05-04T03:17:33.165098+0000","last_undegraded":"2024-05-04T03:17:33.165098+0000","last_fullsized":"2024-05-04T03:17:33.165098+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:50:54.762070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.167642+0000","last_change":"2024-05-04T03:15:48.053233+0000","last_active":"2024-05-04T03:17:33.167642+0000","last_peered":"2024-05-04T03:17:33.167642+0000","last_clean":"2024-05-04T03:17:33.167642+0000","last_became_active":"2024-05-04T03:15:48.052809+0000","last_became_peered":"2024-05-04T03:15:48.052809+0000","last_unstale":"2024-05-04T03:17:33.167642+0000","last_undegraded":"2024-05-04T03:17:33.167642+0000","last_fullsized":"2024-05-04T03:17:33.167642+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:27:20.082694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.156722+0000","last_change":"2024-05-04T03:15:48.045108+0000","last_active":"2024-05-04T03:17:33.156722+0000","last_peered":"2024-05-04T03:17:33.156722+0000","last_clean":"2024-05-04T03:17:33.156722+0000","last_became_active":"2024-05-04T03:15:48.044719+0000","last_became_peered":"2024-05-04T03:15:48.044719+0000","last_unstale":"2024-05-04T03:17:33.156722+0000","last_undegraded":"2024-05-04T03:17:33.156722+0000","last_fullsized":"2024-05-04T03:17:33.156722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:55:02.586294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164990+0000","last_change":"2024-05-04T03:15:48.043559+0000","last_active":"2024-05-04T03:17:33.164990+0000","last_peered":"2024-05-04T03:17:33.164990+0000","last_clean":"2024-05-04T03:17:33.164990+0000","last_became_active":"2024-05-04T03:15:48.043138+0000","last_became_peered":"2024-05-04T03:15:48.043138+0000","last_unstale":"2024-05-04T03:17:33.164990+0000","last_undegraded":"2024-05-04T03:17:33.164990+0000","last_fullsized":"2024-05-04T03:17:33.164990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:05:25.411916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.174068+0000","last_change":"2024-05-04T03:15:48.039632+0000","last_active":"2024-05-04T03:17:33.174068+0000","last_peered":"2024-05-04T03:17:33.174068+0000","last_clean":"2024-05-04T03:17:33.174068+0000","last_became_active":"2024-05-04T03:15:48.039231+0000","last_became_peered":"2024-05-04T03:15:48.039231+0000","last_unstale":"2024-05-04T03:17:33.174068+0000","last_undegraded":"2024-05-04T03:17:33.174068+0000","last_fullsized":"2024-05-04T03:17:33.174068+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:33:12.853172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.168249+0000","last_change":"2024-05-04T03:15:48.055138+0000","last_active":"2024-05-04T03:17:33.168249+0000","last_peered":"2024-05-04T03:17:33.168249+0000","last_clean":"2024-05-04T03:17:33.168249+0000","last_became_active":"2024-05-04T03:15:48.054791+0000","last_became_peered":"2024-05-04T03:15:48.054791+0000","last_unstale":"2024-05-04T03:17:33.168249+0000","last_undegraded":"2024-05-04T03:17:33.168249+0000","last_fullsized":"2024-05-04T03:17:33.168249+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:33:36.822549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165773+0000","last_change":"2024-05-04T03:15:48.050125+0000","last_active":"2024-05-04T03:17:33.165773+0000","last_peered":"2024-05-04T03:17:33.165773+0000","last_clean":"2024-05-04T03:17:33.165773+0000","last_became_active":"2024-05-04T03:15:48.049390+0000","last_became_peered":"2024-05-04T03:15:48.049390+0000","last_unstale":"2024-05-04T03:17:33.165773+0000","last_undegraded":"2024-05-04T03:17:33.165773+0000","last_fullsized":"2024-05-04T03:17:33.165773+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:27:05.740383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.163474+0000","last_change":"2024-05-04T03:15:48.050700+0000","last_active":"2024-05-04T03:17:33.163474+0000","last_peered":"2024-05-04T03:17:33.163474+0000","last_clean":"2024-05-04T03:17:33.163474+0000","last_became_active":"2024-05-04T03:15:48.049288+0000","last_became_peered":"2024-05-04T03:15:48.049288+0000","last_unstale":"2024-05-04T03:17:33.163474+0000","last_undegraded":"2024-05-04T03:17:33.163474+0000","last_fullsized":"2024-05-04T03:17:33.163474+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:40:07.072006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.168343+0000","last_change":"2024-05-04T03:16:34.337123+0000","last_active":"2024-05-04T03:17:33.168343+0000","last_peered":"2024-05-04T03:17:33.168343+0000","last_clean":"2024-05-04T03:17:33.168343+0000","last_became_active":"2024-05-04T03:16:34.336657+0000","last_became_peered":"2024-05-04T03:16:34.336657+0000","last_unstale":"2024-05-04T03:17:33.168343+0000","last_undegraded":"2024-05-04T03:17:33.168343+0000","last_fullsized":"2024-05-04T03:17:33.168343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:58:15.939941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166400+0000","last_change":"2024-05-04T03:15:48.047258+0000","last_active":"2024-05-04T03:17:33.166400+0000","last_peered":"2024-05-04T03:17:33.166400+0000","last_clean":"2024-05-04T03:17:33.166400+0000","last_became_active":"2024-05-04T03:15:48.046897+0000","last_became_peered":"2024-05-04T03:15:48.046897+0000","last_unstale":"2024-05-04T03:17:33.166400+0000","last_undegraded":"2024-05-04T03:17:33.166400+0000","last_fullsized":"2024-05-04T03:17:33.166400+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:14:39.217475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166318+0000","last_change":"2024-05-04T03:15:48.051240+0000","last_active":"2024-05-04T03:17:33.166318+0000","last_peered":"2024-05-04T03:17:33.166318+0000","last_clean":"2024-05-04T03:17:33.166318+0000","last_became_active":"2024-05-04T03:15:48.050081+0000","last_became_peered":"2024-05-04T03:15:48.050081+0000","last_unstale":"2024-05-04T03:17:33.166318+0000","last_undegraded":"2024-05-04T03:17:33.166318+0000","last_fullsized":"2024-05-04T03:17:33.166318+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:39:06.773121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.171066+0000","last_change":"2024-05-04T03:15:48.050936+0000","last_active":"2024-05-04T03:17:33.171066+0000","last_peered":"2024-05-04T03:17:33.171066+0000","last_clean":"2024-05-04T03:17:33.171066+0000","last_became_active":"2024-05-04T03:15:48.050567+0000","last_became_peered":"2024-05-04T03:15:48.050567+0000","last_unstale":"2024-05-04T03:17:33.171066+0000","last_undegraded":"2024-05-04T03:17:33.171066+0000","last_fullsized":"2024-05-04T03:17:33.171066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:17:43.469889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.173267+0000","last_change":"2024-05-04T03:15:48.047200+0000","last_active":"2024-05-04T03:17:33.173267+0000","last_peered":"2024-05-04T03:17:33.173267+0000","last_clean":"2024-05-04T03:17:33.173267+0000","last_became_active":"2024-05-04T03:15:48.046782+0000","last_became_peered":"2024-05-04T03:15:48.046782+0000","last_unstale":"2024-05-04T03:17:33.173267+0000","last_undegraded":"2024-05-04T03:17:33.173267+0000","last_fullsized":"2024-05-04T03:17:33.173267+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:32:56.773916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.154784+0000","last_change":"2024-05-04T03:15:48.042114+0000","last_active":"2024-05-04T03:17:33.154784+0000","last_peered":"2024-05-04T03:17:33.154784+0000","last_clean":"2024-05-04T03:17:33.154784+0000","last_became_active":"2024-05-04T03:15:48.040921+0000","last_became_peered":"2024-05-04T03:15:48.040921+0000","last_unstale":"2024-05-04T03:17:33.154784+0000","last_undegraded":"2024-05-04T03:17:33.154784+0000","last_fullsized":"2024-05-04T03:17:33.154784+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:16:20.130996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164439+0000","last_change":"2024-05-04T03:15:48.051060+0000","last_active":"2024-05-04T03:17:33.164439+0000","last_peered":"2024-05-04T03:17:33.164439+0000","last_clean":"2024-05-04T03:17:33.164439+0000","last_became_active":"2024-05-04T03:15:48.050642+0000","last_became_peered":"2024-05-04T03:15:48.050642+0000","last_unstale":"2024-05-04T03:17:33.164439+0000","last_undegraded":"2024-05-04T03:17:33.164439+0000","last_fullsized":"2024-05-04T03:17:33.164439+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:23:31.193072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.329355+0000","last_change":"2024-05-04T03:15:45.013477+0000","last_active":"2024-05-04T03:16:34.329355+0000","last_peered":"2024-05-04T03:16:34.329355+0000","last_clean":"2024-05-04T03:16:34.329355+0000","last_became_active":"2024-05-04T03:15:45.013161+0000","last_became_peered":"2024-05-04T03:15:45.013161+0000","last_unstale":"2024-05-04T03:16:34.329355+0000","last_undegraded":"2024-05-04T03:16:34.329355+0000","last_fullsized":"2024-05-04T03:16:34.329355+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:08:14.087090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164546+0000","last_change":"2024-05-04T03:15:48.045810+0000","last_active":"2024-05-04T03:17:33.164546+0000","last_peered":"2024-05-04T03:17:33.164546+0000","last_clean":"2024-05-04T03:17:33.164546+0000","last_became_active":"2024-05-04T03:15:48.045449+0000","last_became_peered":"2024-05-04T03:15:48.045449+0000","last_unstale":"2024-05-04T03:17:33.164546+0000","last_undegraded":"2024-05-04T03:17:33.164546+0000","last_fullsized":"2024-05-04T03:17:33.164546+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:48:05.084208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.417032+0000","last_change":"2024-05-04T03:15:45.204196+0000","last_active":"2024-05-04T03:16:34.417032+0000","last_peered":"2024-05-04T03:16:34.417032+0000","last_clean":"2024-05-04T03:16:34.417032+0000","last_became_active":"2024-05-04T03:15:45.203821+0000","last_became_peered":"2024-05-04T03:15:45.203821+0000","last_unstale":"2024-05-04T03:16:34.417032+0000","last_undegraded":"2024-05-04T03:16:34.417032+0000","last_fullsized":"2024-05-04T03:16:34.417032+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:17:05.074450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.167259+0000","last_change":"2024-05-04T03:15:48.045131+0000","last_active":"2024-05-04T03:17:33.167259+0000","last_peered":"2024-05-04T03:17:33.167259+0000","last_clean":"2024-05-04T03:17:33.167259+0000","last_became_active":"2024-05-04T03:15:48.044793+0000","last_became_peered":"2024-05-04T03:15:48.044793+0000","last_unstale":"2024-05-04T03:17:33.167259+0000","last_undegraded":"2024-05-04T03:17:33.167259+0000","last_fullsized":"2024-05-04T03:17:33.167259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:03:37.769324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334752+0000","last_change":"2024-05-04T03:15:45.013644+0000","last_active":"2024-05-04T03:16:34.334752+0000","last_peered":"2024-05-04T03:16:34.334752+0000","last_clean":"2024-05-04T03:16:34.334752+0000","last_became_active":"2024-05-04T03:15:45.013282+0000","last_became_peered":"2024-05-04T03:15:45.013282+0000","last_unstale":"2024-05-04T03:16:34.334752+0000","last_undegraded":"2024-05-04T03:16:34.334752+0000","last_fullsized":"2024-05-04T03:16:34.334752+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:21:21.204433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.174582+0000","last_change":"2024-05-04T03:15:48.043046+0000","last_active":"2024-05-04T03:17:33.174582+0000","last_peered":"2024-05-04T03:17:33.174582+0000","last_clean":"2024-05-04T03:17:33.174582+0000","last_became_active":"2024-05-04T03:15:48.042621+0000","last_became_peered":"2024-05-04T03:15:48.042621+0000","last_unstale":"2024-05-04T03:17:33.174582+0000","last_undegraded":"2024-05-04T03:17:33.174582+0000","last_fullsized":"2024-05-04T03:17:33.174582+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:03:58.916423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334940+0000","last_change":"2024-05-04T03:15:45.010618+0000","last_active":"2024-05-04T03:16:34.334940+0000","last_peered":"2024-05-04T03:16:34.334940+0000","last_clean":"2024-05-04T03:16:34.334940+0000","last_became_active":"2024-05-04T03:15:45.009239+0000","last_became_peered":"2024-05-04T03:15:45.009239+0000","last_unstale":"2024-05-04T03:16:34.334940+0000","last_undegraded":"2024-05-04T03:16:34.334940+0000","last_fullsized":"2024-05-04T03:16:34.334940+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:49:46.777571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.145981+0000","last_change":"2024-05-04T03:15:48.049334+0000","last_active":"2024-05-04T03:17:33.145981+0000","last_peered":"2024-05-04T03:17:33.145981+0000","last_clean":"2024-05-04T03:17:33.145981+0000","last_became_active":"2024-05-04T03:15:48.048984+0000","last_became_peered":"2024-05-04T03:15:48.048984+0000","last_unstale":"2024-05-04T03:17:33.145981+0000","last_undegraded":"2024-05-04T03:17:33.145981+0000","last_fullsized":"2024-05-04T03:17:33.145981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:28:25.366106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":194,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:18:43.505278+0000","last_change":"2024-05-04T03:15:45.015277+0000","last_active":"2024-05-04T03:18:43.505278+0000","last_peered":"2024-05-04T03:18:43.505278+0000","last_clean":"2024-05-04T03:18:43.505278+0000","last_became_active":"2024-05-04T03:15:45.014542+0000","last_became_peered":"2024-05-04T03:15:45.014542+0000","last_unstale":"2024-05-04T03:18:43.505278+0000","last_undegraded":"2024-05-04T03:18:43.505278+0000","last_fullsized":"2024-05-04T03:18:43.505278+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:22:15.594866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":161,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.147597+0000","last_change":"2024-05-04T03:15:48.045029+0000","last_active":"2024-05-04T03:17:33.147597+0000","last_peered":"2024-05-04T03:17:33.147597+0000","last_clean":"2024-05-04T03:17:33.147597+0000","last_became_active":"2024-05-04T03:15:48.041262+0000","last_became_peered":"2024-05-04T03:15:48.041262+0000","last_unstale":"2024-05-04T03:17:33.147597+0000","last_undegraded":"2024-05-04T03:17:33.147597+0000","last_fullsized":"2024-05-04T03:17:33.147597+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:45:56.882957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334215+0000","last_change":"2024-05-04T03:15:45.007530+0000","last_active":"2024-05-04T03:16:34.334215+0000","last_peered":"2024-05-04T03:16:34.334215+0000","last_clean":"2024-05-04T03:16:34.334215+0000","last_became_active":"2024-05-04T03:15:45.007054+0000","last_became_peered":"2024-05-04T03:15:45.007054+0000","last_unstale":"2024-05-04T03:16:34.334215+0000","last_undegraded":"2024-05-04T03:16:34.334215+0000","last_fullsized":"2024-05-04T03:16:34.334215+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:09:13.845679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.142714+0000","last_change":"2024-05-04T03:15:48.050956+0000","last_active":"2024-05-04T03:17:33.142714+0000","last_peered":"2024-05-04T03:17:33.142714+0000","last_clean":"2024-05-04T03:17:33.142714+0000","last_became_active":"2024-05-04T03:15:48.050597+0000","last_became_peered":"2024-05-04T03:15:48.050597+0000","last_unstale":"2024-05-04T03:17:33.142714+0000","last_undegraded":"2024-05-04T03:17:33.142714+0000","last_fullsized":"2024-05-04T03:17:33.142714+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:27:34.127889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.331600+0000","last_change":"2024-05-04T03:15:45.007027+0000","last_active":"2024-05-04T03:16:34.331600+0000","last_peered":"2024-05-04T03:16:34.331600+0000","last_clean":"2024-05-04T03:16:34.331600+0000","last_became_active":"2024-05-04T03:15:45.006659+0000","last_became_peered":"2024-05-04T03:15:45.006659+0000","last_unstale":"2024-05-04T03:16:34.331600+0000","last_undegraded":"2024-05-04T03:16:34.331600+0000","last_fullsized":"2024-05-04T03:16:34.331600+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:06:11.827093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164377+0000","last_change":"2024-05-04T03:15:48.046196+0000","last_active":"2024-05-04T03:17:33.164377+0000","last_peered":"2024-05-04T03:17:33.164377+0000","last_clean":"2024-05-04T03:17:33.164377+0000","last_became_active":"2024-05-04T03:15:48.044316+0000","last_became_peered":"2024-05-04T03:15:48.044316+0000","last_unstale":"2024-05-04T03:17:33.164377+0000","last_undegraded":"2024-05-04T03:17:33.164377+0000","last_fullsized":"2024-05-04T03:17:33.164377+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:04:39.301705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.416682+0000","last_change":"2024-05-04T03:15:45.015360+0000","last_active":"2024-05-04T03:16:34.416682+0000","last_peered":"2024-05-04T03:16:34.416682+0000","last_clean":"2024-05-04T03:16:34.416682+0000","last_became_active":"2024-05-04T03:15:45.014696+0000","last_became_peered":"2024-05-04T03:15:45.014696+0000","last_unstale":"2024-05-04T03:16:34.416682+0000","last_undegraded":"2024-05-04T03:16:34.416682+0000","last_fullsized":"2024-05-04T03:16:34.416682+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:17:28.838651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.163052+0000","last_change":"2024-05-04T03:15:48.039823+0000","last_active":"2024-05-04T03:17:33.163052+0000","last_peered":"2024-05-04T03:17:33.163052+0000","last_clean":"2024-05-04T03:17:33.163052+0000","last_became_active":"2024-05-04T03:15:48.039447+0000","last_became_peered":"2024-05-04T03:15:48.039447+0000","last_unstale":"2024-05-04T03:17:33.163052+0000","last_undegraded":"2024-05-04T03:17:33.163052+0000","last_fullsized":"2024-05-04T03:17:33.163052+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:45:55.136946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":194,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:18:46.510193+0000","last_change":"2024-05-04T03:15:45.004662+0000","last_active":"2024-05-04T03:18:46.510193+0000","last_peered":"2024-05-04T03:18:46.510193+0000","last_clean":"2024-05-04T03:18:46.510193+0000","last_became_active":"2024-05-04T03:15:45.004265+0000","last_became_peered":"2024-05-04T03:15:45.004265+0000","last_unstale":"2024-05-04T03:18:46.510193+0000","last_undegraded":"2024-05-04T03:18:46.510193+0000","last_fullsized":"2024-05-04T03:18:46.510193+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:14:03.065346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":163,"num_read_kb":163,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.173995+0000","last_change":"2024-05-04T03:15:48.042093+0000","last_active":"2024-05-04T03:17:33.173995+0000","last_peered":"2024-05-04T03:17:33.173995+0000","last_clean":"2024-05-04T03:17:33.173995+0000","last_became_active":"2024-05-04T03:15:48.041646+0000","last_became_peered":"2024-05-04T03:15:48.041646+0000","last_unstale":"2024-05-04T03:17:33.173995+0000","last_undegraded":"2024-05-04T03:17:33.173995+0000","last_fullsized":"2024-05-04T03:17:33.173995+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:06:32.083493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.335070+0000","last_change":"2024-05-04T03:15:45.016731+0000","last_active":"2024-05-04T03:16:34.335070+0000","last_peered":"2024-05-04T03:16:34.335070+0000","last_clean":"2024-05-04T03:16:34.335070+0000","last_became_active":"2024-05-04T03:15:45.016313+0000","last_became_peered":"2024-05-04T03:15:45.016313+0000","last_unstale":"2024-05-04T03:16:34.335070+0000","last_undegraded":"2024-05-04T03:16:34.335070+0000","last_fullsized":"2024-05-04T03:16:34.335070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:42:40.830986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166326+0000","last_change":"2024-05-04T03:15:48.053649+0000","last_active":"2024-05-04T03:17:33.166326+0000","last_peered":"2024-05-04T03:17:33.166326+0000","last_clean":"2024-05-04T03:17:33.166326+0000","last_became_active":"2024-05-04T03:15:48.052407+0000","last_became_peered":"2024-05-04T03:15:48.052407+0000","last_unstale":"2024-05-04T03:17:33.166326+0000","last_undegraded":"2024-05-04T03:17:33.166326+0000","last_fullsized":"2024-05-04T03:17:33.166326+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:37:28.359272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.417301+0000","last_change":"2024-05-04T03:15:45.205180+0000","last_active":"2024-05-04T03:16:34.417301+0000","last_peered":"2024-05-04T03:16:34.417301+0000","last_clean":"2024-05-04T03:16:34.417301+0000","last_became_active":"2024-05-04T03:15:45.204828+0000","last_became_peered":"2024-05-04T03:15:45.204828+0000","last_unstale":"2024-05-04T03:16:34.417301+0000","last_undegraded":"2024-05-04T03:16:34.417301+0000","last_fullsized":"2024-05-04T03:16:34.417301+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:27:34.063540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.171291+0000","last_change":"2024-05-04T03:15:48.047154+0000","last_active":"2024-05-04T03:17:33.171291+0000","last_peered":"2024-05-04T03:17:33.171291+0000","last_clean":"2024-05-04T03:17:33.171291+0000","last_became_active":"2024-05-04T03:15:48.046794+0000","last_became_peered":"2024-05-04T03:15:48.046794+0000","last_unstale":"2024-05-04T03:17:33.171291+0000","last_undegraded":"2024-05-04T03:17:33.171291+0000","last_fullsized":"2024-05-04T03:17:33.171291+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:21:42.723252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'39","reported_seq":404,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:18:47.511646+0000","last_change":"2024-05-04T03:15:45.008930+0000","last_active":"2024-05-04T03:18:47.511646+0000","last_peered":"2024-05-04T03:18:47.511646+0000","last_clean":"2024-05-04T03:18:47.511646+0000","last_became_active":"2024-05-04T03:15:45.008459+0000","last_became_peered":"2024-05-04T03:15:45.008459+0000","last_unstale":"2024-05-04T03:18:47.511646+0000","last_undegraded":"2024-05-04T03:18:47.511646+0000","last_fullsized":"2024-05-04T03:18:47.511646+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:07:52.609156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71394,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":338,"num_read_kb":239,"num_write":40,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.143904+0000","last_change":"2024-05-04T03:15:48.051848+0000","last_active":"2024-05-04T03:17:33.143904+0000","last_peered":"2024-05-04T03:17:33.143904+0000","last_clean":"2024-05-04T03:17:33.143904+0000","last_became_active":"2024-05-04T03:15:48.051171+0000","last_became_peered":"2024-05-04T03:15:48.051171+0000","last_unstale":"2024-05-04T03:17:33.143904+0000","last_undegraded":"2024-05-04T03:17:33.143904+0000","last_fullsized":"2024-05-04T03:17:33.143904+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:20:32.722845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.330397+0000","last_change":"2024-05-04T03:15:45.007970+0000","last_active":"2024-05-04T03:16:34.330397+0000","last_peered":"2024-05-04T03:16:34.330397+0000","last_clean":"2024-05-04T03:16:34.330397+0000","last_became_active":"2024-05-04T03:15:45.007605+0000","last_became_peered":"2024-05-04T03:15:45.007605+0000","last_unstale":"2024-05-04T03:16:34.330397+0000","last_undegraded":"2024-05-04T03:16:34.330397+0000","last_fullsized":"2024-05-04T03:16:34.330397+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:52:08.466858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.169073+0000","last_change":"2024-05-04T03:15:48.050647+0000","last_active":"2024-05-04T03:17:33.169073+0000","last_peered":"2024-05-04T03:17:33.169073+0000","last_clean":"2024-05-04T03:17:33.169073+0000","last_became_active":"2024-05-04T03:15:48.050296+0000","last_became_peered":"2024-05-04T03:15:48.050296+0000","last_unstale":"2024-05-04T03:17:33.169073+0000","last_undegraded":"2024-05-04T03:17:33.169073+0000","last_fullsized":"2024-05-04T03:17:33.169073+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:14:23.902094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.416945+0000","last_change":"2024-05-04T03:15:45.014936+0000","last_active":"2024-05-04T03:16:34.416945+0000","last_peered":"2024-05-04T03:16:34.416945+0000","last_clean":"2024-05-04T03:16:34.416945+0000","last_became_active":"2024-05-04T03:15:45.014474+0000","last_became_peered":"2024-05-04T03:15:45.014474+0000","last_unstale":"2024-05-04T03:16:34.416945+0000","last_undegraded":"2024-05-04T03:16:34.416945+0000","last_fullsized":"2024-05-04T03:16:34.416945+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:26:37.972523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.169574+0000","last_change":"2024-05-04T03:16:34.336836+0000","last_active":"2024-05-04T03:17:33.169574+0000","last_peered":"2024-05-04T03:17:33.169574+0000","last_clean":"2024-05-04T03:17:33.169574+0000","last_became_active":"2024-05-04T03:16:34.336008+0000","last_became_peered":"2024-05-04T03:16:34.336008+0000","last_unstale":"2024-05-04T03:17:33.169574+0000","last_undegraded":"2024-05-04T03:17:33.169574+0000","last_fullsized":"2024-05-04T03:17:33.169574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:51:52.803138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'6","reported_seq":202,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:18:47.511553+0000","last_change":"2024-05-04T03:15:45.013177+0000","last_active":"2024-05-04T03:18:47.511553+0000","last_peered":"2024-05-04T03:18:47.511553+0000","last_clean":"2024-05-04T03:18:47.511553+0000","last_became_active":"2024-05-04T03:15:45.012764+0000","last_became_peered":"2024-05-04T03:15:45.012764+0000","last_unstale":"2024-05-04T03:18:47.511553+0000","last_undegraded":"2024-05-04T03:18:47.511553+0000","last_fullsized":"2024-05-04T03:18:47.511553+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:07:43.578443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":170,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.159654+0000","last_change":"2024-05-04T03:15:48.046453+0000","last_active":"2024-05-04T03:17:33.159654+0000","last_peered":"2024-05-04T03:17:33.159654+0000","last_clean":"2024-05-04T03:17:33.159654+0000","last_became_active":"2024-05-04T03:15:48.045990+0000","last_became_peered":"2024-05-04T03:15:48.045990+0000","last_unstale":"2024-05-04T03:17:33.159654+0000","last_undegraded":"2024-05-04T03:17:33.159654+0000","last_fullsized":"2024-05-04T03:17:33.159654+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:54:57.285122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334414+0000","last_change":"2024-05-04T03:15:45.205539+0000","last_active":"2024-05-04T03:16:34.334414+0000","last_peered":"2024-05-04T03:16:34.334414+0000","last_clean":"2024-05-04T03:16:34.334414+0000","last_became_active":"2024-05-04T03:15:45.205174+0000","last_became_peered":"2024-05-04T03:15:45.205174+0000","last_unstale":"2024-05-04T03:16:34.334414+0000","last_undegraded":"2024-05-04T03:16:34.334414+0000","last_fullsized":"2024-05-04T03:16:34.334414+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:03:53.398368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.141832+0000","last_change":"2024-05-04T03:15:48.051824+0000","last_active":"2024-05-04T03:17:33.141832+0000","last_peered":"2024-05-04T03:17:33.141832+0000","last_clean":"2024-05-04T03:17:33.141832+0000","last_became_active":"2024-05-04T03:15:48.051483+0000","last_became_peered":"2024-05-04T03:15:48.051483+0000","last_unstale":"2024-05-04T03:17:33.141832+0000","last_undegraded":"2024-05-04T03:17:33.141832+0000","last_fullsized":"2024-05-04T03:17:33.141832+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:57:56.644486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'3","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:18:03.949283+0000","last_change":"2024-05-04T03:15:45.011062+0000","last_active":"2024-05-04T03:18:03.949283+0000","last_peered":"2024-05-04T03:18:03.949283+0000","last_clean":"2024-05-04T03:18:03.949283+0000","last_became_active":"2024-05-04T03:15:45.010730+0000","last_became_peered":"2024-05-04T03:15:45.010730+0000","last_unstale":"2024-05-04T03:18:03.949283+0000","last_undegraded":"2024-05-04T03:18:03.949283+0000","last_fullsized":"2024-05-04T03:18:03.949283+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:45:55.979067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.167833+0000","last_change":"2024-05-04T03:15:48.045965+0000","last_active":"2024-05-04T03:17:33.167833+0000","last_peered":"2024-05-04T03:17:33.167833+0000","last_clean":"2024-05-04T03:17:33.167833+0000","last_became_active":"2024-05-04T03:15:48.045615+0000","last_became_peered":"2024-05-04T03:15:48.045615+0000","last_unstale":"2024-05-04T03:17:33.167833+0000","last_undegraded":"2024-05-04T03:17:33.167833+0000","last_fullsized":"2024-05-04T03:17:33.167833+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:14:23.497484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.329258+0000","last_change":"2024-05-04T03:15:45.204223+0000","last_active":"2024-05-04T03:16:34.329258+0000","last_peered":"2024-05-04T03:16:34.329258+0000","last_clean":"2024-05-04T03:16:34.329258+0000","last_became_active":"2024-05-04T03:15:45.203774+0000","last_became_peered":"2024-05-04T03:15:45.203774+0000","last_unstale":"2024-05-04T03:16:34.329258+0000","last_undegraded":"2024-05-04T03:16:34.329258+0000","last_fullsized":"2024-05-04T03:16:34.329258+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:02:34.456055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.167144+0000","last_change":"2024-05-04T03:15:48.045574+0000","last_active":"2024-05-04T03:17:33.167144+0000","last_peered":"2024-05-04T03:17:33.167144+0000","last_clean":"2024-05-04T03:17:33.167144+0000","last_became_active":"2024-05-04T03:15:48.045237+0000","last_became_peered":"2024-05-04T03:15:48.045237+0000","last_unstale":"2024-05-04T03:17:33.167144+0000","last_undegraded":"2024-05-04T03:17:33.167144+0000","last_fullsized":"2024-05-04T03:17:33.167144+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:53:46.346199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.333946+0000","last_change":"2024-05-04T03:15:45.013895+0000","last_active":"2024-05-04T03:16:34.333946+0000","last_peered":"2024-05-04T03:16:34.333946+0000","last_clean":"2024-05-04T03:16:34.333946+0000","last_became_active":"2024-05-04T03:15:45.013306+0000","last_became_peered":"2024-05-04T03:15:45.013306+0000","last_unstale":"2024-05-04T03:16:34.333946+0000","last_undegraded":"2024-05-04T03:16:34.333946+0000","last_fullsized":"2024-05-04T03:16:34.333946+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:39:18.616432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.158996+0000","last_change":"2024-05-04T03:16:34.336997+0000","last_active":"2024-05-04T03:17:33.158996+0000","last_peered":"2024-05-04T03:17:33.158996+0000","last_clean":"2024-05-04T03:17:33.158996+0000","last_became_active":"2024-05-04T03:16:34.336505+0000","last_became_peered":"2024-05-04T03:16:34.336505+0000","last_unstale":"2024-05-04T03:17:33.158996+0000","last_undegraded":"2024-05-04T03:17:33.158996+0000","last_fullsized":"2024-05-04T03:17:33.158996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:23:38.979216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.149672+0000","last_change":"2024-05-04T03:15:48.050122+0000","last_active":"2024-05-04T03:17:33.149672+0000","last_peered":"2024-05-04T03:17:33.149672+0000","last_clean":"2024-05-04T03:17:33.149672+0000","last_became_active":"2024-05-04T03:15:48.049794+0000","last_became_peered":"2024-05-04T03:15:48.049794+0000","last_unstale":"2024-05-04T03:17:33.149672+0000","last_undegraded":"2024-05-04T03:17:33.149672+0000","last_fullsized":"2024-05-04T03:17:33.149672+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:29:11.066068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":196,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:18:46.508923+0000","last_change":"2024-05-04T03:15:45.205143+0000","last_active":"2024-05-04T03:18:46.508923+0000","last_peered":"2024-05-04T03:18:46.508923+0000","last_clean":"2024-05-04T03:18:46.508923+0000","last_became_active":"2024-05-04T03:15:45.204727+0000","last_became_peered":"2024-05-04T03:15:45.204727+0000","last_unstale":"2024-05-04T03:18:46.508923+0000","last_undegraded":"2024-05-04T03:18:46.508923+0000","last_fullsized":"2024-05-04T03:18:46.508923+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:15:05.799556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":163,"num_read_kb":163,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.163750+0000","last_change":"2024-05-04T03:15:48.042914+0000","last_active":"2024-05-04T03:17:33.163750+0000","last_peered":"2024-05-04T03:17:33.163750+0000","last_clean":"2024-05-04T03:17:33.163750+0000","last_became_active":"2024-05-04T03:15:48.041936+0000","last_became_peered":"2024-05-04T03:15:48.041936+0000","last_unstale":"2024-05-04T03:17:33.163750+0000","last_undegraded":"2024-05-04T03:17:33.163750+0000","last_fullsized":"2024-05-04T03:17:33.163750+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:11:48.807602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.331287+0000","last_change":"2024-05-04T03:15:45.011045+0000","last_active":"2024-05-04T03:16:34.331287+0000","last_peered":"2024-05-04T03:16:34.331287+0000","last_clean":"2024-05-04T03:16:34.331287+0000","last_became_active":"2024-05-04T03:15:45.010608+0000","last_became_peered":"2024-05-04T03:15:45.010608+0000","last_unstale":"2024-05-04T03:16:34.331287+0000","last_undegraded":"2024-05-04T03:16:34.331287+0000","last_fullsized":"2024-05-04T03:16:34.331287+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:00:36.243588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":102,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.333935+0000","last_change":"2024-05-04T03:14:19.895070+0000","last_active":"2024-05-04T03:16:34.333935+0000","last_peered":"2024-05-04T03:16:34.333935+0000","last_clean":"2024-05-04T03:16:34.333935+0000","last_became_active":"2024-05-04T03:14:19.889987+0000","last_became_peered":"2024-05-04T03:14:19.889987+0000","last_unstale":"2024-05-04T03:16:34.333935+0000","last_undegraded":"2024-05-04T03:16:34.333935+0000","last_fullsized":"2024-05-04T03:16:34.333935+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:11:01.712629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:11:01.712629+0000","last_clean_scrub_stamp":"2024-05-04T03:11:01.712629+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:03:02.769719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166064+0000","last_change":"2024-05-04T03:15:48.044928+0000","last_active":"2024-05-04T03:17:33.166064+0000","last_peered":"2024-05-04T03:17:33.166064+0000","last_clean":"2024-05-04T03:17:33.166064+0000","last_became_active":"2024-05-04T03:15:48.044532+0000","last_became_peered":"2024-05-04T03:15:48.044532+0000","last_unstale":"2024-05-04T03:17:33.166064+0000","last_undegraded":"2024-05-04T03:17:33.166064+0000","last_fullsized":"2024-05-04T03:17:33.166064+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:27:00.825692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334103+0000","last_change":"2024-05-04T03:15:45.014469+0000","last_active":"2024-05-04T03:16:34.334103+0000","last_peered":"2024-05-04T03:16:34.334103+0000","last_clean":"2024-05-04T03:16:34.334103+0000","last_became_active":"2024-05-04T03:15:45.013763+0000","last_became_peered":"2024-05-04T03:15:45.013763+0000","last_unstale":"2024-05-04T03:16:34.334103+0000","last_undegraded":"2024-05-04T03:16:34.334103+0000","last_fullsized":"2024-05-04T03:16:34.334103+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:19:36.379930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334623+0000","last_change":"2024-05-04T03:15:45.204536+0000","last_active":"2024-05-04T03:16:34.334623+0000","last_peered":"2024-05-04T03:16:34.334623+0000","last_clean":"2024-05-04T03:16:34.334623+0000","last_became_active":"2024-05-04T03:15:45.204072+0000","last_became_peered":"2024-05-04T03:15:45.204072+0000","last_unstale":"2024-05-04T03:16:34.334623+0000","last_undegraded":"2024-05-04T03:16:34.334623+0000","last_fullsized":"2024-05-04T03:16:34.334623+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:49:12.726220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.162922+0000","last_change":"2024-05-04T03:15:48.044982+0000","last_active":"2024-05-04T03:17:33.162922+0000","last_peered":"2024-05-04T03:17:33.162922+0000","last_clean":"2024-05-04T03:17:33.162922+0000","last_became_active":"2024-05-04T03:15:48.043996+0000","last_became_peered":"2024-05-04T03:15:48.043996+0000","last_unstale":"2024-05-04T03:17:33.162922+0000","last_undegraded":"2024-05-04T03:17:33.162922+0000","last_fullsized":"2024-05-04T03:17:33.162922+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:09:39.770023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'13","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:18:45.067828+0000","last_change":"2024-05-04T03:15:45.008153+0000","last_active":"2024-05-04T03:18:45.067828+0000","last_peered":"2024-05-04T03:18:45.067828+0000","last_clean":"2024-05-04T03:18:45.067828+0000","last_became_active":"2024-05-04T03:15:45.007698+0000","last_became_peered":"2024-05-04T03:15:45.007698+0000","last_unstale":"2024-05-04T03:18:45.067828+0000","last_undegraded":"2024-05-04T03:18:45.067828+0000","last_fullsized":"2024-05-04T03:18:45.067828+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:56:52.322049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.168750+0000","last_change":"2024-05-04T03:15:48.055767+0000","last_active":"2024-05-04T03:17:33.168750+0000","last_peered":"2024-05-04T03:17:33.168750+0000","last_clean":"2024-05-04T03:17:33.168750+0000","last_became_active":"2024-05-04T03:15:48.055252+0000","last_became_peered":"2024-05-04T03:15:48.055252+0000","last_unstale":"2024-05-04T03:17:33.168750+0000","last_undegraded":"2024-05-04T03:17:33.168750+0000","last_fullsized":"2024-05-04T03:17:33.168750+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:28:11.736803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.331539+0000","last_change":"2024-05-04T03:15:45.012838+0000","last_active":"2024-05-04T03:16:34.331539+0000","last_peered":"2024-05-04T03:16:34.331539+0000","last_clean":"2024-05-04T03:16:34.331539+0000","last_became_active":"2024-05-04T03:15:45.012429+0000","last_became_peered":"2024-05-04T03:15:45.012429+0000","last_unstale":"2024-05-04T03:16:34.331539+0000","last_undegraded":"2024-05-04T03:16:34.331539+0000","last_fullsized":"2024-05-04T03:16:34.331539+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:22:22.602551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165546+0000","last_change":"2024-05-04T03:15:48.056993+0000","last_active":"2024-05-04T03:17:33.165546+0000","last_peered":"2024-05-04T03:17:33.165546+0000","last_clean":"2024-05-04T03:17:33.165546+0000","last_became_active":"2024-05-04T03:15:48.055938+0000","last_became_peered":"2024-05-04T03:15:48.055938+0000","last_unstale":"2024-05-04T03:17:33.165546+0000","last_undegraded":"2024-05-04T03:17:33.165546+0000","last_fullsized":"2024-05-04T03:17:33.165546+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:23:59.521692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.340536+0000","last_change":"2024-05-04T03:16:34.340048+0000","last_active":"2024-05-04T03:16:34.340536+0000","last_peered":"2024-05-04T03:16:34.340536+0000","last_clean":"2024-05-04T03:16:34.340536+0000","last_became_active":"2024-05-04T03:16:34.336386+0000","last_became_peered":"2024-05-04T03:16:34.336386+0000","last_unstale":"2024-05-04T03:16:34.340536+0000","last_undegraded":"2024-05-04T03:16:34.340536+0000","last_fullsized":"2024-05-04T03:16:34.340536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:45:43.734097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.157539+0000","last_change":"2024-05-04T03:15:48.043614+0000","last_active":"2024-05-04T03:17:33.157539+0000","last_peered":"2024-05-04T03:17:33.157539+0000","last_clean":"2024-05-04T03:17:33.157539+0000","last_became_active":"2024-05-04T03:15:48.042390+0000","last_became_peered":"2024-05-04T03:15:48.042390+0000","last_unstale":"2024-05-04T03:17:33.157539+0000","last_undegraded":"2024-05-04T03:17:33.157539+0000","last_fullsized":"2024-05-04T03:17:33.157539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:56:22.791368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.416585+0000","last_change":"2024-05-04T03:15:45.005570+0000","last_active":"2024-05-04T03:16:34.416585+0000","last_peered":"2024-05-04T03:16:34.416585+0000","last_clean":"2024-05-04T03:16:34.416585+0000","last_became_active":"2024-05-04T03:15:45.005105+0000","last_became_peered":"2024-05-04T03:15:45.005105+0000","last_unstale":"2024-05-04T03:16:34.416585+0000","last_undegraded":"2024-05-04T03:16:34.416585+0000","last_fullsized":"2024-05-04T03:16:34.416585+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:40:17.275618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166899+0000","last_change":"2024-05-04T03:15:48.045081+0000","last_active":"2024-05-04T03:17:33.166899+0000","last_peered":"2024-05-04T03:17:33.166899+0000","last_clean":"2024-05-04T03:17:33.166899+0000","last_became_active":"2024-05-04T03:15:48.044745+0000","last_became_peered":"2024-05-04T03:15:48.044745+0000","last_unstale":"2024-05-04T03:17:33.166899+0000","last_undegraded":"2024-05-04T03:17:33.166899+0000","last_fullsized":"2024-05-04T03:17:33.166899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:40:53.486222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:56.748549+0000","last_change":"2024-05-04T03:15:45.008926+0000","last_active":"2024-05-04T03:17:56.748549+0000","last_peered":"2024-05-04T03:17:56.748549+0000","last_clean":"2024-05-04T03:17:56.748549+0000","last_became_active":"2024-05-04T03:15:45.008412+0000","last_became_peered":"2024-05-04T03:15:45.008412+0000","last_unstale":"2024-05-04T03:17:56.748549+0000","last_undegraded":"2024-05-04T03:17:56.748549+0000","last_fullsized":"2024-05-04T03:17:56.748549+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:34:33.107866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164466+0000","last_change":"2024-05-04T03:15:48.054580+0000","last_active":"2024-05-04T03:17:33.164466+0000","last_peered":"2024-05-04T03:17:33.164466+0000","last_clean":"2024-05-04T03:17:33.164466+0000","last_became_active":"2024-05-04T03:15:48.054242+0000","last_became_peered":"2024-05-04T03:15:48.054242+0000","last_unstale":"2024-05-04T03:17:33.164466+0000","last_undegraded":"2024-05-04T03:17:33.164466+0000","last_fullsized":"2024-05-04T03:17:33.164466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:16:19.426571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.341316+0000","last_change":"2024-05-04T03:16:34.340794+0000","last_active":"2024-05-04T03:16:34.341316+0000","last_peered":"2024-05-04T03:16:34.341316+0000","last_clean":"2024-05-04T03:16:34.341316+0000","last_became_active":"2024-05-04T03:16:34.336038+0000","last_became_peered":"2024-05-04T03:16:34.336038+0000","last_unstale":"2024-05-04T03:16:34.341316+0000","last_undegraded":"2024-05-04T03:16:34.341316+0000","last_fullsized":"2024-05-04T03:16:34.341316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:45:19.323938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.172190+0000","last_change":"2024-05-04T03:15:48.045284+0000","last_active":"2024-05-04T03:17:33.172190+0000","last_peered":"2024-05-04T03:17:33.172190+0000","last_clean":"2024-05-04T03:17:33.172190+0000","last_became_active":"2024-05-04T03:15:48.044673+0000","last_became_peered":"2024-05-04T03:15:48.044673+0000","last_unstale":"2024-05-04T03:17:33.172190+0000","last_undegraded":"2024-05-04T03:17:33.172190+0000","last_fullsized":"2024-05-04T03:17:33.172190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:51:52.389748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.330835+0000","last_change":"2024-05-04T03:15:45.205648+0000","last_active":"2024-05-04T03:16:34.330835+0000","last_peered":"2024-05-04T03:16:34.330835+0000","last_clean":"2024-05-04T03:16:34.330835+0000","last_became_active":"2024-05-04T03:15:45.204169+0000","last_became_peered":"2024-05-04T03:15:45.204169+0000","last_unstale":"2024-05-04T03:16:34.330835+0000","last_undegraded":"2024-05-04T03:16:34.330835+0000","last_fullsized":"2024-05-04T03:16:34.330835+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:20:48.859276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.171852+0000","last_change":"2024-05-04T03:15:48.049808+0000","last_active":"2024-05-04T03:17:33.171852+0000","last_peered":"2024-05-04T03:17:33.171852+0000","last_clean":"2024-05-04T03:17:33.171852+0000","last_became_active":"2024-05-04T03:15:48.048998+0000","last_became_peered":"2024-05-04T03:15:48.048998+0000","last_unstale":"2024-05-04T03:17:33.171852+0000","last_undegraded":"2024-05-04T03:17:33.171852+0000","last_fullsized":"2024-05-04T03:17:33.171852+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:54:46.786406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.417121+0000","last_change":"2024-05-04T03:15:45.008874+0000","last_active":"2024-05-04T03:16:34.417121+0000","last_peered":"2024-05-04T03:16:34.417121+0000","last_clean":"2024-05-04T03:16:34.417121+0000","last_became_active":"2024-05-04T03:15:45.008530+0000","last_became_peered":"2024-05-04T03:15:45.008530+0000","last_unstale":"2024-05-04T03:16:34.417121+0000","last_undegraded":"2024-05-04T03:16:34.417121+0000","last_fullsized":"2024-05-04T03:16:34.417121+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:31:41.697386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.168872+0000","last_change":"2024-05-04T03:15:48.046042+0000","last_active":"2024-05-04T03:17:33.168872+0000","last_peered":"2024-05-04T03:17:33.168872+0000","last_clean":"2024-05-04T03:17:33.168872+0000","last_became_active":"2024-05-04T03:15:48.045387+0000","last_became_peered":"2024-05-04T03:15:48.045387+0000","last_unstale":"2024-05-04T03:17:33.168872+0000","last_undegraded":"2024-05-04T03:17:33.168872+0000","last_fullsized":"2024-05-04T03:17:33.168872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:43:50.782623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.331686+0000","last_change":"2024-05-04T03:15:45.009126+0000","last_active":"2024-05-04T03:16:34.331686+0000","last_peered":"2024-05-04T03:16:34.331686+0000","last_clean":"2024-05-04T03:16:34.331686+0000","last_became_active":"2024-05-04T03:15:45.008802+0000","last_became_peered":"2024-05-04T03:15:45.008802+0000","last_unstale":"2024-05-04T03:16:34.331686+0000","last_undegraded":"2024-05-04T03:16:34.331686+0000","last_fullsized":"2024-05-04T03:16:34.331686+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:25:30.867870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.169092+0000","last_change":"2024-05-04T03:15:48.050478+0000","last_active":"2024-05-04T03:17:33.169092+0000","last_peered":"2024-05-04T03:17:33.169092+0000","last_clean":"2024-05-04T03:17:33.169092+0000","last_became_active":"2024-05-04T03:15:48.045756+0000","last_became_peered":"2024-05-04T03:15:48.045756+0000","last_unstale":"2024-05-04T03:17:33.169092+0000","last_undegraded":"2024-05-04T03:17:33.169092+0000","last_fullsized":"2024-05-04T03:17:33.169092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:05:11.098381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165891+0000","last_change":"2024-05-04T03:15:48.042778+0000","last_active":"2024-05-04T03:17:33.165891+0000","last_peered":"2024-05-04T03:17:33.165891+0000","last_clean":"2024-05-04T03:17:33.165891+0000","last_became_active":"2024-05-04T03:15:48.042349+0000","last_became_peered":"2024-05-04T03:15:48.042349+0000","last_unstale":"2024-05-04T03:17:33.165891+0000","last_undegraded":"2024-05-04T03:17:33.165891+0000","last_fullsized":"2024-05-04T03:17:33.165891+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:40:12.260026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.161439+0000","last_change":"2024-05-04T03:15:48.048544+0000","last_active":"2024-05-04T03:17:33.161439+0000","last_peered":"2024-05-04T03:17:33.161439+0000","last_clean":"2024-05-04T03:17:33.161439+0000","last_became_active":"2024-05-04T03:15:48.047873+0000","last_became_peered":"2024-05-04T03:15:48.047873+0000","last_unstale":"2024-05-04T03:17:33.161439+0000","last_undegraded":"2024-05-04T03:17:33.161439+0000","last_fullsized":"2024-05-04T03:17:33.161439+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:48:02.290401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166682+0000","last_change":"2024-05-04T03:15:48.053467+0000","last_active":"2024-05-04T03:17:33.166682+0000","last_peered":"2024-05-04T03:17:33.166682+0000","last_clean":"2024-05-04T03:17:33.166682+0000","last_became_active":"2024-05-04T03:15:48.050796+0000","last_became_peered":"2024-05-04T03:15:48.050796+0000","last_unstale":"2024-05-04T03:17:33.166682+0000","last_undegraded":"2024-05-04T03:17:33.166682+0000","last_fullsized":"2024-05-04T03:17:33.166682+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:11:15.009467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.159030+0000","last_change":"2024-05-04T03:15:48.054616+0000","last_active":"2024-05-04T03:17:33.159030+0000","last_peered":"2024-05-04T03:17:33.159030+0000","last_clean":"2024-05-04T03:17:33.159030+0000","last_became_active":"2024-05-04T03:15:48.054142+0000","last_became_peered":"2024-05-04T03:15:48.054142+0000","last_unstale":"2024-05-04T03:17:33.159030+0000","last_undegraded":"2024-05-04T03:17:33.159030+0000","last_fullsized":"2024-05-04T03:17:33.159030+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:18:24.067666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166465+0000","last_change":"2024-05-04T03:15:48.046969+0000","last_active":"2024-05-04T03:17:33.166465+0000","last_peered":"2024-05-04T03:17:33.166465+0000","last_clean":"2024-05-04T03:17:33.166465+0000","last_became_active":"2024-05-04T03:15:48.046637+0000","last_became_peered":"2024-05-04T03:15:48.046637+0000","last_unstale":"2024-05-04T03:17:33.166465+0000","last_undegraded":"2024-05-04T03:17:33.166465+0000","last_fullsized":"2024-05-04T03:17:33.166465+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:03:01.763037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164220+0000","last_change":"2024-05-04T03:15:48.041753+0000","last_active":"2024-05-04T03:17:33.164220+0000","last_peered":"2024-05-04T03:17:33.164220+0000","last_clean":"2024-05-04T03:17:33.164220+0000","last_became_active":"2024-05-04T03:15:48.041236+0000","last_became_peered":"2024-05-04T03:15:48.041236+0000","last_unstale":"2024-05-04T03:17:33.164220+0000","last_undegraded":"2024-05-04T03:17:33.164220+0000","last_fullsized":"2024-05-04T03:17:33.164220+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:41:02.928257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.162074+0000","last_change":"2024-05-04T03:15:48.044192+0000","last_active":"2024-05-04T03:17:33.162074+0000","last_peered":"2024-05-04T03:17:33.162074+0000","last_clean":"2024-05-04T03:17:33.162074+0000","last_became_active":"2024-05-04T03:15:48.043849+0000","last_became_peered":"2024-05-04T03:15:48.043849+0000","last_unstale":"2024-05-04T03:17:33.162074+0000","last_undegraded":"2024-05-04T03:17:33.162074+0000","last_fullsized":"2024-05-04T03:17:33.162074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:55:22.000788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164261+0000","last_change":"2024-05-04T03:15:48.045675+0000","last_active":"2024-05-04T03:17:33.164261+0000","last_peered":"2024-05-04T03:17:33.164261+0000","last_clean":"2024-05-04T03:17:33.164261+0000","last_became_active":"2024-05-04T03:15:48.045321+0000","last_became_peered":"2024-05-04T03:15:48.045321+0000","last_unstale":"2024-05-04T03:17:33.164261+0000","last_undegraded":"2024-05-04T03:17:33.164261+0000","last_fullsized":"2024-05-04T03:17:33.164261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:12:06.060729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.143245+0000","last_change":"2024-05-04T03:15:48.040872+0000","last_active":"2024-05-04T03:17:33.143245+0000","last_peered":"2024-05-04T03:17:33.143245+0000","last_clean":"2024-05-04T03:17:33.143245+0000","last_became_active":"2024-05-04T03:15:48.040485+0000","last_became_peered":"2024-05-04T03:15:48.040485+0000","last_unstale":"2024-05-04T03:17:33.143245+0000","last_undegraded":"2024-05-04T03:17:33.143245+0000","last_fullsized":"2024-05-04T03:17:33.143245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:58:01.789270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.161920+0000","last_change":"2024-05-04T03:15:48.049080+0000","last_active":"2024-05-04T03:17:33.161920+0000","last_peered":"2024-05-04T03:17:33.161920+0000","last_clean":"2024-05-04T03:17:33.161920+0000","last_became_active":"2024-05-04T03:15:48.044624+0000","last_became_peered":"2024-05-04T03:15:48.044624+0000","last_unstale":"2024-05-04T03:17:33.161920+0000","last_undegraded":"2024-05-04T03:17:33.161920+0000","last_fullsized":"2024-05-04T03:17:33.161920+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:47:08.647401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.171410+0000","last_change":"2024-05-04T03:15:48.049394+0000","last_active":"2024-05-04T03:17:33.171410+0000","last_peered":"2024-05-04T03:17:33.171410+0000","last_clean":"2024-05-04T03:17:33.171410+0000","last_became_active":"2024-05-04T03:15:48.048980+0000","last_became_peered":"2024-05-04T03:15:48.048980+0000","last_unstale":"2024-05-04T03:17:33.171410+0000","last_undegraded":"2024-05-04T03:17:33.171410+0000","last_fullsized":"2024-05-04T03:17:33.171410+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:43:24.914665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.171163+0000","last_change":"2024-05-04T03:15:48.051672+0000","last_active":"2024-05-04T03:17:33.171163+0000","last_peered":"2024-05-04T03:17:33.171163+0000","last_clean":"2024-05-04T03:17:33.171163+0000","last_became_active":"2024-05-04T03:15:48.051314+0000","last_became_peered":"2024-05-04T03:15:48.051314+0000","last_unstale":"2024-05-04T03:17:33.171163+0000","last_undegraded":"2024-05-04T03:17:33.171163+0000","last_fullsized":"2024-05-04T03:17:33.171163+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:19:22.808102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.167760+0000","last_change":"2024-05-04T03:15:48.053045+0000","last_active":"2024-05-04T03:17:33.167760+0000","last_peered":"2024-05-04T03:17:33.167760+0000","last_clean":"2024-05-04T03:17:33.167760+0000","last_became_active":"2024-05-04T03:15:48.052659+0000","last_became_peered":"2024-05-04T03:15:48.052659+0000","last_unstale":"2024-05-04T03:17:33.167760+0000","last_undegraded":"2024-05-04T03:17:33.167760+0000","last_fullsized":"2024-05-04T03:17:33.167760+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:53:25.624540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":76235,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1006,"num_read_kb":584,"num_write":188,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":228705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612660,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":440503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808889,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1609728,"data_stored":1104301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972414,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":440739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168645,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":440659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397577,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32652,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712500,"statfs":{"total":95995035648,"available":95961600000,"internally_reserved":0,"allocated":1511424,"data_stored":1033429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593806,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40380,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704772,"statfs":{"total":95995035648,"available":95953686528,"internally_reserved":0,"allocated":905216,"data_stored":440128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":511315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40400,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704752,"statfs":{"total":95995035648,"available":95953666048,"internally_reserved":0,"allocated":925696,"data_stored":441399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":1003520,"data_stored":512973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182491,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40396,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704756,"statfs":{"total":95995035648,"available":95953670144,"internally_reserved":0,"allocated":921600,"data_stored":440439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986264,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":441488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790035,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36824,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708328,"statfs":{"total":95995035648,"available":95957327872,"internally_reserved":0,"allocated":1523712,"data_stored":1031882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":73052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":73895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-04T03:18:51.594 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-04T03:18:51.594 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-04T03:18:51.854 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:18:52.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:52 smithi072 ceph-mon[33332]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:18:52.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:52 smithi072 ceph-mon[33332]: pgmap v382: 97 pgs: 97 active+clean; 651 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-04T03:18:52.861 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:52 smithi001 ceph-mon[29731]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:18:52.861 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:52 smithi001 ceph-mon[29731]: pgmap v382: 97 pgs: 97 active+clean; 651 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-04T03:18:52.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:52 smithi077 ceph-mon[33531]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:18:52.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:52 smithi077 ceph-mon[33531]: pgmap v382: 97 pgs: 97 active+clean; 651 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-05-04T03:18:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.485+0000 7f4f4e58b700 1 -- 172.21.15.1:0/3046409854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f48108740 msgr2=0x7f4f481100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.485+0000 7f4f4e58b700 1 --2- 172.21.15.1:0/3046409854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f48108740 0x7f4f481100a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4f4000b0d0 tx=0x7f4f4000b3e0 comp rx=0 tx=0).stop 2024-05-04T03:18:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.485+0000 7f4f4e58b700 1 -- 172.21.15.1:0/3046409854 shutdown_connections 2024-05-04T03:18:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.485+0000 7f4f4e58b700 1 --2- 172.21.15.1:0/3046409854 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f481034c0 0x7f4f481038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.485+0000 7f4f4e58b700 1 --2- 172.21.15.1:0/3046409854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f48108740 0x7f4f481100a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.485+0000 7f4f4e58b700 1 --2- 172.21.15.1:0/3046409854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f48107df0 0x7f4f48108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.485+0000 7f4f4e58b700 1 -- 172.21.15.1:0/3046409854 >> 172.21.15.1:0/3046409854 conn(0x7f4f480fedc0 msgr2=0x7f4f481011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.485+0000 7f4f4e58b700 1 -- 172.21.15.1:0/3046409854 shutdown_connections 2024-05-04T03:18:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.485+0000 7f4f4e58b700 1 -- 172.21.15.1:0/3046409854 wait complete. 2024-05-04T03:18:53.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.486+0000 7f4f4e58b700 1 Processor -- start 2024-05-04T03:18:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.486+0000 7f4f4e58b700 1 -- start start 2024-05-04T03:18:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.487+0000 7f4f4e58b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f481034c0 0x7f4f4819bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.487+0000 7f4f4e58b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f48107df0 0x7f4f4819c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.487+0000 7f4f4e58b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f48108740 0x7f4f4819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.487+0000 7f4f4e58b700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4f48113370 con 0x7f4f48107df0 2024-05-04T03:18:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.487+0000 7f4f4e58b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4f481131f0 con 0x7f4f481034c0 2024-05-04T03:18:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.487+0000 7f4f4e58b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4f481134f0 con 0x7f4f48108740 2024-05-04T03:18:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.487+0000 7f4f477fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f48107df0 0x7f4f4819c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.487+0000 7f4f47fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f481034c0 0x7f4f4819bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.487+0000 7f4f477fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f48107df0 0x7f4f4819c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54410/0 (socket says 172.21.15.1:54410) 2024-05-04T03:18:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.487+0000 7f4f477fe700 1 -- 172.21.15.1:0/1532746946 learned_addr learned my addr 172.21.15.1:0/1532746946 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.487+0000 7f4f4cb28700 1 --2- 172.21.15.1:0/1532746946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f48108740 0x7f4f4819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:53.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.487+0000 7f4f477fe700 1 -- 172.21.15.1:0/1532746946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f48108740 msgr2=0x7f4f4819c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.487+0000 7f4f477fe700 1 --2- 172.21.15.1:0/1532746946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f48108740 0x7f4f4819c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.487+0000 7f4f477fe700 1 -- 172.21.15.1:0/1532746946 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f481034c0 msgr2=0x7f4f4819bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.487+0000 7f4f477fe700 1 --2- 172.21.15.1:0/1532746946 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f481034c0 0x7f4f4819bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.487+0000 7f4f477fe700 1 -- 172.21.15.1:0/1532746946 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f481a6f80 con 0x7f4f48107df0 2024-05-04T03:18:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.487+0000 7f4f4cb28700 1 --2- 172.21.15.1:0/1532746946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f48108740 0x7f4f4819c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:18:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.487+0000 7f4f47fff700 1 --2- 172.21.15.1:0/1532746946 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f481034c0 0x7f4f4819bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:18:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.487+0000 7f4f477fe700 1 --2- 172.21.15.1:0/1532746946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f48107df0 0x7f4f4819c150 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800c8f0 tx=0x7f4f3800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.488+0000 7f4f457fa700 1 -- 172.21.15.1:0/1532746946 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f38007940 con 0x7f4f48107df0 2024-05-04T03:18:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.488+0000 7f4f4e58b700 1 -- 172.21.15.1:0/1532746946 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f481a7270 con 0x7f4f48107df0 2024-05-04T03:18:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.488+0000 7f4f457fa700 1 -- 172.21.15.1:0/1532746946 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f3800f450 con 0x7f4f48107df0 2024-05-04T03:18:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.488+0000 7f4f457fa700 1 -- 172.21.15.1:0/1532746946 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f38018660 con 0x7f4f48107df0 2024-05-04T03:18:53.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.488+0000 7f4f4e58b700 1 -- 172.21.15.1:0/1532746946 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f481a7800 con 0x7f4f48107df0 2024-05-04T03:18:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.489+0000 7f4f4e58b700 1 -- 172.21.15.1:0/1532746946 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f480512c0 con 0x7f4f48107df0 2024-05-04T03:18:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.490+0000 7f4f457fa700 1 -- 172.21.15.1:0/1532746946 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7f4f3800f890 con 0x7f4f48107df0 2024-05-04T03:18:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.491+0000 7f4f457fa700 1 --2- 172.21.15.1:0/1532746946 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4f340b2910 0x7f4f340b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:53.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.491+0000 7f4f47fff700 1 --2- 172.21.15.1:0/1532746946 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4f340b2910 0x7f4f340b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:53.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.491+0000 7f4f457fa700 1 -- 172.21.15.1:0/1532746946 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f4f380e2220 con 0x7f4f48107df0 2024-05-04T03:18:53.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.492+0000 7f4f47fff700 1 --2- 172.21.15.1:0/1532746946 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4f340b2910 0x7f4f340b4e20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4f30005950 tx=0x7f4f3000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:53.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.495+0000 7f4f457fa700 1 -- 172.21.15.1:0/1532746946 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4f38014030 con 0x7f4f48107df0 2024-05-04T03:18:53.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.707+0000 7f4f4e58b700 1 -- 172.21.15.1:0/1532746946 --> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4f48063c00 con 0x7f4f340b2910 2024-05-04T03:18:53.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.716+0000 7f4f457fa700 1 -- 172.21.15.1:0/1532746946 <== mgr.14150 v2:172.21.15.1:6800/402647351 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258827 (secure 0 0 0) 0x7f4f48063c00 con 0x7f4f340b2910 2024-05-04T03:18:53.718 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:18:53.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.718+0000 7f4f4e58b700 1 -- 172.21.15.1:0/1532746946 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4f340b2910 msgr2=0x7f4f340b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:53.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.718+0000 7f4f4e58b700 1 --2- 172.21.15.1:0/1532746946 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4f340b2910 0x7f4f340b4e20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4f30005950 tx=0x7f4f3000b410 comp rx=0 tx=0).stop 2024-05-04T03:18:53.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.718+0000 7f4f4e58b700 1 -- 172.21.15.1:0/1532746946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f48107df0 msgr2=0x7f4f4819c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:53.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.718+0000 7f4f4e58b700 1 --2- 172.21.15.1:0/1532746946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f48107df0 0x7f4f4819c150 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800c8f0 tx=0x7f4f3800cc00 comp rx=0 tx=0).stop 2024-05-04T03:18:53.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.718+0000 7f4f4e58b700 1 -- 172.21.15.1:0/1532746946 shutdown_connections 2024-05-04T03:18:53.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.718+0000 7f4f4e58b700 1 --2- 172.21.15.1:0/1532746946 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f481034c0 0x7f4f4819bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:53.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.718+0000 7f4f4e58b700 1 --2- 172.21.15.1:0/1532746946 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4f340b2910 0x7f4f340b4e20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:53.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.718+0000 7f4f4e58b700 1 --2- 172.21.15.1:0/1532746946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f48108740 0x7f4f4819c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:53.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.718+0000 7f4f4e58b700 1 --2- 172.21.15.1:0/1532746946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f48107df0 0x7f4f4819c150 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:53.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.718+0000 7f4f4e58b700 1 -- 172.21.15.1:0/1532746946 >> 172.21.15.1:0/1532746946 conn(0x7f4f480fedc0 msgr2=0x7f4f48107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:53.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.718+0000 7f4f4e58b700 1 -- 172.21.15.1:0/1532746946 shutdown_connections 2024-05-04T03:18:53.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:53.718+0000 7f4f4e58b700 1 -- 172.21.15.1:0/1532746946 wait complete. 2024-05-04T03:18:53.726 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-05-04T03:18:54.505 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-05-04T03:18:53.187408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669357,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1104,"num_read_kb":662,"num_write":314,"num_write_kb":2293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":754,"ondisk_log_size":754,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":411792,"kb_used_data":12720,"kb_used_omap":18,"kb_used_meta":398957,"kb_avail":1124530032,"statfs":{"total":1151940427776,"available":1151518752768,"internally_reserved":0,"allocated":13025280,"data_stored":7287123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":408532281},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8232,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":51,"num_write":17,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003244"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.173150+0000","last_change":"2024-05-04T03:15:48.049361+0000","last_active":"2024-05-04T03:17:33.173150+0000","last_peered":"2024-05-04T03:17:33.173150+0000","last_clean":"2024-05-04T03:17:33.173150+0000","last_became_active":"2024-05-04T03:15:48.048876+0000","last_became_peered":"2024-05-04T03:15:48.048876+0000","last_unstale":"2024-05-04T03:17:33.173150+0000","last_undegraded":"2024-05-04T03:17:33.173150+0000","last_fullsized":"2024-05-04T03:17:33.173150+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:50:52.568516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165921+0000","last_change":"2024-05-04T03:15:48.055554+0000","last_active":"2024-05-04T03:17:33.165921+0000","last_peered":"2024-05-04T03:17:33.165921+0000","last_clean":"2024-05-04T03:17:33.165921+0000","last_became_active":"2024-05-04T03:15:48.055155+0000","last_became_peered":"2024-05-04T03:15:48.055155+0000","last_unstale":"2024-05-04T03:17:33.165921+0000","last_undegraded":"2024-05-04T03:17:33.165921+0000","last_fullsized":"2024-05-04T03:17:33.165921+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:38:07.824031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.167129+0000","last_change":"2024-05-04T03:15:48.047974+0000","last_active":"2024-05-04T03:17:33.167129+0000","last_peered":"2024-05-04T03:17:33.167129+0000","last_clean":"2024-05-04T03:17:33.167129+0000","last_became_active":"2024-05-04T03:15:48.044299+0000","last_became_peered":"2024-05-04T03:15:48.044299+0000","last_unstale":"2024-05-04T03:17:33.167129+0000","last_undegraded":"2024-05-04T03:17:33.167129+0000","last_fullsized":"2024-05-04T03:17:33.167129+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:49:08.014987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.156171+0000","last_change":"2024-05-04T03:15:48.050899+0000","last_active":"2024-05-04T03:17:33.156171+0000","last_peered":"2024-05-04T03:17:33.156171+0000","last_clean":"2024-05-04T03:17:33.156171+0000","last_became_active":"2024-05-04T03:15:48.048812+0000","last_became_peered":"2024-05-04T03:15:48.048812+0000","last_unstale":"2024-05-04T03:17:33.156171+0000","last_undegraded":"2024-05-04T03:17:33.156171+0000","last_fullsized":"2024-05-04T03:17:33.156171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:21:01.829349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165098+0000","last_change":"2024-05-04T03:15:48.044403+0000","last_active":"2024-05-04T03:17:33.165098+0000","last_peered":"2024-05-04T03:17:33.165098+0000","last_clean":"2024-05-04T03:17:33.165098+0000","last_became_active":"2024-05-04T03:15:48.044021+0000","last_became_peered":"2024-05-04T03:15:48.044021+0000","last_unstale":"2024-05-04T03:17:33.165098+0000","last_undegraded":"2024-05-04T03:17:33.165098+0000","last_fullsized":"2024-05-04T03:17:33.165098+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:50:54.762070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.167642+0000","last_change":"2024-05-04T03:15:48.053233+0000","last_active":"2024-05-04T03:17:33.167642+0000","last_peered":"2024-05-04T03:17:33.167642+0000","last_clean":"2024-05-04T03:17:33.167642+0000","last_became_active":"2024-05-04T03:15:48.052809+0000","last_became_peered":"2024-05-04T03:15:48.052809+0000","last_unstale":"2024-05-04T03:17:33.167642+0000","last_undegraded":"2024-05-04T03:17:33.167642+0000","last_fullsized":"2024-05-04T03:17:33.167642+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:27:20.082694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.156722+0000","last_change":"2024-05-04T03:15:48.045108+0000","last_active":"2024-05-04T03:17:33.156722+0000","last_peered":"2024-05-04T03:17:33.156722+0000","last_clean":"2024-05-04T03:17:33.156722+0000","last_became_active":"2024-05-04T03:15:48.044719+0000","last_became_peered":"2024-05-04T03:15:48.044719+0000","last_unstale":"2024-05-04T03:17:33.156722+0000","last_undegraded":"2024-05-04T03:17:33.156722+0000","last_fullsized":"2024-05-04T03:17:33.156722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:55:02.586294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164990+0000","last_change":"2024-05-04T03:15:48.043559+0000","last_active":"2024-05-04T03:17:33.164990+0000","last_peered":"2024-05-04T03:17:33.164990+0000","last_clean":"2024-05-04T03:17:33.164990+0000","last_became_active":"2024-05-04T03:15:48.043138+0000","last_became_peered":"2024-05-04T03:15:48.043138+0000","last_unstale":"2024-05-04T03:17:33.164990+0000","last_undegraded":"2024-05-04T03:17:33.164990+0000","last_fullsized":"2024-05-04T03:17:33.164990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:05:25.411916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.174068+0000","last_change":"2024-05-04T03:15:48.039632+0000","last_active":"2024-05-04T03:17:33.174068+0000","last_peered":"2024-05-04T03:17:33.174068+0000","last_clean":"2024-05-04T03:17:33.174068+0000","last_became_active":"2024-05-04T03:15:48.039231+0000","last_became_peered":"2024-05-04T03:15:48.039231+0000","last_unstale":"2024-05-04T03:17:33.174068+0000","last_undegraded":"2024-05-04T03:17:33.174068+0000","last_fullsized":"2024-05-04T03:17:33.174068+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:33:12.853172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.168249+0000","last_change":"2024-05-04T03:15:48.055138+0000","last_active":"2024-05-04T03:17:33.168249+0000","last_peered":"2024-05-04T03:17:33.168249+0000","last_clean":"2024-05-04T03:17:33.168249+0000","last_became_active":"2024-05-04T03:15:48.054791+0000","last_became_peered":"2024-05-04T03:15:48.054791+0000","last_unstale":"2024-05-04T03:17:33.168249+0000","last_undegraded":"2024-05-04T03:17:33.168249+0000","last_fullsized":"2024-05-04T03:17:33.168249+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:33:36.822549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165773+0000","last_change":"2024-05-04T03:15:48.050125+0000","last_active":"2024-05-04T03:17:33.165773+0000","last_peered":"2024-05-04T03:17:33.165773+0000","last_clean":"2024-05-04T03:17:33.165773+0000","last_became_active":"2024-05-04T03:15:48.049390+0000","last_became_peered":"2024-05-04T03:15:48.049390+0000","last_unstale":"2024-05-04T03:17:33.165773+0000","last_undegraded":"2024-05-04T03:17:33.165773+0000","last_fullsized":"2024-05-04T03:17:33.165773+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:27:05.740383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.163474+0000","last_change":"2024-05-04T03:15:48.050700+0000","last_active":"2024-05-04T03:17:33.163474+0000","last_peered":"2024-05-04T03:17:33.163474+0000","last_clean":"2024-05-04T03:17:33.163474+0000","last_became_active":"2024-05-04T03:15:48.049288+0000","last_became_peered":"2024-05-04T03:15:48.049288+0000","last_unstale":"2024-05-04T03:17:33.163474+0000","last_undegraded":"2024-05-04T03:17:33.163474+0000","last_fullsized":"2024-05-04T03:17:33.163474+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:40:07.072006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.168343+0000","last_change":"2024-05-04T03:16:34.337123+0000","last_active":"2024-05-04T03:17:33.168343+0000","last_peered":"2024-05-04T03:17:33.168343+0000","last_clean":"2024-05-04T03:17:33.168343+0000","last_became_active":"2024-05-04T03:16:34.336657+0000","last_became_peered":"2024-05-04T03:16:34.336657+0000","last_unstale":"2024-05-04T03:17:33.168343+0000","last_undegraded":"2024-05-04T03:17:33.168343+0000","last_fullsized":"2024-05-04T03:17:33.168343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:58:15.939941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166400+0000","last_change":"2024-05-04T03:15:48.047258+0000","last_active":"2024-05-04T03:17:33.166400+0000","last_peered":"2024-05-04T03:17:33.166400+0000","last_clean":"2024-05-04T03:17:33.166400+0000","last_became_active":"2024-05-04T03:15:48.046897+0000","last_became_peered":"2024-05-04T03:15:48.046897+0000","last_unstale":"2024-05-04T03:17:33.166400+0000","last_undegraded":"2024-05-04T03:17:33.166400+0000","last_fullsized":"2024-05-04T03:17:33.166400+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:14:39.217475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166318+0000","last_change":"2024-05-04T03:15:48.051240+0000","last_active":"2024-05-04T03:17:33.166318+0000","last_peered":"2024-05-04T03:17:33.166318+0000","last_clean":"2024-05-04T03:17:33.166318+0000","last_became_active":"2024-05-04T03:15:48.050081+0000","last_became_peered":"2024-05-04T03:15:48.050081+0000","last_unstale":"2024-05-04T03:17:33.166318+0000","last_undegraded":"2024-05-04T03:17:33.166318+0000","last_fullsized":"2024-05-04T03:17:33.166318+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:39:06.773121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.171066+0000","last_change":"2024-05-04T03:15:48.050936+0000","last_active":"2024-05-04T03:17:33.171066+0000","last_peered":"2024-05-04T03:17:33.171066+0000","last_clean":"2024-05-04T03:17:33.171066+0000","last_became_active":"2024-05-04T03:15:48.050567+0000","last_became_peered":"2024-05-04T03:15:48.050567+0000","last_unstale":"2024-05-04T03:17:33.171066+0000","last_undegraded":"2024-05-04T03:17:33.171066+0000","last_fullsized":"2024-05-04T03:17:33.171066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:17:43.469889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.173267+0000","last_change":"2024-05-04T03:15:48.047200+0000","last_active":"2024-05-04T03:17:33.173267+0000","last_peered":"2024-05-04T03:17:33.173267+0000","last_clean":"2024-05-04T03:17:33.173267+0000","last_became_active":"2024-05-04T03:15:48.046782+0000","last_became_peered":"2024-05-04T03:15:48.046782+0000","last_unstale":"2024-05-04T03:17:33.173267+0000","last_undegraded":"2024-05-04T03:17:33.173267+0000","last_fullsized":"2024-05-04T03:17:33.173267+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:32:56.773916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.154784+0000","last_change":"2024-05-04T03:15:48.042114+0000","last_active":"2024-05-04T03:17:33.154784+0000","last_peered":"2024-05-04T03:17:33.154784+0000","last_clean":"2024-05-04T03:17:33.154784+0000","last_became_active":"2024-05-04T03:15:48.040921+0000","last_became_peered":"2024-05-04T03:15:48.040921+0000","last_unstale":"2024-05-04T03:17:33.154784+0000","last_undegraded":"2024-05-04T03:17:33.154784+0000","last_fullsized":"2024-05-04T03:17:33.154784+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:16:20.130996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164439+0000","last_change":"2024-05-04T03:15:48.051060+0000","last_active":"2024-05-04T03:17:33.164439+0000","last_peered":"2024-05-04T03:17:33.164439+0000","last_clean":"2024-05-04T03:17:33.164439+0000","last_became_active":"2024-05-04T03:15:48.050642+0000","last_became_peered":"2024-05-04T03:15:48.050642+0000","last_unstale":"2024-05-04T03:17:33.164439+0000","last_undegraded":"2024-05-04T03:17:33.164439+0000","last_fullsized":"2024-05-04T03:17:33.164439+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:23:31.193072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.329355+0000","last_change":"2024-05-04T03:15:45.013477+0000","last_active":"2024-05-04T03:16:34.329355+0000","last_peered":"2024-05-04T03:16:34.329355+0000","last_clean":"2024-05-04T03:16:34.329355+0000","last_became_active":"2024-05-04T03:15:45.013161+0000","last_became_peered":"2024-05-04T03:15:45.013161+0000","last_unstale":"2024-05-04T03:16:34.329355+0000","last_undegraded":"2024-05-04T03:16:34.329355+0000","last_fullsized":"2024-05-04T03:16:34.329355+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:08:14.087090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164546+0000","last_change":"2024-05-04T03:15:48.045810+0000","last_active":"2024-05-04T03:17:33.164546+0000","last_peered":"2024-05-04T03:17:33.164546+0000","last_clean":"2024-05-04T03:17:33.164546+0000","last_became_active":"2024-05-04T03:15:48.045449+0000","last_became_peered":"2024-05-04T03:15:48.045449+0000","last_unstale":"2024-05-04T03:17:33.164546+0000","last_undegraded":"2024-05-04T03:17:33.164546+0000","last_fullsized":"2024-05-04T03:17:33.164546+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:48:05.084208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.417032+0000","last_change":"2024-05-04T03:15:45.204196+0000","last_active":"2024-05-04T03:16:34.417032+0000","last_peered":"2024-05-04T03:16:34.417032+0000","last_clean":"2024-05-04T03:16:34.417032+0000","last_became_active":"2024-05-04T03:15:45.203821+0000","last_became_peered":"2024-05-04T03:15:45.203821+0000","last_unstale":"2024-05-04T03:16:34.417032+0000","last_undegraded":"2024-05-04T03:16:34.417032+0000","last_fullsized":"2024-05-04T03:16:34.417032+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:17:05.074450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.167259+0000","last_change":"2024-05-04T03:15:48.045131+0000","last_active":"2024-05-04T03:17:33.167259+0000","last_peered":"2024-05-04T03:17:33.167259+0000","last_clean":"2024-05-04T03:17:33.167259+0000","last_became_active":"2024-05-04T03:15:48.044793+0000","last_became_peered":"2024-05-04T03:15:48.044793+0000","last_unstale":"2024-05-04T03:17:33.167259+0000","last_undegraded":"2024-05-04T03:17:33.167259+0000","last_fullsized":"2024-05-04T03:17:33.167259+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:03:37.769324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334752+0000","last_change":"2024-05-04T03:15:45.013644+0000","last_active":"2024-05-04T03:16:34.334752+0000","last_peered":"2024-05-04T03:16:34.334752+0000","last_clean":"2024-05-04T03:16:34.334752+0000","last_became_active":"2024-05-04T03:15:45.013282+0000","last_became_peered":"2024-05-04T03:15:45.013282+0000","last_unstale":"2024-05-04T03:16:34.334752+0000","last_undegraded":"2024-05-04T03:16:34.334752+0000","last_fullsized":"2024-05-04T03:16:34.334752+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:21:21.204433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.174582+0000","last_change":"2024-05-04T03:15:48.043046+0000","last_active":"2024-05-04T03:17:33.174582+0000","last_peered":"2024-05-04T03:17:33.174582+0000","last_clean":"2024-05-04T03:17:33.174582+0000","last_became_active":"2024-05-04T03:15:48.042621+0000","last_became_peered":"2024-05-04T03:15:48.042621+0000","last_unstale":"2024-05-04T03:17:33.174582+0000","last_undegraded":"2024-05-04T03:17:33.174582+0000","last_fullsized":"2024-05-04T03:17:33.174582+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:03:58.916423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334940+0000","last_change":"2024-05-04T03:15:45.010618+0000","last_active":"2024-05-04T03:16:34.334940+0000","last_peered":"2024-05-04T03:16:34.334940+0000","last_clean":"2024-05-04T03:16:34.334940+0000","last_became_active":"2024-05-04T03:15:45.009239+0000","last_became_peered":"2024-05-04T03:15:45.009239+0000","last_unstale":"2024-05-04T03:16:34.334940+0000","last_undegraded":"2024-05-04T03:16:34.334940+0000","last_fullsized":"2024-05-04T03:16:34.334940+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:49:46.777571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.145981+0000","last_change":"2024-05-04T03:15:48.049334+0000","last_active":"2024-05-04T03:17:33.145981+0000","last_peered":"2024-05-04T03:17:33.145981+0000","last_clean":"2024-05-04T03:17:33.145981+0000","last_became_active":"2024-05-04T03:15:48.048984+0000","last_became_peered":"2024-05-04T03:15:48.048984+0000","last_unstale":"2024-05-04T03:17:33.145981+0000","last_undegraded":"2024-05-04T03:17:33.145981+0000","last_fullsized":"2024-05-04T03:17:33.145981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:28:25.366106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":200,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:18:47.512444+0000","last_change":"2024-05-04T03:15:45.015277+0000","last_active":"2024-05-04T03:18:47.512444+0000","last_peered":"2024-05-04T03:18:47.512444+0000","last_clean":"2024-05-04T03:18:47.512444+0000","last_became_active":"2024-05-04T03:15:45.014542+0000","last_became_peered":"2024-05-04T03:15:45.014542+0000","last_unstale":"2024-05-04T03:18:47.512444+0000","last_undegraded":"2024-05-04T03:18:47.512444+0000","last_fullsized":"2024-05-04T03:18:47.512444+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:22:15.594866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":167,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.147597+0000","last_change":"2024-05-04T03:15:48.045029+0000","last_active":"2024-05-04T03:17:33.147597+0000","last_peered":"2024-05-04T03:17:33.147597+0000","last_clean":"2024-05-04T03:17:33.147597+0000","last_became_active":"2024-05-04T03:15:48.041262+0000","last_became_peered":"2024-05-04T03:15:48.041262+0000","last_unstale":"2024-05-04T03:17:33.147597+0000","last_undegraded":"2024-05-04T03:17:33.147597+0000","last_fullsized":"2024-05-04T03:17:33.147597+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:45:56.882957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334215+0000","last_change":"2024-05-04T03:15:45.007530+0000","last_active":"2024-05-04T03:16:34.334215+0000","last_peered":"2024-05-04T03:16:34.334215+0000","last_clean":"2024-05-04T03:16:34.334215+0000","last_became_active":"2024-05-04T03:15:45.007054+0000","last_became_peered":"2024-05-04T03:15:45.007054+0000","last_unstale":"2024-05-04T03:16:34.334215+0000","last_undegraded":"2024-05-04T03:16:34.334215+0000","last_fullsized":"2024-05-04T03:16:34.334215+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:09:13.845679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.142714+0000","last_change":"2024-05-04T03:15:48.050956+0000","last_active":"2024-05-04T03:17:33.142714+0000","last_peered":"2024-05-04T03:17:33.142714+0000","last_clean":"2024-05-04T03:17:33.142714+0000","last_became_active":"2024-05-04T03:15:48.050597+0000","last_became_peered":"2024-05-04T03:15:48.050597+0000","last_unstale":"2024-05-04T03:17:33.142714+0000","last_undegraded":"2024-05-04T03:17:33.142714+0000","last_fullsized":"2024-05-04T03:17:33.142714+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:27:34.127889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.331600+0000","last_change":"2024-05-04T03:15:45.007027+0000","last_active":"2024-05-04T03:16:34.331600+0000","last_peered":"2024-05-04T03:16:34.331600+0000","last_clean":"2024-05-04T03:16:34.331600+0000","last_became_active":"2024-05-04T03:15:45.006659+0000","last_became_peered":"2024-05-04T03:15:45.006659+0000","last_unstale":"2024-05-04T03:16:34.331600+0000","last_undegraded":"2024-05-04T03:16:34.331600+0000","last_fullsized":"2024-05-04T03:16:34.331600+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:06:11.827093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164377+0000","last_change":"2024-05-04T03:15:48.046196+0000","last_active":"2024-05-04T03:17:33.164377+0000","last_peered":"2024-05-04T03:17:33.164377+0000","last_clean":"2024-05-04T03:17:33.164377+0000","last_became_active":"2024-05-04T03:15:48.044316+0000","last_became_peered":"2024-05-04T03:15:48.044316+0000","last_unstale":"2024-05-04T03:17:33.164377+0000","last_undegraded":"2024-05-04T03:17:33.164377+0000","last_fullsized":"2024-05-04T03:17:33.164377+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:04:39.301705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.416682+0000","last_change":"2024-05-04T03:15:45.015360+0000","last_active":"2024-05-04T03:16:34.416682+0000","last_peered":"2024-05-04T03:16:34.416682+0000","last_clean":"2024-05-04T03:16:34.416682+0000","last_became_active":"2024-05-04T03:15:45.014696+0000","last_became_peered":"2024-05-04T03:15:45.014696+0000","last_unstale":"2024-05-04T03:16:34.416682+0000","last_undegraded":"2024-05-04T03:16:34.416682+0000","last_fullsized":"2024-05-04T03:16:34.416682+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:17:28.838651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.163052+0000","last_change":"2024-05-04T03:15:48.039823+0000","last_active":"2024-05-04T03:17:33.163052+0000","last_peered":"2024-05-04T03:17:33.163052+0000","last_clean":"2024-05-04T03:17:33.163052+0000","last_became_active":"2024-05-04T03:15:48.039447+0000","last_became_peered":"2024-05-04T03:15:48.039447+0000","last_unstale":"2024-05-04T03:17:33.163052+0000","last_undegraded":"2024-05-04T03:17:33.163052+0000","last_fullsized":"2024-05-04T03:17:33.163052+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:45:55.136946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":199,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:18:51.518317+0000","last_change":"2024-05-04T03:15:45.004662+0000","last_active":"2024-05-04T03:18:51.518317+0000","last_peered":"2024-05-04T03:18:51.518317+0000","last_clean":"2024-05-04T03:18:51.518317+0000","last_became_active":"2024-05-04T03:15:45.004265+0000","last_became_peered":"2024-05-04T03:15:45.004265+0000","last_unstale":"2024-05-04T03:18:51.518317+0000","last_undegraded":"2024-05-04T03:18:51.518317+0000","last_fullsized":"2024-05-04T03:18:51.518317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:14:03.065346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":168,"num_read_kb":168,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.173995+0000","last_change":"2024-05-04T03:15:48.042093+0000","last_active":"2024-05-04T03:17:33.173995+0000","last_peered":"2024-05-04T03:17:33.173995+0000","last_clean":"2024-05-04T03:17:33.173995+0000","last_became_active":"2024-05-04T03:15:48.041646+0000","last_became_peered":"2024-05-04T03:15:48.041646+0000","last_unstale":"2024-05-04T03:17:33.173995+0000","last_undegraded":"2024-05-04T03:17:33.173995+0000","last_fullsized":"2024-05-04T03:17:33.173995+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:06:32.083493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.335070+0000","last_change":"2024-05-04T03:15:45.016731+0000","last_active":"2024-05-04T03:16:34.335070+0000","last_peered":"2024-05-04T03:16:34.335070+0000","last_clean":"2024-05-04T03:16:34.335070+0000","last_became_active":"2024-05-04T03:15:45.016313+0000","last_became_peered":"2024-05-04T03:15:45.016313+0000","last_unstale":"2024-05-04T03:16:34.335070+0000","last_undegraded":"2024-05-04T03:16:34.335070+0000","last_fullsized":"2024-05-04T03:16:34.335070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:42:40.830986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166326+0000","last_change":"2024-05-04T03:15:48.053649+0000","last_active":"2024-05-04T03:17:33.166326+0000","last_peered":"2024-05-04T03:17:33.166326+0000","last_clean":"2024-05-04T03:17:33.166326+0000","last_became_active":"2024-05-04T03:15:48.052407+0000","last_became_peered":"2024-05-04T03:15:48.052407+0000","last_unstale":"2024-05-04T03:17:33.166326+0000","last_undegraded":"2024-05-04T03:17:33.166326+0000","last_fullsized":"2024-05-04T03:17:33.166326+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:37:28.359272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.417301+0000","last_change":"2024-05-04T03:15:45.205180+0000","last_active":"2024-05-04T03:16:34.417301+0000","last_peered":"2024-05-04T03:16:34.417301+0000","last_clean":"2024-05-04T03:16:34.417301+0000","last_became_active":"2024-05-04T03:15:45.204828+0000","last_became_peered":"2024-05-04T03:15:45.204828+0000","last_unstale":"2024-05-04T03:16:34.417301+0000","last_undegraded":"2024-05-04T03:16:34.417301+0000","last_fullsized":"2024-05-04T03:16:34.417301+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:27:34.063540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.171291+0000","last_change":"2024-05-04T03:15:48.047154+0000","last_active":"2024-05-04T03:17:33.171291+0000","last_peered":"2024-05-04T03:17:33.171291+0000","last_clean":"2024-05-04T03:17:33.171291+0000","last_became_active":"2024-05-04T03:15:48.046794+0000","last_became_peered":"2024-05-04T03:15:48.046794+0000","last_unstale":"2024-05-04T03:17:33.171291+0000","last_undegraded":"2024-05-04T03:17:33.171291+0000","last_fullsized":"2024-05-04T03:17:33.171291+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:21:42.723252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'40","reported_seq":416,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:18:52.519902+0000","last_change":"2024-05-04T03:15:45.008930+0000","last_active":"2024-05-04T03:18:52.519902+0000","last_peered":"2024-05-04T03:18:52.519902+0000","last_clean":"2024-05-04T03:18:52.519902+0000","last_became_active":"2024-05-04T03:15:45.008459+0000","last_became_peered":"2024-05-04T03:15:45.008459+0000","last_unstale":"2024-05-04T03:18:52.519902+0000","last_undegraded":"2024-05-04T03:18:52.519902+0000","last_fullsized":"2024-05-04T03:18:52.519902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:07:52.609156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73914,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":349,"num_read_kb":247,"num_write":41,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.143904+0000","last_change":"2024-05-04T03:15:48.051848+0000","last_active":"2024-05-04T03:17:33.143904+0000","last_peered":"2024-05-04T03:17:33.143904+0000","last_clean":"2024-05-04T03:17:33.143904+0000","last_became_active":"2024-05-04T03:15:48.051171+0000","last_became_peered":"2024-05-04T03:15:48.051171+0000","last_unstale":"2024-05-04T03:17:33.143904+0000","last_undegraded":"2024-05-04T03:17:33.143904+0000","last_fullsized":"2024-05-04T03:17:33.143904+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:20:32.722845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.330397+0000","last_change":"2024-05-04T03:15:45.007970+0000","last_active":"2024-05-04T03:16:34.330397+0000","last_peered":"2024-05-04T03:16:34.330397+0000","last_clean":"2024-05-04T03:16:34.330397+0000","last_became_active":"2024-05-04T03:15:45.007605+0000","last_became_peered":"2024-05-04T03:15:45.007605+0000","last_unstale":"2024-05-04T03:16:34.330397+0000","last_undegraded":"2024-05-04T03:16:34.330397+0000","last_fullsized":"2024-05-04T03:16:34.330397+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:52:08.466858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.169073+0000","last_change":"2024-05-04T03:15:48.050647+0000","last_active":"2024-05-04T03:17:33.169073+0000","last_peered":"2024-05-04T03:17:33.169073+0000","last_clean":"2024-05-04T03:17:33.169073+0000","last_became_active":"2024-05-04T03:15:48.050296+0000","last_became_peered":"2024-05-04T03:15:48.050296+0000","last_unstale":"2024-05-04T03:17:33.169073+0000","last_undegraded":"2024-05-04T03:17:33.169073+0000","last_fullsized":"2024-05-04T03:17:33.169073+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:14:23.902094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.416945+0000","last_change":"2024-05-04T03:15:45.014936+0000","last_active":"2024-05-04T03:16:34.416945+0000","last_peered":"2024-05-04T03:16:34.416945+0000","last_clean":"2024-05-04T03:16:34.416945+0000","last_became_active":"2024-05-04T03:15:45.014474+0000","last_became_peered":"2024-05-04T03:15:45.014474+0000","last_unstale":"2024-05-04T03:16:34.416945+0000","last_undegraded":"2024-05-04T03:16:34.416945+0000","last_fullsized":"2024-05-04T03:16:34.416945+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:26:37.972523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.169574+0000","last_change":"2024-05-04T03:16:34.336836+0000","last_active":"2024-05-04T03:17:33.169574+0000","last_peered":"2024-05-04T03:17:33.169574+0000","last_clean":"2024-05-04T03:17:33.169574+0000","last_became_active":"2024-05-04T03:16:34.336008+0000","last_became_peered":"2024-05-04T03:16:34.336008+0000","last_unstale":"2024-05-04T03:17:33.169574+0000","last_undegraded":"2024-05-04T03:17:33.169574+0000","last_fullsized":"2024-05-04T03:17:33.169574+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:51:52.803138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'6","reported_seq":202,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:18:47.511553+0000","last_change":"2024-05-04T03:15:45.013177+0000","last_active":"2024-05-04T03:18:47.511553+0000","last_peered":"2024-05-04T03:18:47.511553+0000","last_clean":"2024-05-04T03:18:47.511553+0000","last_became_active":"2024-05-04T03:15:45.012764+0000","last_became_peered":"2024-05-04T03:15:45.012764+0000","last_unstale":"2024-05-04T03:18:47.511553+0000","last_undegraded":"2024-05-04T03:18:47.511553+0000","last_fullsized":"2024-05-04T03:18:47.511553+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:07:43.578443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":170,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.159654+0000","last_change":"2024-05-04T03:15:48.046453+0000","last_active":"2024-05-04T03:17:33.159654+0000","last_peered":"2024-05-04T03:17:33.159654+0000","last_clean":"2024-05-04T03:17:33.159654+0000","last_became_active":"2024-05-04T03:15:48.045990+0000","last_became_peered":"2024-05-04T03:15:48.045990+0000","last_unstale":"2024-05-04T03:17:33.159654+0000","last_undegraded":"2024-05-04T03:17:33.159654+0000","last_fullsized":"2024-05-04T03:17:33.159654+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:54:57.285122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334414+0000","last_change":"2024-05-04T03:15:45.205539+0000","last_active":"2024-05-04T03:16:34.334414+0000","last_peered":"2024-05-04T03:16:34.334414+0000","last_clean":"2024-05-04T03:16:34.334414+0000","last_became_active":"2024-05-04T03:15:45.205174+0000","last_became_peered":"2024-05-04T03:15:45.205174+0000","last_unstale":"2024-05-04T03:16:34.334414+0000","last_undegraded":"2024-05-04T03:16:34.334414+0000","last_fullsized":"2024-05-04T03:16:34.334414+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:03:53.398368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.141832+0000","last_change":"2024-05-04T03:15:48.051824+0000","last_active":"2024-05-04T03:17:33.141832+0000","last_peered":"2024-05-04T03:17:33.141832+0000","last_clean":"2024-05-04T03:17:33.141832+0000","last_became_active":"2024-05-04T03:15:48.051483+0000","last_became_peered":"2024-05-04T03:15:48.051483+0000","last_unstale":"2024-05-04T03:17:33.141832+0000","last_undegraded":"2024-05-04T03:17:33.141832+0000","last_fullsized":"2024-05-04T03:17:33.141832+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:57:56.644486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'3","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:18:03.949283+0000","last_change":"2024-05-04T03:15:45.011062+0000","last_active":"2024-05-04T03:18:03.949283+0000","last_peered":"2024-05-04T03:18:03.949283+0000","last_clean":"2024-05-04T03:18:03.949283+0000","last_became_active":"2024-05-04T03:15:45.010730+0000","last_became_peered":"2024-05-04T03:15:45.010730+0000","last_unstale":"2024-05-04T03:18:03.949283+0000","last_undegraded":"2024-05-04T03:18:03.949283+0000","last_fullsized":"2024-05-04T03:18:03.949283+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:45:55.979067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.167833+0000","last_change":"2024-05-04T03:15:48.045965+0000","last_active":"2024-05-04T03:17:33.167833+0000","last_peered":"2024-05-04T03:17:33.167833+0000","last_clean":"2024-05-04T03:17:33.167833+0000","last_became_active":"2024-05-04T03:15:48.045615+0000","last_became_peered":"2024-05-04T03:15:48.045615+0000","last_unstale":"2024-05-04T03:17:33.167833+0000","last_undegraded":"2024-05-04T03:17:33.167833+0000","last_fullsized":"2024-05-04T03:17:33.167833+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:14:23.497484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.329258+0000","last_change":"2024-05-04T03:15:45.204223+0000","last_active":"2024-05-04T03:16:34.329258+0000","last_peered":"2024-05-04T03:16:34.329258+0000","last_clean":"2024-05-04T03:16:34.329258+0000","last_became_active":"2024-05-04T03:15:45.203774+0000","last_became_peered":"2024-05-04T03:15:45.203774+0000","last_unstale":"2024-05-04T03:16:34.329258+0000","last_undegraded":"2024-05-04T03:16:34.329258+0000","last_fullsized":"2024-05-04T03:16:34.329258+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:02:34.456055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.167144+0000","last_change":"2024-05-04T03:15:48.045574+0000","last_active":"2024-05-04T03:17:33.167144+0000","last_peered":"2024-05-04T03:17:33.167144+0000","last_clean":"2024-05-04T03:17:33.167144+0000","last_became_active":"2024-05-04T03:15:48.045237+0000","last_became_peered":"2024-05-04T03:15:48.045237+0000","last_unstale":"2024-05-04T03:17:33.167144+0000","last_undegraded":"2024-05-04T03:17:33.167144+0000","last_fullsized":"2024-05-04T03:17:33.167144+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:53:46.346199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.333946+0000","last_change":"2024-05-04T03:15:45.013895+0000","last_active":"2024-05-04T03:16:34.333946+0000","last_peered":"2024-05-04T03:16:34.333946+0000","last_clean":"2024-05-04T03:16:34.333946+0000","last_became_active":"2024-05-04T03:15:45.013306+0000","last_became_peered":"2024-05-04T03:15:45.013306+0000","last_unstale":"2024-05-04T03:16:34.333946+0000","last_undegraded":"2024-05-04T03:16:34.333946+0000","last_fullsized":"2024-05-04T03:16:34.333946+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:39:18.616432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.158996+0000","last_change":"2024-05-04T03:16:34.336997+0000","last_active":"2024-05-04T03:17:33.158996+0000","last_peered":"2024-05-04T03:17:33.158996+0000","last_clean":"2024-05-04T03:17:33.158996+0000","last_became_active":"2024-05-04T03:16:34.336505+0000","last_became_peered":"2024-05-04T03:16:34.336505+0000","last_unstale":"2024-05-04T03:17:33.158996+0000","last_undegraded":"2024-05-04T03:17:33.158996+0000","last_fullsized":"2024-05-04T03:17:33.158996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:23:38.979216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.149672+0000","last_change":"2024-05-04T03:15:48.050122+0000","last_active":"2024-05-04T03:17:33.149672+0000","last_peered":"2024-05-04T03:17:33.149672+0000","last_clean":"2024-05-04T03:17:33.149672+0000","last_became_active":"2024-05-04T03:15:48.049794+0000","last_became_peered":"2024-05-04T03:15:48.049794+0000","last_unstale":"2024-05-04T03:17:33.149672+0000","last_undegraded":"2024-05-04T03:17:33.149672+0000","last_fullsized":"2024-05-04T03:17:33.149672+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:29:11.066068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":196,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:18:46.508923+0000","last_change":"2024-05-04T03:15:45.205143+0000","last_active":"2024-05-04T03:18:46.508923+0000","last_peered":"2024-05-04T03:18:46.508923+0000","last_clean":"2024-05-04T03:18:46.508923+0000","last_became_active":"2024-05-04T03:15:45.204727+0000","last_became_peered":"2024-05-04T03:15:45.204727+0000","last_unstale":"2024-05-04T03:18:46.508923+0000","last_undegraded":"2024-05-04T03:18:46.508923+0000","last_fullsized":"2024-05-04T03:18:46.508923+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:15:05.799556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":163,"num_read_kb":163,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.163750+0000","last_change":"2024-05-04T03:15:48.042914+0000","last_active":"2024-05-04T03:17:33.163750+0000","last_peered":"2024-05-04T03:17:33.163750+0000","last_clean":"2024-05-04T03:17:33.163750+0000","last_became_active":"2024-05-04T03:15:48.041936+0000","last_became_peered":"2024-05-04T03:15:48.041936+0000","last_unstale":"2024-05-04T03:17:33.163750+0000","last_undegraded":"2024-05-04T03:17:33.163750+0000","last_fullsized":"2024-05-04T03:17:33.163750+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:11:48.807602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.331287+0000","last_change":"2024-05-04T03:15:45.011045+0000","last_active":"2024-05-04T03:16:34.331287+0000","last_peered":"2024-05-04T03:16:34.331287+0000","last_clean":"2024-05-04T03:16:34.331287+0000","last_became_active":"2024-05-04T03:15:45.010608+0000","last_became_peered":"2024-05-04T03:15:45.010608+0000","last_unstale":"2024-05-04T03:16:34.331287+0000","last_undegraded":"2024-05-04T03:16:34.331287+0000","last_fullsized":"2024-05-04T03:16:34.331287+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:00:36.243588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":102,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.333935+0000","last_change":"2024-05-04T03:14:19.895070+0000","last_active":"2024-05-04T03:16:34.333935+0000","last_peered":"2024-05-04T03:16:34.333935+0000","last_clean":"2024-05-04T03:16:34.333935+0000","last_became_active":"2024-05-04T03:14:19.889987+0000","last_became_peered":"2024-05-04T03:14:19.889987+0000","last_unstale":"2024-05-04T03:16:34.333935+0000","last_undegraded":"2024-05-04T03:16:34.333935+0000","last_fullsized":"2024-05-04T03:16:34.333935+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:11:01.712629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:11:01.712629+0000","last_clean_scrub_stamp":"2024-05-04T03:11:01.712629+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:03:02.769719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166064+0000","last_change":"2024-05-04T03:15:48.044928+0000","last_active":"2024-05-04T03:17:33.166064+0000","last_peered":"2024-05-04T03:17:33.166064+0000","last_clean":"2024-05-04T03:17:33.166064+0000","last_became_active":"2024-05-04T03:15:48.044532+0000","last_became_peered":"2024-05-04T03:15:48.044532+0000","last_unstale":"2024-05-04T03:17:33.166064+0000","last_undegraded":"2024-05-04T03:17:33.166064+0000","last_fullsized":"2024-05-04T03:17:33.166064+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:27:00.825692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334103+0000","last_change":"2024-05-04T03:15:45.014469+0000","last_active":"2024-05-04T03:16:34.334103+0000","last_peered":"2024-05-04T03:16:34.334103+0000","last_clean":"2024-05-04T03:16:34.334103+0000","last_became_active":"2024-05-04T03:15:45.013763+0000","last_became_peered":"2024-05-04T03:15:45.013763+0000","last_unstale":"2024-05-04T03:16:34.334103+0000","last_undegraded":"2024-05-04T03:16:34.334103+0000","last_fullsized":"2024-05-04T03:16:34.334103+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:19:36.379930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.334623+0000","last_change":"2024-05-04T03:15:45.204536+0000","last_active":"2024-05-04T03:16:34.334623+0000","last_peered":"2024-05-04T03:16:34.334623+0000","last_clean":"2024-05-04T03:16:34.334623+0000","last_became_active":"2024-05-04T03:15:45.204072+0000","last_became_peered":"2024-05-04T03:15:45.204072+0000","last_unstale":"2024-05-04T03:16:34.334623+0000","last_undegraded":"2024-05-04T03:16:34.334623+0000","last_fullsized":"2024-05-04T03:16:34.334623+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T09:49:12.726220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.162922+0000","last_change":"2024-05-04T03:15:48.044982+0000","last_active":"2024-05-04T03:17:33.162922+0000","last_peered":"2024-05-04T03:17:33.162922+0000","last_clean":"2024-05-04T03:17:33.162922+0000","last_became_active":"2024-05-04T03:15:48.043996+0000","last_became_peered":"2024-05-04T03:15:48.043996+0000","last_unstale":"2024-05-04T03:17:33.162922+0000","last_undegraded":"2024-05-04T03:17:33.162922+0000","last_fullsized":"2024-05-04T03:17:33.162922+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:09:39.770023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'13","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:18:45.067828+0000","last_change":"2024-05-04T03:15:45.008153+0000","last_active":"2024-05-04T03:18:45.067828+0000","last_peered":"2024-05-04T03:18:45.067828+0000","last_clean":"2024-05-04T03:18:45.067828+0000","last_became_active":"2024-05-04T03:15:45.007698+0000","last_became_peered":"2024-05-04T03:15:45.007698+0000","last_unstale":"2024-05-04T03:18:45.067828+0000","last_undegraded":"2024-05-04T03:18:45.067828+0000","last_fullsized":"2024-05-04T03:18:45.067828+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:56:52.322049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.168750+0000","last_change":"2024-05-04T03:15:48.055767+0000","last_active":"2024-05-04T03:17:33.168750+0000","last_peered":"2024-05-04T03:17:33.168750+0000","last_clean":"2024-05-04T03:17:33.168750+0000","last_became_active":"2024-05-04T03:15:48.055252+0000","last_became_peered":"2024-05-04T03:15:48.055252+0000","last_unstale":"2024-05-04T03:17:33.168750+0000","last_undegraded":"2024-05-04T03:17:33.168750+0000","last_fullsized":"2024-05-04T03:17:33.168750+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:28:11.736803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.331539+0000","last_change":"2024-05-04T03:15:45.012838+0000","last_active":"2024-05-04T03:16:34.331539+0000","last_peered":"2024-05-04T03:16:34.331539+0000","last_clean":"2024-05-04T03:16:34.331539+0000","last_became_active":"2024-05-04T03:15:45.012429+0000","last_became_peered":"2024-05-04T03:15:45.012429+0000","last_unstale":"2024-05-04T03:16:34.331539+0000","last_undegraded":"2024-05-04T03:16:34.331539+0000","last_fullsized":"2024-05-04T03:16:34.331539+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:22:22.602551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165546+0000","last_change":"2024-05-04T03:15:48.056993+0000","last_active":"2024-05-04T03:17:33.165546+0000","last_peered":"2024-05-04T03:17:33.165546+0000","last_clean":"2024-05-04T03:17:33.165546+0000","last_became_active":"2024-05-04T03:15:48.055938+0000","last_became_peered":"2024-05-04T03:15:48.055938+0000","last_unstale":"2024-05-04T03:17:33.165546+0000","last_undegraded":"2024-05-04T03:17:33.165546+0000","last_fullsized":"2024-05-04T03:17:33.165546+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:23:59.521692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.340536+0000","last_change":"2024-05-04T03:16:34.340048+0000","last_active":"2024-05-04T03:16:34.340536+0000","last_peered":"2024-05-04T03:16:34.340536+0000","last_clean":"2024-05-04T03:16:34.340536+0000","last_became_active":"2024-05-04T03:16:34.336386+0000","last_became_peered":"2024-05-04T03:16:34.336386+0000","last_unstale":"2024-05-04T03:16:34.340536+0000","last_undegraded":"2024-05-04T03:16:34.340536+0000","last_fullsized":"2024-05-04T03:16:34.340536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:45:43.734097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.157539+0000","last_change":"2024-05-04T03:15:48.043614+0000","last_active":"2024-05-04T03:17:33.157539+0000","last_peered":"2024-05-04T03:17:33.157539+0000","last_clean":"2024-05-04T03:17:33.157539+0000","last_became_active":"2024-05-04T03:15:48.042390+0000","last_became_peered":"2024-05-04T03:15:48.042390+0000","last_unstale":"2024-05-04T03:17:33.157539+0000","last_undegraded":"2024-05-04T03:17:33.157539+0000","last_fullsized":"2024-05-04T03:17:33.157539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:56:22.791368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.416585+0000","last_change":"2024-05-04T03:15:45.005570+0000","last_active":"2024-05-04T03:16:34.416585+0000","last_peered":"2024-05-04T03:16:34.416585+0000","last_clean":"2024-05-04T03:16:34.416585+0000","last_became_active":"2024-05-04T03:15:45.005105+0000","last_became_peered":"2024-05-04T03:15:45.005105+0000","last_unstale":"2024-05-04T03:16:34.416585+0000","last_undegraded":"2024-05-04T03:16:34.416585+0000","last_fullsized":"2024-05-04T03:16:34.416585+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:40:17.275618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166899+0000","last_change":"2024-05-04T03:15:48.045081+0000","last_active":"2024-05-04T03:17:33.166899+0000","last_peered":"2024-05-04T03:17:33.166899+0000","last_clean":"2024-05-04T03:17:33.166899+0000","last_became_active":"2024-05-04T03:15:48.044745+0000","last_became_peered":"2024-05-04T03:15:48.044745+0000","last_unstale":"2024-05-04T03:17:33.166899+0000","last_undegraded":"2024-05-04T03:17:33.166899+0000","last_fullsized":"2024-05-04T03:17:33.166899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:40:53.486222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:56.748549+0000","last_change":"2024-05-04T03:15:45.008926+0000","last_active":"2024-05-04T03:17:56.748549+0000","last_peered":"2024-05-04T03:17:56.748549+0000","last_clean":"2024-05-04T03:17:56.748549+0000","last_became_active":"2024-05-04T03:15:45.008412+0000","last_became_peered":"2024-05-04T03:15:45.008412+0000","last_unstale":"2024-05-04T03:17:56.748549+0000","last_undegraded":"2024-05-04T03:17:56.748549+0000","last_fullsized":"2024-05-04T03:17:56.748549+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:34:33.107866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164466+0000","last_change":"2024-05-04T03:15:48.054580+0000","last_active":"2024-05-04T03:17:33.164466+0000","last_peered":"2024-05-04T03:17:33.164466+0000","last_clean":"2024-05-04T03:17:33.164466+0000","last_became_active":"2024-05-04T03:15:48.054242+0000","last_became_peered":"2024-05-04T03:15:48.054242+0000","last_unstale":"2024-05-04T03:17:33.164466+0000","last_undegraded":"2024-05-04T03:17:33.164466+0000","last_fullsized":"2024-05-04T03:17:33.164466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:16:19.426571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:35.207003+0000","last_change":"2024-05-04T03:16:34.340794+0000","last_active":"2024-05-04T03:16:35.207003+0000","last_peered":"2024-05-04T03:16:35.207003+0000","last_clean":"2024-05-04T03:16:35.207003+0000","last_became_active":"2024-05-04T03:16:34.336038+0000","last_became_peered":"2024-05-04T03:16:34.336038+0000","last_unstale":"2024-05-04T03:16:35.207003+0000","last_undegraded":"2024-05-04T03:16:35.207003+0000","last_fullsized":"2024-05-04T03:16:35.207003+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:45:19.323938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.172190+0000","last_change":"2024-05-04T03:15:48.045284+0000","last_active":"2024-05-04T03:17:33.172190+0000","last_peered":"2024-05-04T03:17:33.172190+0000","last_clean":"2024-05-04T03:17:33.172190+0000","last_became_active":"2024-05-04T03:15:48.044673+0000","last_became_peered":"2024-05-04T03:15:48.044673+0000","last_unstale":"2024-05-04T03:17:33.172190+0000","last_undegraded":"2024-05-04T03:17:33.172190+0000","last_fullsized":"2024-05-04T03:17:33.172190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:51:52.389748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.330835+0000","last_change":"2024-05-04T03:15:45.205648+0000","last_active":"2024-05-04T03:16:34.330835+0000","last_peered":"2024-05-04T03:16:34.330835+0000","last_clean":"2024-05-04T03:16:34.330835+0000","last_became_active":"2024-05-04T03:15:45.204169+0000","last_became_peered":"2024-05-04T03:15:45.204169+0000","last_unstale":"2024-05-04T03:16:34.330835+0000","last_undegraded":"2024-05-04T03:16:34.330835+0000","last_fullsized":"2024-05-04T03:16:34.330835+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:20:48.859276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.171852+0000","last_change":"2024-05-04T03:15:48.049808+0000","last_active":"2024-05-04T03:17:33.171852+0000","last_peered":"2024-05-04T03:17:33.171852+0000","last_clean":"2024-05-04T03:17:33.171852+0000","last_became_active":"2024-05-04T03:15:48.048998+0000","last_became_peered":"2024-05-04T03:15:48.048998+0000","last_unstale":"2024-05-04T03:17:33.171852+0000","last_undegraded":"2024-05-04T03:17:33.171852+0000","last_fullsized":"2024-05-04T03:17:33.171852+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:54:46.786406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.417121+0000","last_change":"2024-05-04T03:15:45.008874+0000","last_active":"2024-05-04T03:16:34.417121+0000","last_peered":"2024-05-04T03:16:34.417121+0000","last_clean":"2024-05-04T03:16:34.417121+0000","last_became_active":"2024-05-04T03:15:45.008530+0000","last_became_peered":"2024-05-04T03:15:45.008530+0000","last_unstale":"2024-05-04T03:16:34.417121+0000","last_undegraded":"2024-05-04T03:16:34.417121+0000","last_fullsized":"2024-05-04T03:16:34.417121+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:31:41.697386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.168872+0000","last_change":"2024-05-04T03:15:48.046042+0000","last_active":"2024-05-04T03:17:33.168872+0000","last_peered":"2024-05-04T03:17:33.168872+0000","last_clean":"2024-05-04T03:17:33.168872+0000","last_became_active":"2024-05-04T03:15:48.045387+0000","last_became_peered":"2024-05-04T03:15:48.045387+0000","last_unstale":"2024-05-04T03:17:33.168872+0000","last_undegraded":"2024-05-04T03:17:33.168872+0000","last_fullsized":"2024-05-04T03:17:33.168872+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:43:50.782623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:16:34.331686+0000","last_change":"2024-05-04T03:15:45.009126+0000","last_active":"2024-05-04T03:16:34.331686+0000","last_peered":"2024-05-04T03:16:34.331686+0000","last_clean":"2024-05-04T03:16:34.331686+0000","last_became_active":"2024-05-04T03:15:45.008802+0000","last_became_peered":"2024-05-04T03:15:45.008802+0000","last_unstale":"2024-05-04T03:16:34.331686+0000","last_undegraded":"2024-05-04T03:16:34.331686+0000","last_fullsized":"2024-05-04T03:16:34.331686+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:43.989709+0000","last_clean_scrub_stamp":"2024-05-04T03:15:43.989709+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T14:25:30.867870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.169092+0000","last_change":"2024-05-04T03:15:48.050478+0000","last_active":"2024-05-04T03:17:33.169092+0000","last_peered":"2024-05-04T03:17:33.169092+0000","last_clean":"2024-05-04T03:17:33.169092+0000","last_became_active":"2024-05-04T03:15:48.045756+0000","last_became_peered":"2024-05-04T03:15:48.045756+0000","last_unstale":"2024-05-04T03:17:33.169092+0000","last_undegraded":"2024-05-04T03:17:33.169092+0000","last_fullsized":"2024-05-04T03:17:33.169092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T10:05:11.098381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.165891+0000","last_change":"2024-05-04T03:15:48.042778+0000","last_active":"2024-05-04T03:17:33.165891+0000","last_peered":"2024-05-04T03:17:33.165891+0000","last_clean":"2024-05-04T03:17:33.165891+0000","last_became_active":"2024-05-04T03:15:48.042349+0000","last_became_peered":"2024-05-04T03:15:48.042349+0000","last_unstale":"2024-05-04T03:17:33.165891+0000","last_undegraded":"2024-05-04T03:17:33.165891+0000","last_fullsized":"2024-05-04T03:17:33.165891+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:40:12.260026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.161439+0000","last_change":"2024-05-04T03:15:48.048544+0000","last_active":"2024-05-04T03:17:33.161439+0000","last_peered":"2024-05-04T03:17:33.161439+0000","last_clean":"2024-05-04T03:17:33.161439+0000","last_became_active":"2024-05-04T03:15:48.047873+0000","last_became_peered":"2024-05-04T03:15:48.047873+0000","last_unstale":"2024-05-04T03:17:33.161439+0000","last_undegraded":"2024-05-04T03:17:33.161439+0000","last_fullsized":"2024-05-04T03:17:33.161439+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:48:02.290401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166682+0000","last_change":"2024-05-04T03:15:48.053467+0000","last_active":"2024-05-04T03:17:33.166682+0000","last_peered":"2024-05-04T03:17:33.166682+0000","last_clean":"2024-05-04T03:17:33.166682+0000","last_became_active":"2024-05-04T03:15:48.050796+0000","last_became_peered":"2024-05-04T03:15:48.050796+0000","last_unstale":"2024-05-04T03:17:33.166682+0000","last_undegraded":"2024-05-04T03:17:33.166682+0000","last_fullsized":"2024-05-04T03:17:33.166682+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:11:15.009467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.159030+0000","last_change":"2024-05-04T03:15:48.054616+0000","last_active":"2024-05-04T03:17:33.159030+0000","last_peered":"2024-05-04T03:17:33.159030+0000","last_clean":"2024-05-04T03:17:33.159030+0000","last_became_active":"2024-05-04T03:15:48.054142+0000","last_became_peered":"2024-05-04T03:15:48.054142+0000","last_unstale":"2024-05-04T03:17:33.159030+0000","last_undegraded":"2024-05-04T03:17:33.159030+0000","last_fullsized":"2024-05-04T03:17:33.159030+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:18:24.067666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.166465+0000","last_change":"2024-05-04T03:15:48.046969+0000","last_active":"2024-05-04T03:17:33.166465+0000","last_peered":"2024-05-04T03:17:33.166465+0000","last_clean":"2024-05-04T03:17:33.166465+0000","last_became_active":"2024-05-04T03:15:48.046637+0000","last_became_peered":"2024-05-04T03:15:48.046637+0000","last_unstale":"2024-05-04T03:17:33.166465+0000","last_undegraded":"2024-05-04T03:17:33.166465+0000","last_fullsized":"2024-05-04T03:17:33.166465+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:03:01.763037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164220+0000","last_change":"2024-05-04T03:15:48.041753+0000","last_active":"2024-05-04T03:17:33.164220+0000","last_peered":"2024-05-04T03:17:33.164220+0000","last_clean":"2024-05-04T03:17:33.164220+0000","last_became_active":"2024-05-04T03:15:48.041236+0000","last_became_peered":"2024-05-04T03:15:48.041236+0000","last_unstale":"2024-05-04T03:17:33.164220+0000","last_undegraded":"2024-05-04T03:17:33.164220+0000","last_fullsized":"2024-05-04T03:17:33.164220+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T13:41:02.928257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.162074+0000","last_change":"2024-05-04T03:15:48.044192+0000","last_active":"2024-05-04T03:17:33.162074+0000","last_peered":"2024-05-04T03:17:33.162074+0000","last_clean":"2024-05-04T03:17:33.162074+0000","last_became_active":"2024-05-04T03:15:48.043849+0000","last_became_peered":"2024-05-04T03:15:48.043849+0000","last_unstale":"2024-05-04T03:17:33.162074+0000","last_undegraded":"2024-05-04T03:17:33.162074+0000","last_fullsized":"2024-05-04T03:17:33.162074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T11:55:22.000788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.164261+0000","last_change":"2024-05-04T03:15:48.045675+0000","last_active":"2024-05-04T03:17:33.164261+0000","last_peered":"2024-05-04T03:17:33.164261+0000","last_clean":"2024-05-04T03:17:33.164261+0000","last_became_active":"2024-05-04T03:15:48.045321+0000","last_became_peered":"2024-05-04T03:15:48.045321+0000","last_unstale":"2024-05-04T03:17:33.164261+0000","last_undegraded":"2024-05-04T03:17:33.164261+0000","last_fullsized":"2024-05-04T03:17:33.164261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T08:12:06.060729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.143245+0000","last_change":"2024-05-04T03:15:48.040872+0000","last_active":"2024-05-04T03:17:33.143245+0000","last_peered":"2024-05-04T03:17:33.143245+0000","last_clean":"2024-05-04T03:17:33.143245+0000","last_became_active":"2024-05-04T03:15:48.040485+0000","last_became_peered":"2024-05-04T03:15:48.040485+0000","last_unstale":"2024-05-04T03:17:33.143245+0000","last_undegraded":"2024-05-04T03:17:33.143245+0000","last_fullsized":"2024-05-04T03:17:33.143245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:58:01.789270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.161920+0000","last_change":"2024-05-04T03:15:48.049080+0000","last_active":"2024-05-04T03:17:33.161920+0000","last_peered":"2024-05-04T03:17:33.161920+0000","last_clean":"2024-05-04T03:17:33.161920+0000","last_became_active":"2024-05-04T03:15:48.044624+0000","last_became_peered":"2024-05-04T03:15:48.044624+0000","last_unstale":"2024-05-04T03:17:33.161920+0000","last_undegraded":"2024-05-04T03:17:33.161920+0000","last_fullsized":"2024-05-04T03:17:33.161920+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:47:08.647401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.171410+0000","last_change":"2024-05-04T03:15:48.049394+0000","last_active":"2024-05-04T03:17:33.171410+0000","last_peered":"2024-05-04T03:17:33.171410+0000","last_clean":"2024-05-04T03:17:33.171410+0000","last_became_active":"2024-05-04T03:15:48.048980+0000","last_became_peered":"2024-05-04T03:15:48.048980+0000","last_unstale":"2024-05-04T03:17:33.171410+0000","last_undegraded":"2024-05-04T03:17:33.171410+0000","last_fullsized":"2024-05-04T03:17:33.171410+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:43:24.914665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.171163+0000","last_change":"2024-05-04T03:15:48.051672+0000","last_active":"2024-05-04T03:17:33.171163+0000","last_peered":"2024-05-04T03:17:33.171163+0000","last_clean":"2024-05-04T03:17:33.171163+0000","last_became_active":"2024-05-04T03:15:48.051314+0000","last_became_peered":"2024-05-04T03:15:48.051314+0000","last_unstale":"2024-05-04T03:17:33.171163+0000","last_undegraded":"2024-05-04T03:17:33.171163+0000","last_fullsized":"2024-05-04T03:17:33.171163+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T12:19:22.808102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-04T03:17:33.167760+0000","last_change":"2024-05-04T03:15:48.053045+0000","last_active":"2024-05-04T03:17:33.167760+0000","last_peered":"2024-05-04T03:17:33.167760+0000","last_clean":"2024-05-04T03:17:33.167760+0000","last_became_active":"2024-05-04T03:15:48.052659+0000","last_became_peered":"2024-05-04T03:15:48.052659+0000","last_unstale":"2024-05-04T03:17:33.167760+0000","last_undegraded":"2024-05-04T03:17:33.167760+0000","last_fullsized":"2024-05-04T03:17:33.167760+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-04T03:15:47.011188+0000","last_clean_scrub_stamp":"2024-05-04T03:15:47.011188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T07:53:25.624540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":78755,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":1028,"num_read_kb":598,"num_write":189,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":236265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":73,"seq":313532612661,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":440503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808890,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28588,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716564,"statfs":{"total":95995035648,"available":95965761536,"internally_reserved":0,"allocated":1609728,"data_stored":1106821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972415,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":440739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168645,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":917504,"data_stored":440659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397577,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32652,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712500,"statfs":{"total":95995035648,"available":95961600000,"internally_reserved":0,"allocated":1511424,"data_stored":1033429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593806,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40380,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704772,"statfs":{"total":95995035648,"available":95953686528,"internally_reserved":0,"allocated":905216,"data_stored":440128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":978944,"data_stored":513835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40400,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704752,"statfs":{"total":95995035648,"available":95953666048,"internally_reserved":0,"allocated":925696,"data_stored":441399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411427,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":1003520,"data_stored":515493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182492,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40396,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704756,"statfs":{"total":95995035648,"available":95953670144,"internally_reserved":0,"allocated":921600,"data_stored":440439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986264,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":441488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790036,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36824,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708328,"statfs":{"total":95995035648,"available":95957327872,"internally_reserved":0,"allocated":1523712,"data_stored":1032190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":73914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":75572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":76415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-04T03:18:54.509 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:18:54.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:54 smithi072 ceph-mon[33332]: pgmap v383: 97 pgs: 97 active+clean; 654 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-04T03:18:54.766 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:18:54.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:54 smithi001 ceph-mon[29731]: pgmap v383: 97 pgs: 97 active+clean; 654 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-04T03:18:54.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:54 smithi077 ceph-mon[33531]: pgmap v383: 97 pgs: 97 active+clean; 654 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-04T03:18:55.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:55 smithi072 ceph-mon[33332]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:18:55.818 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:55 smithi001 ceph-mon[29731]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:18:55.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:55 smithi077 ceph-mon[33531]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-04T03:18:56.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.378+0000 7fd0376a2700 1 -- 172.21.15.1:0/1232097292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0301034c0 msgr2=0x7fd0301038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:56.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.378+0000 7fd0376a2700 1 --2- 172.21.15.1:0/1232097292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0301034c0 0x7fd0301038d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd02800b0d0 tx=0x7fd02800b3e0 comp rx=0 tx=0).stop 2024-05-04T03:18:56.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.378+0000 7fd0376a2700 1 -- 172.21.15.1:0/1232097292 shutdown_connections 2024-05-04T03:18:56.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.378+0000 7fd0376a2700 1 --2- 172.21.15.1:0/1232097292 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd030108740 0x7fd0301100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:56.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.378+0000 7fd0376a2700 1 --2- 172.21.15.1:0/1232097292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0301034c0 0x7fd0301038d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:56.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.378+0000 7fd0376a2700 1 --2- 172.21.15.1:0/1232097292 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd030107df0 0x7fd030108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:56.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.379+0000 7fd0376a2700 1 -- 172.21.15.1:0/1232097292 >> 172.21.15.1:0/1232097292 conn(0x7fd0300fedf0 msgr2=0x7fd030101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:56.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.379+0000 7fd0376a2700 1 -- 172.21.15.1:0/1232097292 shutdown_connections 2024-05-04T03:18:56.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.379+0000 7fd0376a2700 1 -- 172.21.15.1:0/1232097292 wait complete. 2024-05-04T03:18:56.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.380+0000 7fd0376a2700 1 Processor -- start 2024-05-04T03:18:56.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.380+0000 7fd0376a2700 1 -- start start 2024-05-04T03:18:56.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.380+0000 7fd0376a2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0301034c0 0x7fd03019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:56.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.380+0000 7fd0376a2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd030107df0 0x7fd03019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:56.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.380+0000 7fd0376a2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd030108740 0x7fd03019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:56.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.380+0000 7fd0376a2700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd030113370 con 0x7fd0301034c0 2024-05-04T03:18:56.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.380+0000 7fd0376a2700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd0301131f0 con 0x7fd030107df0 2024-05-04T03:18:56.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.380+0000 7fd0376a2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd0301134f0 con 0x7fd030108740 2024-05-04T03:18:56.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.380+0000 7fd034c3d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd030107df0 0x7fd03019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:56.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.380+0000 7fd03543e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0301034c0 0x7fd03019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:56.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.380+0000 7fd034c3d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd030107df0 0x7fd03019b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:48850/0 (socket says 172.21.15.1:48850) 2024-05-04T03:18:56.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.380+0000 7fd03543e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0301034c0 0x7fd03019adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54426/0 (socket says 172.21.15.1:54426) 2024-05-04T03:18:56.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.380+0000 7fd034c3d700 1 -- 172.21.15.1:0/2058964273 learned_addr learned my addr 172.21.15.1:0/2058964273 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:56.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.380+0000 7fd035c3f700 1 --2- 172.21.15.1:0/2058964273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd030108740 0x7fd03019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:56.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.381+0000 7fd034c3d700 1 -- 172.21.15.1:0/2058964273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd030108740 msgr2=0x7fd03019f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:56.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.381+0000 7fd034c3d700 1 --2- 172.21.15.1:0/2058964273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd030108740 0x7fd03019f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:56.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.381+0000 7fd034c3d700 1 -- 172.21.15.1:0/2058964273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0301034c0 msgr2=0x7fd03019adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:56.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.381+0000 7fd034c3d700 1 --2- 172.21.15.1:0/2058964273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0301034c0 0x7fd03019adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:56.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.381+0000 7fd034c3d700 1 -- 172.21.15.1:0/2058964273 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03019fcc0 con 0x7fd030107df0 2024-05-04T03:18:56.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.381+0000 7fd035c3f700 1 --2- 172.21.15.1:0/2058964273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd030108740 0x7fd03019f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:18:56.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.381+0000 7fd03543e700 1 --2- 172.21.15.1:0/2058964273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0301034c0 0x7fd03019adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:18:56.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.381+0000 7fd034c3d700 1 --2- 172.21.15.1:0/2058964273 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd030107df0 0x7fd03019b390 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd02400d980 tx=0x7fd02400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:56.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.381+0000 7fd02e7fc700 1 -- 172.21.15.1:0/2058964273 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd024017070 con 0x7fd030107df0 2024-05-04T03:18:56.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.381+0000 7fd0376a2700 1 -- 172.21.15.1:0/2058964273 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd03019ffb0 con 0x7fd030107df0 2024-05-04T03:18:56.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.381+0000 7fd02e7fc700 1 -- 172.21.15.1:0/2058964273 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd024009de0 con 0x7fd030107df0 2024-05-04T03:18:56.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.382+0000 7fd02e7fc700 1 -- 172.21.15.1:0/2058964273 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02400f3c0 con 0x7fd030107df0 2024-05-04T03:18:56.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.382+0000 7fd0376a2700 1 -- 172.21.15.1:0/2058964273 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0301a78d0 con 0x7fd030107df0 2024-05-04T03:18:56.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.382+0000 7fd0376a2700 1 -- 172.21.15.1:0/2058964273 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0300512c0 con 0x7fd030107df0 2024-05-04T03:18:56.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.384+0000 7fd02e7fc700 1 -- 172.21.15.1:0/2058964273 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7fd024009930 con 0x7fd030107df0 2024-05-04T03:18:56.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.385+0000 7fd02e7fc700 1 --2- 172.21.15.1:0/2058964273 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd0180b2910 0x7fd0180b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:56.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.385+0000 7fd03543e700 1 --2- 172.21.15.1:0/2058964273 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd0180b2910 0x7fd0180b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:56.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.385+0000 7fd02e7fc700 1 -- 172.21.15.1:0/2058964273 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fd0240e1dc0 con 0x7fd030107df0 2024-05-04T03:18:56.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.385+0000 7fd03543e700 1 --2- 172.21.15.1:0/2058964273 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd0180b2910 0x7fd0180b4e20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd028009900 tx=0x7fd02800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:56.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.388+0000 7fd02e7fc700 1 -- 172.21.15.1:0/2058964273 <== mon.2 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd0240a99a0 con 0x7fd030107df0 2024-05-04T03:18:56.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.657+0000 7fd0376a2700 1 -- 172.21.15.1:0/2058964273 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd03019c150 con 0x7fd030107df0 2024-05-04T03:18:56.659 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:56 smithi001 ceph-mon[29731]: pgmap v384: 97 pgs: 97 active+clean; 654 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-04T03:18:56.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.658+0000 7fd02e7fc700 1 -- 172.21.15.1:0/2058964273 <== mon.2 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11844 (secure 0 0 0) 0x7fd0240a92a0 con 0x7fd030107df0 2024-05-04T03:18:56.660 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:18:56.660 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2186681910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2186681910},{"type":"v1","addr":"172.21.15.1:6835","nonce":2186681910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2747191047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2747191047},{"type":"v1","addr":"172.21.15.1:6837","nonce":2747191047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4115120540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4115120540},{"type":"v1","addr":"172.21.15.1:6839","nonce":4115120540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3405709780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3405709780},{"type":"v1","addr":"172.21.15.1:6841","nonce":3405709780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/3816098761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":3816098761},{"type":"v1","addr":"172.21.15.72:6833","nonce":3816098761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/379026128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":379026128},{"type":"v1","addr":"172.21.15.77:6833","nonce":379026128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:15:49.823471+0000","modified":"2024-05-04T03:18:45.058419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24472,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/3930984976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3930984976},{"type":"v1","addr":"172.21.15.72:6835","nonce":3930984976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/4279951305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4279951305},{"type":"v1","addr":"172.21.15.77:6835","nonce":4279951305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2003521417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2003521417},{"type":"v1","addr":"172.21.15.77:6837","nonce":2003521417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/1417614203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1417614203},{"type":"v1","addr":"172.21.15.72:6837","nonce":1417614203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.77:6839/922723189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":922723189},{"type":"v1","addr":"172.21.15.77:6839","nonce":922723189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2976050650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2976050650},{"type":"v1","addr":"172.21.15.72:6839","nonce":2976050650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24467,24472,24494]},"id":1}]} 2024-05-04T03:18:56.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.661+0000 7fd0376a2700 1 -- 172.21.15.1:0/2058964273 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd0180b2910 msgr2=0x7fd0180b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:56.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.661+0000 7fd0376a2700 1 --2- 172.21.15.1:0/2058964273 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd0180b2910 0x7fd0180b4e20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd028009900 tx=0x7fd02800b040 comp rx=0 tx=0).stop 2024-05-04T03:18:56.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.661+0000 7fd0376a2700 1 -- 172.21.15.1:0/2058964273 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd030107df0 msgr2=0x7fd03019b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:56.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.661+0000 7fd0376a2700 1 --2- 172.21.15.1:0/2058964273 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd030107df0 0x7fd03019b390 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd02400d980 tx=0x7fd02400dc90 comp rx=0 tx=0).stop 2024-05-04T03:18:56.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.661+0000 7fd0376a2700 1 -- 172.21.15.1:0/2058964273 shutdown_connections 2024-05-04T03:18:56.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.661+0000 7fd0376a2700 1 --2- 172.21.15.1:0/2058964273 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd0180b2910 0x7fd0180b4e20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:56.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.661+0000 7fd0376a2700 1 --2- 172.21.15.1:0/2058964273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd030108740 0x7fd03019f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:56.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.661+0000 7fd0376a2700 1 --2- 172.21.15.1:0/2058964273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd0301034c0 0x7fd03019adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:56.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.661+0000 7fd0376a2700 1 --2- 172.21.15.1:0/2058964273 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd030107df0 0x7fd03019b390 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:56.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.661+0000 7fd0376a2700 1 -- 172.21.15.1:0/2058964273 >> 172.21.15.1:0/2058964273 conn(0x7fd0300fedf0 msgr2=0x7fd030075cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:56.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.661+0000 7fd0376a2700 1 -- 172.21.15.1:0/2058964273 shutdown_connections 2024-05-04T03:18:56.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:56.661+0000 7fd0376a2700 1 -- 172.21.15.1:0/2058964273 wait complete. 2024-05-04T03:18:56.664 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 26 2024-05-04T03:18:56.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:56 smithi072 ceph-mon[33332]: pgmap v384: 97 pgs: 97 active+clean; 654 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-04T03:18:56.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:56 smithi077 ceph-mon[33531]: pgmap v384: 97 pgs: 97 active+clean; 654 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-04T03:18:57.309 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:18:57.564 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:18:57.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:57 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2058964273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:18:57.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:57 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/2058964273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:18:57.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:57 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/2058964273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:18:58.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:58 smithi072 ceph-mon[33332]: pgmap v385: 97 pgs: 97 active+clean; 654 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-04T03:18:58.809 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:58 smithi001 ceph-mon[29731]: pgmap v385: 97 pgs: 97 active+clean; 654 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-04T03:18:58.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:58 smithi077 ceph-mon[33531]: pgmap v385: 97 pgs: 97 active+clean; 654 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-05-04T03:18:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.102+0000 7f869a696640 1 -- 172.21.15.1:0/2306295771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8694108aa0 msgr2=0x7f869410aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.102+0000 7f869a696640 1 --2- 172.21.15.1:0/2306295771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8694108aa0 0x7f869410aec0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f868c009ca0 tx=0x7f868c02a8c0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.102+0000 7f869a696640 1 -- 172.21.15.1:0/2306295771 shutdown_connections 2024-05-04T03:18:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.102+0000 7f869a696640 1 --2- 172.21.15.1:0/2306295771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8694108aa0 0x7f869410aec0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.102+0000 7f869a696640 1 --2- 172.21.15.1:0/2306295771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8694103da0 0x7f86941041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.102+0000 7f869a696640 1 --2- 172.21.15.1:0/2306295771 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8694102b60 0x7f8694103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.102+0000 7f869a696640 1 -- 172.21.15.1:0/2306295771 >> 172.21.15.1:0/2306295771 conn(0x7f86940fdd50 msgr2=0x7f8694100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.102+0000 7f869a696640 1 -- 172.21.15.1:0/2306295771 shutdown_connections 2024-05-04T03:18:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.103+0000 7f869a696640 1 -- 172.21.15.1:0/2306295771 wait complete. 2024-05-04T03:18:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.103+0000 7f869a696640 1 Processor -- start 2024-05-04T03:18:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.103+0000 7f869a696640 1 -- start start 2024-05-04T03:18:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.103+0000 7f869a696640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8694102b60 0x7f86940ffbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.103+0000 7f869a696640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8694103da0 0x7f86940fe280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.104+0000 7f869a696640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8694108aa0 0x7f86940fe7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.104+0000 7f869a696640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f869410e690 con 0x7f8694102b60 2024-05-04T03:18:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.104+0000 7f869a696640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f869410e510 con 0x7f8694103da0 2024-05-04T03:18:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.104+0000 7f869a696640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f869410e810 con 0x7f8694108aa0 2024-05-04T03:18:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.104+0000 7f8693fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8694102b60 0x7f86940ffbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.104+0000 7f8693fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8694102b60 0x7f86940ffbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54450/0 (socket says 172.21.15.1:54450) 2024-05-04T03:18:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.104+0000 7f8693fff640 1 -- 172.21.15.1:0/3317911758 learned_addr learned my addr 172.21.15.1:0/3317911758 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.104+0000 7f86937fe640 1 --2- 172.21.15.1:0/3317911758 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8694103da0 0x7f86940fe280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.104+0000 7f8698c33640 1 --2- 172.21.15.1:0/3317911758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8694108aa0 0x7f86940fe7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.104+0000 7f8693fff640 1 -- 172.21.15.1:0/3317911758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8694108aa0 msgr2=0x7f86940fe7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.104+0000 7f8693fff640 1 --2- 172.21.15.1:0/3317911758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8694108aa0 0x7f86940fe7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.104+0000 7f8693fff640 1 -- 172.21.15.1:0/3317911758 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8694103da0 msgr2=0x7f86940fe280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.104+0000 7f8693fff640 1 --2- 172.21.15.1:0/3317911758 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8694103da0 0x7f86940fe280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.104+0000 7f8693fff640 1 -- 172.21.15.1:0/3317911758 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86940feef0 con 0x7f8694102b60 2024-05-04T03:18:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.104+0000 7f8693fff640 1 --2- 172.21.15.1:0/3317911758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8694102b60 0x7f86940ffbe0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f868400c970 tx=0x7f868400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.104+0000 7f86917fa640 1 -- 172.21.15.1:0/3317911758 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684004220 con 0x7f8694102b60 2024-05-04T03:18:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.104+0000 7f869a696640 1 -- 172.21.15.1:0/3317911758 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86941a4330 con 0x7f8694102b60 2024-05-04T03:18:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.104+0000 7f86917fa640 1 -- 172.21.15.1:0/3317911758 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86840043c0 con 0x7f8694102b60 2024-05-04T03:18:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.104+0000 7f869a696640 1 -- 172.21.15.1:0/3317911758 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86941a4820 con 0x7f8694102b60 2024-05-04T03:18:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.104+0000 7f86917fa640 1 -- 172.21.15.1:0/3317911758 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868400f450 con 0x7f8694102b60 2024-05-04T03:18:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.107+0000 7f86a1627740 1 Processor -- start 2024-05-04T03:18:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.107+0000 7f86a1627740 1 -- start start 2024-05-04T03:18:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.107+0000 7f86917fa640 1 -- 172.21.15.1:0/3317911758 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7f868400f5f0 con 0x7f8694102b60 2024-05-04T03:18:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.107+0000 7f86a1627740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c359771b0 0x556c35d18a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.107+0000 7f86a1627740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c3594e2f0 0x556c35d18fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.107+0000 7f86a1627740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c359564d0 0x556c35d1fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.107+0000 7f86a1627740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556c35c7d5d0 con 0x556c359771b0 2024-05-04T03:18:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.107+0000 7f86a1627740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x556c35c36890 con 0x556c3594e2f0 2024-05-04T03:18:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.107+0000 7f86a1627740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556c35c79590 con 0x556c359564d0 2024-05-04T03:18:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.108+0000 7f8698c33640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c359771b0 0x556c35d18a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.108+0000 7f8698c33640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c359771b0 0x556c35d18a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54460/0 (socket says 172.21.15.1:54460) 2024-05-04T03:18:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.108+0000 7f8698c33640 1 -- 172.21.15.1:0/3139512316 learned_addr learned my addr 172.21.15.1:0/3139512316 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.108+0000 7f8693fff640 1 --2- 172.21.15.1:0/3139512316 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c3594e2f0 0x556c35d18fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.108+0000 7f86937fe640 1 --2- 172.21.15.1:0/3139512316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c359564d0 0x556c35d1fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.108+0000 7f8698c33640 1 -- 172.21.15.1:0/3139512316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c359564d0 msgr2=0x556c35d1fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.108+0000 7f8698c33640 1 --2- 172.21.15.1:0/3139512316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c359564d0 0x556c35d1fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.108+0000 7f8698c33640 1 -- 172.21.15.1:0/3139512316 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c3594e2f0 msgr2=0x556c35d18fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.108+0000 7f8698c33640 1 --2- 172.21.15.1:0/3139512316 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c3594e2f0 0x556c35d18fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.108+0000 7f8698c33640 1 -- 172.21.15.1:0/3139512316 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c3598f210 con 0x556c359771b0 2024-05-04T03:18:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.108+0000 7f86917fa640 1 --2- 172.21.15.1:0/3317911758 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f867c0b2920 0x7f867c0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.108+0000 7f8698c33640 1 --2- 172.21.15.1:0/3139512316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c359771b0 0x556c35d18a60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f868c006010 tx=0x7f868c02f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3a8e9453e5f455 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.108+0000 7f8693fff640 1 --2- 172.21.15.1:0/3317911758 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f867c0b2920 0x7f867c0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.108+0000 7f86917fa640 1 -- 172.21.15.1:0/3317911758 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f86840e2450 con 0x7f8694102b60 2024-05-04T03:18:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.108+0000 7f867a7fc640 1 -- 172.21.15.1:0/3139512316 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c031650 con 0x556c359771b0 2024-05-04T03:18:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.109+0000 7f867a7fc640 1 -- 172.21.15.1:0/3139512316 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f868c036070 con 0x556c359771b0 2024-05-04T03:18:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.109+0000 7f867a7fc640 1 -- 172.21.15.1:0/3139512316 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c004700 con 0x556c359771b0 2024-05-04T03:18:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.109+0000 7f86a1627740 1 -- 172.21.15.1:0/3139512316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c359771b0 msgr2=0x556c35d18a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.109+0000 7f8693fff640 1 --2- 172.21.15.1:0/3317911758 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f867c0b2920 0x7f867c0b4e30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x556c35d19e40 tx=0x7f8684004570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.109+0000 7f86a1627740 1 --2- 172.21.15.1:0/3139512316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c359771b0 0x556c35d18a60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f868c006010 tx=0x7f868c02f450 comp rx=0 tx=0).stop 2024-05-04T03:18:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.109+0000 7f86a1627740 1 -- 172.21.15.1:0/3139512316 shutdown_connections 2024-05-04T03:18:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.109+0000 7f86a1627740 1 --2- 172.21.15.1:0/3139512316 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c3594e2f0 0x556c35d18fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.109+0000 7f86a1627740 1 --2- 172.21.15.1:0/3139512316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c359564d0 0x556c35d1fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.109+0000 7f86a1627740 1 --2- 172.21.15.1:0/3139512316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c359771b0 0x556c35d18a60 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.109+0000 7f86a1627740 1 -- 172.21.15.1:0/3139512316 >> 172.21.15.1:0/3139512316 conn(0x556c3596b300 msgr2=0x556c35d143a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.110+0000 7f86a1627740 1 -- 172.21.15.1:0/3139512316 shutdown_connections 2024-05-04T03:18:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.110+0000 7f86a1627740 1 -- 172.21.15.1:0/3139512316 wait complete. 2024-05-04T03:18:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.110+0000 7f86a1627740 1 -- start start 2024-05-04T03:18:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.110+0000 7f86a1627740 1 Processor -- start 2024-05-04T03:18:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.110+0000 7f86a1627740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c3594e2f0 0x556c35d22650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.110+0000 7f86a1627740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c359564d0 0x556c35d22b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.110+0000 7f86a1627740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c359771b0 0x556c35d29af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.110+0000 7f86a1627740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556c35d208d0 con 0x556c359564d0 2024-05-04T03:18:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.110+0000 7f86a1627740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x556c35c3a370 con 0x556c3594e2f0 2024-05-04T03:18:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.110+0000 7f86a1627740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556c35d20a50 con 0x556c359771b0 2024-05-04T03:18:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.111+0000 7f86a1627740 10 client.0 _ll_register_callbacks cb 0x556c3598ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:18:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.111+0000 7f86937fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c359564d0 0x556c35d22b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.111+0000 7f86937fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c359564d0 0x556c35d22b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54470/0 (socket says 172.21.15.1:54470) 2024-05-04T03:18:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.111+0000 7f86937fe640 1 -- 172.21.15.1:0/1717211696 learned_addr learned my addr 172.21.15.1:0/1717211696 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.111+0000 7f8698c33640 1 --2- 172.21.15.1:0/1717211696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c359771b0 0x556c35d29af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.111+0000 7f8698c33640 1 --2- 172.21.15.1:0/1717211696 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c3594e2f0 0x556c35d22650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.111+0000 7f86937fe640 1 -- 172.21.15.1:0/1717211696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c359771b0 msgr2=0x556c35d29af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.111+0000 7f86937fe640 1 --2- 172.21.15.1:0/1717211696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c359771b0 0x556c35d29af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.111+0000 7f86937fe640 1 -- 172.21.15.1:0/1717211696 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c3594e2f0 msgr2=0x556c35d22650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.111+0000 7f86937fe640 1 --2- 172.21.15.1:0/1717211696 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c3594e2f0 0x556c35d22650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.111+0000 7f86937fe640 1 -- 172.21.15.1:0/1717211696 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c35a5b620 con 0x556c359564d0 2024-05-04T03:18:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.111+0000 7f8698c33640 1 --2- 172.21.15.1:0/1717211696 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c3594e2f0 0x556c35d22650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:18:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.111+0000 7f86937fe640 1 --2- 172.21.15.1:0/1717211696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c359564d0 0x556c35d22b90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f868800bea0 tx=0x7f868800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3800302bc8b52d76 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.111+0000 7f86737fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:18:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.111+0000 7f86737fe640 1 -- 172.21.15.1:0/1717211696 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8688012650 con 0x556c359564d0 2024-05-04T03:18:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.111+0000 7f86737fe640 1 -- 172.21.15.1:0/1717211696 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f86880127f0 con 0x556c359564d0 2024-05-04T03:18:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.111+0000 7f86a1627740 1 -- 172.21.15.1:0/1717211696 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f868c036070 con 0x556c359564d0 2024-05-04T03:18:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.112+0000 7f86737fe640 1 -- 172.21.15.1:0/1717211696 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8688021030 con 0x556c359564d0 2024-05-04T03:18:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.112+0000 7f86a1627740 10 client.15147 fetch_fsmap learned FSMap version 26 2024-05-04T03:18:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.112+0000 7f86a1627740 1 -- 172.21.15.1:0/1717211696 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556c35d2cb20 con 0x556c359564d0 2024-05-04T03:18:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.112+0000 7f86737fe640 1 -- 172.21.15.1:0/1717211696 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f868c036070 con 0x556c359564d0 2024-05-04T03:18:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.112+0000 7f86737fe640 1 -- 172.21.15.1:0/1717211696 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8688026b40 con 0x556c359564d0 2024-05-04T03:18:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.112+0000 7f86a1627740 10 client.15147 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:18:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.112+0000 7f86a1627740 10 client.15147 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:18:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.112+0000 7f86a1627740 20 client.15147 populate_metadata read hostname 'smithi001' 2024-05-04T03:18:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.112+0000 7f86a1627740 1 --2- 172.21.15.1:0/1717211696 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x556c35935d80 0x556c35d302b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.112+0000 7f86a1627740 4 client.15147 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:18:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.112+0000 7f86a1627740 1 -- 172.21.15.1:0/1717211696 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c35c8b1e0 con 0x556c35935d80 2024-05-04T03:18:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.113+0000 7f8698c33640 1 --2- 172.21.15.1:0/1717211696 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x556c35935d80 0x556c35d302b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.113+0000 7f86737fe640 10 client.15147 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:18:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.113+0000 7f8698c33640 1 --2- 172.21.15.1:0/1717211696 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x556c35935d80 0x556c35d302b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acce71f61800918e server_cookie=f830830991178df6 in_seq=0 out_seq=0 2024-05-04T03:18:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.114+0000 7f86737fe640 1 -- 172.21.15.1:0/1717211696 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556c35c8b1e0 con 0x556c35935d80 2024-05-04T03:18:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.114+0000 7f86737fe640 10 client.15147 handle_command_reply: tid=0 2024-05-04T03:18:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.114+0000 7f86a1627740 1 client.15147 shutdown 2024-05-04T03:18:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.114+0000 7f86a1627740 1 -- 172.21.15.1:0/1717211696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c359564d0 msgr2=0x556c35d22b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.114+0000 7f86a1627740 1 --2- 172.21.15.1:0/1717211696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c359564d0 0x556c35d22b90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f868800bea0 tx=0x7f868800bf90 comp rx=0 tx=0).stop 2024-05-04T03:18:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.114+0000 7f86a1627740 1 -- 172.21.15.1:0/1717211696 shutdown_connections 2024-05-04T03:18:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.114+0000 7f86a1627740 1 --2- 172.21.15.1:0/1717211696 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x556c35935d80 0x556c35d302b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.114+0000 7f86a1627740 1 --2- 172.21.15.1:0/1717211696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c359771b0 0x556c35d29af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.114+0000 7f86a1627740 1 --2- 172.21.15.1:0/1717211696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c359564d0 0x556c35d22b90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.114+0000 7f86737fe640 0 client.15147 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:18:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.114+0000 7f86a1627740 1 --2- 172.21.15.1:0/1717211696 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c3594e2f0 0x556c35d22650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.114+0000 7f86a1627740 1 -- 172.21.15.1:0/1717211696 >> 172.21.15.1:0/1717211696 conn(0x556c3596b300 msgr2=0x556c35d133a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:59.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.114+0000 7f86a1627740 1 -- 172.21.15.1:0/1717211696 shutdown_connections 2024-05-04T03:18:59.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.114+0000 7f86a1627740 1 -- 172.21.15.1:0/1717211696 wait complete. 2024-05-04T03:18:59.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.114+0000 7f86a1627740 20 client.15147 trim_cache size 0 max 16384 2024-05-04T03:18:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.130+0000 7f86a1627740 1 Processor -- start 2024-05-04T03:18:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.130+0000 7f86a1627740 1 -- start start 2024-05-04T03:18:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.130+0000 7f86a1627740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c3594e2f0 0x556c35d13cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.130+0000 7f86a1627740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c359564d0 0x556c35d14230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.130+0000 7f86a1627740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c359771b0 0x556c35d29a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.130+0000 7f86a1627740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f86941afbb0 con 0x556c359771b0 2024-05-04T03:18:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.130+0000 7f86a1627740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f86941ad480 con 0x556c3594e2f0 2024-05-04T03:18:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.130+0000 7f86a1627740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f86941ab2d0 con 0x556c359564d0 2024-05-04T03:18:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.130+0000 7f8698c33640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c359771b0 0x556c35d29a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.130+0000 7f8698c33640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c359771b0 0x556c35d29a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54478/0 (socket says 172.21.15.1:54478) 2024-05-04T03:18:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.130+0000 7f8698c33640 1 -- 172.21.15.1:0/1119914682 learned_addr learned my addr 172.21.15.1:0/1119914682 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.131+0000 7f8698c33640 1 --2- 172.21.15.1:0/1119914682 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c3594e2f0 0x556c35d13cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.131+0000 7f86937fe640 1 --2- 172.21.15.1:0/1119914682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c359564d0 0x556c35d14230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.131+0000 7f8698c33640 1 -- 172.21.15.1:0/1119914682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c359564d0 msgr2=0x556c35d14230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.131+0000 7f8698c33640 1 --2- 172.21.15.1:0/1119914682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c359564d0 0x556c35d14230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.131+0000 7f8698c33640 1 -- 172.21.15.1:0/1119914682 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c3594e2f0 msgr2=0x556c35d13cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.131+0000 7f8698c33640 1 --2- 172.21.15.1:0/1119914682 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c3594e2f0 0x556c35d13cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.131+0000 7f8698c33640 1 -- 172.21.15.1:0/1119914682 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c35d14d00 con 0x556c359771b0 2024-05-04T03:18:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.131+0000 7f8698c33640 1 --2- 172.21.15.1:0/1119914682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c359771b0 0x556c35d29a10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f868c031a90 tx=0x7f868c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a5beb914841e08 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.131+0000 7f8679ffb640 1 -- 172.21.15.1:0/1119914682 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c049040 con 0x556c359771b0 2024-05-04T03:18:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.131+0000 7f8679ffb640 1 -- 172.21.15.1:0/1119914682 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f868c036050 con 0x556c359771b0 2024-05-04T03:18:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.131+0000 7f8679ffb640 1 -- 172.21.15.1:0/1119914682 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c02fb00 con 0x556c359771b0 2024-05-04T03:18:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.132+0000 7f86a1627740 1 -- 172.21.15.1:0/1119914682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c359771b0 msgr2=0x556c35d29a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.132+0000 7f86a1627740 1 --2- 172.21.15.1:0/1119914682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c359771b0 0x556c35d29a10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f868c031a90 tx=0x7f868c02a8c0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.132+0000 7f86a1627740 1 -- 172.21.15.1:0/1119914682 shutdown_connections 2024-05-04T03:18:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.132+0000 7f86a1627740 1 --2- 172.21.15.1:0/1119914682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c359771b0 0x556c35d29a10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.132+0000 7f86a1627740 1 --2- 172.21.15.1:0/1119914682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c359564d0 0x556c35d14230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.132+0000 7f86a1627740 1 --2- 172.21.15.1:0/1119914682 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c3594e2f0 0x556c35d13cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.132+0000 7f86a1627740 1 -- 172.21.15.1:0/1119914682 >> 172.21.15.1:0/1119914682 conn(0x556c3596b300 msgr2=0x556c35d1ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.132+0000 7f86a1627740 1 -- 172.21.15.1:0/1119914682 shutdown_connections 2024-05-04T03:18:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.132+0000 7f86a1627740 1 -- 172.21.15.1:0/1119914682 wait complete. 2024-05-04T03:18:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.133+0000 7f86a1627740 1 -- start start 2024-05-04T03:18:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.133+0000 7f86a1627740 1 Processor -- start 2024-05-04T03:18:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.133+0000 7f86a1627740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c3594e2f0 0x556c35d26f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.133+0000 7f86a1627740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c359564d0 0x556c35d27450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.133+0000 7f86a1627740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c359771b0 0x556c35d96650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.133+0000 7f86a1627740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f86941ab940 con 0x556c359771b0 2024-05-04T03:18:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.133+0000 7f86a1627740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f86941acdb0 con 0x556c359564d0 2024-05-04T03:18:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.133+0000 7f86a1627740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f86941ac010 con 0x556c3594e2f0 2024-05-04T03:18:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.133+0000 7f86a1627740 10 client.0 _ll_register_callbacks cb 0x7f868800b3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:18:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.133+0000 7f8698c33640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c359771b0 0x556c35d96650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.134+0000 7f8698c33640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c359771b0 0x556c35d96650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54494/0 (socket says 172.21.15.1:54494) 2024-05-04T03:18:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.134+0000 7f8698c33640 1 -- 172.21.15.1:0/2793208339 learned_addr learned my addr 172.21.15.1:0/2793208339 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.134+0000 7f86937fe640 1 --2- 172.21.15.1:0/2793208339 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c359564d0 0x556c35d27450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.134+0000 7f8698c33640 1 --2- 172.21.15.1:0/2793208339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c3594e2f0 0x556c35d26f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.134+0000 7f8698c33640 1 -- 172.21.15.1:0/2793208339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c3594e2f0 msgr2=0x556c35d26f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.134+0000 7f8698c33640 1 --2- 172.21.15.1:0/2793208339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c3594e2f0 0x556c35d26f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.134+0000 7f8698c33640 1 -- 172.21.15.1:0/2793208339 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c359564d0 msgr2=0x556c35d27450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:18:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.134+0000 7f8698c33640 1 --2- 172.21.15.1:0/2793208339 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c359564d0 0x556c35d27450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.134+0000 7f8698c33640 1 -- 172.21.15.1:0/2793208339 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c35d2a410 con 0x556c359771b0 2024-05-04T03:18:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.134+0000 7f8698c33640 1 --2- 172.21.15.1:0/2793208339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c359771b0 0x556c35d96650 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f868c009ca0 tx=0x7f868c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e676af2659f2d184 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.134+0000 7f86737fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:18:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.134+0000 7f86737fe640 1 -- 172.21.15.1:0/2793208339 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c0044b0 con 0x556c359771b0 2024-05-04T03:18:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.134+0000 7f86737fe640 1 -- 172.21.15.1:0/2793208339 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f868c004650 con 0x556c359771b0 2024-05-04T03:18:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.134+0000 7f86a1627740 1 -- 172.21.15.1:0/2793208339 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f868c036050 con 0x556c359771b0 2024-05-04T03:18:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.135+0000 7f86737fe640 1 -- 172.21.15.1:0/2793208339 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f868c06b020 con 0x556c359771b0 2024-05-04T03:18:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.135+0000 7f86a1627740 10 client.15153 fetch_fsmap learned FSMap version 26 2024-05-04T03:18:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.135+0000 7f86a1627740 1 -- 172.21.15.1:0/2793208339 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556c35d97a30 con 0x556c359771b0 2024-05-04T03:18:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.135+0000 7f86737fe640 1 -- 172.21.15.1:0/2793208339 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f868c036050 con 0x556c359771b0 2024-05-04T03:18:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.135+0000 7f86737fe640 1 -- 172.21.15.1:0/2793208339 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f868c04f040 con 0x556c359771b0 2024-05-04T03:18:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.135+0000 7f86a1627740 10 client.15153 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:18:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.135+0000 7f86a1627740 10 client.15153 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:18:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.135+0000 7f86a1627740 20 client.15153 populate_metadata read hostname 'smithi001' 2024-05-04T03:18:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.135+0000 7f86a1627740 1 --2- 172.21.15.1:0/2793208339 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x556c3590b1c0 0x556c35d99f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.135+0000 7f86a1627740 4 client.15153 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:18:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.135+0000 7f86a1627740 1 -- 172.21.15.1:0/2793208339 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556c35c85050 con 0x556c3590b1c0 2024-05-04T03:18:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.135+0000 7f86937fe640 1 --2- 172.21.15.1:0/2793208339 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x556c3590b1c0 0x556c35d99f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:59.142 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.136+0000 7f86737fe640 10 client.15153 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:18:59.142 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.136+0000 7f86937fe640 1 --2- 172.21.15.1:0/2793208339 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x556c3590b1c0 0x556c35d99f90 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efa66426eb5cb03b server_cookie=46eb284ed59adff in_seq=0 out_seq=0 2024-05-04T03:18:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.148+0000 7f86737fe640 1 -- 172.21.15.1:0/2793208339 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556c35c85050 con 0x556c3590b1c0 2024-05-04T03:18:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.148+0000 7f86737fe640 10 client.15153 handle_command_reply: tid=0 2024-05-04T03:18:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.148+0000 7f86a1627740 1 client.15153 shutdown 2024-05-04T03:18:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.148+0000 7f86a1627740 1 -- 172.21.15.1:0/2793208339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c359771b0 msgr2=0x556c35d96650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.148+0000 7f86a1627740 1 --2- 172.21.15.1:0/2793208339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c359771b0 0x556c35d96650 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f868c009ca0 tx=0x7f868c000f40 comp rx=0 tx=0).stop 2024-05-04T03:18:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.148+0000 7f86a1627740 1 -- 172.21.15.1:0/2793208339 shutdown_connections 2024-05-04T03:18:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.148+0000 7f86a1627740 1 --2- 172.21.15.1:0/2793208339 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x556c3590b1c0 0x556c35d99f90 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.148+0000 7f86a1627740 1 --2- 172.21.15.1:0/2793208339 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c359564d0 0x556c35d27450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.148+0000 7f86a1627740 1 --2- 172.21.15.1:0/2793208339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c359771b0 0x556c35d96650 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.148+0000 7f86737fe640 0 client.15153 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:18:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.148+0000 7f86a1627740 1 --2- 172.21.15.1:0/2793208339 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556c3594e2f0 0x556c35d26f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.148+0000 7f86a1627740 1 -- 172.21.15.1:0/2793208339 >> 172.21.15.1:0/2793208339 conn(0x556c3596b300 msgr2=0x556c35d13cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.148+0000 7f86a1627740 1 -- 172.21.15.1:0/2793208339 shutdown_connections 2024-05-04T03:18:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.148+0000 7f86a1627740 1 -- 172.21.15.1:0/2793208339 wait complete. 2024-05-04T03:18:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.149+0000 7f86a1627740 20 client.15153 trim_cache size 0 max 16384 2024-05-04T03:18:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:18:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:18:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:18:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:18:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.149+0000 7f869a696640 1 -- 172.21.15.1:0/3317911758 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f867c0b2920 msgr2=0x7f867c0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.149+0000 7f869a696640 1 --2- 172.21.15.1:0/3317911758 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f867c0b2920 0x7f867c0b4e30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x556c35d19e40 tx=0x7f8684004570 comp rx=0 tx=0).stop 2024-05-04T03:18:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.149+0000 7f869a696640 1 -- 172.21.15.1:0/3317911758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8694102b60 msgr2=0x7f86940ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.149+0000 7f869a696640 1 --2- 172.21.15.1:0/3317911758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8694102b60 0x7f86940ffbe0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f868400c970 tx=0x7f868400ce30 comp rx=0 tx=0).stop 2024-05-04T03:18:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.149+0000 7f869a696640 1 -- 172.21.15.1:0/3317911758 shutdown_connections 2024-05-04T03:18:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.149+0000 7f869a696640 1 --2- 172.21.15.1:0/3317911758 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f867c0b2920 0x7f867c0b4e30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.149+0000 7f869a696640 1 --2- 172.21.15.1:0/3317911758 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8694103da0 0x7f86940fe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.149+0000 7f869a696640 1 --2- 172.21.15.1:0/3317911758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8694108aa0 0x7f86940fe7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.149+0000 7f869a696640 1 --2- 172.21.15.1:0/3317911758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8694102b60 0x7f86940ffbe0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.149+0000 7f869a696640 1 -- 172.21.15.1:0/3317911758 >> 172.21.15.1:0/3317911758 conn(0x7f86940fdd50 msgr2=0x7f869406bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.149+0000 7f869a696640 1 -- 172.21.15.1:0/3317911758 shutdown_connections 2024-05-04T03:18:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:18:59.149+0000 7f869a696640 1 -- 172.21.15.1:0/3317911758 wait complete. 2024-05-04T03:18:59.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.174+0000 7f4b26820700 1 -- 172.21.15.1:0/3635030472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b20108f30 msgr2=0x7f4b20109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:59.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.174+0000 7f4b26820700 1 --2- 172.21.15.1:0/3635030472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b20108f30 0x7f4b20109340 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4b140099c0 tx=0x7f4b14009cd0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.174+0000 7f4b26820700 1 -- 172.21.15.1:0/3635030472 shutdown_connections 2024-05-04T03:18:59.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.174+0000 7f4b26820700 1 --2- 172.21.15.1:0/3635030472 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b20104650 0x7f4b20104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.174+0000 7f4b26820700 1 --2- 172.21.15.1:0/3635030472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b20109880 0x7f4b201111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.174+0000 7f4b26820700 1 --2- 172.21.15.1:0/3635030472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b20108f30 0x7f4b20109340 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.174+0000 7f4b26820700 1 -- 172.21.15.1:0/3635030472 >> 172.21.15.1:0/3635030472 conn(0x7f4b20079f00 msgr2=0x7f4b2007a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:59.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.174+0000 7f4b26820700 1 -- 172.21.15.1:0/3635030472 shutdown_connections 2024-05-04T03:18:59.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.174+0000 7f4b26820700 1 -- 172.21.15.1:0/3635030472 wait complete. 2024-05-04T03:18:59.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.175+0000 7f4b26820700 1 Processor -- start 2024-05-04T03:18:59.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.175+0000 7f4b26820700 1 -- start start 2024-05-04T03:18:59.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.175+0000 7f4b26820700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b20104650 0x7f4b2019aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:59.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.175+0000 7f4b26820700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b20108f30 0x7f4b2019b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:59.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.176+0000 7f4b26820700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b20109880 0x7f4b2019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:59.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.176+0000 7f4b26820700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4b20114490 con 0x7f4b20109880 2024-05-04T03:18:59.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.176+0000 7f4b26820700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4b20114310 con 0x7f4b20108f30 2024-05-04T03:18:59.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.176+0000 7f4b26820700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4b20114610 con 0x7f4b20104650 2024-05-04T03:18:59.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.176+0000 7f4b24dbd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b20109880 0x7f4b2019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:59.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.176+0000 7f4b24dbd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b20109880 0x7f4b2019f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54518/0 (socket says 172.21.15.1:54518) 2024-05-04T03:18:59.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.176+0000 7f4b24dbd700 1 -- 172.21.15.1:0/3640170445 learned_addr learned my addr 172.21.15.1:0/3640170445 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:18:59.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.176+0000 7f4b1f7fe700 1 --2- 172.21.15.1:0/3640170445 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b20108f30 0x7f4b2019b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:59.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.176+0000 7f4b1ffff700 1 --2- 172.21.15.1:0/3640170445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b20104650 0x7f4b2019aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:59.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.176+0000 7f4b24dbd700 1 -- 172.21.15.1:0/3640170445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b20104650 msgr2=0x7f4b2019aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:59.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.176+0000 7f4b24dbd700 1 --2- 172.21.15.1:0/3640170445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b20104650 0x7f4b2019aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.176+0000 7f4b24dbd700 1 -- 172.21.15.1:0/3640170445 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b20108f30 msgr2=0x7f4b2019b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:59.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.176+0000 7f4b24dbd700 1 --2- 172.21.15.1:0/3640170445 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b20108f30 0x7f4b2019b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.176+0000 7f4b24dbd700 1 -- 172.21.15.1:0/3640170445 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b2019fd90 con 0x7f4b20109880 2024-05-04T03:18:59.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.176+0000 7f4b24dbd700 1 --2- 172.21.15.1:0/3640170445 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b20109880 0x7f4b2019f6f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800ef30 tx=0x7f4b1800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:59.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.176+0000 7f4b1d7fa700 1 -- 172.21.15.1:0/3640170445 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b180092e0 con 0x7f4b20109880 2024-05-04T03:18:59.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.176+0000 7f4b26820700 1 -- 172.21.15.1:0/3640170445 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b201a0020 con 0x7f4b20109880 2024-05-04T03:18:59.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.176+0000 7f4b1d7fa700 1 -- 172.21.15.1:0/3640170445 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b18004bb0 con 0x7f4b20109880 2024-05-04T03:18:59.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.177+0000 7f4b1d7fa700 1 -- 172.21.15.1:0/3640170445 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b18004020 con 0x7f4b20109880 2024-05-04T03:18:59.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.177+0000 7f4b26820700 1 -- 172.21.15.1:0/3640170445 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b201a0530 con 0x7f4b20109880 2024-05-04T03:18:59.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.178+0000 7f4b26820700 1 -- 172.21.15.1:0/3640170445 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b200512c0 con 0x7f4b20109880 2024-05-04T03:18:59.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.180+0000 7f4b1d7fa700 1 -- 172.21.15.1:0/3640170445 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7f4b1801b030 con 0x7f4b20109880 2024-05-04T03:18:59.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.181+0000 7f4b1d7fa700 1 --2- 172.21.15.1:0/3640170445 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4b080b2910 0x7f4b080b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:18:59.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.181+0000 7f4b1ffff700 1 --2- 172.21.15.1:0/3640170445 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4b080b2910 0x7f4b080b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:18:59.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.181+0000 7f4b1d7fa700 1 -- 172.21.15.1:0/3640170445 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f4b180e27f0 con 0x7f4b20109880 2024-05-04T03:18:59.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.181+0000 7f4b1ffff700 1 --2- 172.21.15.1:0/3640170445 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4b080b2910 0x7f4b080b4e20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4b10005950 tx=0x7f4b1000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:18:59.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.184+0000 7f4b1d7fa700 1 -- 172.21.15.1:0/3640170445 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4b18022020 con 0x7f4b20109880 2024-05-04T03:18:59.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.453+0000 7f4b26820700 1 -- 172.21.15.1:0/3640170445 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b2002e5f0 con 0x7f4b20109880 2024-05-04T03:18:59.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.454+0000 7f4b1d7fa700 1 -- 172.21.15.1:0/3640170445 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11844 (secure 0 0 0) 0x7f4b18010040 con 0x7f4b20109880 2024-05-04T03:18:59.456 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:18:59.456 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2186681910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2186681910},{"type":"v1","addr":"172.21.15.1:6835","nonce":2186681910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2747191047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2747191047},{"type":"v1","addr":"172.21.15.1:6837","nonce":2747191047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4115120540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4115120540},{"type":"v1","addr":"172.21.15.1:6839","nonce":4115120540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3405709780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3405709780},{"type":"v1","addr":"172.21.15.1:6841","nonce":3405709780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/3816098761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":3816098761},{"type":"v1","addr":"172.21.15.72:6833","nonce":3816098761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/379026128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":379026128},{"type":"v1","addr":"172.21.15.77:6833","nonce":379026128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:15:49.823471+0000","modified":"2024-05-04T03:18:45.058419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24472,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/3930984976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3930984976},{"type":"v1","addr":"172.21.15.72:6835","nonce":3930984976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/4279951305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4279951305},{"type":"v1","addr":"172.21.15.77:6835","nonce":4279951305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2003521417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2003521417},{"type":"v1","addr":"172.21.15.77:6837","nonce":2003521417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/1417614203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1417614203},{"type":"v1","addr":"172.21.15.72:6837","nonce":1417614203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.77:6839/922723189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":922723189},{"type":"v1","addr":"172.21.15.77:6839","nonce":922723189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2976050650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2976050650},{"type":"v1","addr":"172.21.15.72:6839","nonce":2976050650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24467,24472,24494]},"id":1}]} 2024-05-04T03:18:59.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.456+0000 7f4b26820700 1 -- 172.21.15.1:0/3640170445 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4b080b2910 msgr2=0x7f4b080b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:59.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.456+0000 7f4b26820700 1 --2- 172.21.15.1:0/3640170445 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4b080b2910 0x7f4b080b4e20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4b10005950 tx=0x7f4b1000b410 comp rx=0 tx=0).stop 2024-05-04T03:18:59.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.456+0000 7f4b26820700 1 -- 172.21.15.1:0/3640170445 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b20109880 msgr2=0x7f4b2019f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:18:59.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.456+0000 7f4b26820700 1 --2- 172.21.15.1:0/3640170445 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b20109880 0x7f4b2019f6f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800ef30 tx=0x7f4b1800c3a0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.457+0000 7f4b26820700 1 -- 172.21.15.1:0/3640170445 shutdown_connections 2024-05-04T03:18:59.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.457+0000 7f4b26820700 1 --2- 172.21.15.1:0/3640170445 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4b080b2910 0x7f4b080b4e20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.457+0000 7f4b26820700 1 --2- 172.21.15.1:0/3640170445 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b20109880 0x7f4b2019f6f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.457+0000 7f4b26820700 1 --2- 172.21.15.1:0/3640170445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b20104650 0x7f4b2019aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.457+0000 7f4b26820700 1 --2- 172.21.15.1:0/3640170445 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4b20108f30 0x7f4b2019b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:18:59.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.457+0000 7f4b26820700 1 -- 172.21.15.1:0/3640170445 >> 172.21.15.1:0/3640170445 conn(0x7f4b20079f00 msgr2=0x7f4b201020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:18:59.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.457+0000 7f4b26820700 1 -- 172.21.15.1:0/3640170445 shutdown_connections 2024-05-04T03:18:59.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:18:59.457+0000 7f4b26820700 1 -- 172.21.15.1:0/3640170445 wait complete. 2024-05-04T03:18:59.460 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 26 2024-05-04T03:18:59.797 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:18:59 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3640170445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:18:59.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:18:59 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/3640170445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:18:59.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:18:59 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/3640170445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:19:00.155 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-04T03:19:00.407 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:19:00.826 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:00 smithi001 ceph-mon[29731]: pgmap v386: 97 pgs: 97 active+clean; 654 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-05-04T03:19:00.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:00 smithi077 ceph-mon[33531]: pgmap v386: 97 pgs: 97 active+clean; 654 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-05-04T03:19:00.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:00 smithi072 ceph-mon[33332]: pgmap v386: 97 pgs: 97 active+clean; 654 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-05-04T03:19:01.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.946+0000 7fe3746aa700 1 -- 172.21.15.1:0/3925200173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe36c109860 msgr2=0x7fe36c1111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:01.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.946+0000 7fe3746aa700 1 --2- 172.21.15.1:0/3925200173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe36c109860 0x7fe36c1111c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe36400b0d0 tx=0x7fe36400b3e0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.946+0000 7fe3746aa700 1 -- 172.21.15.1:0/3925200173 shutdown_connections 2024-05-04T03:19:01.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.946+0000 7fe3746aa700 1 --2- 172.21.15.1:0/3925200173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe36c109860 0x7fe36c1111c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.946+0000 7fe3746aa700 1 --2- 172.21.15.1:0/3925200173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe36c1045e0 0x7fe36c1049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.946+0000 7fe3746aa700 1 --2- 172.21.15.1:0/3925200173 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe36c108f10 0x7fe36c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.946+0000 7fe3746aa700 1 -- 172.21.15.1:0/3925200173 >> 172.21.15.1:0/3925200173 conn(0x7fe36c0ffed0 msgr2=0x7fe36c102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:01.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.946+0000 7fe3746aa700 1 -- 172.21.15.1:0/3925200173 shutdown_connections 2024-05-04T03:19:01.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.946+0000 7fe3746aa700 1 -- 172.21.15.1:0/3925200173 wait complete. 2024-05-04T03:19:01.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.946+0000 7fe3746aa700 1 Processor -- start 2024-05-04T03:19:01.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.947+0000 7fe3746aa700 1 -- start start 2024-05-04T03:19:01.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.947+0000 7fe3746aa700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe36c1045e0 0x7fe36c19ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:01.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.947+0000 7fe3746aa700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe36c108f10 0x7fe36c19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:01.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.947+0000 7fe3746aa700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe36c109860 0x7fe36c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:01.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.947+0000 7fe3746aa700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c1144e0 con 0x7fe36c108f10 2024-05-04T03:19:01.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.947+0000 7fe3746aa700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c114360 con 0x7fe36c1045e0 2024-05-04T03:19:01.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.947+0000 7fe3746aa700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c114660 con 0x7fe36c109860 2024-05-04T03:19:01.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.947+0000 7fe371c45700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe36c108f10 0x7fe36c19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:01.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.947+0000 7fe371c45700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe36c108f10 0x7fe36c19b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36930/0 (socket says 172.21.15.1:36930) 2024-05-04T03:19:01.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.947+0000 7fe371c45700 1 -- 172.21.15.1:0/2432775139 learned_addr learned my addr 172.21.15.1:0/2432775139 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:01.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.947+0000 7fe372446700 1 --2- 172.21.15.1:0/2432775139 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe36c1045e0 0x7fe36c19ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:01.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.947+0000 7fe372c47700 1 --2- 172.21.15.1:0/2432775139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe36c109860 0x7fe36c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:01.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.947+0000 7fe371c45700 1 -- 172.21.15.1:0/2432775139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe36c109860 msgr2=0x7fe36c19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:01.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.947+0000 7fe371c45700 1 --2- 172.21.15.1:0/2432775139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe36c109860 0x7fe36c19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.947+0000 7fe371c45700 1 -- 172.21.15.1:0/2432775139 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe36c1045e0 msgr2=0x7fe36c19ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:01.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.947+0000 7fe371c45700 1 --2- 172.21.15.1:0/2432775139 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe36c1045e0 0x7fe36c19ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:01.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.947+0000 7fe371c45700 1 -- 172.21.15.1:0/2432775139 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36c19fc70 con 0x7fe36c108f10 2024-05-04T03:19:01.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.948+0000 7fe371c45700 1 --2- 172.21.15.1:0/2432775139 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe36c108f10 0x7fe36c19b3f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe35c00d980 tx=0x7fe35c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:01.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.948+0000 7fe3637fe700 1 -- 172.21.15.1:0/2432775139 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c017070 con 0x7fe36c108f10 2024-05-04T03:19:01.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.948+0000 7fe3746aa700 1 -- 172.21.15.1:0/2432775139 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe36c19ff60 con 0x7fe36c108f10 2024-05-04T03:19:01.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.948+0000 7fe3637fe700 1 -- 172.21.15.1:0/2432775139 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe35c00fd40 con 0x7fe36c108f10 2024-05-04T03:19:01.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.948+0000 7fe3637fe700 1 -- 172.21.15.1:0/2432775139 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c00e3b0 con 0x7fe36c108f10 2024-05-04T03:19:01.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.948+0000 7fe3746aa700 1 -- 172.21.15.1:0/2432775139 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe36c1a7930 con 0x7fe36c108f10 2024-05-04T03:19:01.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.949+0000 7fe3746aa700 1 -- 172.21.15.1:0/2432775139 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe36c0512c0 con 0x7fe36c108f10 2024-05-04T03:19:01.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.950+0000 7fe3637fe700 1 -- 172.21.15.1:0/2432775139 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7fe35c010610 con 0x7fe36c108f10 2024-05-04T03:19:01.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.951+0000 7fe3637fe700 1 --2- 172.21.15.1:0/2432775139 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe3580b2910 0x7fe3580b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:01.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.951+0000 7fe3637fe700 1 -- 172.21.15.1:0/2432775139 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fe35c0e1c40 con 0x7fe36c108f10 2024-05-04T03:19:01.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.951+0000 7fe372446700 1 --2- 172.21.15.1:0/2432775139 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe3580b2910 0x7fe3580b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:01.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.952+0000 7fe372446700 1 --2- 172.21.15.1:0/2432775139 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe3580b2910 0x7fe3580b4e20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe36800ac50 tx=0x7fe36800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:01.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:01.955+0000 7fe3637fe700 1 -- 172.21.15.1:0/2432775139 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe35c0a9920 con 0x7fe36c108f10 2024-05-04T03:19:02.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:02.170+0000 7fe3746aa700 1 -- 172.21.15.1:0/2432775139 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe36c19c1b0 con 0x7fe36c108f10 2024-05-04T03:19:02.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:02.172+0000 7fe3637fe700 1 -- 172.21.15.1:0/2432775139 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22380 (secure 0 0 0) 0x7fe35c0a9220 con 0x7fe36c108f10 2024-05-04T03:19:02.174 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:19:02.174 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":83,"fsid":"4a572aa8-09c3-11ef-bc97-c7b262605968","created":"2024-05-04T03:07:32.845547+0000","modified":"2024-05-04T03:16:34.321580+0000","last_up_change":"2024-05-04T03:14:44.192571+0000","last_in_change":"2024-05-04T03:14:23.353342+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T03:11:01.139189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T03:15:43.151696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T03:15:46.753539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"100fe035-2454-4c95-90c7-759956ec36a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6803","nonce":3479011484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6805","nonce":3479011484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6809","nonce":3479011484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6807","nonce":3479011484}]},"public_addr":"172.21.15.1:6803/3479011484","cluster_addr":"172.21.15.1:6805/3479011484","heartbeat_back_addr":"172.21.15.1:6809/3479011484","heartbeat_front_addr":"172.21.15.1:6807/3479011484","state":["exists","up"]},{"osd":1,"uuid":"5826ac76-ef6b-4040-8ad1-305db355af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6801","nonce":3725706149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6803","nonce":3725706149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6807","nonce":3725706149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6805","nonce":3725706149}]},"public_addr":"172.21.15.72:6801/3725706149","cluster_addr":"172.21.15.72:6803/3725706149","heartbeat_back_addr":"172.21.15.72:6807/3725706149","heartbeat_front_addr":"172.21.15.72:6805/3725706149","state":["exists","up"]},{"osd":2,"uuid":"923d76e5-b032-4190-9e35-546d27722a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6801","nonce":515038343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6803","nonce":515038343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6807","nonce":515038343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6805","nonce":515038343}]},"public_addr":"172.21.15.77:6801/515038343","cluster_addr":"172.21.15.77:6803/515038343","heartbeat_back_addr":"172.21.15.77:6807/515038343","heartbeat_front_addr":"172.21.15.77:6805/515038343","state":["exists","up"]},{"osd":3,"uuid":"495cc0c5-6a99-481f-ad9e-38f05fb84356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6811","nonce":4050439365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6813","nonce":4050439365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6817","nonce":4050439365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6815","nonce":4050439365}]},"public_addr":"172.21.15.1:6811/4050439365","cluster_addr":"172.21.15.1:6813/4050439365","heartbeat_back_addr":"172.21.15.1:6817/4050439365","heartbeat_front_addr":"172.21.15.1:6815/4050439365","state":["exists","up"]},{"osd":4,"uuid":"b86b8026-79c0-4056-9e67-a61f54b6ab5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6809","nonce":2501732391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6811","nonce":2501732391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6815","nonce":2501732391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6813","nonce":2501732391}]},"public_addr":"172.21.15.72:6809/2501732391","cluster_addr":"172.21.15.72:6811/2501732391","heartbeat_back_addr":"172.21.15.72:6815/2501732391","heartbeat_front_addr":"172.21.15.72:6813/2501732391","state":["exists","up"]},{"osd":5,"uuid":"80975cb9-39e0-46b8-a343-9ff69194a14e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6809","nonce":3582852165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6811","nonce":3582852165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6815","nonce":3582852165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6813","nonce":3582852165}]},"public_addr":"172.21.15.77:6809/3582852165","cluster_addr":"172.21.15.77:6811/3582852165","heartbeat_back_addr":"172.21.15.77:6815/3582852165","heartbeat_front_addr":"172.21.15.77:6813/3582852165","state":["exists","up"]},{"osd":6,"uuid":"aca3fac5-fbb5-42de-bf0e-e4a0db7381e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6819","nonce":1799033649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6821","nonce":1799033649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6825","nonce":1799033649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6823","nonce":1799033649}]},"public_addr":"172.21.15.1:6819/1799033649","cluster_addr":"172.21.15.1:6821/1799033649","heartbeat_back_addr":"172.21.15.1:6825/1799033649","heartbeat_front_addr":"172.21.15.1:6823/1799033649","state":["exists","up"]},{"osd":7,"uuid":"cbff6b8a-c463-4249-ae45-fdc6f99a5489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6817","nonce":3460835447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6819","nonce":3460835447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6823","nonce":3460835447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6821","nonce":3460835447}]},"public_addr":"172.21.15.72:6817/3460835447","cluster_addr":"172.21.15.72:6819/3460835447","heartbeat_back_addr":"172.21.15.72:6823/3460835447","heartbeat_front_addr":"172.21.15.72:6821/3460835447","state":["exists","up"]},{"osd":8,"uuid":"e140bf13-4887-4d68-9373-3d4fc1c4584a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6817","nonce":1124479323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6819","nonce":1124479323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6823","nonce":1124479323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6821","nonce":1124479323}]},"public_addr":"172.21.15.77:6817/1124479323","cluster_addr":"172.21.15.77:6819/1124479323","heartbeat_back_addr":"172.21.15.77:6823/1124479323","heartbeat_front_addr":"172.21.15.77:6821/1124479323","state":["exists","up"]},{"osd":9,"uuid":"f1e8a232-cb11-4b9d-8789-e1637fa5aaa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6827","nonce":2681322272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6829","nonce":2681322272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6833","nonce":2681322272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6831","nonce":2681322272}]},"public_addr":"172.21.15.1:6827/2681322272","cluster_addr":"172.21.15.1:6829/2681322272","heartbeat_back_addr":"172.21.15.1:6833/2681322272","heartbeat_front_addr":"172.21.15.1:6831/2681322272","state":["exists","up"]},{"osd":10,"uuid":"79b570af-3d1e-439d-a328-9e6bc7d67839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6825","nonce":3293703569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6827","nonce":3293703569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6831","nonce":3293703569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6829","nonce":3293703569}]},"public_addr":"172.21.15.72:6825/3293703569","cluster_addr":"172.21.15.72:6827/3293703569","heartbeat_back_addr":"172.21.15.72:6831/3293703569","heartbeat_front_addr":"172.21.15.72:6829/3293703569","state":["exists","up"]},{"osd":11,"uuid":"e35bf83d-9ee4-4745-96c2-1c35cc8eefba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6825","nonce":4155947548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6827","nonce":4155947548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6831","nonce":4155947548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6829","nonce":4155947548}]},"public_addr":"172.21.15.77:6825/4155947548","cluster_addr":"172.21.15.77:6827/4155947548","heartbeat_back_addr":"172.21.15.77:6831/4155947548","heartbeat_front_addr":"172.21.15.77:6829/4155947548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:09.467126+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:33.861139+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:57.741962+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:11:21.746021+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:11:46.323252+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:12:10.640180+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:12:35.750591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:00.306549+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:25.012495+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:51.185229+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:14:16.403850+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:14:41.653316+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/2029897106":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/402488450":"2024-05-05T03:08:33.045884+0000","172.21.15.1:6801/128793132":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/2233526606":"2024-05-05T03:08:06.135783+0000","172.21.15.1:0/2715541855":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/2415697325":"2024-05-05T03:08:06.135783+0000","172.21.15.1:0/2305298106":"2024-05-05T03:08:06.135783+0000","172.21.15.1:6800/128793132":"2024-05-05T03:08:33.045884+0000","172.21.15.1:6800/4200529863":"2024-05-05T03:08:06.135783+0000","172.21.15.1:6801/4200529863":"2024-05-05T03:08:06.135783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T03:19:02.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:02.174+0000 7fe3746aa700 1 -- 172.21.15.1:0/2432775139 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe3580b2910 msgr2=0x7fe3580b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:02.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:02.174+0000 7fe3746aa700 1 --2- 172.21.15.1:0/2432775139 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe3580b2910 0x7fe3580b4e20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe36800ac50 tx=0x7fe36800a380 comp rx=0 tx=0).stop 2024-05-04T03:19:02.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:02.174+0000 7fe3746aa700 1 -- 172.21.15.1:0/2432775139 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe36c108f10 msgr2=0x7fe36c19b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:02.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:02.174+0000 7fe3746aa700 1 --2- 172.21.15.1:0/2432775139 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe36c108f10 0x7fe36c19b3f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe35c00d980 tx=0x7fe35c00dc90 comp rx=0 tx=0).stop 2024-05-04T03:19:02.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:02.174+0000 7fe3746aa700 1 -- 172.21.15.1:0/2432775139 shutdown_connections 2024-05-04T03:19:02.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:02.174+0000 7fe3746aa700 1 --2- 172.21.15.1:0/2432775139 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe36c1045e0 0x7fe36c19ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:02.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:02.174+0000 7fe3746aa700 1 --2- 172.21.15.1:0/2432775139 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe3580b2910 0x7fe3580b4e20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:02.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:02.174+0000 7fe3746aa700 1 --2- 172.21.15.1:0/2432775139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe36c109860 0x7fe36c19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:02.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:02.174+0000 7fe3746aa700 1 --2- 172.21.15.1:0/2432775139 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe36c108f10 0x7fe36c19b3f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:02.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:02.174+0000 7fe3746aa700 1 -- 172.21.15.1:0/2432775139 >> 172.21.15.1:0/2432775139 conn(0x7fe36c0ffed0 msgr2=0x7fe36c108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:02.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:02.175+0000 7fe3746aa700 1 -- 172.21.15.1:0/2432775139 shutdown_connections 2024-05-04T03:19:02.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:02.175+0000 7fe3746aa700 1 -- 172.21.15.1:0/2432775139 wait complete. 2024-05-04T03:19:02.788 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:02 smithi001 ceph-mon[29731]: pgmap v387: 97 pgs: 97 active+clean; 654 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-04T03:19:02.789 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:02 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2432775139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:19:02.821 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-05-04T03:19:02.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:02 smithi077 ceph-mon[33531]: pgmap v387: 97 pgs: 97 active+clean; 654 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-04T03:19:02.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:02 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/2432775139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:19:02.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:02 smithi072 ceph-mon[33332]: pgmap v387: 97 pgs: 97 active+clean; 654 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-04T03:19:02.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:02 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/2432775139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:19:03.074 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:19:04.681 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:04 smithi001 ceph-mon[29731]: pgmap v388: 97 pgs: 97 active+clean; 656 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-05-04T03:19:04.930 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:04 smithi077 ceph-mon[33531]: pgmap v388: 97 pgs: 97 active+clean; 656 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-05-04T03:19:04.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:04 smithi072 ceph-mon[33332]: pgmap v388: 97 pgs: 97 active+clean; 656 KiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-05-04T03:19:05.352 INFO:teuthology.orchestra.run.smithi001.stdout:{"run_dir":"/var/run/ceph"} 2024-05-04T03:19:05.353 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-05-04T03:19:05.353 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 1842 2024-05-04T03:19:05.353 DEBUG:tasks.quiescer:Yielding 2024-05-04T03:19:05.353 INFO:teuthology.run_tasks:Running task workunit... 2024-05-04T03:19:05.366 INFO:tasks.workunit:Pulling workunits from ref bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-04T03:19:05.366 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-04T03:19:05.366 INFO:tasks.workunit:timeout=3h 2024-05-04T03:19:05.366 INFO:tasks.workunit:cleanup=True 2024-05-04T03:19:05.367 DEBUG:teuthology.orchestra.run.smithi001:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-04T03:19:05.367 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:19:05.386 INFO:teuthology.orchestra.run.smithi001.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-04T03:19:05.386 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-04T03:19:05.386 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 68h/104d Inode: 1099511628283 Links: 2 2024-05-04T03:19:05.386 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-04T03:19:05.386 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-04T03:19:05.387 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-05-04 03:16:19.711084702 +0000 2024-05-04T03:19:05.387 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-05-04 03:16:19.711084702 +0000 2024-05-04T03:19:05.387 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-05-04 03:18:44.346986719 +0000 2024-05-04T03:19:05.387 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-05-04 03:16:19.711084702 +0000 2024-05-04T03:19:05.387 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-04T03:19:05.387 DEBUG:teuthology.orchestra.run.smithi001:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-04T03:19:05.457 DEBUG:teuthology.orchestra.run.smithi001:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout bd4e00a39d766328b458e4c953f4d722fed96a31 2024-05-04T03:19:05.513 INFO:tasks.workunit.client.0.smithi001.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-04T03:19:05.624 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:19:06.716 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:06 smithi001 ceph-mon[29731]: pgmap v389: 97 pgs: 97 active+clean; 656 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-05-04T03:19:06.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:06 smithi077 ceph-mon[33531]: pgmap v389: 97 pgs: 97 active+clean; 656 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-05-04T03:19:06.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:06 smithi072 ceph-mon[33332]: pgmap v389: 97 pgs: 97 active+clean; 656 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-05-04T03:19:07.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.300+0000 7f671c395700 1 -- 172.21.15.1:0/751175247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6714107e10 msgr2=0x7f6714108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:07.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.300+0000 7f671c395700 1 --2- 172.21.15.1:0/751175247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6714107e10 0x7f6714108220 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f67100099c0 tx=0x7f6710009cd0 comp rx=0 tx=0).stop 2024-05-04T03:19:07.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.300+0000 7f671c395700 1 -- 172.21.15.1:0/751175247 shutdown_connections 2024-05-04T03:19:07.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.300+0000 7f671c395700 1 --2- 172.21.15.1:0/751175247 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6714108760 0x7f67141100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:07.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.300+0000 7f671c395700 1 --2- 172.21.15.1:0/751175247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67141034e0 0x7f67141038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:07.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.300+0000 7f671c395700 1 --2- 172.21.15.1:0/751175247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6714107e10 0x7f6714108220 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:07.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.300+0000 7f671c395700 1 -- 172.21.15.1:0/751175247 >> 172.21.15.1:0/751175247 conn(0x7f67140fef30 msgr2=0x7f6714101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:07.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.301+0000 7f671c395700 1 -- 172.21.15.1:0/751175247 shutdown_connections 2024-05-04T03:19:07.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.301+0000 7f671c395700 1 -- 172.21.15.1:0/751175247 wait complete. 2024-05-04T03:19:07.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.301+0000 7f671c395700 1 Processor -- start 2024-05-04T03:19:07.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.301+0000 7f671c395700 1 -- start start 2024-05-04T03:19:07.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.302+0000 7f671c395700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67141034e0 0x7f671410a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:07.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.302+0000 7f671c395700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6714107e10 0x7f671410a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:07.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.302+0000 7f671c395700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6714108760 0x7f671410abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:07.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.302+0000 7f671c395700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6714113370 con 0x7f6714107e10 2024-05-04T03:19:07.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.302+0000 7f671c395700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f67141131f0 con 0x7f6714108760 2024-05-04T03:19:07.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.302+0000 7f671c395700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f67141134f0 con 0x7f67141034e0 2024-05-04T03:19:07.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.302+0000 7f6719930700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6714107e10 0x7f671410a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:07.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.302+0000 7f6719930700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6714107e10 0x7f671410a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36942/0 (socket says 172.21.15.1:36942) 2024-05-04T03:19:07.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.302+0000 7f6719930700 1 -- 172.21.15.1:0/495252184 learned_addr learned my addr 172.21.15.1:0/495252184 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:07.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.302+0000 7f671a131700 1 --2- 172.21.15.1:0/495252184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67141034e0 0x7f671410a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:07.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.303+0000 7f671a932700 1 --2- 172.21.15.1:0/495252184 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6714108760 0x7f671410abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:07.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.303+0000 7f6719930700 1 -- 172.21.15.1:0/495252184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67141034e0 msgr2=0x7f671410a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:07.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.303+0000 7f6719930700 1 --2- 172.21.15.1:0/495252184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67141034e0 0x7f671410a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:07.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.303+0000 7f6719930700 1 -- 172.21.15.1:0/495252184 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6714108760 msgr2=0x7f671410abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:07.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.303+0000 7f6719930700 1 --2- 172.21.15.1:0/495252184 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6714108760 0x7f671410abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:07.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.303+0000 7f6719930700 1 -- 172.21.15.1:0/495252184 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67141ad670 con 0x7f6714107e10 2024-05-04T03:19:07.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.303+0000 7f671a932700 1 --2- 172.21.15.1:0/495252184 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6714108760 0x7f671410abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:19:07.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.303+0000 7f6719930700 1 --2- 172.21.15.1:0/495252184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6714107e10 0x7f671410a680 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f67100098e0 tx=0x7f67100057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:07.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.303+0000 7f67077fe700 1 -- 172.21.15.1:0/495252184 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710027070 con 0x7f6714107e10 2024-05-04T03:19:07.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.303+0000 7f671c395700 1 -- 172.21.15.1:0/495252184 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67141ad8a0 con 0x7f6714107e10 2024-05-04T03:19:07.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.303+0000 7f67077fe700 1 -- 172.21.15.1:0/495252184 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6710003f20 con 0x7f6714107e10 2024-05-04T03:19:07.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.303+0000 7f671c395700 1 -- 172.21.15.1:0/495252184 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67141adcf0 con 0x7f6714107e10 2024-05-04T03:19:07.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.303+0000 7f67077fe700 1 -- 172.21.15.1:0/495252184 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710031060 con 0x7f6714107e10 2024-05-04T03:19:07.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.305+0000 7f671c395700 1 -- 172.21.15.1:0/495252184 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6714104990 con 0x7f6714107e10 2024-05-04T03:19:07.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.306+0000 7f67077fe700 1 -- 172.21.15.1:0/495252184 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7f671002a070 con 0x7f6714107e10 2024-05-04T03:19:07.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.307+0000 7f67077fe700 1 --2- 172.21.15.1:0/495252184 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f67000b2910 0x7f67000b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:07.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.307+0000 7f671a131700 1 --2- 172.21.15.1:0/495252184 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f67000b2910 0x7f67000b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:07.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.307+0000 7f67077fe700 1 -- 172.21.15.1:0/495252184 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f67100e3550 con 0x7f6714107e10 2024-05-04T03:19:07.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.308+0000 7f671a131700 1 --2- 172.21.15.1:0/495252184 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f67000b2910 0x7f67000b4e20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6708007be0 tx=0x7f67080073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:07.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.311+0000 7f67077fe700 1 -- 172.21.15.1:0/495252184 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6710013070 con 0x7f6714107e10 2024-05-04T03:19:07.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.584+0000 7f671c395700 1 -- 172.21.15.1:0/495252184 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f671402e5f0 con 0x7f6714107e10 2024-05-04T03:19:07.586 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:07 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:19:07.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.588+0000 7f67077fe700 1 -- 172.21.15.1:0/495252184 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11844 (secure 0 0 0) 0x7f67100ab2c0 con 0x7f6714107e10 2024-05-04T03:19:07.590 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:19:07.590 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2186681910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2186681910},{"type":"v1","addr":"172.21.15.1:6835","nonce":2186681910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2747191047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2747191047},{"type":"v1","addr":"172.21.15.1:6837","nonce":2747191047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4115120540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4115120540},{"type":"v1","addr":"172.21.15.1:6839","nonce":4115120540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3405709780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3405709780},{"type":"v1","addr":"172.21.15.1:6841","nonce":3405709780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/3816098761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":3816098761},{"type":"v1","addr":"172.21.15.72:6833","nonce":3816098761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/379026128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":379026128},{"type":"v1","addr":"172.21.15.77:6833","nonce":379026128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:15:49.823471+0000","modified":"2024-05-04T03:18:45.058419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24472,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/3930984976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3930984976},{"type":"v1","addr":"172.21.15.72:6835","nonce":3930984976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/4279951305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4279951305},{"type":"v1","addr":"172.21.15.77:6835","nonce":4279951305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2003521417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2003521417},{"type":"v1","addr":"172.21.15.77:6837","nonce":2003521417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/1417614203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1417614203},{"type":"v1","addr":"172.21.15.72:6837","nonce":1417614203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.77:6839/922723189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":922723189},{"type":"v1","addr":"172.21.15.77:6839","nonce":922723189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2976050650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2976050650},{"type":"v1","addr":"172.21.15.72:6839","nonce":2976050650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24467,24472,24494]},"id":1}]} 2024-05-04T03:19:07.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.592+0000 7f671c395700 1 -- 172.21.15.1:0/495252184 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f67000b2910 msgr2=0x7f67000b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:07.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.592+0000 7f671c395700 1 --2- 172.21.15.1:0/495252184 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f67000b2910 0x7f67000b4e20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6708007be0 tx=0x7f67080073d0 comp rx=0 tx=0).stop 2024-05-04T03:19:07.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.592+0000 7f671c395700 1 -- 172.21.15.1:0/495252184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6714107e10 msgr2=0x7f671410a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:07.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.592+0000 7f671c395700 1 --2- 172.21.15.1:0/495252184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6714107e10 0x7f671410a680 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f67100098e0 tx=0x7f67100057c0 comp rx=0 tx=0).stop 2024-05-04T03:19:07.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.592+0000 7f671c395700 1 -- 172.21.15.1:0/495252184 shutdown_connections 2024-05-04T03:19:07.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.592+0000 7f671c395700 1 --2- 172.21.15.1:0/495252184 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6714108760 0x7f671410abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:07.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.592+0000 7f671c395700 1 --2- 172.21.15.1:0/495252184 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f67000b2910 0x7f67000b4e20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:07.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.592+0000 7f671c395700 1 --2- 172.21.15.1:0/495252184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67141034e0 0x7f671410a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:07.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.592+0000 7f671c395700 1 --2- 172.21.15.1:0/495252184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6714107e10 0x7f671410a680 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:07.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.592+0000 7f671c395700 1 -- 172.21.15.1:0/495252184 >> 172.21.15.1:0/495252184 conn(0x7f67140fef30 msgr2=0x7f671410ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:07.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.592+0000 7f671c395700 1 -- 172.21.15.1:0/495252184 shutdown_connections 2024-05-04T03:19:07.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:07.592+0000 7f671c395700 1 -- 172.21.15.1:0/495252184 wait complete. 2024-05-04T03:19:07.595 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 26 2024-05-04T03:19:07.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:07 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:19:07.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:07 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:19:08.313 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-04T03:15:49.823471+0000', 'modified': '2024-05-04T03:18:45.058419+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24494, 'mds_1': 24472, 'mds_2': 24467}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.72:6835/3930984976', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.72:6834', 'nonce': 3930984976}, {'type': 'v1', 'addr': '172.21.15.72:6835', 'nonce': 3930984976}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6835/4279951305', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6834', 'nonce': 4279951305}, {'type': 'v1', 'addr': '172.21.15.77:6835', 'nonce': 4279951305}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6837/2003521417', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6836', 'nonce': 2003521417}, {'type': 'v1', 'addr': '172.21.15.77:6837', 'nonce': 2003521417}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.72:6837/1417614203', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.72:6836', 'nonce': 1417614203}, {'type': 'v1', 'addr': '172.21.15.72:6837', 'nonce': 1417614203}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.77:6839/922723189', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6838', 'nonce': 922723189}, {'type': 'v1', 'addr': '172.21.15.77:6839', 'nonce': 922723189}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.72:6839/2976050650', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.72:6838', 'nonce': 2976050650}, {'type': 'v1', 'addr': '172.21.15.72:6839', 'nonce': 2976050650}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24494, 'qdb_cluster': [24467, 24472, 24494]} 2024-05-04T03:19:08.313 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-04T03:19:08.314 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph tell mds.h status 2024-05-04T03:19:08.615 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:19:08.859 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:08 smithi001 ceph-mon[29731]: pgmap v390: 97 pgs: 97 active+clean; 656 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-04T03:19:08.859 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:08 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:19:08.859 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:08 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:19:08.859 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:08 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:19:08.860 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:08 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/495252184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:19:08.860 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:08 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:19:08.860 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:08 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:19:08.860 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:08 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:19:08.860 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:08 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:19:08.860 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:08 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:19:08.860 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:08 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:19:08.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:08 smithi077 ceph-mon[33531]: pgmap v390: 97 pgs: 97 active+clean; 656 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-04T03:19:08.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:08 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:19:08.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:08 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:19:08.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:08 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:19:08.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:08 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/495252184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:19:08.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:08 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:19:08.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:08 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:19:08.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:08 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:19:08.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:08 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:19:08.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:08 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:19:08.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:08 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:19:08.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:08 smithi072 ceph-mon[33332]: pgmap v390: 97 pgs: 97 active+clean; 656 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-04T03:19:08.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:08 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:19:08.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:08 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:19:08.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:08 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:19:08.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:08 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/495252184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:19:08.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:08 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:19:08.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:08 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:19:08.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:08 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:19:08.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:08 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:19:08.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:08 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:19:08.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:08 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:19:10.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.223+0000 7f6acb1b9700 1 -- 172.21.15.1:0/1869344004 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ac4108740 msgr2=0x7f6ac41100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:10.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.223+0000 7f6acb1b9700 1 --2- 172.21.15.1:0/1869344004 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ac4108740 0x7f6ac41100a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00b0d0 tx=0x7f6abc00b3e0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.223+0000 7f6acb1b9700 1 -- 172.21.15.1:0/1869344004 shutdown_connections 2024-05-04T03:19:10.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.223+0000 7f6acb1b9700 1 --2- 172.21.15.1:0/1869344004 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ac41034c0 0x7f6ac41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.223+0000 7f6acb1b9700 1 --2- 172.21.15.1:0/1869344004 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ac4108740 0x7f6ac41100a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.223+0000 7f6acb1b9700 1 --2- 172.21.15.1:0/1869344004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ac4107df0 0x7f6ac4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.223+0000 7f6acb1b9700 1 -- 172.21.15.1:0/1869344004 >> 172.21.15.1:0/1869344004 conn(0x7f6ac40fedf0 msgr2=0x7f6ac4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:10.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.223+0000 7f6acb1b9700 1 -- 172.21.15.1:0/1869344004 shutdown_connections 2024-05-04T03:19:10.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.223+0000 7f6acb1b9700 1 -- 172.21.15.1:0/1869344004 wait complete. 2024-05-04T03:19:10.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.224+0000 7f6acb1b9700 1 Processor -- start 2024-05-04T03:19:10.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.224+0000 7f6acb1b9700 1 -- start start 2024-05-04T03:19:10.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.224+0000 7f6acb1b9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ac41034c0 0x7f6ac419bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:10.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.224+0000 7f6acb1b9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ac4107df0 0x7f6ac419c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:10.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.224+0000 7f6acb1b9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ac4108740 0x7f6ac419c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:10.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.224+0000 7f6acb1b9700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac4113370 con 0x7f6ac4108740 2024-05-04T03:19:10.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.224+0000 7f6acb1b9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac41131f0 con 0x7f6ac41034c0 2024-05-04T03:19:10.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.224+0000 7f6acb1b9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac41134f0 con 0x7f6ac4107df0 2024-05-04T03:19:10.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.224+0000 7f6ac9756700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ac4108740 0x7f6ac419c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:10.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.224+0000 7f6ac9756700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ac4108740 0x7f6ac419c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36964/0 (socket says 172.21.15.1:36964) 2024-05-04T03:19:10.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.224+0000 7f6ac9756700 1 -- 172.21.15.1:0/2510317163 learned_addr learned my addr 172.21.15.1:0/2510317163 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:10.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.225+0000 7f6ac3fff700 1 --2- 172.21.15.1:0/2510317163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ac4107df0 0x7f6ac419c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:10.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.225+0000 7f6ac8f55700 1 --2- 172.21.15.1:0/2510317163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ac41034c0 0x7f6ac419bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:10.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.225+0000 7f6ac9756700 1 -- 172.21.15.1:0/2510317163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ac4107df0 msgr2=0x7f6ac419c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:10.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.225+0000 7f6ac9756700 1 --2- 172.21.15.1:0/2510317163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ac4107df0 0x7f6ac419c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.225+0000 7f6ac9756700 1 -- 172.21.15.1:0/2510317163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ac41034c0 msgr2=0x7f6ac419bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:10.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.225+0000 7f6ac9756700 1 --2- 172.21.15.1:0/2510317163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ac41034c0 0x7f6ac419bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.225+0000 7f6ac9756700 1 -- 172.21.15.1:0/2510317163 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac41a6f80 con 0x7f6ac4108740 2024-05-04T03:19:10.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.225+0000 7f6ac8f55700 1 --2- 172.21.15.1:0/2510317163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ac41034c0 0x7f6ac419bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:19:10.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.225+0000 7f6ac9756700 1 --2- 172.21.15.1:0/2510317163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ac4108740 0x7f6ac419c6d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6abc0095c0 tx=0x7f6abc007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:10.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.225+0000 7f6ac1ffb700 1 -- 172.21.15.1:0/2510317163 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6abc01f070 con 0x7f6ac4108740 2024-05-04T03:19:10.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.225+0000 7f6ac1ffb700 1 -- 172.21.15.1:0/2510317163 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6abc003c00 con 0x7f6ac4108740 2024-05-04T03:19:10.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.225+0000 7f6ac1ffb700 1 -- 172.21.15.1:0/2510317163 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6abc01b6a0 con 0x7f6ac4108740 2024-05-04T03:19:10.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.225+0000 7f6acb1b9700 1 -- 172.21.15.1:0/2510317163 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ac41a7210 con 0x7f6ac4108740 2024-05-04T03:19:10.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.226+0000 7f6acb1b9700 1 -- 172.21.15.1:0/2510317163 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ac41a7660 con 0x7f6ac4108740 2024-05-04T03:19:10.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.228+0000 7f6ac1ffb700 1 -- 172.21.15.1:0/2510317163 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7f6abc022070 con 0x7f6ac4108740 2024-05-04T03:19:10.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.229+0000 7f6ac1ffb700 1 --2- 172.21.15.1:0/2510317163 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6ab40b2910 0x7f6ab40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:10.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.229+0000 7f6ac8f55700 1 --2- 172.21.15.1:0/2510317163 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6ab40b2910 0x7f6ab40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:10.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.229+0000 7f6ac1ffb700 1 -- 172.21.15.1:0/2510317163 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f6abc00f020 con 0x7f6ac4108740 2024-05-04T03:19:10.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.229+0000 7f6ad9bcdb80 1 Processor -- start 2024-05-04T03:19:10.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.229+0000 7f6ad9bcdb80 1 -- start start 2024-05-04T03:19:10.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.229+0000 7f6ad9bcdb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5560956060d0 0x5560956064e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:10.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.229+0000 7f6ad9bcdb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5560956001c0 0x5560956005d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:10.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.229+0000 7f6ad9bcdb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556095600b10 0x55609560cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:10.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.229+0000 7f6ad9bcdb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5560955228f0 con 0x5560956001c0 2024-05-04T03:19:10.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.229+0000 7f6ad9bcdb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x556095487d50 con 0x5560956060d0 2024-05-04T03:19:10.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.229+0000 7f6ad9bcdb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55609551e8b0 con 0x556095600b10 2024-05-04T03:19:10.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.229+0000 7f6ac8f55700 1 --2- 172.21.15.1:0/2510317163 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6ab40b2910 0x7f6ab40b4e20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8005950 tx=0x7f6ab80058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:10.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.230+0000 7f6ac3fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5560956001c0 0x5560956005d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:10.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.230+0000 7f6ac3fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5560956001c0 0x5560956005d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36974/0 (socket says 172.21.15.1:36974) 2024-05-04T03:19:10.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.230+0000 7f6ac3fff700 1 -- 172.21.15.1:0/2860929413 learned_addr learned my addr 172.21.15.1:0/2860929413 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:10.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.230+0000 7f6ac8f55700 1 --2- 172.21.15.1:0/2860929413 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5560956060d0 0x5560956064e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:10.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.230+0000 7f6ac9756700 1 --2- 172.21.15.1:0/2860929413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556095600b10 0x55609560cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:10.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.230+0000 7f6ac3fff700 1 -- 172.21.15.1:0/2860929413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556095600b10 msgr2=0x55609560cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:10.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.230+0000 7f6ac3fff700 1 --2- 172.21.15.1:0/2860929413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556095600b10 0x55609560cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.230+0000 7f6ac3fff700 1 -- 172.21.15.1:0/2860929413 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5560956060d0 msgr2=0x5560956064e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:10.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.230+0000 7f6ac3fff700 1 --2- 172.21.15.1:0/2860929413 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5560956060d0 0x5560956064e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.230+0000 7f6ac3fff700 1 -- 172.21.15.1:0/2860929413 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55609546f880 con 0x5560956001c0 2024-05-04T03:19:10.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.230+0000 7f6ac3fff700 1 --2- 172.21.15.1:0/2860929413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5560956001c0 0x5560956005d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6ab000dc80 tx=0x7f6ab000df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51d53ae5c4f42e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:10.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.230+0000 7f6aaeffd700 1 -- 172.21.15.1:0/2860929413 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab000fb70 con 0x5560956001c0 2024-05-04T03:19:10.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.230+0000 7f6aaeffd700 1 -- 172.21.15.1:0/2860929413 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ab000fd10 con 0x5560956001c0 2024-05-04T03:19:10.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.230+0000 7f6aaeffd700 1 -- 172.21.15.1:0/2860929413 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab001b700 con 0x5560956001c0 2024-05-04T03:19:10.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.231+0000 7f6ad9bcdb80 1 -- 172.21.15.1:0/2860929413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5560956001c0 msgr2=0x5560956005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:10.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.231+0000 7f6ad9bcdb80 1 --2- 172.21.15.1:0/2860929413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5560956001c0 0x5560956005d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6ab000dc80 tx=0x7f6ab000df90 comp rx=0 tx=0).stop 2024-05-04T03:19:10.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.231+0000 7f6ad9bcdb80 1 -- 172.21.15.1:0/2860929413 shutdown_connections 2024-05-04T03:19:10.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.231+0000 7f6ad9bcdb80 1 --2- 172.21.15.1:0/2860929413 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5560956060d0 0x5560956064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.231+0000 7f6ad9bcdb80 1 --2- 172.21.15.1:0/2860929413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556095600b10 0x55609560cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.231+0000 7f6ad9bcdb80 1 --2- 172.21.15.1:0/2860929413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5560956001c0 0x5560956005d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.231+0000 7f6ad9bcdb80 1 -- 172.21.15.1:0/2860929413 >> 172.21.15.1:0/2860929413 conn(0x556095482df0 msgr2=0x5560955fb210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:10.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.231+0000 7f6ad9bcdb80 1 -- 172.21.15.1:0/2860929413 shutdown_connections 2024-05-04T03:19:10.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.231+0000 7f6ad9bcdb80 1 -- 172.21.15.1:0/2860929413 wait complete. 2024-05-04T03:19:10.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.232+0000 7f6ad9bcdb80 1 -- start start 2024-05-04T03:19:10.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.232+0000 7f6ad9bcdb80 1 Processor -- start 2024-05-04T03:19:10.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.232+0000 7f6ad9bcdb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5560956001c0 0x55609560a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:10.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.232+0000 7f6ad9bcdb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556095600b10 0x55609560a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:10.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.232+0000 7f6ad9bcdb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5560956060d0 0x55609560ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:10.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.232+0000 7f6ad9bcdb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55609560d4a0 con 0x5560956001c0 2024-05-04T03:19:10.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.232+0000 7f6ad9bcdb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5560954dbe80 con 0x5560956060d0 2024-05-04T03:19:10.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.232+0000 7f6ad9bcdb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55609560d620 con 0x556095600b10 2024-05-04T03:19:10.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.233+0000 7f6ad9bcdb80 10 client.0 _ll_register_callbacks cb 0x5560954ba900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:10.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.233+0000 7f6ac8f55700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5560956001c0 0x55609560a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:10.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.233+0000 7f6ac8f55700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5560956001c0 0x55609560a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36990/0 (socket says 172.21.15.1:36990) 2024-05-04T03:19:10.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.233+0000 7f6ac8f55700 1 -- 172.21.15.1:0/3859945377 learned_addr learned my addr 172.21.15.1:0/3859945377 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:10.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.233+0000 7f6ac9756700 1 --2- 172.21.15.1:0/3859945377 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5560956060d0 0x55609560ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:10.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.233+0000 7f6ac3fff700 1 --2- 172.21.15.1:0/3859945377 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556095600b10 0x55609560a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:10.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.233+0000 7f6ac8f55700 1 -- 172.21.15.1:0/3859945377 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556095600b10 msgr2=0x55609560a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:10.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.233+0000 7f6ac8f55700 1 --2- 172.21.15.1:0/3859945377 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556095600b10 0x55609560a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.233+0000 7f6ac8f55700 1 -- 172.21.15.1:0/3859945377 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5560956060d0 msgr2=0x55609560ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:10.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.233+0000 7f6ac8f55700 1 --2- 172.21.15.1:0/3859945377 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5560956060d0 0x55609560ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.233+0000 7f6ac8f55700 1 -- 172.21.15.1:0/3859945377 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560954d9710 con 0x5560956001c0 2024-05-04T03:19:10.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.233+0000 7f6ac8f55700 1 --2- 172.21.15.1:0/3859945377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5560956001c0 0x55609560a260 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80042b0 tx=0x7f6ab8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a9e4c4352ebe3a server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:10.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.233+0000 7f6aa3fff700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:19:10.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.233+0000 7f6aa3fff700 1 -- 172.21.15.1:0/3859945377 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab8024070 con 0x5560956001c0 2024-05-04T03:19:10.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.233+0000 7f6aa3fff700 1 -- 172.21.15.1:0/3859945377 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f6ab8003a90 con 0x5560956001c0 2024-05-04T03:19:10.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.233+0000 7f6ad9bcdb80 1 -- 172.21.15.1:0/3859945377 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ab000fd10 con 0x5560956001c0 2024-05-04T03:19:10.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.234+0000 7f6aa3fff700 1 -- 172.21.15.1:0/3859945377 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6ab8030050 con 0x5560956001c0 2024-05-04T03:19:10.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.234+0000 7f6ad9bcdb80 10 client.15183 fetch_fsmap learned FSMap version 26 2024-05-04T03:19:10.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.234+0000 7f6ad9bcdb80 1 -- 172.21.15.1:0/3859945377 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556095618130 con 0x5560956001c0 2024-05-04T03:19:10.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.234+0000 7f6aa3fff700 1 -- 172.21.15.1:0/3859945377 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ab000fd10 con 0x5560956001c0 2024-05-04T03:19:10.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.234+0000 7f6aa3fff700 1 -- 172.21.15.1:0/3859945377 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6ab8020410 con 0x5560956001c0 2024-05-04T03:19:10.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.234+0000 7f6ad9bcdb80 10 client.15183 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:19:10.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.234+0000 7f6ad9bcdb80 10 client.15183 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-04T03:19:10.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.234+0000 7f6ad9bcdb80 20 client.15183 populate_metadata read hostname 'smithi001' 2024-05-04T03:19:10.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.234+0000 7f6ad9bcdb80 1 --2- 172.21.15.1:0/3859945377 >> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] conn(0x55609561a800 0x55609561bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:10.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.234+0000 7f6ad9bcdb80 4 client.15183 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:19:10.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.234+0000 7f6ad9bcdb80 1 -- 172.21.15.1:0/3859945377 --> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560955302d0 con 0x55609561a800 2024-05-04T03:19:10.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.234+0000 7f6ac3fff700 1 --2- 172.21.15.1:0/3859945377 >> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] conn(0x55609561a800 0x55609561bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:10.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.235+0000 7f6aa3fff700 10 client.15183 ms_handle_connect on v2:172.21.15.72:6836/1417614203 2024-05-04T03:19:10.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.235+0000 7f6ac3fff700 1 --2- 172.21.15.1:0/3859945377 >> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] conn(0x55609561a800 0x55609561bc70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cbaa9a6cac598a43 server_cookie=e0f43d7602132de in_seq=0 out_seq=0 2024-05-04T03:19:10.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.236+0000 7f6aa3fff700 1 -- 172.21.15.1:0/3859945377 <== mds.2 v2:172.21.15.72:6836/1417614203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5560955302d0 con 0x55609561a800 2024-05-04T03:19:10.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.236+0000 7f6aa3fff700 10 client.15183 handle_command_reply: tid=0 2024-05-04T03:19:10.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.236+0000 7f6ad9bcdb80 1 client.15183 shutdown 2024-05-04T03:19:10.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.236+0000 7f6ad9bcdb80 1 -- 172.21.15.1:0/3859945377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5560956001c0 msgr2=0x55609560a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:10.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.236+0000 7f6ad9bcdb80 1 --2- 172.21.15.1:0/3859945377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5560956001c0 0x55609560a260 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80042b0 tx=0x7f6ab8004750 comp rx=0 tx=0).stop 2024-05-04T03:19:10.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.236+0000 7f6ad9bcdb80 1 -- 172.21.15.1:0/3859945377 shutdown_connections 2024-05-04T03:19:10.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.236+0000 7f6ad9bcdb80 1 --2- 172.21.15.1:0/3859945377 >> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] conn(0x55609561a800 0x55609561bc70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.236+0000 7f6ad9bcdb80 1 --2- 172.21.15.1:0/3859945377 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5560956060d0 0x55609560ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.236+0000 7f6ad9bcdb80 1 --2- 172.21.15.1:0/3859945377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5560956001c0 0x55609560a260 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.236+0000 7f6ad9bcdb80 1 --2- 172.21.15.1:0/3859945377 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556095600b10 0x55609560a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.236+0000 7f6aa3fff700 0 client.15183 ms_handle_reset on v2:172.21.15.72:6836/1417614203 2024-05-04T03:19:10.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.236+0000 7f6ad9bcdb80 1 -- 172.21.15.1:0/3859945377 >> 172.21.15.1:0/3859945377 conn(0x556095482df0 msgr2=0x5560955ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:10.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.236+0000 7f6ad9bcdb80 1 -- 172.21.15.1:0/3859945377 shutdown_connections 2024-05-04T03:19:10.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.236+0000 7f6ad9bcdb80 1 -- 172.21.15.1:0/3859945377 wait complete. 2024-05-04T03:19:10.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.236+0000 7f6ad9bcdb80 20 client.15183 trim_cache size 0 max 16384 2024-05-04T03:19:10.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.257+0000 7f6ad9bcdb80 1 Processor -- start 2024-05-04T03:19:10.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.257+0000 7f6ad9bcdb80 1 -- start start 2024-05-04T03:19:10.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.258+0000 7f6ad9bcdb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5560956001c0 0x556095609220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:10.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.258+0000 7f6ad9bcdb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556095600b10 0x5560955ffbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:10.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.258+0000 7f6ad9bcdb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5560956060d0 0x5560956049f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:10.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.258+0000 7f6ad9bcdb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x556095619fb0 con 0x556095600b10 2024-05-04T03:19:10.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.258+0000 7f6ad9bcdb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55609561a680 con 0x5560956001c0 2024-05-04T03:19:10.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.258+0000 7f6ad9bcdb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55609561ad50 con 0x5560956060d0 2024-05-04T03:19:10.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.258+0000 7f6ac3fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556095600b10 0x5560955ffbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:10.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.258+0000 7f6ac3fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556095600b10 0x5560955ffbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36996/0 (socket says 172.21.15.1:36996) 2024-05-04T03:19:10.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.258+0000 7f6ac3fff700 1 -- 172.21.15.1:0/4100049163 learned_addr learned my addr 172.21.15.1:0/4100049163 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:10.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.258+0000 7f6ac9756700 1 --2- 172.21.15.1:0/4100049163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5560956060d0 0x5560956049f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:10.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.258+0000 7f6ac8f55700 1 --2- 172.21.15.1:0/4100049163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5560956001c0 0x556095609220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:10.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.258+0000 7f6ac3fff700 1 -- 172.21.15.1:0/4100049163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5560956060d0 msgr2=0x5560956049f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:10.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.258+0000 7f6ac3fff700 1 --2- 172.21.15.1:0/4100049163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5560956060d0 0x5560956049f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.258+0000 7f6ac3fff700 1 -- 172.21.15.1:0/4100049163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5560956001c0 msgr2=0x556095609220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:10.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.258+0000 7f6ac3fff700 1 --2- 172.21.15.1:0/4100049163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5560956001c0 0x556095609220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.258+0000 7f6ac3fff700 1 -- 172.21.15.1:0/4100049163 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55609560cb00 con 0x556095600b10 2024-05-04T03:19:10.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.258+0000 7f6ac8f55700 1 --2- 172.21.15.1:0/4100049163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5560956001c0 0x556095609220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:19:10.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.258+0000 7f6ac9756700 1 --2- 172.21.15.1:0/4100049163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5560956060d0 0x5560956049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:19:10.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.258+0000 7f6ac3fff700 1 --2- 172.21.15.1:0/4100049163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556095600b10 0x5560955ffbe0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0011e60 tx=0x7f6ab0011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0e8990789911a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:10.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.258+0000 7f6aae7fc700 1 -- 172.21.15.1:0/4100049163 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0004040 con 0x556095600b10 2024-05-04T03:19:10.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.259+0000 7f6aae7fc700 1 -- 172.21.15.1:0/4100049163 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ab0005040 con 0x556095600b10 2024-05-04T03:19:10.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.259+0000 7f6aae7fc700 1 -- 172.21.15.1:0/4100049163 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0011680 con 0x556095600b10 2024-05-04T03:19:10.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.259+0000 7f6ad9bcdb80 1 -- 172.21.15.1:0/4100049163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556095600b10 msgr2=0x5560955ffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:10.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.259+0000 7f6ad9bcdb80 1 --2- 172.21.15.1:0/4100049163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556095600b10 0x5560955ffbe0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0011e60 tx=0x7f6ab0011c50 comp rx=0 tx=0).stop 2024-05-04T03:19:10.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.260+0000 7f6ad9bcdb80 1 -- 172.21.15.1:0/4100049163 shutdown_connections 2024-05-04T03:19:10.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.260+0000 7f6ad9bcdb80 1 --2- 172.21.15.1:0/4100049163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5560956001c0 0x556095609220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.260+0000 7f6ad9bcdb80 1 --2- 172.21.15.1:0/4100049163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5560956060d0 0x5560956049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.260+0000 7f6ad9bcdb80 1 --2- 172.21.15.1:0/4100049163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556095600b10 0x5560955ffbe0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.260+0000 7f6ad9bcdb80 1 -- 172.21.15.1:0/4100049163 >> 172.21.15.1:0/4100049163 conn(0x556095482df0 msgr2=0x5560955fb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:10.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.260+0000 7f6ad9bcdb80 1 -- 172.21.15.1:0/4100049163 shutdown_connections 2024-05-04T03:19:10.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.260+0000 7f6ad9bcdb80 1 -- 172.21.15.1:0/4100049163 wait complete. 2024-05-04T03:19:10.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.260+0000 7f6ad9bcdb80 1 -- start start 2024-05-04T03:19:10.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.260+0000 7f6ad9bcdb80 1 Processor -- start 2024-05-04T03:19:10.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.261+0000 7f6ad9bcdb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5560956001c0 0x5560956806c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:10.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.261+0000 7f6ad9bcdb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556095600b10 0x556095680c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:10.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.261+0000 7f6ad9bcdb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5560956060d0 0x5560956240c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:10.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.261+0000 7f6ad9bcdb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55609561baf0 con 0x556095600b10 2024-05-04T03:19:10.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.261+0000 7f6ad9bcdb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55609561b420 con 0x5560956001c0 2024-05-04T03:19:10.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.261+0000 7f6ad9bcdb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55609561c1c0 con 0x5560956060d0 2024-05-04T03:19:10.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.261+0000 7f6ad9bcdb80 10 client.0 _ll_register_callbacks cb 0x5560954ba900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:10.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.261+0000 7f6ac8f55700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5560956001c0 0x5560956806c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:10.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.261+0000 7f6ac8f55700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5560956001c0 0x5560956806c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:38480/0 (socket says 172.21.15.1:38480) 2024-05-04T03:19:10.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.261+0000 7f6ac8f55700 1 -- 172.21.15.1:0/1577125209 learned_addr learned my addr 172.21.15.1:0/1577125209 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:10.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.261+0000 7f6ac9756700 1 --2- 172.21.15.1:0/1577125209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5560956060d0 0x5560956240c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:10.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.261+0000 7f6ac3fff700 1 --2- 172.21.15.1:0/1577125209 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556095600b10 0x556095680c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:10.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.261+0000 7f6ac8f55700 1 -- 172.21.15.1:0/1577125209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5560956060d0 msgr2=0x5560956240c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:10.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.261+0000 7f6ac8f55700 1 --2- 172.21.15.1:0/1577125209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5560956060d0 0x5560956240c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.261+0000 7f6ac8f55700 1 -- 172.21.15.1:0/1577125209 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556095600b10 msgr2=0x556095680c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:10.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.261+0000 7f6ac8f55700 1 --2- 172.21.15.1:0/1577125209 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556095600b10 0x556095680c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.261+0000 7f6ac8f55700 1 -- 172.21.15.1:0/1577125209 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556095624600 con 0x5560956001c0 2024-05-04T03:19:10.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.262+0000 7f6ac8f55700 1 --2- 172.21.15.1:0/1577125209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5560956001c0 0x5560956806c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6ab800fc50 tx=0x7f6ab800f9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cd5c82f9b3e883c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:10.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.262+0000 7f6aa3fff700 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-04T03:19:10.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.262+0000 7f6aa3fff700 1 -- 172.21.15.1:0/1577125209 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab8024040 con 0x5560956001c0 2024-05-04T03:19:10.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.262+0000 7f6ad9bcdb80 1 -- 172.21.15.1:0/1577125209 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ab0005040 con 0x5560956001c0 2024-05-04T03:19:10.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.262+0000 7f6aa3fff700 1 -- 172.21.15.1:0/1577125209 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f6ab8003890 con 0x5560956001c0 2024-05-04T03:19:10.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.262+0000 7f6aa3fff700 1 -- 172.21.15.1:0/1577125209 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6ab8003a90 con 0x5560956001c0 2024-05-04T03:19:10.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.262+0000 7f6ad9bcdb80 10 client.24950 fetch_fsmap learned FSMap version 26 2024-05-04T03:19:10.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.262+0000 7f6ad9bcdb80 1 -- 172.21.15.1:0/1577125209 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556095625700 con 0x5560956001c0 2024-05-04T03:19:10.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.262+0000 7f6aa3fff700 1 -- 172.21.15.1:0/1577125209 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ab0005040 con 0x5560956001c0 2024-05-04T03:19:10.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.262+0000 7f6aa3fff700 1 -- 172.21.15.1:0/1577125209 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6ab8046020 con 0x5560956001c0 2024-05-04T03:19:10.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.262+0000 7f6ad9bcdb80 10 client.24950 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:19:10.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.263+0000 7f6ad9bcdb80 10 client.24950 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-04T03:19:10.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.263+0000 7f6ad9bcdb80 20 client.24950 populate_metadata read hostname 'smithi001' 2024-05-04T03:19:10.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.263+0000 7f6ad9bcdb80 1 --2- 172.21.15.1:0/1577125209 >> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] conn(0x556095690720 0x556095691b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:10.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.263+0000 7f6ad9bcdb80 4 client.24950 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-04T03:19:10.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.263+0000 7f6ad9bcdb80 1 -- 172.21.15.1:0/1577125209 --> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] -- command(tid 0: {"prefix": "status"}) -- 0x55609552a370 con 0x556095690720 2024-05-04T03:19:10.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.263+0000 7f6ac3fff700 1 --2- 172.21.15.1:0/1577125209 >> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] conn(0x556095690720 0x556095691b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:10.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.263+0000 7f6ac3fff700 1 --2- 172.21.15.1:0/1577125209 >> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] conn(0x556095690720 0x556095691b90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a5175309283e5222 server_cookie=f68d41c20d7f171d in_seq=0 out_seq=0 2024-05-04T03:19:10.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.263+0000 7f6aa3fff700 10 client.24950 ms_handle_connect on v2:172.21.15.72:6836/1417614203 2024-05-04T03:19:10.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.263+0000 7f6aa3fff700 1 -- 172.21.15.1:0/1577125209 <== mds.2 v2:172.21.15.72:6836/1417614203 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55609552a370 con 0x556095690720 2024-05-04T03:19:10.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.263+0000 7f6aa3fff700 10 client.24950 handle_command_reply: tid=0 2024-05-04T03:19:10.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.263+0000 7f6ad9bcdb80 1 client.24950 shutdown 2024-05-04T03:19:10.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.264+0000 7f6ad9bcdb80 1 -- 172.21.15.1:0/1577125209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5560956001c0 msgr2=0x5560956806c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:10.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.264+0000 7f6ad9bcdb80 1 --2- 172.21.15.1:0/1577125209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5560956001c0 0x5560956806c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6ab800fc50 tx=0x7f6ab800f9d0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.264+0000 7f6ad9bcdb80 1 -- 172.21.15.1:0/1577125209 shutdown_connections 2024-05-04T03:19:10.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.264+0000 7f6ad9bcdb80 1 --2- 172.21.15.1:0/1577125209 >> [v2:172.21.15.72:6836/1417614203,v1:172.21.15.72:6837/1417614203] conn(0x556095690720 0x556095691b90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.264+0000 7f6ad9bcdb80 1 --2- 172.21.15.1:0/1577125209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5560956001c0 0x5560956806c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.264+0000 7f6ad9bcdb80 1 --2- 172.21.15.1:0/1577125209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5560956060d0 0x5560956240c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.264+0000 7f6ad9bcdb80 1 --2- 172.21.15.1:0/1577125209 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556095600b10 0x556095680c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.264+0000 7f6aa3fff700 0 client.24950 ms_handle_reset on v2:172.21.15.72:6836/1417614203 2024-05-04T03:19:10.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.264+0000 7f6ad9bcdb80 1 -- 172.21.15.1:0/1577125209 >> 172.21.15.1:0/1577125209 conn(0x556095482df0 msgr2=0x55609560c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:10.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.264+0000 7f6ad9bcdb80 1 -- 172.21.15.1:0/1577125209 shutdown_connections 2024-05-04T03:19:10.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.264+0000 7f6ad9bcdb80 1 -- 172.21.15.1:0/1577125209 wait complete. 2024-05-04T03:19:10.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.264+0000 7f6ad9bcdb80 20 client.24950 trim_cache size 0 max 16384 2024-05-04T03:19:10.271 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-04T03:19:10.271 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "4a572aa8-09c3-11ef-bc97-c7b262605968", 2024-05-04T03:19:10.271 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 2, 2024-05-04T03:19:10.271 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24467, 2024-05-04T03:19:10.271 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-05-04T03:19:10.271 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-05-04T03:19:10.271 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-05-04T03:19:10.271 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 187.03038165500001, 2024-05-04T03:19:10.272 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 26, 2024-05-04T03:19:10.272 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 83, 2024-05-04T03:19:10.272 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 83, 2024-05-04T03:19:10.272 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 228.705414574 2024-05-04T03:19:10.272 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-04T03:19:10.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.265+0000 7f6acb1b9700 1 -- 172.21.15.1:0/2510317163 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6ab40b2910 msgr2=0x7f6ab40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:10.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.265+0000 7f6acb1b9700 1 --2- 172.21.15.1:0/2510317163 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6ab40b2910 0x7f6ab40b4e20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8005950 tx=0x7f6ab80058e0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.265+0000 7f6acb1b9700 1 -- 172.21.15.1:0/2510317163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ac4108740 msgr2=0x7f6ac419c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:10.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.265+0000 7f6acb1b9700 1 --2- 172.21.15.1:0/2510317163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ac4108740 0x7f6ac419c6d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6abc0095c0 tx=0x7f6abc007ad0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.265+0000 7f6acb1b9700 1 -- 172.21.15.1:0/2510317163 shutdown_connections 2024-05-04T03:19:10.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.265+0000 7f6acb1b9700 1 --2- 172.21.15.1:0/2510317163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ac41034c0 0x7f6ac419bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.265+0000 7f6acb1b9700 1 --2- 172.21.15.1:0/2510317163 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6ab40b2910 0x7f6ab40b4e20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.265+0000 7f6acb1b9700 1 --2- 172.21.15.1:0/2510317163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6ac4108740 0x7f6ac419c6d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.265+0000 7f6acb1b9700 1 --2- 172.21.15.1:0/2510317163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ac4107df0 0x7f6ac419c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:10.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.265+0000 7f6acb1b9700 1 -- 172.21.15.1:0/2510317163 >> 172.21.15.1:0/2510317163 conn(0x7f6ac40fedf0 msgr2=0x7f6ac4107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:10.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.265+0000 7f6acb1b9700 1 -- 172.21.15.1:0/2510317163 shutdown_connections 2024-05-04T03:19:10.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:10.265+0000 7f6acb1b9700 1 -- 172.21.15.1:0/2510317163 wait complete. 2024-05-04T03:19:10.829 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:10 smithi001 ceph-mon[29731]: pgmap v391: 97 pgs: 97 active+clean; 665 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-05-04T03:19:10.858 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-04T03:19:10.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:10 smithi077 ceph-mon[33531]: pgmap v391: 97 pgs: 97 active+clean; 665 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-05-04T03:19:10.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:10 smithi072 ceph-mon[33332]: pgmap v391: 97 pgs: 97 active+clean; 665 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-05-04T03:19:11.106 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:19:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.261+0000 7f9cef049640 1 -- 172.21.15.1:0/343933024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ce81034f0 msgr2=0x7f9ce8103900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.261+0000 7f9cef049640 1 --2- 172.21.15.1:0/343933024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ce81034f0 0x7f9ce8103900 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4009900 tx=0x7f9cd402b520 comp rx=0 tx=0).stop 2024-05-04T03:19:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.262+0000 7f9cef049640 1 -- 172.21.15.1:0/343933024 shutdown_connections 2024-05-04T03:19:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.262+0000 7f9cef049640 1 --2- 172.21.15.1:0/343933024 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ce81081f0 0x7f9ce810a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.262+0000 7f9cef049640 1 --2- 172.21.15.1:0/343933024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ce81034f0 0x7f9ce8103900 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.262+0000 7f9cef049640 1 --2- 172.21.15.1:0/343933024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ce81022b0 0x7f9ce8102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.262+0000 7f9cef049640 1 -- 172.21.15.1:0/343933024 >> 172.21.15.1:0/343933024 conn(0x7f9ce80fd480 msgr2=0x7f9ce80ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.262+0000 7f9cef049640 1 -- 172.21.15.1:0/343933024 shutdown_connections 2024-05-04T03:19:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.262+0000 7f9cef049640 1 -- 172.21.15.1:0/343933024 wait complete. 2024-05-04T03:19:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.262+0000 7f9cef049640 1 Processor -- start 2024-05-04T03:19:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.262+0000 7f9cef049640 1 -- start start 2024-05-04T03:19:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.263+0000 7f9cef049640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ce81022b0 0x7f9ce819a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.263+0000 7f9cef049640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ce81034f0 0x7f9ce819a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.263+0000 7f9cef049640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ce81081f0 0x7f9ce81a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.263+0000 7f9cef049640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce810d540 con 0x7f9ce81081f0 2024-05-04T03:19:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.263+0000 7f9cef049640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce810d3c0 con 0x7f9ce81022b0 2024-05-04T03:19:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.263+0000 7f9cef049640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce810d6c0 con 0x7f9ce81034f0 2024-05-04T03:19:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.263+0000 7f9ced5e6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ce81081f0 0x7f9ce81a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.263+0000 7f9ced5e6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ce81081f0 0x7f9ce81a1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44732/0 (socket says 172.21.15.1:44732) 2024-05-04T03:19:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.263+0000 7f9ced5e6640 1 -- 172.21.15.1:0/1792198776 learned_addr learned my addr 172.21.15.1:0/1792198776 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.263+0000 7f9cecde5640 1 --2- 172.21.15.1:0/1792198776 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ce81022b0 0x7f9ce819a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.263+0000 7f9ce7fff640 1 --2- 172.21.15.1:0/1792198776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ce81034f0 0x7f9ce819a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.263+0000 7f9ced5e6640 1 -- 172.21.15.1:0/1792198776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ce81034f0 msgr2=0x7f9ce819a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.263+0000 7f9ced5e6640 1 --2- 172.21.15.1:0/1792198776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ce81034f0 0x7f9ce819a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.263+0000 7f9ced5e6640 1 -- 172.21.15.1:0/1792198776 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ce81022b0 msgr2=0x7f9ce819a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.263+0000 7f9ced5e6640 1 --2- 172.21.15.1:0/1792198776 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ce81022b0 0x7f9ce819a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.263+0000 7f9ced5e6640 1 -- 172.21.15.1:0/1792198776 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce81a1e00 con 0x7f9ce81081f0 2024-05-04T03:19:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.263+0000 7f9ce7fff640 1 --2- 172.21.15.1:0/1792198776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ce81034f0 0x7f9ce819a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:19:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.264+0000 7f9ced5e6640 1 --2- 172.21.15.1:0/1792198776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ce81081f0 0x7f9ce81a1790 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000eaa0 tx=0x7f9ce000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.264+0000 7f9ce5ffb640 1 -- 172.21.15.1:0/1792198776 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce000cdb0 con 0x7f9ce81081f0 2024-05-04T03:19:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.264+0000 7f9ce5ffb640 1 -- 172.21.15.1:0/1792198776 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ce0005020 con 0x7f9ce81081f0 2024-05-04T03:19:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.264+0000 7f9cef049640 1 -- 172.21.15.1:0/1792198776 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce81a20f0 con 0x7f9ce81081f0 2024-05-04T03:19:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.264+0000 7f9ce5ffb640 1 -- 172.21.15.1:0/1792198776 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce00116d0 con 0x7f9ce81081f0 2024-05-04T03:19:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.264+0000 7f9cef049640 1 -- 172.21.15.1:0/1792198776 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce81a2640 con 0x7f9ce81081f0 2024-05-04T03:19:12.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.267+0000 7f9ce5ffb640 1 -- 172.21.15.1:0/1792198776 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7f9ce00051d0 con 0x7f9ce81081f0 2024-05-04T03:19:12.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.267+0000 7f9cf5fd4740 1 Processor -- start 2024-05-04T03:19:12.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.267+0000 7f9cf5fd4740 1 -- start start 2024-05-04T03:19:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.267+0000 7f9cf5fd4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c4730541b0 0x55c4733f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.267+0000 7f9cf5fd4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c47302b2f0 0x55c4733f5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.267+0000 7f9cf5fd4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4730334d0 0x55c4733fcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.267+0000 7f9cf5fd4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c47335a5d0 con 0x55c4730334d0 2024-05-04T03:19:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.267+0000 7f9cf5fd4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55c473313890 con 0x55c4730541b0 2024-05-04T03:19:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.267+0000 7f9cf5fd4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c473356590 con 0x55c47302b2f0 2024-05-04T03:19:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.268+0000 7f9ce7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4730334d0 0x55c4733fcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.268+0000 7f9ce7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4730334d0 0x55c4733fcee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44748/0 (socket says 172.21.15.1:44748) 2024-05-04T03:19:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.268+0000 7f9ce7fff640 1 -- 172.21.15.1:0/2105159401 learned_addr learned my addr 172.21.15.1:0/2105159401 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.268+0000 7f9ce7fff640 1 --2- 172.21.15.1:0/2105159401 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c4730541b0 0x55c4733f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.268+0000 7f9cecde5640 1 --2- 172.21.15.1:0/2105159401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c47302b2f0 0x55c4733f5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.268+0000 7f9ce5ffb640 1 --2- 172.21.15.1:0/1792198776 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9cd80b29a0 0x7f9cd80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.268+0000 7f9ce7fff640 1 -- 172.21.15.1:0/2105159401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c47302b2f0 msgr2=0x55c4733f5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.268+0000 7f9ce7fff640 1 --2- 172.21.15.1:0/2105159401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c47302b2f0 0x55c4733f5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.268+0000 7f9ce7fff640 1 -- 172.21.15.1:0/2105159401 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c4730541b0 msgr2=0x55c4733f5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.268+0000 7f9ce7fff640 1 --2- 172.21.15.1:0/2105159401 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c4730541b0 0x55c4733f5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.268+0000 7f9ce7fff640 1 -- 172.21.15.1:0/2105159401 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c47306c210 con 0x55c4730334d0 2024-05-04T03:19:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.268+0000 7f9ced5e6640 1 --2- 172.21.15.1:0/1792198776 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9cd80b29a0 0x7f9cd80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.268+0000 7f9ce5ffb640 1 -- 172.21.15.1:0/1792198776 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f9ce00e23e0 con 0x7f9ce81081f0 2024-05-04T03:19:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.268+0000 7f9ce7fff640 1 --2- 172.21.15.1:0/2105159401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4730334d0 0x55c4733fcee0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4005940 tx=0x7f9cd4005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a516b71e5afa55 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.269+0000 7f9cd2ffd640 1 -- 172.21.15.1:0/2105159401 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4036040 con 0x55c4730334d0 2024-05-04T03:19:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.269+0000 7f9cd2ffd640 1 -- 172.21.15.1:0/2105159401 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cd4005020 con 0x55c4730334d0 2024-05-04T03:19:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.269+0000 7f9cd2ffd640 1 -- 172.21.15.1:0/2105159401 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4004920 con 0x55c4730334d0 2024-05-04T03:19:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.270+0000 7f9cf5fd4740 1 -- 172.21.15.1:0/2105159401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4730334d0 msgr2=0x55c4733fcee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.270+0000 7f9ced5e6640 1 --2- 172.21.15.1:0/1792198776 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9cd80b29a0 0x7f9cd80b4eb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0009270 tx=0x7f9ce001ed30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.270+0000 7f9cf5fd4740 1 --2- 172.21.15.1:0/2105159401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4730334d0 0x55c4733fcee0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4005940 tx=0x7f9cd4005a40 comp rx=0 tx=0).stop 2024-05-04T03:19:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.270+0000 7f9cf5fd4740 1 -- 172.21.15.1:0/2105159401 shutdown_connections 2024-05-04T03:19:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.270+0000 7f9cf5fd4740 1 --2- 172.21.15.1:0/2105159401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4730334d0 0x55c4733fcee0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.270+0000 7f9cf5fd4740 1 --2- 172.21.15.1:0/2105159401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c47302b2f0 0x55c4733f5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.270+0000 7f9cf5fd4740 1 --2- 172.21.15.1:0/2105159401 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c4730541b0 0x55c4733f5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.270+0000 7f9cf5fd4740 1 -- 172.21.15.1:0/2105159401 >> 172.21.15.1:0/2105159401 conn(0x55c473048300 msgr2=0x55c4733f13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.270+0000 7f9cf5fd4740 1 -- 172.21.15.1:0/2105159401 shutdown_connections 2024-05-04T03:19:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.270+0000 7f9cf5fd4740 1 -- 172.21.15.1:0/2105159401 wait complete. 2024-05-04T03:19:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.271+0000 7f9cf5fd4740 1 -- start start 2024-05-04T03:19:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.271+0000 7f9cf5fd4740 1 Processor -- start 2024-05-04T03:19:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.272+0000 7f9cf5fd4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47302b2f0 0x55c4733ff610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.272+0000 7f9cf5fd4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4730334d0 0x55c4733ffb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.272+0000 7f9cf5fd4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c4730541b0 0x55c473406ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.272+0000 7f9cf5fd4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55c4733fd8d0 con 0x55c47302b2f0 2024-05-04T03:19:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.272+0000 7f9cf5fd4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55c473317370 con 0x55c4730541b0 2024-05-04T03:19:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.272+0000 7f9cf5fd4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c4733fda50 con 0x55c4730334d0 2024-05-04T03:19:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.272+0000 7f9ce7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47302b2f0 0x55c4733ff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.272+0000 7f9cf5fd4740 10 client.0 _ll_register_callbacks cb 0x55c47306ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.272+0000 7f9ce7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47302b2f0 0x55c4733ff610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44752/0 (socket says 172.21.15.1:44752) 2024-05-04T03:19:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.272+0000 7f9ce7fff640 1 -- 172.21.15.1:0/1999041464 learned_addr learned my addr 172.21.15.1:0/1999041464 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.272+0000 7f9cecde5640 1 --2- 172.21.15.1:0/1999041464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4730334d0 0x55c4733ffb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.272+0000 7f9ce7fff640 1 --2- 172.21.15.1:0/1999041464 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c4730541b0 0x55c473406ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.272+0000 7f9ce7fff640 1 -- 172.21.15.1:0/1999041464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4730334d0 msgr2=0x55c4733ffb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.272+0000 7f9ce7fff640 1 --2- 172.21.15.1:0/1999041464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4730334d0 0x55c4733ffb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.272+0000 7f9ce7fff640 1 -- 172.21.15.1:0/1999041464 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c4730541b0 msgr2=0x55c473406ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.272+0000 7f9ce7fff640 1 --2- 172.21.15.1:0/1999041464 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c4730541b0 0x55c473406ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.272+0000 7f9ce7fff640 1 -- 172.21.15.1:0/1999041464 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c473138620 con 0x55c47302b2f0 2024-05-04T03:19:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.272+0000 7f9ce7fff640 1 --2- 172.21.15.1:0/1999041464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47302b2f0 0x55c4733ff610 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9cd402b560 tx=0x7f9cd4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d4ae222c9814c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.272+0000 7f9cc7fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:19:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.272+0000 7f9cc7fff640 1 -- 172.21.15.1:0/1999041464 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4036040 con 0x55c47302b2f0 2024-05-04T03:19:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.272+0000 7f9cc7fff640 1 -- 172.21.15.1:0/1999041464 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f9cd4004ad0 con 0x55c47302b2f0 2024-05-04T03:19:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.272+0000 7f9cf5fd4740 1 -- 172.21.15.1:0/1999041464 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cd4005020 con 0x55c47302b2f0 2024-05-04T03:19:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.273+0000 7f9cc7fff640 1 -- 172.21.15.1:0/1999041464 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9cd4056030 con 0x55c47302b2f0 2024-05-04T03:19:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.273+0000 7f9cf5fd4740 10 client.15201 fetch_fsmap learned FSMap version 26 2024-05-04T03:19:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.273+0000 7f9cf5fd4740 1 -- 172.21.15.1:0/1999041464 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c4734099e0 con 0x55c47302b2f0 2024-05-04T03:19:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.273+0000 7f9cc7fff640 1 -- 172.21.15.1:0/1999041464 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cd4005020 con 0x55c47302b2f0 2024-05-04T03:19:12.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.273+0000 7f9cc7fff640 1 -- 172.21.15.1:0/1999041464 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9cd404e040 con 0x55c47302b2f0 2024-05-04T03:19:12.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.273+0000 7f9cf5fd4740 10 client.15201 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:19:12.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.273+0000 7f9cf5fd4740 10 client.15201 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:19:12.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.273+0000 7f9cf5fd4740 20 client.15201 populate_metadata read hostname 'smithi001' 2024-05-04T03:19:12.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.273+0000 7f9cf5fd4740 1 --2- 172.21.15.1:0/1999041464 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55c473012d80 0x55c47340cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.273+0000 7f9cf5fd4740 4 client.15201 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:19:12.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.273+0000 7f9cf5fd4740 1 -- 172.21.15.1:0/1999041464 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4733681e0 con 0x55c473012d80 2024-05-04T03:19:12.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.274+0000 7f9cecde5640 1 --2- 172.21.15.1:0/1999041464 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55c473012d80 0x55c47340cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.274+0000 7f9cecde5640 1 --2- 172.21.15.1:0/1999041464 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55c473012d80 0x55c47340cd40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adba07fa69de3d4d server_cookie=223310df0465a3d7 in_seq=0 out_seq=0 2024-05-04T03:19:12.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.274+0000 7f9cc7fff640 10 client.15201 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:19:12.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.275+0000 7f9cc7fff640 1 -- 172.21.15.1:0/1999041464 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c4733681e0 con 0x55c473012d80 2024-05-04T03:19:12.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.275+0000 7f9cc7fff640 10 client.15201 handle_command_reply: tid=0 2024-05-04T03:19:12.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.275+0000 7f9cf5fd4740 1 client.15201 shutdown 2024-05-04T03:19:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.275+0000 7f9cf5fd4740 1 -- 172.21.15.1:0/1999041464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47302b2f0 msgr2=0x55c4733ff610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.275+0000 7f9cf5fd4740 1 --2- 172.21.15.1:0/1999041464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47302b2f0 0x55c4733ff610 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9cd402b560 tx=0x7f9cd4001e70 comp rx=0 tx=0).stop 2024-05-04T03:19:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.275+0000 7f9cf5fd4740 1 -- 172.21.15.1:0/1999041464 shutdown_connections 2024-05-04T03:19:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.275+0000 7f9cf5fd4740 1 --2- 172.21.15.1:0/1999041464 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55c473012d80 0x55c47340cd40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.275+0000 7f9cf5fd4740 1 --2- 172.21.15.1:0/1999041464 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c4730541b0 0x55c473406ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.275+0000 7f9cf5fd4740 1 --2- 172.21.15.1:0/1999041464 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4730334d0 0x55c4733ffb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.275+0000 7f9cc7fff640 0 client.15201 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:19:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.275+0000 7f9cf5fd4740 1 --2- 172.21.15.1:0/1999041464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47302b2f0 0x55c4733ff610 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.275+0000 7f9cf5fd4740 1 -- 172.21.15.1:0/1999041464 >> 172.21.15.1:0/1999041464 conn(0x55c473048300 msgr2=0x55c4733f4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.275+0000 7f9cf5fd4740 1 -- 172.21.15.1:0/1999041464 shutdown_connections 2024-05-04T03:19:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.275+0000 7f9cf5fd4740 1 -- 172.21.15.1:0/1999041464 wait complete. 2024-05-04T03:19:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.275+0000 7f9cf5fd4740 20 client.15201 trim_cache size 0 max 16384 2024-05-04T03:19:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.291+0000 7f9cf5fd4740 1 Processor -- start 2024-05-04T03:19:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.291+0000 7f9cf5fd4740 1 -- start start 2024-05-04T03:19:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.291+0000 7f9cf5fd4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c47302b2f0 0x55c4733fa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.291+0000 7f9cf5fd4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4730334d0 0x55c473406a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.291+0000 7f9cf5fd4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c4730541b0 0x55c4733fe500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.291+0000 7f9cf5fd4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce81ad9f0 con 0x55c4730334d0 2024-05-04T03:19:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.291+0000 7f9cf5fd4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce81ab2c0 con 0x55c4730541b0 2024-05-04T03:19:12.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.291+0000 7f9cf5fd4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce81a9130 con 0x55c47302b2f0 2024-05-04T03:19:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.291+0000 7f9cecde5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4730334d0 0x55c473406a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.291+0000 7f9cecde5640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4730334d0 0x55c473406a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44754/0 (socket says 172.21.15.1:44754) 2024-05-04T03:19:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.291+0000 7f9cecde5640 1 -- 172.21.15.1:0/472736205 learned_addr learned my addr 172.21.15.1:0/472736205 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.292+0000 7f9ce7fff640 1 --2- 172.21.15.1:0/472736205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c47302b2f0 0x55c4733fa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.292+0000 7f9ce7fff640 1 --2- 172.21.15.1:0/472736205 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c4730541b0 0x55c4733fe500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.292+0000 7f9cecde5640 1 -- 172.21.15.1:0/472736205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c47302b2f0 msgr2=0x55c4733fa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.292+0000 7f9cecde5640 1 --2- 172.21.15.1:0/472736205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c47302b2f0 0x55c4733fa130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.292+0000 7f9cecde5640 1 -- 172.21.15.1:0/472736205 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c4730541b0 msgr2=0x55c4733fe500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.292+0000 7f9cecde5640 1 --2- 172.21.15.1:0/472736205 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c4730541b0 0x55c4733fe500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.292+0000 7f9cecde5640 1 -- 172.21.15.1:0/472736205 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4733f0980 con 0x55c4730334d0 2024-05-04T03:19:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.292+0000 7f9ce7fff640 1 --2- 172.21.15.1:0/472736205 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c4730541b0 0x55c4733fe500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:19:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.292+0000 7f9cecde5640 1 --2- 172.21.15.1:0/472736205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4730334d0 0x55c473406a40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8102eb0 tx=0x7f9cdc00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d025f736187d7d9b server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:12.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.292+0000 7f9cd27fc640 1 -- 172.21.15.1:0/472736205 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc005220 con 0x55c4730334d0 2024-05-04T03:19:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.292+0000 7f9cd27fc640 1 -- 172.21.15.1:0/472736205 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cdc024030 con 0x55c4730334d0 2024-05-04T03:19:12.296 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.292+0000 7f9cd27fc640 1 -- 172.21.15.1:0/472736205 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc020520 con 0x55c4730334d0 2024-05-04T03:19:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.293+0000 7f9cf5fd4740 1 -- 172.21.15.1:0/472736205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4730334d0 msgr2=0x55c473406a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.293+0000 7f9cf5fd4740 1 --2- 172.21.15.1:0/472736205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4730334d0 0x55c473406a40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8102eb0 tx=0x7f9cdc00b390 comp rx=0 tx=0).stop 2024-05-04T03:19:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.293+0000 7f9cf5fd4740 1 -- 172.21.15.1:0/472736205 shutdown_connections 2024-05-04T03:19:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.293+0000 7f9cf5fd4740 1 --2- 172.21.15.1:0/472736205 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c4730541b0 0x55c4733fe500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.299 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.293+0000 7f9cf5fd4740 1 --2- 172.21.15.1:0/472736205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c4730334d0 0x55c473406a40 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.293+0000 7f9cf5fd4740 1 --2- 172.21.15.1:0/472736205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c47302b2f0 0x55c4733fa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.293+0000 7f9cf5fd4740 1 -- 172.21.15.1:0/472736205 >> 172.21.15.1:0/472736205 conn(0x55c473048300 msgr2=0x55c4733fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.293+0000 7f9cf5fd4740 1 -- 172.21.15.1:0/472736205 shutdown_connections 2024-05-04T03:19:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.293+0000 7f9cf5fd4740 1 -- 172.21.15.1:0/472736205 wait complete. 2024-05-04T03:19:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.293+0000 7f9cf5fd4740 1 -- start start 2024-05-04T03:19:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.294+0000 7f9cf5fd4740 1 Processor -- start 2024-05-04T03:19:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.294+0000 7f9cf5fd4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47302b2f0 0x55c473404350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.294+0000 7f9cf5fd4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4730334d0 0x55c473404890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.294+0000 7f9cf5fd4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c4730541b0 0x55c4734057c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.294+0000 7f9cf5fd4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce81a9780 con 0x55c47302b2f0 2024-05-04T03:19:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.294+0000 7f9cf5fd4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce81aabf0 con 0x55c4730541b0 2024-05-04T03:19:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.294+0000 7f9cf5fd4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce81a9e50 con 0x55c4730334d0 2024-05-04T03:19:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.294+0000 7f9cf5fd4740 10 client.0 _ll_register_callbacks cb 0x7f9cd4032d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:12.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.294+0000 7f9ce7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47302b2f0 0x55c473404350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.294+0000 7f9ce7fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47302b2f0 0x55c473404350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44764/0 (socket says 172.21.15.1:44764) 2024-05-04T03:19:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.294+0000 7f9ce7fff640 1 -- 172.21.15.1:0/1582001729 learned_addr learned my addr 172.21.15.1:0/1582001729 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.294+0000 7f9cecde5640 1 --2- 172.21.15.1:0/1582001729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4730334d0 0x55c473404890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.294+0000 7f9ce7fff640 1 --2- 172.21.15.1:0/1582001729 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c4730541b0 0x55c4734057c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.294+0000 7f9ce7fff640 1 -- 172.21.15.1:0/1582001729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4730334d0 msgr2=0x55c473404890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.294+0000 7f9ce7fff640 1 --2- 172.21.15.1:0/1582001729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4730334d0 0x55c473404890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.294+0000 7f9ce7fff640 1 -- 172.21.15.1:0/1582001729 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c4730541b0 msgr2=0x55c4734057c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.294+0000 7f9ce7fff640 1 --2- 172.21.15.1:0/1582001729 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c4730541b0 0x55c4734057c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.294+0000 7f9ce7fff640 1 -- 172.21.15.1:0/1582001729 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c473405d80 con 0x55c47302b2f0 2024-05-04T03:19:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.295+0000 7f9ce7fff640 1 --2- 172.21.15.1:0/1582001729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47302b2f0 0x55c473404350 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40096c0 tx=0x7f9cd4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9782e3ed467ad21c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.295+0000 7f9cc7fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:19:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.295+0000 7f9cc7fff640 1 -- 172.21.15.1:0/1582001729 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4043040 con 0x55c47302b2f0 2024-05-04T03:19:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.295+0000 7f9cc7fff640 1 -- 172.21.15.1:0/1582001729 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f9cd405b740 con 0x55c47302b2f0 2024-05-04T03:19:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.295+0000 7f9cf5fd4740 1 -- 172.21.15.1:0/1582001729 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cdc024030 con 0x55c47302b2f0 2024-05-04T03:19:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.295+0000 7f9cc7fff640 1 -- 172.21.15.1:0/1582001729 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9cd4036040 con 0x55c47302b2f0 2024-05-04T03:19:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.295+0000 7f9cf5fd4740 10 client.15207 fetch_fsmap learned FSMap version 26 2024-05-04T03:19:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.295+0000 7f9cf5fd4740 1 -- 172.21.15.1:0/1582001729 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c473474f20 con 0x55c47302b2f0 2024-05-04T03:19:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.295+0000 7f9cc7fff640 1 -- 172.21.15.1:0/1582001729 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cdc024030 con 0x55c47302b2f0 2024-05-04T03:19:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.295+0000 7f9cc7fff640 1 -- 172.21.15.1:0/1582001729 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9cd4071070 con 0x55c47302b2f0 2024-05-04T03:19:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.295+0000 7f9cf5fd4740 10 client.15207 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:19:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.295+0000 7f9cf5fd4740 10 client.15207 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:19:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.295+0000 7f9cf5fd4740 20 client.15207 populate_metadata read hostname 'smithi001' 2024-05-04T03:19:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.295+0000 7f9cf5fd4740 1 --2- 172.21.15.1:0/1582001729 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55c472fe81c0 0x55c473477850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.295+0000 7f9cf5fd4740 4 client.15207 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:19:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.295+0000 7f9cf5fd4740 1 -- 172.21.15.1:0/1582001729 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c473362050 con 0x55c472fe81c0 2024-05-04T03:19:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.296+0000 7f9cecde5640 1 --2- 172.21.15.1:0/1582001729 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55c472fe81c0 0x55c473477850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.302 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.296+0000 7f9cc7fff640 10 client.15207 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:19:12.303 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.296+0000 7f9cecde5640 1 --2- 172.21.15.1:0/1582001729 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55c472fe81c0 0x55c473477850 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f26d7b9e19eeb5f server_cookie=b6972db3a418d8de in_seq=0 out_seq=0 2024-05-04T03:19:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.305+0000 7f9cc7fff640 1 -- 172.21.15.1:0/1582001729 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c473362050 con 0x55c472fe81c0 2024-05-04T03:19:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.305+0000 7f9cc7fff640 10 client.15207 handle_command_reply: tid=0 2024-05-04T03:19:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.305+0000 7f9cf5fd4740 1 client.15207 shutdown 2024-05-04T03:19:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.305+0000 7f9cf5fd4740 1 -- 172.21.15.1:0/1582001729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47302b2f0 msgr2=0x55c473404350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.305+0000 7f9cf5fd4740 1 --2- 172.21.15.1:0/1582001729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47302b2f0 0x55c473404350 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40096c0 tx=0x7f9cd4001e70 comp rx=0 tx=0).stop 2024-05-04T03:19:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.305+0000 7f9cf5fd4740 1 -- 172.21.15.1:0/1582001729 shutdown_connections 2024-05-04T03:19:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.305+0000 7f9cf5fd4740 1 --2- 172.21.15.1:0/1582001729 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55c472fe81c0 0x55c473477850 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.305+0000 7f9cf5fd4740 1 --2- 172.21.15.1:0/1582001729 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c4730541b0 0x55c4734057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.305+0000 7f9cc7fff640 0 client.15207 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:19:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.305+0000 7f9cf5fd4740 1 --2- 172.21.15.1:0/1582001729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c4730334d0 0x55c473404890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.305+0000 7f9cf5fd4740 1 --2- 172.21.15.1:0/1582001729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47302b2f0 0x55c473404350 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.305+0000 7f9cf5fd4740 1 -- 172.21.15.1:0/1582001729 >> 172.21.15.1:0/1582001729 conn(0x55c473048300 msgr2=0x55c4733f41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.305+0000 7f9cf5fd4740 1 -- 172.21.15.1:0/1582001729 shutdown_connections 2024-05-04T03:19:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.305+0000 7f9cf5fd4740 1 -- 172.21.15.1:0/1582001729 wait complete. 2024-05-04T03:19:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.305+0000 7f9cf5fd4740 20 client.15207 trim_cache size 0 max 16384 2024-05-04T03:19:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:19:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:19:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:19:12.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:19:12.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.306+0000 7f9cef049640 1 -- 172.21.15.1:0/1792198776 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9cd80b29a0 msgr2=0x7f9cd80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:12.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.306+0000 7f9cef049640 1 --2- 172.21.15.1:0/1792198776 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9cd80b29a0 0x7f9cd80b4eb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0009270 tx=0x7f9ce001ed30 comp rx=0 tx=0).stop 2024-05-04T03:19:12.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.306+0000 7f9cef049640 1 -- 172.21.15.1:0/1792198776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ce81081f0 msgr2=0x7f9ce81a1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:12.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.306+0000 7f9cef049640 1 --2- 172.21.15.1:0/1792198776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ce81081f0 0x7f9ce81a1790 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000eaa0 tx=0x7f9ce000ef60 comp rx=0 tx=0).stop 2024-05-04T03:19:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.306+0000 7f9cef049640 1 -- 172.21.15.1:0/1792198776 shutdown_connections 2024-05-04T03:19:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.306+0000 7f9cef049640 1 --2- 172.21.15.1:0/1792198776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ce81081f0 0x7f9ce81a1790 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.306+0000 7f9cef049640 1 --2- 172.21.15.1:0/1792198776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ce81034f0 0x7f9ce819a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.306+0000 7f9cef049640 1 --2- 172.21.15.1:0/1792198776 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9cd80b29a0 0x7f9cd80b4eb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.306+0000 7f9cef049640 1 --2- 172.21.15.1:0/1792198776 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ce81022b0 0x7f9ce819a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.306+0000 7f9cef049640 1 -- 172.21.15.1:0/1792198776 >> 172.21.15.1:0/1792198776 conn(0x7f9ce80fd480 msgr2=0x7f9ce81061e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.306+0000 7f9cef049640 1 -- 172.21.15.1:0/1792198776 shutdown_connections 2024-05-04T03:19:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:12.306+0000 7f9cef049640 1 -- 172.21.15.1:0/1792198776 wait complete. 2024-05-04T03:19:12.722 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:12 smithi001 ceph-mon[29731]: pgmap v392: 97 pgs: 97 active+clean; 665 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-05-04T03:19:12.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.744+0000 7fabc2d33700 1 -- 172.21.15.1:0/2856368560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabbc108740 msgr2=0x7fabbc1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:12.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.744+0000 7fabc2d33700 1 --2- 172.21.15.1:0/2856368560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabbc108740 0x7fabbc1100a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fabb400b0d0 tx=0x7fabb400b3e0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.744+0000 7fabc2d33700 1 -- 172.21.15.1:0/2856368560 shutdown_connections 2024-05-04T03:19:12.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.744+0000 7fabc2d33700 1 --2- 172.21.15.1:0/2856368560 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabbc1034c0 0x7fabbc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.744+0000 7fabc2d33700 1 --2- 172.21.15.1:0/2856368560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabbc108740 0x7fabbc1100a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.744+0000 7fabc2d33700 1 --2- 172.21.15.1:0/2856368560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabbc107df0 0x7fabbc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.744+0000 7fabc2d33700 1 -- 172.21.15.1:0/2856368560 >> 172.21.15.1:0/2856368560 conn(0x7fabbc0fedf0 msgr2=0x7fabbc101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:12.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.744+0000 7fabc2d33700 1 -- 172.21.15.1:0/2856368560 shutdown_connections 2024-05-04T03:19:12.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.744+0000 7fabc2d33700 1 -- 172.21.15.1:0/2856368560 wait complete. 2024-05-04T03:19:12.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.745+0000 7fabc2d33700 1 Processor -- start 2024-05-04T03:19:12.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.745+0000 7fabc2d33700 1 -- start start 2024-05-04T03:19:12.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.745+0000 7fabc2d33700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabbc1034c0 0x7fabbc19bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.745+0000 7fabc2d33700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabbc107df0 0x7fabbc19c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.745+0000 7fabc2d33700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabbc108740 0x7fabbc1a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.745+0000 7fabc2d33700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc1133c0 con 0x7fabbc107df0 2024-05-04T03:19:12.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.745+0000 7fabc2d33700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc113240 con 0x7fabbc108740 2024-05-04T03:19:12.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.745+0000 7fabc2d33700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc113540 con 0x7fabbc1034c0 2024-05-04T03:19:12.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.745+0000 7fabbbfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabbc107df0 0x7fabbc19c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.745+0000 7fabbbfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabbc107df0 0x7fabbc19c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44770/0 (socket says 172.21.15.1:44770) 2024-05-04T03:19:12.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.745+0000 7fabbbfff700 1 -- 172.21.15.1:0/3334894284 learned_addr learned my addr 172.21.15.1:0/3334894284 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:12.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.745+0000 7fabc0acf700 1 --2- 172.21.15.1:0/3334894284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabbc1034c0 0x7fabbc19bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.745+0000 7fabc12d0700 1 --2- 172.21.15.1:0/3334894284 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabbc108740 0x7fabbc1a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.745+0000 7fabbbfff700 1 -- 172.21.15.1:0/3334894284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabbc1034c0 msgr2=0x7fabbc19bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:12.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.745+0000 7fabbbfff700 1 --2- 172.21.15.1:0/3334894284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabbc1034c0 0x7fabbc19bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.745+0000 7fabbbfff700 1 -- 172.21.15.1:0/3334894284 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabbc108740 msgr2=0x7fabbc1a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:12.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.745+0000 7fabbbfff700 1 --2- 172.21.15.1:0/3334894284 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabbc108740 0x7fabbc1a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.745+0000 7fabbbfff700 1 -- 172.21.15.1:0/3334894284 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabbc19ca30 con 0x7fabbc107df0 2024-05-04T03:19:12.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.746+0000 7fabbbfff700 1 --2- 172.21.15.1:0/3334894284 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabbc107df0 0x7fabbc19c180 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7faba800d980 tx=0x7faba800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:12.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.746+0000 7fabb9ffb700 1 -- 172.21.15.1:0/3334894284 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8017070 con 0x7fabbc107df0 2024-05-04T03:19:12.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.746+0000 7fabb9ffb700 1 -- 172.21.15.1:0/3334894284 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba800fd40 con 0x7fabbc107df0 2024-05-04T03:19:12.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.746+0000 7fabb9ffb700 1 -- 172.21.15.1:0/3334894284 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba800e3b0 con 0x7fabbc107df0 2024-05-04T03:19:12.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.746+0000 7fabc2d33700 1 -- 172.21.15.1:0/3334894284 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabbc1a7320 con 0x7fabbc107df0 2024-05-04T03:19:12.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.746+0000 7fabc2d33700 1 -- 172.21.15.1:0/3334894284 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabbc1a7770 con 0x7fabbc107df0 2024-05-04T03:19:12.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.749+0000 7fabb9ffb700 1 -- 172.21.15.1:0/3334894284 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7faba8010610 con 0x7fabbc107df0 2024-05-04T03:19:12.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.749+0000 7fabb9ffb700 1 --2- 172.21.15.1:0/3334894284 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fabac0b2910 0x7fabac0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.749+0000 7fabc0acf700 1 --2- 172.21.15.1:0/3334894284 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fabac0b2910 0x7fabac0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.749+0000 7fabb9ffb700 1 -- 172.21.15.1:0/3334894284 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7faba80e1c80 con 0x7fabbc107df0 2024-05-04T03:19:12.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.750+0000 7fabd1747b80 1 Processor -- start 2024-05-04T03:19:12.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.750+0000 7fabc0acf700 1 --2- 172.21.15.1:0/3334894284 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fabac0b2910 0x7fabac0b4e20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fabb0005950 tx=0x7fabb000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:12.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.750+0000 7fabd1747b80 1 -- start start 2024-05-04T03:19:12.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.750+0000 7fabd1747b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55886faa9870 0x55886faa9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.750+0000 7fabd1747b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55886faaa1c0 0x55886faae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.750+0000 7fabd1747b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55886faae600 0x55886fab6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.750+0000 7fabd1747b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55886f9c78f0 con 0x55886faaa1c0 2024-05-04T03:19:12.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.750+0000 7fabd1747b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55886f92cd50 con 0x55886faa9870 2024-05-04T03:19:12.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.750+0000 7fabd1747b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55886f9c38b0 con 0x55886faae600 2024-05-04T03:19:12.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.750+0000 7fabbbfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55886faaa1c0 0x55886faae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.751+0000 7fabbbfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55886faaa1c0 0x55886faae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44772/0 (socket says 172.21.15.1:44772) 2024-05-04T03:19:12.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.751+0000 7fabbbfff700 1 -- 172.21.15.1:0/2236674294 learned_addr learned my addr 172.21.15.1:0/2236674294 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:12.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.751+0000 7fabc0acf700 1 --2- 172.21.15.1:0/2236674294 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55886faa9870 0x55886faa9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.751+0000 7fabc12d0700 1 --2- 172.21.15.1:0/2236674294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55886faae600 0x55886fab6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.751+0000 7fabbbfff700 1 -- 172.21.15.1:0/2236674294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55886faae600 msgr2=0x55886fab6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:12.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.751+0000 7fabbbfff700 1 --2- 172.21.15.1:0/2236674294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55886faae600 0x55886fab6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.751+0000 7fabbbfff700 1 -- 172.21.15.1:0/2236674294 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55886faa9870 msgr2=0x55886faa9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:12.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.751+0000 7fabbbfff700 1 --2- 172.21.15.1:0/2236674294 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55886faa9870 0x55886faa9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.751+0000 7fabbbfff700 1 -- 172.21.15.1:0/2236674294 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55886f914880 con 0x55886faaa1c0 2024-05-04T03:19:12.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.751+0000 7fabc0acf700 1 --2- 172.21.15.1:0/2236674294 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55886faa9870 0x55886faa9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:19:12.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.751+0000 7fabbbfff700 1 --2- 172.21.15.1:0/2236674294 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55886faaa1c0 0x55886faae000 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7faba80a91f0 tx=0x7faba80a8aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46e027b7315b70f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:12.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.751+0000 7faba6ffd700 1 -- 172.21.15.1:0/2236674294 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8020040 con 0x55886faaa1c0 2024-05-04T03:19:12.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.751+0000 7faba6ffd700 1 -- 172.21.15.1:0/2236674294 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba80a9260 con 0x55886faaa1c0 2024-05-04T03:19:12.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.751+0000 7faba6ffd700 1 -- 172.21.15.1:0/2236674294 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba80a84c0 con 0x55886faaa1c0 2024-05-04T03:19:12.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.752+0000 7fabd1747b80 1 -- 172.21.15.1:0/2236674294 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55886faaa1c0 msgr2=0x55886faae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:12.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.752+0000 7fabd1747b80 1 --2- 172.21.15.1:0/2236674294 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55886faaa1c0 0x55886faae000 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7faba80a91f0 tx=0x7faba80a8aa0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.752+0000 7fabd1747b80 1 -- 172.21.15.1:0/2236674294 shutdown_connections 2024-05-04T03:19:12.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.752+0000 7fabd1747b80 1 --2- 172.21.15.1:0/2236674294 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55886faa9870 0x55886faa9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.752+0000 7fabd1747b80 1 --2- 172.21.15.1:0/2236674294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55886faae600 0x55886fab6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.752+0000 7fabd1747b80 1 --2- 172.21.15.1:0/2236674294 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55886faaa1c0 0x55886faae000 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.752+0000 7fabd1747b80 1 -- 172.21.15.1:0/2236674294 >> 172.21.15.1:0/2236674294 conn(0x55886f927df0 msgr2=0x55886faa0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:12.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.752+0000 7fabd1747b80 1 -- 172.21.15.1:0/2236674294 shutdown_connections 2024-05-04T03:19:12.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.753+0000 7fabd1747b80 1 -- 172.21.15.1:0/2236674294 wait complete. 2024-05-04T03:19:12.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.753+0000 7fabd1747b80 1 -- start start 2024-05-04T03:19:12.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.753+0000 7fabd1747b80 1 Processor -- start 2024-05-04T03:19:12.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.754+0000 7fabd1747b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55886faa9870 0x55886fab5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.754+0000 7fabd1747b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55886faae600 0x55886fab33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.754+0000 7fabd1747b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55886fab38f0 0x55886fab3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.754+0000 7fabd1747b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55886fab67f0 con 0x55886faae600 2024-05-04T03:19:12.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.754+0000 7fabd1747b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55886f980e80 con 0x55886faa9870 2024-05-04T03:19:12.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.754+0000 7fabd1747b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55886fab6970 con 0x55886fab38f0 2024-05-04T03:19:12.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.754+0000 7fabd1747b80 10 client.0 _ll_register_callbacks cb 0x55886f95f900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:12.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.754+0000 7fabbbfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55886faae600 0x55886fab33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.754+0000 7fabbbfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55886faae600 0x55886fab33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44774/0 (socket says 172.21.15.1:44774) 2024-05-04T03:19:12.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.754+0000 7fabbbfff700 1 -- 172.21.15.1:0/15387274 learned_addr learned my addr 172.21.15.1:0/15387274 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:12.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.754+0000 7fabc0acf700 1 --2- 172.21.15.1:0/15387274 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55886faa9870 0x55886fab5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.754+0000 7fabbbfff700 1 -- 172.21.15.1:0/15387274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55886fab38f0 msgr2=0x55886fab3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:12.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.754+0000 7fabbbfff700 1 --2- 172.21.15.1:0/15387274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55886fab38f0 0x55886fab3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.754+0000 7fabbbfff700 1 -- 172.21.15.1:0/15387274 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55886faa9870 msgr2=0x55886fab5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:12.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.754+0000 7fabbbfff700 1 --2- 172.21.15.1:0/15387274 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55886faa9870 0x55886fab5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.754+0000 7fabbbfff700 1 -- 172.21.15.1:0/15387274 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55886f97e710 con 0x55886faae600 2024-05-04T03:19:12.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.754+0000 7fabbbfff700 1 --2- 172.21.15.1:0/15387274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55886faae600 0x55886fab33b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7faba80df7d0 tx=0x7faba80a5230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ef645adf906fb server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:12.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.754+0000 7fab9bfff700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:19:12.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.754+0000 7fab9bfff700 1 -- 172.21.15.1:0/15387274 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba806d060 con 0x55886faae600 2024-05-04T03:19:12.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.754+0000 7fab9bfff700 1 -- 172.21.15.1:0/15387274 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7faba80a5850 con 0x55886faae600 2024-05-04T03:19:12.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.754+0000 7fabd1747b80 1 -- 172.21.15.1:0/15387274 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faba80a9260 con 0x55886faae600 2024-05-04T03:19:12.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.755+0000 7fab9bfff700 1 -- 172.21.15.1:0/15387274 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faba80c2030 con 0x55886faae600 2024-05-04T03:19:12.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.755+0000 7fabd1747b80 10 client.15219 fetch_fsmap learned FSMap version 26 2024-05-04T03:19:12.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.755+0000 7fabd1747b80 1 -- 172.21.15.1:0/15387274 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55886fac1490 con 0x55886faae600 2024-05-04T03:19:12.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.755+0000 7fab9bfff700 1 -- 172.21.15.1:0/15387274 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba80a9260 con 0x55886faae600 2024-05-04T03:19:12.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.755+0000 7fab9bfff700 1 -- 172.21.15.1:0/15387274 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7faba80a4bd0 con 0x55886faae600 2024-05-04T03:19:12.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.755+0000 7fabd1747b80 10 client.15219 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:19:12.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.755+0000 7fabd1747b80 10 client.15219 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-04T03:19:12.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.755+0000 7fabd1747b80 20 client.15219 populate_metadata read hostname 'smithi001' 2024-05-04T03:19:12.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.755+0000 7fabd1747b80 1 --2- 172.21.15.1:0/15387274 >> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] conn(0x55886fac35a0 0x55886fac4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.755+0000 7fabd1747b80 4 client.15219 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:19:12.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.755+0000 7fabd1747b80 1 -- 172.21.15.1:0/15387274 --> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55886f9d52d0 con 0x55886fac35a0 2024-05-04T03:19:12.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.756+0000 7fabc0acf700 1 --2- 172.21.15.1:0/15387274 >> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] conn(0x55886fac35a0 0x55886fac4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.756+0000 7fab9bfff700 10 client.15219 ms_handle_connect on v2:172.21.15.77:6838/922723189 2024-05-04T03:19:12.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.756+0000 7fabc0acf700 1 --2- 172.21.15.1:0/15387274 >> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] conn(0x55886fac35a0 0x55886fac4a10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f148e1108a4049e7 server_cookie=9924c68b963d2f74 in_seq=0 out_seq=0 2024-05-04T03:19:12.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.757+0000 7fab9bfff700 1 -- 172.21.15.1:0/15387274 <== mds.1 v2:172.21.15.77:6838/922723189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55886f9d52d0 con 0x55886fac35a0 2024-05-04T03:19:12.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.757+0000 7fab9bfff700 10 client.15219 handle_command_reply: tid=0 2024-05-04T03:19:12.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.757+0000 7fabd1747b80 1 client.15219 shutdown 2024-05-04T03:19:12.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.757+0000 7fabd1747b80 1 -- 172.21.15.1:0/15387274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55886faae600 msgr2=0x55886fab33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:12.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.757+0000 7fabd1747b80 1 --2- 172.21.15.1:0/15387274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55886faae600 0x55886fab33b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7faba80df7d0 tx=0x7faba80a5230 comp rx=0 tx=0).stop 2024-05-04T03:19:12.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.757+0000 7fabd1747b80 1 -- 172.21.15.1:0/15387274 shutdown_connections 2024-05-04T03:19:12.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.757+0000 7fabd1747b80 1 --2- 172.21.15.1:0/15387274 >> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] conn(0x55886fac35a0 0x55886fac4a10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.757+0000 7fabd1747b80 1 --2- 172.21.15.1:0/15387274 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55886faa9870 0x55886fab5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.757+0000 7fabd1747b80 1 --2- 172.21.15.1:0/15387274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55886fab38f0 0x55886fab3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.757+0000 7fabd1747b80 1 --2- 172.21.15.1:0/15387274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55886faae600 0x55886fab33b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.757+0000 7fabd1747b80 1 -- 172.21.15.1:0/15387274 >> 172.21.15.1:0/15387274 conn(0x55886f927df0 msgr2=0x55886faa9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:12.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.757+0000 7fab9bfff700 0 client.15219 ms_handle_reset on v2:172.21.15.77:6838/922723189 2024-05-04T03:19:12.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.757+0000 7fabd1747b80 1 -- 172.21.15.1:0/15387274 shutdown_connections 2024-05-04T03:19:12.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.757+0000 7fabd1747b80 1 -- 172.21.15.1:0/15387274 wait complete. 2024-05-04T03:19:12.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.757+0000 7fabd1747b80 20 client.15219 trim_cache size 0 max 16384 2024-05-04T03:19:12.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.778+0000 7fabd1747b80 1 Processor -- start 2024-05-04T03:19:12.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.778+0000 7fabd1747b80 1 -- start start 2024-05-04T03:19:12.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.779+0000 7fabd1747b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55886faa9870 0x55886fab0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.779+0000 7fabd1747b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55886faae600 0x55886fabfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.779+0000 7fabd1747b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55886fab38f0 0x55886fa9a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.779+0000 7fabd1747b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55886fac2d50 con 0x55886faa9870 2024-05-04T03:19:12.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.779+0000 7fabd1747b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55886fac3420 con 0x55886fab38f0 2024-05-04T03:19:12.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.779+0000 7fabd1747b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55886fac3af0 con 0x55886faae600 2024-05-04T03:19:12.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.779+0000 7fabc0acf700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55886faa9870 0x55886fab0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.779+0000 7fabc0acf700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55886faa9870 0x55886fab0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44790/0 (socket says 172.21.15.1:44790) 2024-05-04T03:19:12.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.779+0000 7fabc0acf700 1 -- 172.21.15.1:0/656279748 learned_addr learned my addr 172.21.15.1:0/656279748 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:12.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.779+0000 7fabc12d0700 1 --2- 172.21.15.1:0/656279748 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55886fab38f0 0x55886fa9a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.779+0000 7fabc0acf700 1 -- 172.21.15.1:0/656279748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55886faae600 msgr2=0x55886fabfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:12.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.779+0000 7fabbbfff700 1 --2- 172.21.15.1:0/656279748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55886faae600 0x55886fabfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.779+0000 7fabc0acf700 1 --2- 172.21.15.1:0/656279748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55886faae600 0x55886fabfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.779+0000 7fabc0acf700 1 -- 172.21.15.1:0/656279748 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55886fab38f0 msgr2=0x55886fa9a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:12.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.779+0000 7fabc0acf700 1 --2- 172.21.15.1:0/656279748 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55886fab38f0 0x55886fa9a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.779+0000 7fabc0acf700 1 -- 172.21.15.1:0/656279748 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55886faa1ca0 con 0x55886faa9870 2024-05-04T03:19:12.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.779+0000 7fabc12d0700 1 --2- 172.21.15.1:0/656279748 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55886fab38f0 0x55886fa9a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:19:12.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.779+0000 7fabbbfff700 1 --2- 172.21.15.1:0/656279748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55886faae600 0x55886fabfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:19:12.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.779+0000 7fabc0acf700 1 --2- 172.21.15.1:0/656279748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55886faa9870 0x55886fab0f20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fabb000bfd0 tx=0x7fabb0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f5723d9e6a3d77 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:12.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.780+0000 7faba67fc700 1 -- 172.21.15.1:0/656279748 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb0011e20 con 0x55886faa9870 2024-05-04T03:19:12.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.780+0000 7faba67fc700 1 -- 172.21.15.1:0/656279748 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabb0004ce0 con 0x55886faa9870 2024-05-04T03:19:12.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.780+0000 7faba67fc700 1 -- 172.21.15.1:0/656279748 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb00274c0 con 0x55886faa9870 2024-05-04T03:19:12.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.780+0000 7fabd1747b80 1 -- 172.21.15.1:0/656279748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55886faa9870 msgr2=0x55886fab0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:12.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.780+0000 7fabd1747b80 1 --2- 172.21.15.1:0/656279748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55886faa9870 0x55886fab0f20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fabb000bfd0 tx=0x7fabb0013000 comp rx=0 tx=0).stop 2024-05-04T03:19:12.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.780+0000 7fabd1747b80 1 -- 172.21.15.1:0/656279748 shutdown_connections 2024-05-04T03:19:12.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.780+0000 7fabd1747b80 1 --2- 172.21.15.1:0/656279748 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55886fab38f0 0x55886fa9a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.780+0000 7fabd1747b80 1 --2- 172.21.15.1:0/656279748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55886faa9870 0x55886fab0f20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.780+0000 7fabd1747b80 1 --2- 172.21.15.1:0/656279748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55886faae600 0x55886fabfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.780+0000 7fabd1747b80 1 -- 172.21.15.1:0/656279748 >> 172.21.15.1:0/656279748 conn(0x55886f927df0 msgr2=0x55886fab1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:12.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.780+0000 7fabd1747b80 1 -- 172.21.15.1:0/656279748 shutdown_connections 2024-05-04T03:19:12.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.780+0000 7fabd1747b80 1 -- 172.21.15.1:0/656279748 wait complete. 2024-05-04T03:19:12.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.781+0000 7fabd1747b80 1 -- start start 2024-05-04T03:19:12.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.781+0000 7fabd1747b80 1 Processor -- start 2024-05-04T03:19:12.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.781+0000 7fabd1747b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55886faa9870 0x55886fac9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.781+0000 7fabd1747b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55886faae600 0x55886fac9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.781+0000 7fabd1747b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55886fab38f0 0x55886faca530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.781+0000 7fabd1747b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55886fac4890 con 0x55886faae600 2024-05-04T03:19:12.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.781+0000 7fabd1747b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55886fac41c0 con 0x55886fab38f0 2024-05-04T03:19:12.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.781+0000 7fabd1747b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55886fac4f60 con 0x55886faa9870 2024-05-04T03:19:12.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.781+0000 7fabd1747b80 10 client.0 _ll_register_callbacks cb 0x55886f95f900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:12.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.781+0000 7fabbbfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55886faae600 0x55886fac9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.782+0000 7fabbbfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55886faae600 0x55886fac9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44798/0 (socket says 172.21.15.1:44798) 2024-05-04T03:19:12.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.782+0000 7fabbbfff700 1 -- 172.21.15.1:0/318639243 learned_addr learned my addr 172.21.15.1:0/318639243 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:12.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.782+0000 7fabc0acf700 1 --2- 172.21.15.1:0/318639243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55886faa9870 0x55886fac9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.782+0000 7fabc12d0700 1 --2- 172.21.15.1:0/318639243 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55886fab38f0 0x55886faca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.782+0000 7fabbbfff700 1 -- 172.21.15.1:0/318639243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55886faa9870 msgr2=0x55886fac9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:12.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.782+0000 7fabbbfff700 1 --2- 172.21.15.1:0/318639243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55886faa9870 0x55886fac9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.782+0000 7fabbbfff700 1 -- 172.21.15.1:0/318639243 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55886fab38f0 msgr2=0x55886faca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:12.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.782+0000 7fabbbfff700 1 --2- 172.21.15.1:0/318639243 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55886fab38f0 0x55886faca530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.782+0000 7fabbbfff700 1 -- 172.21.15.1:0/318639243 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55886fac0170 con 0x55886faae600 2024-05-04T03:19:12.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.782+0000 7fabbbfff700 1 --2- 172.21.15.1:0/318639243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55886faae600 0x55886fac9ff0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7faba80a64d0 tx=0x7faba8010460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e1abe0463fdc server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:12.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.782+0000 7fab9bfff700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:19:12.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.782+0000 7fab9bfff700 1 -- 172.21.15.1:0/318639243 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8020070 con 0x55886faae600 2024-05-04T03:19:12.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.782+0000 7fab9bfff700 1 -- 172.21.15.1:0/318639243 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7faba80a5850 con 0x55886faae600 2024-05-04T03:19:12.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.782+0000 7fabd1747b80 1 -- 172.21.15.1:0/318639243 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabb0004ce0 con 0x55886faae600 2024-05-04T03:19:12.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.782+0000 7fab9bfff700 1 -- 172.21.15.1:0/318639243 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faba80a89d0 con 0x55886faae600 2024-05-04T03:19:12.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.783+0000 7fabd1747b80 10 client.15225 fetch_fsmap learned FSMap version 26 2024-05-04T03:19:12.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.783+0000 7fabd1747b80 1 -- 172.21.15.1:0/318639243 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55886fac0170 con 0x55886faae600 2024-05-04T03:19:12.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.783+0000 7fab9bfff700 1 -- 172.21.15.1:0/318639243 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabb0004ce0 con 0x55886faae600 2024-05-04T03:19:12.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.783+0000 7fab9bfff700 1 -- 172.21.15.1:0/318639243 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7faba80a84c0 con 0x55886faae600 2024-05-04T03:19:12.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.783+0000 7fabd1747b80 10 client.15225 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:19:12.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.783+0000 7fabd1747b80 10 client.15225 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-04T03:19:12.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.783+0000 7fabd1747b80 20 client.15225 populate_metadata read hostname 'smithi001' 2024-05-04T03:19:12.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.783+0000 7fabd1747b80 1 --2- 172.21.15.1:0/318639243 >> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] conn(0x55886fb30990 0x55886fb31e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:12.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.783+0000 7fabd1747b80 4 client.15225 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-04T03:19:12.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.783+0000 7fabd1747b80 1 -- 172.21.15.1:0/318639243 --> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] -- command(tid 0: {"prefix": "status"}) -- 0x55886f9cf370 con 0x55886fb30990 2024-05-04T03:19:12.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.783+0000 7fabc0acf700 1 --2- 172.21.15.1:0/318639243 >> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] conn(0x55886fb30990 0x55886fb31e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:12.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.784+0000 7fabc0acf700 1 --2- 172.21.15.1:0/318639243 >> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] conn(0x55886fb30990 0x55886fb31e00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d3ff23d15717f901 server_cookie=984b983ab4c35ed in_seq=0 out_seq=0 2024-05-04T03:19:12.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.784+0000 7fab9bfff700 10 client.15225 ms_handle_connect on v2:172.21.15.77:6838/922723189 2024-05-04T03:19:12.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.784+0000 7fab9bfff700 1 -- 172.21.15.1:0/318639243 <== mds.1 v2:172.21.15.77:6838/922723189 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55886f9cf370 con 0x55886fb30990 2024-05-04T03:19:12.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.784+0000 7fab9bfff700 10 client.15225 handle_command_reply: tid=0 2024-05-04T03:19:12.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.784+0000 7fabd1747b80 1 client.15225 shutdown 2024-05-04T03:19:12.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.784+0000 7fabd1747b80 1 -- 172.21.15.1:0/318639243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55886faae600 msgr2=0x55886fac9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:12.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.784+0000 7fabd1747b80 1 --2- 172.21.15.1:0/318639243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55886faae600 0x55886fac9ff0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7faba80a64d0 tx=0x7faba8010460 comp rx=0 tx=0).stop 2024-05-04T03:19:12.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.784+0000 7fabd1747b80 1 -- 172.21.15.1:0/318639243 shutdown_connections 2024-05-04T03:19:12.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.784+0000 7fabd1747b80 1 --2- 172.21.15.1:0/318639243 >> [v2:172.21.15.77:6838/922723189,v1:172.21.15.77:6839/922723189] conn(0x55886fb30990 0x55886fb31e00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.784+0000 7fabd1747b80 1 --2- 172.21.15.1:0/318639243 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55886fab38f0 0x55886faca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.784+0000 7fabd1747b80 1 --2- 172.21.15.1:0/318639243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55886faa9870 0x55886fac9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.784+0000 7fab9bfff700 0 client.15225 ms_handle_reset on v2:172.21.15.77:6838/922723189 2024-05-04T03:19:12.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.784+0000 7fabd1747b80 1 --2- 172.21.15.1:0/318639243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55886faae600 0x55886fac9ff0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.784+0000 7fabd1747b80 1 -- 172.21.15.1:0/318639243 >> 172.21.15.1:0/318639243 conn(0x55886f927df0 msgr2=0x55886faa9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:12.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.784+0000 7fabd1747b80 1 -- 172.21.15.1:0/318639243 shutdown_connections 2024-05-04T03:19:12.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.784+0000 7fabd1747b80 1 -- 172.21.15.1:0/318639243 wait complete. 2024-05-04T03:19:12.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.784+0000 7fabd1747b80 20 client.15225 trim_cache size 0 max 16384 2024-05-04T03:19:12.791 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-04T03:19:12.792 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "4a572aa8-09c3-11ef-bc97-c7b262605968", 2024-05-04T03:19:12.792 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 1, 2024-05-04T03:19:12.792 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24472, 2024-05-04T03:19:12.792 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-05-04T03:19:12.792 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-05-04T03:19:12.792 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-05-04T03:19:12.792 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 190.56254943100001, 2024-05-04T03:19:12.792 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 26, 2024-05-04T03:19:12.792 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 83, 2024-05-04T03:19:12.792 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 83, 2024-05-04T03:19:12.792 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 228.52535164599999 2024-05-04T03:19:12.792 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-04T03:19:12.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.785+0000 7fabc2d33700 1 -- 172.21.15.1:0/3334894284 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fabac0b2910 msgr2=0x7fabac0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:12.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.785+0000 7fabc2d33700 1 --2- 172.21.15.1:0/3334894284 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fabac0b2910 0x7fabac0b4e20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fabb0005950 tx=0x7fabb000b410 comp rx=0 tx=0).stop 2024-05-04T03:19:12.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.785+0000 7fabc2d33700 1 -- 172.21.15.1:0/3334894284 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabbc107df0 msgr2=0x7fabbc19c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:12.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.785+0000 7fabc2d33700 1 --2- 172.21.15.1:0/3334894284 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabbc107df0 0x7fabbc19c180 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7faba800d980 tx=0x7faba800dc90 comp rx=0 tx=0).stop 2024-05-04T03:19:12.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.785+0000 7fabc2d33700 1 -- 172.21.15.1:0/3334894284 shutdown_connections 2024-05-04T03:19:12.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.785+0000 7fabc2d33700 1 --2- 172.21.15.1:0/3334894284 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fabbc108740 0x7fabbc1a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.785+0000 7fabc2d33700 1 --2- 172.21.15.1:0/3334894284 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fabac0b2910 0x7fabac0b4e20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.785+0000 7fabc2d33700 1 --2- 172.21.15.1:0/3334894284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fabbc1034c0 0x7fabbc19bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.785+0000 7fabc2d33700 1 --2- 172.21.15.1:0/3334894284 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fabbc107df0 0x7fabbc19c180 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:12.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.785+0000 7fabc2d33700 1 -- 172.21.15.1:0/3334894284 >> 172.21.15.1:0/3334894284 conn(0x7fabbc0fedf0 msgr2=0x7fabbc107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:12.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.785+0000 7fabc2d33700 1 -- 172.21.15.1:0/3334894284 shutdown_connections 2024-05-04T03:19:12.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:12.785+0000 7fabc2d33700 1 -- 172.21.15.1:0/3334894284 wait complete. 2024-05-04T03:19:12.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:12 smithi077 ceph-mon[33531]: pgmap v392: 97 pgs: 97 active+clean; 665 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-05-04T03:19:12.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:12 smithi072 ceph-mon[33332]: pgmap v392: 97 pgs: 97 active+clean; 665 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-05-04T03:19:13.515 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-04T03:19:13.766 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:19:14.859 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:14 smithi001 ceph-mon[29731]: pgmap v393: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-05-04T03:19:14.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:14 smithi077 ceph-mon[33531]: pgmap v393: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-05-04T03:19:14.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:14 smithi072 ceph-mon[33332]: pgmap v393: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-05-04T03:19:15.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.419+0000 7f1ab937f700 1 -- 172.21.15.1:0/536844968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ab4107df0 msgr2=0x7f1ab4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:15.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.419+0000 7f1ab937f700 1 --2- 172.21.15.1:0/536844968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ab4107df0 0x7f1ab4108200 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c0099c0 tx=0x7f1a9c009cd0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.420+0000 7f1ab937f700 1 -- 172.21.15.1:0/536844968 shutdown_connections 2024-05-04T03:19:15.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.420+0000 7f1ab937f700 1 --2- 172.21.15.1:0/536844968 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1ab4108740 0x7f1ab41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.420+0000 7f1ab937f700 1 --2- 172.21.15.1:0/536844968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ab41034c0 0x7f1ab41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.420+0000 7f1ab937f700 1 --2- 172.21.15.1:0/536844968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ab4107df0 0x7f1ab4108200 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.420+0000 7f1ab937f700 1 -- 172.21.15.1:0/536844968 >> 172.21.15.1:0/536844968 conn(0x7f1ab40fedc0 msgr2=0x7f1ab41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:15.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.420+0000 7f1ab937f700 1 -- 172.21.15.1:0/536844968 shutdown_connections 2024-05-04T03:19:15.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.420+0000 7f1ab937f700 1 -- 172.21.15.1:0/536844968 wait complete. 2024-05-04T03:19:15.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.420+0000 7f1ab937f700 1 Processor -- start 2024-05-04T03:19:15.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.420+0000 7f1ab937f700 1 -- start start 2024-05-04T03:19:15.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.421+0000 7f1ab937f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ab41034c0 0x7f1ab419bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.421+0000 7f1ab937f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1ab4107df0 0x7f1ab419c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.421+0000 7f1ab937f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ab4108740 0x7f1ab419c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.421+0000 7f1ab937f700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab4113350 con 0x7f1ab4108740 2024-05-04T03:19:15.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.421+0000 7f1ab937f700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab41131d0 con 0x7f1ab4107df0 2024-05-04T03:19:15.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.421+0000 7f1ab937f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab41134d0 con 0x7f1ab41034c0 2024-05-04T03:19:15.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.421+0000 7f1ab37fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ab4108740 0x7f1ab419c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.421+0000 7f1ab37fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ab4108740 0x7f1ab419c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44814/0 (socket says 172.21.15.1:44814) 2024-05-04T03:19:15.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.421+0000 7f1ab37fe700 1 -- 172.21.15.1:0/1080638817 learned_addr learned my addr 172.21.15.1:0/1080638817 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:15.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.421+0000 7f1ab2ffd700 1 --2- 172.21.15.1:0/1080638817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ab41034c0 0x7f1ab419bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.421+0000 7f1aabfff700 1 --2- 172.21.15.1:0/1080638817 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1ab4107df0 0x7f1ab419c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.421+0000 7f1ab37fe700 1 -- 172.21.15.1:0/1080638817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ab41034c0 msgr2=0x7f1ab419bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:15.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.421+0000 7f1ab37fe700 1 --2- 172.21.15.1:0/1080638817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ab41034c0 0x7f1ab419bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.421+0000 7f1ab37fe700 1 -- 172.21.15.1:0/1080638817 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1ab4107df0 msgr2=0x7f1ab419c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:15.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.421+0000 7f1ab37fe700 1 --2- 172.21.15.1:0/1080638817 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1ab4107df0 0x7f1ab419c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.421+0000 7f1ab37fe700 1 -- 172.21.15.1:0/1080638817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab41a6f50 con 0x7f1ab4108740 2024-05-04T03:19:15.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.422+0000 7f1ab37fe700 1 --2- 172.21.15.1:0/1080638817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ab4108740 0x7f1ab419c730 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00ef30 tx=0x7f1aac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:15.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.422+0000 7f1ab0ff9700 1 -- 172.21.15.1:0/1080638817 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac0092e0 con 0x7f1ab4108740 2024-05-04T03:19:15.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.422+0000 7f1ab0ff9700 1 -- 172.21.15.1:0/1080638817 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aac004bb0 con 0x7f1ab4108740 2024-05-04T03:19:15.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.422+0000 7f1ab937f700 1 -- 172.21.15.1:0/1080638817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab41a71e0 con 0x7f1ab4108740 2024-05-04T03:19:15.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.422+0000 7f1ab0ff9700 1 -- 172.21.15.1:0/1080638817 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac004020 con 0x7f1ab4108740 2024-05-04T03:19:15.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.422+0000 7f1ab937f700 1 -- 172.21.15.1:0/1080638817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab41a76a0 con 0x7f1ab4108740 2024-05-04T03:19:15.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.425+0000 7f1ab0ff9700 1 -- 172.21.15.1:0/1080638817 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7f1aac01b030 con 0x7f1ab4108740 2024-05-04T03:19:15.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.425+0000 7f1ab0ff9700 1 --2- 172.21.15.1:0/1080638817 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1aa00b2910 0x7f1aa00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.426+0000 7f1ab2ffd700 1 --2- 172.21.15.1:0/1080638817 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1aa00b2910 0x7f1aa00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.426+0000 7f1ab0ff9700 1 -- 172.21.15.1:0/1080638817 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f1aac0e1cd0 con 0x7f1ab4108740 2024-05-04T03:19:15.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.426+0000 7f1ac7d93b80 1 Processor -- start 2024-05-04T03:19:15.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.426+0000 7f1ab2ffd700 1 --2- 172.21.15.1:0/1080638817 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1aa00b2910 0x7f1aa00b4e20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1aa400ac50 tx=0x7f1aa400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:15.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.426+0000 7f1ac7d93b80 1 -- start start 2024-05-04T03:19:15.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.426+0000 7f1ac7d93b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56402db70da0 0x56402db711b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.426+0000 7f1ac7d93b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56402db6aef0 0x56402db6b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.426+0000 7f1ac7d93b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56402db6b840 0x56402db779f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.427+0000 7f1ac7d93b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56402da8d810 con 0x56402db6aef0 2024-05-04T03:19:15.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.427+0000 7f1ac7d93b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x56402da2c770 con 0x56402db6b840 2024-05-04T03:19:15.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.427+0000 7f1ac7d93b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56402da897d0 con 0x56402db70da0 2024-05-04T03:19:15.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.427+0000 7f1aabfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56402db6aef0 0x56402db6b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.427+0000 7f1aabfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56402db6aef0 0x56402db6b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44824/0 (socket says 172.21.15.1:44824) 2024-05-04T03:19:15.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.427+0000 7f1aabfff700 1 -- 172.21.15.1:0/1211727357 learned_addr learned my addr 172.21.15.1:0/1211727357 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:15.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.427+0000 7f1ab37fe700 1 --2- 172.21.15.1:0/1211727357 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56402db6b840 0x56402db779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.427+0000 7f1ab2ffd700 1 --2- 172.21.15.1:0/1211727357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56402db70da0 0x56402db711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.427+0000 7f1aabfff700 1 -- 172.21.15.1:0/1211727357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56402db70da0 msgr2=0x56402db711b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:15.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.427+0000 7f1aabfff700 1 --2- 172.21.15.1:0/1211727357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56402db70da0 0x56402db711b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.427+0000 7f1aabfff700 1 -- 172.21.15.1:0/1211727357 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56402db6b840 msgr2=0x56402db779f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:15.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.427+0000 7f1aabfff700 1 --2- 172.21.15.1:0/1211727357 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56402db6b840 0x56402db779f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.427+0000 7f1aabfff700 1 -- 172.21.15.1:0/1211727357 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56402d9da880 con 0x56402db6aef0 2024-05-04T03:19:15.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.427+0000 7f1ab37fe700 1 --2- 172.21.15.1:0/1211727357 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56402db6b840 0x56402db779f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:19:15.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.427+0000 7f1aabfff700 1 --2- 172.21.15.1:0/1211727357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56402db6aef0 0x56402db6b300 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c0099c0 tx=0x7f1a9c005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3622ce9ff5cc5514 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:15.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.427+0000 7f1aa97fa700 1 -- 172.21.15.1:0/1211727357 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c027070 con 0x56402db6aef0 2024-05-04T03:19:15.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.427+0000 7f1aa97fa700 1 -- 172.21.15.1:0/1211727357 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a9c003c70 con 0x56402db6aef0 2024-05-04T03:19:15.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.427+0000 7f1aa97fa700 1 -- 172.21.15.1:0/1211727357 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c00ba90 con 0x56402db6aef0 2024-05-04T03:19:15.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.428+0000 7f1ac7d93b80 1 -- 172.21.15.1:0/1211727357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56402db6aef0 msgr2=0x56402db6b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:15.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.428+0000 7f1ac7d93b80 1 --2- 172.21.15.1:0/1211727357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56402db6aef0 0x56402db6b300 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c0099c0 tx=0x7f1a9c005040 comp rx=0 tx=0).stop 2024-05-04T03:19:15.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.428+0000 7f1ac7d93b80 1 -- 172.21.15.1:0/1211727357 shutdown_connections 2024-05-04T03:19:15.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.428+0000 7f1ac7d93b80 1 --2- 172.21.15.1:0/1211727357 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56402db6b840 0x56402db779f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.428+0000 7f1ac7d93b80 1 --2- 172.21.15.1:0/1211727357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56402db70da0 0x56402db711b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.428+0000 7f1ac7d93b80 1 --2- 172.21.15.1:0/1211727357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56402db6aef0 0x56402db6b300 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.428+0000 7f1ac7d93b80 1 -- 172.21.15.1:0/1211727357 >> 172.21.15.1:0/1211727357 conn(0x56402d9eddf0 msgr2=0x56402db65bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:15.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.429+0000 7f1ac7d93b80 1 -- 172.21.15.1:0/1211727357 shutdown_connections 2024-05-04T03:19:15.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.429+0000 7f1ac7d93b80 1 -- 172.21.15.1:0/1211727357 wait complete. 2024-05-04T03:19:15.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.429+0000 7f1ac7d93b80 1 -- start start 2024-05-04T03:19:15.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.429+0000 7f1ac7d93b80 1 Processor -- start 2024-05-04T03:19:15.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.429+0000 7f1ac7d93b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56402db6aef0 0x56402db74f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.429+0000 7f1ac7d93b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56402db6b840 0x56402db75440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.429+0000 7f1ac7d93b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56402db70da0 0x56402db75980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.429+0000 7f1ac7d93b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56402db78170 con 0x56402db6b840 2024-05-04T03:19:15.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.429+0000 7f1ac7d93b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x56402d9f3be0 con 0x56402db6aef0 2024-05-04T03:19:15.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.429+0000 7f1ac7d93b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56402db782f0 con 0x56402db70da0 2024-05-04T03:19:15.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.429+0000 7f1ac7d93b80 10 client.0 _ll_register_callbacks cb 0x56402da258d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:15.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.429+0000 7f1aabfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56402db6b840 0x56402db75440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.430+0000 7f1aabfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56402db6b840 0x56402db75440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44840/0 (socket says 172.21.15.1:44840) 2024-05-04T03:19:15.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.430+0000 7f1aabfff700 1 -- 172.21.15.1:0/2046277241 learned_addr learned my addr 172.21.15.1:0/2046277241 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:15.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.430+0000 7f1ab2ffd700 1 --2- 172.21.15.1:0/2046277241 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56402db6aef0 0x56402db74f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.430+0000 7f1ab37fe700 1 --2- 172.21.15.1:0/2046277241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56402db70da0 0x56402db75980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.430+0000 7f1aabfff700 1 -- 172.21.15.1:0/2046277241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56402db70da0 msgr2=0x56402db75980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:15.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.430+0000 7f1aabfff700 1 --2- 172.21.15.1:0/2046277241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56402db70da0 0x56402db75980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.430+0000 7f1aabfff700 1 -- 172.21.15.1:0/2046277241 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56402db6aef0 msgr2=0x56402db74f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:15.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.430+0000 7f1aabfff700 1 --2- 172.21.15.1:0/2046277241 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56402db6aef0 0x56402db74f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.430+0000 7f1aabfff700 1 -- 172.21.15.1:0/2046277241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56402da29640 con 0x56402db6b840 2024-05-04T03:19:15.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.430+0000 7f1ab37fe700 1 --2- 172.21.15.1:0/2046277241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56402db70da0 0x56402db75980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:19:15.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.430+0000 7f1aabfff700 1 --2- 172.21.15.1:0/2046277241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56402db6b840 0x56402db75440 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c0098e0 tx=0x7f1a9c003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec70b21603d160f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:15.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.430+0000 7f1a9a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:19:15.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.430+0000 7f1a9a7fc700 1 -- 172.21.15.1:0/2046277241 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c027040 con 0x56402db6b840 2024-05-04T03:19:15.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.430+0000 7f1a9a7fc700 1 -- 172.21.15.1:0/2046277241 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f1a9c00b990 con 0x56402db6b840 2024-05-04T03:19:15.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.430+0000 7f1ac7d93b80 1 -- 172.21.15.1:0/2046277241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a9c003c70 con 0x56402db6b840 2024-05-04T03:19:15.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.431+0000 7f1a9a7fc700 1 -- 172.21.15.1:0/2046277241 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1a9c00bb90 con 0x56402db6b840 2024-05-04T03:19:15.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.431+0000 7f1ac7d93b80 10 client.15237 fetch_fsmap learned FSMap version 26 2024-05-04T03:19:15.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.431+0000 7f1ac7d93b80 1 -- 172.21.15.1:0/2046277241 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56402db82da0 con 0x56402db6b840 2024-05-04T03:19:15.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.431+0000 7f1a9a7fc700 1 -- 172.21.15.1:0/2046277241 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a9c003c70 con 0x56402db6b840 2024-05-04T03:19:15.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.431+0000 7f1a9a7fc700 1 -- 172.21.15.1:0/2046277241 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1a9c021030 con 0x56402db6b840 2024-05-04T03:19:15.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.431+0000 7f1ac7d93b80 10 client.15237 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:19:15.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.431+0000 7f1ac7d93b80 10 client.15237 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-04T03:19:15.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.431+0000 7f1ac7d93b80 20 client.15237 populate_metadata read hostname 'smithi001' 2024-05-04T03:19:15.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.431+0000 7f1ac7d93b80 1 --2- 172.21.15.1:0/2046277241 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x56402db840f0 0x56402db862b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.431+0000 7f1ac7d93b80 4 client.15237 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:19:15.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.431+0000 7f1ac7d93b80 1 -- 172.21.15.1:0/2046277241 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56402da9b1f0 con 0x56402db840f0 2024-05-04T03:19:15.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.431+0000 7f1ab2ffd700 1 --2- 172.21.15.1:0/2046277241 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x56402db840f0 0x56402db862b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.432+0000 7f1a9a7fc700 10 client.15237 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:19:15.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.432+0000 7f1ab2ffd700 1 --2- 172.21.15.1:0/2046277241 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x56402db840f0 0x56402db862b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2330322f559d35fa server_cookie=e100ef8304d8fdcd in_seq=0 out_seq=0 2024-05-04T03:19:15.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.433+0000 7f1a9a7fc700 1 -- 172.21.15.1:0/2046277241 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56402da9b1f0 con 0x56402db840f0 2024-05-04T03:19:15.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.433+0000 7f1a9a7fc700 10 client.15237 handle_command_reply: tid=0 2024-05-04T03:19:15.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.433+0000 7f1ac7d93b80 1 client.15237 shutdown 2024-05-04T03:19:15.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.433+0000 7f1ac7d93b80 1 -- 172.21.15.1:0/2046277241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56402db6b840 msgr2=0x56402db75440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:15.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.433+0000 7f1ac7d93b80 1 --2- 172.21.15.1:0/2046277241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56402db6b840 0x56402db75440 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c0098e0 tx=0x7f1a9c003a70 comp rx=0 tx=0).stop 2024-05-04T03:19:15.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.433+0000 7f1ac7d93b80 1 -- 172.21.15.1:0/2046277241 shutdown_connections 2024-05-04T03:19:15.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.433+0000 7f1ac7d93b80 1 --2- 172.21.15.1:0/2046277241 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x56402db840f0 0x56402db862b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.433+0000 7f1ac7d93b80 1 --2- 172.21.15.1:0/2046277241 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56402db6aef0 0x56402db74f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.433+0000 7f1ac7d93b80 1 --2- 172.21.15.1:0/2046277241 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56402db70da0 0x56402db75980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.433+0000 7f1a9a7fc700 0 client.15237 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:19:15.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.433+0000 7f1ac7d93b80 1 --2- 172.21.15.1:0/2046277241 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56402db6b840 0x56402db75440 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.433+0000 7f1ac7d93b80 1 -- 172.21.15.1:0/2046277241 >> 172.21.15.1:0/2046277241 conn(0x56402d9eddf0 msgr2=0x56402db69ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:15.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.433+0000 7f1ac7d93b80 1 -- 172.21.15.1:0/2046277241 shutdown_connections 2024-05-04T03:19:15.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.433+0000 7f1ac7d93b80 1 -- 172.21.15.1:0/2046277241 wait complete. 2024-05-04T03:19:15.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.433+0000 7f1ac7d93b80 20 client.15237 trim_cache size 0 max 16384 2024-05-04T03:19:15.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.454+0000 7f1ac7d93b80 1 Processor -- start 2024-05-04T03:19:15.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.454+0000 7f1ac7d93b80 1 -- start start 2024-05-04T03:19:15.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.455+0000 7f1ac7d93b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56402db6aef0 0x56402db84770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.455+0000 7f1ac7d93b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56402db6b840 0x56402db604c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.455+0000 7f1ac7d93b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56402db70da0 0x56402db81590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.455+0000 7f1ac7d93b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56402db845f0 con 0x56402db6aef0 2024-05-04T03:19:15.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.455+0000 7f1ac7d93b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x56402db84cc0 con 0x56402db70da0 2024-05-04T03:19:15.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.455+0000 7f1ac7d93b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56402db85390 con 0x56402db6b840 2024-05-04T03:19:15.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.455+0000 7f1ab2ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56402db6aef0 0x56402db84770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.455+0000 7f1ab2ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56402db6aef0 0x56402db84770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44846/0 (socket says 172.21.15.1:44846) 2024-05-04T03:19:15.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.455+0000 7f1ab2ffd700 1 -- 172.21.15.1:0/3119298870 learned_addr learned my addr 172.21.15.1:0/3119298870 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:15.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.455+0000 7f1ab37fe700 1 --2- 172.21.15.1:0/3119298870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56402db70da0 0x56402db81590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.455+0000 7f1aabfff700 1 --2- 172.21.15.1:0/3119298870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56402db6b840 0x56402db604c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.455+0000 7f1ab2ffd700 1 -- 172.21.15.1:0/3119298870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56402db6b840 msgr2=0x56402db604c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:15.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.455+0000 7f1ab2ffd700 1 --2- 172.21.15.1:0/3119298870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56402db6b840 0x56402db604c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.455+0000 7f1ab2ffd700 1 -- 172.21.15.1:0/3119298870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56402db70da0 msgr2=0x56402db81590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:15.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.455+0000 7f1ab2ffd700 1 --2- 172.21.15.1:0/3119298870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56402db70da0 0x56402db81590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.455+0000 7f1ab2ffd700 1 -- 172.21.15.1:0/3119298870 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56402db73470 con 0x56402db6aef0 2024-05-04T03:19:15.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.455+0000 7f1ab2ffd700 1 --2- 172.21.15.1:0/3119298870 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56402db6aef0 0x56402db84770 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x56402db74780 tx=0x7f1aa4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3882e01d4f787e server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:15.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.456+0000 7f1aa8ff9700 1 -- 172.21.15.1:0/3119298870 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4016070 con 0x56402db6aef0 2024-05-04T03:19:15.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.456+0000 7f1aa8ff9700 1 -- 172.21.15.1:0/3119298870 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aa4014070 con 0x56402db6aef0 2024-05-04T03:19:15.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.456+0000 7f1aa8ff9700 1 -- 172.21.15.1:0/3119298870 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4027950 con 0x56402db6aef0 2024-05-04T03:19:15.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.457+0000 7f1ac7d93b80 1 -- 172.21.15.1:0/3119298870 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56402db6aef0 msgr2=0x56402db84770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:15.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.457+0000 7f1ac7d93b80 1 --2- 172.21.15.1:0/3119298870 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56402db6aef0 0x56402db84770 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x56402db74780 tx=0x7f1aa4012000 comp rx=0 tx=0).stop 2024-05-04T03:19:15.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.457+0000 7f1ac7d93b80 1 -- 172.21.15.1:0/3119298870 shutdown_connections 2024-05-04T03:19:15.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.457+0000 7f1ac7d93b80 1 --2- 172.21.15.1:0/3119298870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56402db70da0 0x56402db81590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.457+0000 7f1ac7d93b80 1 --2- 172.21.15.1:0/3119298870 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56402db6aef0 0x56402db84770 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.457+0000 7f1ac7d93b80 1 --2- 172.21.15.1:0/3119298870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56402db6b840 0x56402db604c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.457+0000 7f1ac7d93b80 1 -- 172.21.15.1:0/3119298870 >> 172.21.15.1:0/3119298870 conn(0x56402d9eddf0 msgr2=0x56402db5f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:15.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.457+0000 7f1ac7d93b80 1 -- 172.21.15.1:0/3119298870 shutdown_connections 2024-05-04T03:19:15.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.457+0000 7f1ac7d93b80 1 -- 172.21.15.1:0/3119298870 wait complete. 2024-05-04T03:19:15.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.457+0000 7f1ac7d93b80 1 -- start start 2024-05-04T03:19:15.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.457+0000 7f1ac7d93b80 1 Processor -- start 2024-05-04T03:19:15.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.458+0000 7f1ac7d93b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56402db6aef0 0x56402db8de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.458+0000 7f1ac7d93b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56402db6b840 0x56402db8e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.458+0000 7f1ac7d93b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56402db70da0 0x56402db75190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.458+0000 7f1ac7d93b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56402db86130 con 0x56402db70da0 2024-05-04T03:19:15.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.458+0000 7f1ac7d93b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x56402db85a60 con 0x56402db6b840 2024-05-04T03:19:15.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.458+0000 7f1ac7d93b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56402db86800 con 0x56402db6aef0 2024-05-04T03:19:15.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.458+0000 7f1ac7d93b80 10 client.0 _ll_register_callbacks cb 0x56402da258d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:15.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.458+0000 7f1ab37fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56402db70da0 0x56402db75190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.458+0000 7f1aabfff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56402db6b840 0x56402db8e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.458+0000 7f1ab37fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56402db70da0 0x56402db75190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44852/0 (socket says 172.21.15.1:44852) 2024-05-04T03:19:15.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.458+0000 7f1ab37fe700 1 -- 172.21.15.1:0/2957717471 learned_addr learned my addr 172.21.15.1:0/2957717471 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:15.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.458+0000 7f1ab2ffd700 1 --2- 172.21.15.1:0/2957717471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56402db6aef0 0x56402db8de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.458+0000 7f1ab37fe700 1 -- 172.21.15.1:0/2957717471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56402db6aef0 msgr2=0x56402db8de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:15.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.458+0000 7f1ab37fe700 1 --2- 172.21.15.1:0/2957717471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56402db6aef0 0x56402db8de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.458+0000 7f1ab37fe700 1 -- 172.21.15.1:0/2957717471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56402db6b840 msgr2=0x56402db8e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:15.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.458+0000 7f1ab37fe700 1 --2- 172.21.15.1:0/2957717471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56402db6b840 0x56402db8e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.458+0000 7f1ab37fe700 1 -- 172.21.15.1:0/2957717471 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56402db756d0 con 0x56402db70da0 2024-05-04T03:19:15.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.458+0000 7f1ab37fe700 1 --2- 172.21.15.1:0/2957717471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56402db70da0 0x56402db75190 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0a53c0 tx=0x7f1aac0b1f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56cc4e83233d1a7f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:15.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.458+0000 7f1a9a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:19:15.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.458+0000 7f1a9a7fc700 1 -- 172.21.15.1:0/2957717471 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac0a8e50 con 0x56402db70da0 2024-05-04T03:19:15.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.458+0000 7f1a9a7fc700 1 -- 172.21.15.1:0/2957717471 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f1aac0a4ec0 con 0x56402db70da0 2024-05-04T03:19:15.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.458+0000 7f1ac7d93b80 1 -- 172.21.15.1:0/2957717471 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1aa4014070 con 0x56402db70da0 2024-05-04T03:19:15.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.459+0000 7f1a9a7fc700 1 -- 172.21.15.1:0/2957717471 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1aac0a9a40 con 0x56402db70da0 2024-05-04T03:19:15.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.459+0000 7f1ac7d93b80 10 client.15243 fetch_fsmap learned FSMap version 26 2024-05-04T03:19:15.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.459+0000 7f1ac7d93b80 1 -- 172.21.15.1:0/2957717471 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56402dbf4cb0 con 0x56402db70da0 2024-05-04T03:19:15.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.459+0000 7f1a9a7fc700 1 -- 172.21.15.1:0/2957717471 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aa4014070 con 0x56402db70da0 2024-05-04T03:19:15.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.459+0000 7f1a9a7fc700 1 -- 172.21.15.1:0/2957717471 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1aac003a90 con 0x56402db70da0 2024-05-04T03:19:15.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.459+0000 7f1ac7d93b80 10 client.15243 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:19:15.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.459+0000 7f1ac7d93b80 10 client.15243 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-04T03:19:15.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.459+0000 7f1ac7d93b80 20 client.15243 populate_metadata read hostname 'smithi001' 2024-05-04T03:19:15.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.459+0000 7f1ac7d93b80 1 --2- 172.21.15.1:0/2957717471 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x56402dbf7110 0x56402dbf8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:15.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.459+0000 7f1ac7d93b80 4 client.15243 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-04T03:19:15.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.459+0000 7f1ac7d93b80 1 -- 172.21.15.1:0/2957717471 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "status"}) -- 0x56402da95290 con 0x56402dbf7110 2024-05-04T03:19:15.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.459+0000 7f1ab2ffd700 1 --2- 172.21.15.1:0/2957717471 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x56402dbf7110 0x56402dbf8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:15.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.460+0000 7f1a9a7fc700 10 client.15243 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:19:15.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.460+0000 7f1ab2ffd700 1 --2- 172.21.15.1:0/2957717471 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x56402dbf7110 0x56402dbf8580 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d008f5ebb871fe74 server_cookie=ef51d3820ce67132 in_seq=0 out_seq=0 2024-05-04T03:19:15.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.460+0000 7f1a9a7fc700 1 -- 172.21.15.1:0/2957717471 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56402da95290 con 0x56402dbf7110 2024-05-04T03:19:15.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.460+0000 7f1a9a7fc700 10 client.15243 handle_command_reply: tid=0 2024-05-04T03:19:15.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.460+0000 7f1ac7d93b80 1 client.15243 shutdown 2024-05-04T03:19:15.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.460+0000 7f1ac7d93b80 1 -- 172.21.15.1:0/2957717471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56402db70da0 msgr2=0x56402db75190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:15.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.460+0000 7f1ac7d93b80 1 --2- 172.21.15.1:0/2957717471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56402db70da0 0x56402db75190 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0a53c0 tx=0x7f1aac0b1f60 comp rx=0 tx=0).stop 2024-05-04T03:19:15.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.460+0000 7f1ac7d93b80 1 -- 172.21.15.1:0/2957717471 shutdown_connections 2024-05-04T03:19:15.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.460+0000 7f1ac7d93b80 1 --2- 172.21.15.1:0/2957717471 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x56402dbf7110 0x56402dbf8580 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.460+0000 7f1ac7d93b80 1 --2- 172.21.15.1:0/2957717471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56402db70da0 0x56402db75190 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.460+0000 7f1ac7d93b80 1 --2- 172.21.15.1:0/2957717471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56402db6aef0 0x56402db8de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.460+0000 7f1ac7d93b80 1 --2- 172.21.15.1:0/2957717471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56402db6b840 0x56402db8e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.460+0000 7f1ac7d93b80 1 -- 172.21.15.1:0/2957717471 >> 172.21.15.1:0/2957717471 conn(0x56402d9eddf0 msgr2=0x56402db5f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:15.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.460+0000 7f1a9a7fc700 0 client.15243 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:19:15.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.461+0000 7f1ac7d93b80 1 -- 172.21.15.1:0/2957717471 shutdown_connections 2024-05-04T03:19:15.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.461+0000 7f1ac7d93b80 1 -- 172.21.15.1:0/2957717471 wait complete. 2024-05-04T03:19:15.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.461+0000 7f1ac7d93b80 20 client.15243 trim_cache size 0 max 16384 2024-05-04T03:19:15.473 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-04T03:19:15.473 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "4a572aa8-09c3-11ef-bc97-c7b262605968", 2024-05-04T03:19:15.473 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-05-04T03:19:15.473 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24494, 2024-05-04T03:19:15.474 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-05-04T03:19:15.474 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-05-04T03:19:15.474 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-05-04T03:19:15.474 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 205.42479059999999, 2024-05-04T03:19:15.474 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 26, 2024-05-04T03:19:15.474 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 83, 2024-05-04T03:19:15.474 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 83, 2024-05-04T03:19:15.474 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 221.089628218 2024-05-04T03:19:15.474 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-04T03:19:15.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.461+0000 7f1ab937f700 1 -- 172.21.15.1:0/1080638817 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1aa00b2910 msgr2=0x7f1aa00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:15.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.461+0000 7f1ab937f700 1 --2- 172.21.15.1:0/1080638817 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1aa00b2910 0x7f1aa00b4e20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1aa400ac50 tx=0x7f1aa400a380 comp rx=0 tx=0).stop 2024-05-04T03:19:15.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.461+0000 7f1ab937f700 1 -- 172.21.15.1:0/1080638817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ab4108740 msgr2=0x7f1ab419c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:15.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.461+0000 7f1ab937f700 1 --2- 172.21.15.1:0/1080638817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ab4108740 0x7f1ab419c730 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00ef30 tx=0x7f1aac00c3a0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.461+0000 7f1ab937f700 1 -- 172.21.15.1:0/1080638817 shutdown_connections 2024-05-04T03:19:15.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.461+0000 7f1ab937f700 1 --2- 172.21.15.1:0/1080638817 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1aa00b2910 0x7f1aa00b4e20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.461+0000 7f1ab937f700 1 --2- 172.21.15.1:0/1080638817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ab4108740 0x7f1ab419c730 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.461+0000 7f1ab937f700 1 --2- 172.21.15.1:0/1080638817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ab41034c0 0x7f1ab419bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.461+0000 7f1ab937f700 1 --2- 172.21.15.1:0/1080638817 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1ab4107df0 0x7f1ab419c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:15.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.461+0000 7f1ab937f700 1 -- 172.21.15.1:0/1080638817 >> 172.21.15.1:0/1080638817 conn(0x7f1ab40fedc0 msgr2=0x7f1ab41096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:15.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.461+0000 7f1ab937f700 1 -- 172.21.15.1:0/1080638817 shutdown_connections 2024-05-04T03:19:15.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:15.461+0000 7f1ab937f700 1 -- 172.21.15.1:0/1080638817 wait complete. 2024-05-04T03:19:16.145 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-05-04T03:19:16.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:16 smithi001 ceph-mon[29731]: pgmap v394: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-04T03:19:16.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:16 smithi077 ceph-mon[33531]: pgmap v394: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-04T03:19:16.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:16 smithi072 ceph-mon[33332]: pgmap v394: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-04T03:19:18.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:18 smithi001 ceph-mon[29731]: pgmap v395: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-05-04T03:19:18.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:18 smithi077 ceph-mon[33531]: pgmap v395: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-05-04T03:19:18.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:18 smithi072 ceph-mon[33332]: pgmap v395: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-05-04T03:19:20.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:20 smithi001 ceph-mon[29731]: pgmap v396: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-05-04T03:19:20.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:20 smithi077 ceph-mon[33531]: pgmap v396: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-05-04T03:19:20.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:20 smithi072 ceph-mon[33332]: pgmap v396: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-05-04T03:19:22.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:22 smithi001 ceph-mon[29731]: pgmap v397: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-04T03:19:22.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:22 smithi077 ceph-mon[33531]: pgmap v397: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-04T03:19:22.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:22 smithi072 ceph-mon[33332]: pgmap v397: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-05-04T03:19:24.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:24 smithi001 ceph-mon[29731]: pgmap v398: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-04T03:19:24.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:24 smithi077 ceph-mon[33531]: pgmap v398: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-04T03:19:24.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:24 smithi072 ceph-mon[33332]: pgmap v398: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-04T03:19:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.419+0000 7f5449f00640 1 -- 172.21.15.1:0/2888250424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5444104740 msgr2=0x7f5444106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.419+0000 7f5449f00640 1 --2- 172.21.15.1:0/2888250424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5444104740 0x7f5444106b60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5434009900 tx=0x7f543402a410 comp rx=0 tx=0).stop 2024-05-04T03:19:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.419+0000 7f5449f00640 1 -- 172.21.15.1:0/2888250424 shutdown_connections 2024-05-04T03:19:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.419+0000 7f5449f00640 1 --2- 172.21.15.1:0/2888250424 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54441070a0 0x7f54441094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.419+0000 7f5449f00640 1 --2- 172.21.15.1:0/2888250424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5444109a00 0x7f544410be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.419+0000 7f5449f00640 1 --2- 172.21.15.1:0/2888250424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5444104740 0x7f5444106b60 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.419+0000 7f5449f00640 1 -- 172.21.15.1:0/2888250424 >> 172.21.15.1:0/2888250424 conn(0x7f54440fdd60 msgr2=0x7f54441001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.419+0000 7f5449f00640 1 -- 172.21.15.1:0/2888250424 shutdown_connections 2024-05-04T03:19:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.419+0000 7f5449f00640 1 -- 172.21.15.1:0/2888250424 wait complete. 2024-05-04T03:19:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.420+0000 7f5449f00640 1 Processor -- start 2024-05-04T03:19:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.420+0000 7f5449f00640 1 -- start start 2024-05-04T03:19:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.421+0000 7f5449f00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5444104740 0x7f5444103b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.421+0000 7f5449f00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54441070a0 0x7f5444102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.421+0000 7f5449f00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5444109a00 0x7f5444102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.421+0000 7f5449f00640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f544410f730 con 0x7f5444109a00 2024-05-04T03:19:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.421+0000 7f5449f00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f544410f5b0 con 0x7f5444104740 2024-05-04T03:19:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.421+0000 7f5449f00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f544410f8b0 con 0x7f54441070a0 2024-05-04T03:19:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.421+0000 7f5443fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5444109a00 0x7f5444102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.421+0000 7f5443fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5444109a00 0x7f5444102770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36158/0 (socket says 172.21.15.1:36158) 2024-05-04T03:19:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.421+0000 7f5443fff640 1 -- 172.21.15.1:0/1444890131 learned_addr learned my addr 172.21.15.1:0/1444890131 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.421+0000 7f54437fe640 1 --2- 172.21.15.1:0/1444890131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5444104740 0x7f5444103b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.421+0000 7f5442ffd640 1 --2- 172.21.15.1:0/1444890131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54441070a0 0x7f5444102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.421+0000 7f5443fff640 1 -- 172.21.15.1:0/1444890131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54441070a0 msgr2=0x7f5444102230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.421+0000 7f5443fff640 1 --2- 172.21.15.1:0/1444890131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54441070a0 0x7f5444102230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.421+0000 7f5443fff640 1 -- 172.21.15.1:0/1444890131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5444104740 msgr2=0x7f5444103b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.421+0000 7f5443fff640 1 --2- 172.21.15.1:0/1444890131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5444104740 0x7f5444103b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.421+0000 7f5443fff640 1 -- 172.21.15.1:0/1444890131 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5444102ea0 con 0x7f5444109a00 2024-05-04T03:19:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.421+0000 7f5443fff640 1 --2- 172.21.15.1:0/1444890131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5444109a00 0x7f5444102770 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f543c00dac0 tx=0x7f543c00df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.421+0000 7f5440ff9640 1 -- 172.21.15.1:0/1444890131 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c017070 con 0x7f5444109a00 2024-05-04T03:19:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.422+0000 7f5449f00640 1 -- 172.21.15.1:0/1444890131 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54441a4370 con 0x7f5444109a00 2024-05-04T03:19:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.422+0000 7f5440ff9640 1 -- 172.21.15.1:0/1444890131 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f543c00be30 con 0x7f5444109a00 2024-05-04T03:19:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.422+0000 7f5449f00640 1 -- 172.21.15.1:0/1444890131 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54441a47e0 con 0x7f5444109a00 2024-05-04T03:19:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.422+0000 7f5440ff9640 1 -- 172.21.15.1:0/1444890131 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c020060 con 0x7f5444109a00 2024-05-04T03:19:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.424+0000 7f5450db1740 1 Processor -- start 2024-05-04T03:19:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.424+0000 7f5440ff9640 1 -- 172.21.15.1:0/1444890131 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7f543c010ac0 con 0x7f5444109a00 2024-05-04T03:19:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.424+0000 7f5450db1740 1 -- start start 2024-05-04T03:19:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.424+0000 7f5450db1740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560e69a5c1b0 0x560e69dfda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.424+0000 7f5450db1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e69a332f0 0x560e69dfdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.424+0000 7f5450db1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e69a3b4d0 0x560e69e04ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.424+0000 7f5450db1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560e69d625d0 con 0x560e69a3b4d0 2024-05-04T03:19:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.424+0000 7f5450db1740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x560e69d1b890 con 0x560e69a5c1b0 2024-05-04T03:19:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.424+0000 7f5450db1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560e69d5e590 con 0x560e69a332f0 2024-05-04T03:19:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.425+0000 7f5442ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e69a3b4d0 0x560e69e04ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.425+0000 7f5442ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e69a3b4d0 0x560e69e04ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36164/0 (socket says 172.21.15.1:36164) 2024-05-04T03:19:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.425+0000 7f5442ffd640 1 -- 172.21.15.1:0/2436812217 learned_addr learned my addr 172.21.15.1:0/2436812217 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.425+0000 7f5442ffd640 1 --2- 172.21.15.1:0/2436812217 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560e69a5c1b0 0x560e69dfda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.425+0000 7f54437fe640 1 --2- 172.21.15.1:0/2436812217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e69a332f0 0x560e69dfdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.425+0000 7f5442ffd640 1 -- 172.21.15.1:0/2436812217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e69a332f0 msgr2=0x560e69dfdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.425+0000 7f5442ffd640 1 --2- 172.21.15.1:0/2436812217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e69a332f0 0x560e69dfdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.425+0000 7f5442ffd640 1 -- 172.21.15.1:0/2436812217 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560e69a5c1b0 msgr2=0x560e69dfda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.425+0000 7f5442ffd640 1 --2- 172.21.15.1:0/2436812217 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560e69a5c1b0 0x560e69dfda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.425+0000 7f5442ffd640 1 -- 172.21.15.1:0/2436812217 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e69a74210 con 0x560e69a3b4d0 2024-05-04T03:19:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.425+0000 7f5440ff9640 1 --2- 172.21.15.1:0/1444890131 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f542c0b2920 0x7f542c0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.425+0000 7f54437fe640 1 --2- 172.21.15.1:0/1444890131 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f542c0b2920 0x7f542c0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.425+0000 7f5442ffd640 1 --2- 172.21.15.1:0/2436812217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e69a3b4d0 0x560e69e04ee0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x560e69dfec30 tx=0x7f5438009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7435409ed7a3210 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.425+0000 7f5440ff9640 1 -- 172.21.15.1:0/1444890131 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f543c0e25c0 con 0x7f5444109a00 2024-05-04T03:19:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.425+0000 7f5429ffb640 1 -- 172.21.15.1:0/2436812217 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438005020 con 0x560e69a3b4d0 2024-05-04T03:19:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.426+0000 7f5429ffb640 1 -- 172.21.15.1:0/2436812217 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54380047a0 con 0x560e69a3b4d0 2024-05-04T03:19:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.426+0000 7f5429ffb640 1 -- 172.21.15.1:0/2436812217 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543800f990 con 0x560e69a3b4d0 2024-05-04T03:19:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.426+0000 7f5450db1740 1 -- 172.21.15.1:0/2436812217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e69a3b4d0 msgr2=0x560e69e04ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.426+0000 7f54437fe640 1 --2- 172.21.15.1:0/1444890131 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f542c0b2920 0x7f542c0b4e30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5434000c00 tx=0x7f54340314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.426+0000 7f5450db1740 1 --2- 172.21.15.1:0/2436812217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e69a3b4d0 0x560e69e04ee0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x560e69dfec30 tx=0x7f5438009650 comp rx=0 tx=0).stop 2024-05-04T03:19:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.426+0000 7f5450db1740 1 -- 172.21.15.1:0/2436812217 shutdown_connections 2024-05-04T03:19:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.426+0000 7f5450db1740 1 --2- 172.21.15.1:0/2436812217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e69a3b4d0 0x560e69e04ee0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.426+0000 7f5450db1740 1 --2- 172.21.15.1:0/2436812217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e69a332f0 0x560e69dfdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.426+0000 7f5450db1740 1 --2- 172.21.15.1:0/2436812217 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560e69a5c1b0 0x560e69dfda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.426+0000 7f5450db1740 1 -- 172.21.15.1:0/2436812217 >> 172.21.15.1:0/2436812217 conn(0x560e69a50300 msgr2=0x560e69df93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.427+0000 7f5450db1740 1 -- 172.21.15.1:0/2436812217 shutdown_connections 2024-05-04T03:19:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.427+0000 7f5450db1740 1 -- 172.21.15.1:0/2436812217 wait complete. 2024-05-04T03:19:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.427+0000 7f5450db1740 1 -- start start 2024-05-04T03:19:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.427+0000 7f5450db1740 1 Processor -- start 2024-05-04T03:19:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.427+0000 7f5450db1740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560e69a332f0 0x560e69e07710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.427+0000 7f5450db1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e69a3b4d0 0x560e69e07c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.427+0000 7f5450db1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e69a5c1b0 0x560e69e0ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.427+0000 7f5450db1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560e69e05930 con 0x560e69a3b4d0 2024-05-04T03:19:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.427+0000 7f5450db1740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x560e69d1f370 con 0x560e69a332f0 2024-05-04T03:19:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.427+0000 7f5450db1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560e69e05ab0 con 0x560e69a5c1b0 2024-05-04T03:19:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.427+0000 7f5450db1740 10 client.0 _ll_register_callbacks cb 0x560e69a73a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.428+0000 7f5442ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e69a3b4d0 0x560e69e07c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.428+0000 7f5442ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e69a3b4d0 0x560e69e07c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36178/0 (socket says 172.21.15.1:36178) 2024-05-04T03:19:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.428+0000 7f5442ffd640 1 -- 172.21.15.1:0/189541761 learned_addr learned my addr 172.21.15.1:0/189541761 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.428+0000 7f54437fe640 1 --2- 172.21.15.1:0/189541761 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560e69a332f0 0x560e69e07710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.428+0000 7f5443fff640 1 --2- 172.21.15.1:0/189541761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e69a5c1b0 0x560e69e0ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.428+0000 7f5442ffd640 1 -- 172.21.15.1:0/189541761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e69a5c1b0 msgr2=0x560e69e0ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.428+0000 7f5442ffd640 1 --2- 172.21.15.1:0/189541761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e69a5c1b0 0x560e69e0ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.428+0000 7f5442ffd640 1 -- 172.21.15.1:0/189541761 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560e69a332f0 msgr2=0x560e69e07710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.428+0000 7f5442ffd640 1 --2- 172.21.15.1:0/189541761 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560e69a332f0 0x560e69e07710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.428+0000 7f5442ffd640 1 -- 172.21.15.1:0/189541761 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e69b40620 con 0x560e69a3b4d0 2024-05-04T03:19:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.428+0000 7f5443fff640 1 --2- 172.21.15.1:0/189541761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e69a5c1b0 0x560e69e0ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:19:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.428+0000 7f54437fe640 1 --2- 172.21.15.1:0/189541761 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560e69a332f0 0x560e69e07710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:19:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.428+0000 7f5442ffd640 1 --2- 172.21.15.1:0/189541761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e69a3b4d0 0x560e69e07c50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5438014440 tx=0x7f5438004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa221b34a1f12f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.428+0000 7f5422ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:19:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.428+0000 7f5422ffd640 1 -- 172.21.15.1:0/189541761 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438005430 con 0x560e69a3b4d0 2024-05-04T03:19:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.428+0000 7f5422ffd640 1 -- 172.21.15.1:0/189541761 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f54380055d0 con 0x560e69a3b4d0 2024-05-04T03:19:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.428+0000 7f5450db1740 1 -- 172.21.15.1:0/189541761 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54380047a0 con 0x560e69a3b4d0 2024-05-04T03:19:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.428+0000 7f5422ffd640 1 -- 172.21.15.1:0/189541761 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5438014530 con 0x560e69a3b4d0 2024-05-04T03:19:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.428+0000 7f5450db1740 10 client.15255 fetch_fsmap learned FSMap version 26 2024-05-04T03:19:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.428+0000 7f5450db1740 1 -- 172.21.15.1:0/189541761 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560e69e11ae0 con 0x560e69a3b4d0 2024-05-04T03:19:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.429+0000 7f5422ffd640 1 -- 172.21.15.1:0/189541761 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54380047a0 con 0x560e69a3b4d0 2024-05-04T03:19:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.429+0000 7f5422ffd640 1 -- 172.21.15.1:0/189541761 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f5438012680 con 0x560e69a3b4d0 2024-05-04T03:19:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.429+0000 7f5450db1740 10 client.15255 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:19:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.429+0000 7f5450db1740 10 client.15255 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:19:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.429+0000 7f5450db1740 20 client.15255 populate_metadata read hostname 'smithi001' 2024-05-04T03:19:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.429+0000 7f5450db1740 1 --2- 172.21.15.1:0/189541761 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x560e69a1ad80 0x560e69e15120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.429+0000 7f5450db1740 4 client.15255 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:19:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.429+0000 7f5450db1740 1 -- 172.21.15.1:0/189541761 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e69d701e0 con 0x560e69a1ad80 2024-05-04T03:19:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.429+0000 7f54437fe640 1 --2- 172.21.15.1:0/189541761 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x560e69a1ad80 0x560e69e15120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.430+0000 7f5422ffd640 10 client.15255 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:19:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.430+0000 7f54437fe640 1 --2- 172.21.15.1:0/189541761 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x560e69a1ad80 0x560e69e15120 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9ac09d5b4c62581 server_cookie=112ca0a9cd1dc269 in_seq=0 out_seq=0 2024-05-04T03:19:25.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.431+0000 7f5422ffd640 1 -- 172.21.15.1:0/189541761 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560e69d701e0 con 0x560e69a1ad80 2024-05-04T03:19:25.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.431+0000 7f5422ffd640 10 client.15255 handle_command_reply: tid=0 2024-05-04T03:19:25.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.431+0000 7f5450db1740 1 client.15255 shutdown 2024-05-04T03:19:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.431+0000 7f5450db1740 1 -- 172.21.15.1:0/189541761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e69a3b4d0 msgr2=0x560e69e07c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.431+0000 7f5450db1740 1 --2- 172.21.15.1:0/189541761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e69a3b4d0 0x560e69e07c50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5438014440 tx=0x7f5438004c00 comp rx=0 tx=0).stop 2024-05-04T03:19:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.431+0000 7f5450db1740 1 -- 172.21.15.1:0/189541761 shutdown_connections 2024-05-04T03:19:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.431+0000 7f5450db1740 1 --2- 172.21.15.1:0/189541761 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x560e69a1ad80 0x560e69e15120 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.431+0000 7f5450db1740 1 --2- 172.21.15.1:0/189541761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e69a5c1b0 0x560e69e0ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.431+0000 7f5422ffd640 0 client.15255 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:19:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.431+0000 7f5450db1740 1 --2- 172.21.15.1:0/189541761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e69a3b4d0 0x560e69e07c50 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.431+0000 7f5450db1740 1 --2- 172.21.15.1:0/189541761 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560e69a332f0 0x560e69e07710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.431+0000 7f5450db1740 1 -- 172.21.15.1:0/189541761 >> 172.21.15.1:0/189541761 conn(0x560e69a50300 msgr2=0x560e69dfc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.431+0000 7f5450db1740 1 -- 172.21.15.1:0/189541761 shutdown_connections 2024-05-04T03:19:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.431+0000 7f5450db1740 1 -- 172.21.15.1:0/189541761 wait complete. 2024-05-04T03:19:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.431+0000 7f5450db1740 20 client.15255 trim_cache size 0 max 16384 2024-05-04T03:19:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.446+0000 7f5450db1740 1 Processor -- start 2024-05-04T03:19:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.446+0000 7f5450db1740 1 -- start start 2024-05-04T03:19:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.447+0000 7f5450db1740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560e69a332f0 0x560e69df8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.447+0000 7f5450db1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e69a3b4d0 0x560e69df9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.447+0000 7f5450db1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e69a5c1b0 0x560e69e0ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.447+0000 7f5450db1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f54441afb80 con 0x560e69a3b4d0 2024-05-04T03:19:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.447+0000 7f5450db1740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f54441ad450 con 0x560e69a332f0 2024-05-04T03:19:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.447+0000 7f5450db1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f54441ab2c0 con 0x560e69a5c1b0 2024-05-04T03:19:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.447+0000 7f5442ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e69a3b4d0 0x560e69df9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.447+0000 7f5442ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e69a3b4d0 0x560e69df9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36194/0 (socket says 172.21.15.1:36194) 2024-05-04T03:19:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.447+0000 7f5442ffd640 1 -- 172.21.15.1:0/3461297765 learned_addr learned my addr 172.21.15.1:0/3461297765 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.447+0000 7f5443fff640 1 --2- 172.21.15.1:0/3461297765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e69a5c1b0 0x560e69e0ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.447+0000 7f54437fe640 1 --2- 172.21.15.1:0/3461297765 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560e69a332f0 0x560e69df8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.447+0000 7f5442ffd640 1 -- 172.21.15.1:0/3461297765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e69a5c1b0 msgr2=0x560e69e0ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.447+0000 7f5442ffd640 1 --2- 172.21.15.1:0/3461297765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e69a5c1b0 0x560e69e0ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.447+0000 7f5442ffd640 1 -- 172.21.15.1:0/3461297765 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560e69a332f0 msgr2=0x560e69df8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.447+0000 7f5442ffd640 1 --2- 172.21.15.1:0/3461297765 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560e69a332f0 0x560e69df8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.447+0000 7f5442ffd640 1 -- 172.21.15.1:0/3461297765 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e69df9d00 con 0x560e69a3b4d0 2024-05-04T03:19:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.447+0000 7f5443fff640 1 --2- 172.21.15.1:0/3461297765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e69a5c1b0 0x560e69e0ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:19:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.447+0000 7f54437fe640 1 --2- 172.21.15.1:0/3461297765 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560e69a332f0 0x560e69df8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:19:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.447+0000 7f5442ffd640 1 --2- 172.21.15.1:0/3461297765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e69a3b4d0 0x560e69df9200 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5438012b70 tx=0x7f5438009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969624975c22804d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.447+0000 7f54297fa640 1 -- 172.21.15.1:0/3461297765 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438036030 con 0x560e69a3b4d0 2024-05-04T03:19:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.447+0000 7f54297fa640 1 -- 172.21.15.1:0/3461297765 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5438009530 con 0x560e69a3b4d0 2024-05-04T03:19:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.447+0000 7f54297fa640 1 -- 172.21.15.1:0/3461297765 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438004330 con 0x560e69a3b4d0 2024-05-04T03:19:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.448+0000 7f5450db1740 1 -- 172.21.15.1:0/3461297765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e69a3b4d0 msgr2=0x560e69df9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.448+0000 7f5450db1740 1 --2- 172.21.15.1:0/3461297765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e69a3b4d0 0x560e69df9200 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5438012b70 tx=0x7f5438009ad0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.448+0000 7f5450db1740 1 -- 172.21.15.1:0/3461297765 shutdown_connections 2024-05-04T03:19:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.448+0000 7f5450db1740 1 --2- 172.21.15.1:0/3461297765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e69a5c1b0 0x560e69e0ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.448+0000 7f5450db1740 1 --2- 172.21.15.1:0/3461297765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e69a3b4d0 0x560e69df9200 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.448+0000 7f5450db1740 1 --2- 172.21.15.1:0/3461297765 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560e69a332f0 0x560e69df8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.448+0000 7f5450db1740 1 -- 172.21.15.1:0/3461297765 >> 172.21.15.1:0/3461297765 conn(0x560e69a50300 msgr2=0x560e69e01b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.448+0000 7f5450db1740 1 -- 172.21.15.1:0/3461297765 shutdown_connections 2024-05-04T03:19:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.448+0000 7f5450db1740 1 -- 172.21.15.1:0/3461297765 wait complete. 2024-05-04T03:19:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.449+0000 7f5450db1740 1 -- start start 2024-05-04T03:19:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.449+0000 7f5450db1740 1 Processor -- start 2024-05-04T03:19:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.449+0000 7f5450db1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e69a332f0 0x560e69e0c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.449+0000 7f5450db1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e69a3b4d0 0x560e69e0cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.449+0000 7f5450db1740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560e69a5c1b0 0x560e69e0dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.449+0000 7f5450db1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f54441ab910 con 0x560e69a3b4d0 2024-05-04T03:19:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.449+0000 7f5450db1740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f54441acd80 con 0x560e69a5c1b0 2024-05-04T03:19:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.449+0000 7f5450db1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f54441abfe0 con 0x560e69a332f0 2024-05-04T03:19:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.449+0000 7f5450db1740 10 client.0 _ll_register_callbacks cb 0x7f5438009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.449+0000 7f5442ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e69a3b4d0 0x560e69e0cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.449+0000 7f5442ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e69a3b4d0 0x560e69e0cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36196/0 (socket says 172.21.15.1:36196) 2024-05-04T03:19:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.449+0000 7f5442ffd640 1 -- 172.21.15.1:0/3584058805 learned_addr learned my addr 172.21.15.1:0/3584058805 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.449+0000 7f54437fe640 1 --2- 172.21.15.1:0/3584058805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e69a332f0 0x560e69e0c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.449+0000 7f5443fff640 1 --2- 172.21.15.1:0/3584058805 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560e69a5c1b0 0x560e69e0dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.449+0000 7f5442ffd640 1 -- 172.21.15.1:0/3584058805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e69a332f0 msgr2=0x560e69e0c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.449+0000 7f5442ffd640 1 --2- 172.21.15.1:0/3584058805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e69a332f0 0x560e69e0c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.449+0000 7f5442ffd640 1 -- 172.21.15.1:0/3584058805 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560e69a5c1b0 msgr2=0x560e69e0dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.450+0000 7f5442ffd640 1 --2- 172.21.15.1:0/3584058805 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560e69a5c1b0 0x560e69e0dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.450+0000 7f5442ffd640 1 -- 172.21.15.1:0/3584058805 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e69e0e1f0 con 0x560e69a3b4d0 2024-05-04T03:19:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.450+0000 7f5442ffd640 1 --2- 172.21.15.1:0/3584058805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e69a3b4d0 0x560e69e0cd00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f54380046a0 tx=0x7f5438009bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31137f85a9e4e2ba server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.450+0000 7f5422ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:19:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.450+0000 7f5422ffd640 1 -- 172.21.15.1:0/3584058805 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543803f020 con 0x560e69a3b4d0 2024-05-04T03:19:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.450+0000 7f5422ffd640 1 -- 172.21.15.1:0/3584058805 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f5438012450 con 0x560e69a3b4d0 2024-05-04T03:19:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.450+0000 7f5450db1740 1 -- 172.21.15.1:0/3584058805 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5438009530 con 0x560e69a3b4d0 2024-05-04T03:19:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.450+0000 7f5422ffd640 1 -- 172.21.15.1:0/3584058805 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5438036070 con 0x560e69a3b4d0 2024-05-04T03:19:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.450+0000 7f5450db1740 10 client.15261 fetch_fsmap learned FSMap version 26 2024-05-04T03:19:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.450+0000 7f5450db1740 1 -- 172.21.15.1:0/3584058805 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560e69e0f3f0 con 0x560e69a3b4d0 2024-05-04T03:19:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.450+0000 7f5422ffd640 1 -- 172.21.15.1:0/3584058805 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5438009530 con 0x560e69a3b4d0 2024-05-04T03:19:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.450+0000 7f5422ffd640 1 -- 172.21.15.1:0/3584058805 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f543800fc30 con 0x560e69a3b4d0 2024-05-04T03:19:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.450+0000 7f5450db1740 10 client.15261 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:19:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.450+0000 7f5450db1740 10 client.15261 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:19:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.450+0000 7f5450db1740 20 client.15261 populate_metadata read hostname 'smithi001' 2024-05-04T03:19:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.451+0000 7f5450db1740 1 --2- 172.21.15.1:0/3584058805 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x560e699f01c0 0x560e69e7fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.451+0000 7f5450db1740 4 client.15261 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:19:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.451+0000 7f5450db1740 1 -- 172.21.15.1:0/3584058805 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560e69d6a050 con 0x560e699f01c0 2024-05-04T03:19:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.451+0000 7f54437fe640 1 --2- 172.21.15.1:0/3584058805 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x560e699f01c0 0x560e69e7fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.451+0000 7f54437fe640 1 --2- 172.21.15.1:0/3584058805 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x560e699f01c0 0x560e69e7fa30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c1a3a650b76eeac server_cookie=21a25c044e054af4 in_seq=0 out_seq=0 2024-05-04T03:19:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.451+0000 7f5422ffd640 10 client.15261 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:19:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.456+0000 7f5422ffd640 1 -- 172.21.15.1:0/3584058805 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560e69d6a050 con 0x560e699f01c0 2024-05-04T03:19:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.456+0000 7f5422ffd640 10 client.15261 handle_command_reply: tid=0 2024-05-04T03:19:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.456+0000 7f5450db1740 1 client.15261 shutdown 2024-05-04T03:19:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.456+0000 7f5450db1740 1 -- 172.21.15.1:0/3584058805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e69a3b4d0 msgr2=0x560e69e0cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.456+0000 7f5450db1740 1 --2- 172.21.15.1:0/3584058805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e69a3b4d0 0x560e69e0cd00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f54380046a0 tx=0x7f5438009bc0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.456+0000 7f5450db1740 1 -- 172.21.15.1:0/3584058805 shutdown_connections 2024-05-04T03:19:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.456+0000 7f5450db1740 1 --2- 172.21.15.1:0/3584058805 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x560e699f01c0 0x560e69e7fa30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.456+0000 7f5450db1740 1 --2- 172.21.15.1:0/3584058805 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560e69a5c1b0 0x560e69e0dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.456+0000 7f5450db1740 1 --2- 172.21.15.1:0/3584058805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e69a3b4d0 0x560e69e0cd00 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.456+0000 7f5422ffd640 0 client.15261 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:19:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.456+0000 7f5450db1740 1 --2- 172.21.15.1:0/3584058805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e69a332f0 0x560e69e0c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.456+0000 7f5450db1740 1 -- 172.21.15.1:0/3584058805 >> 172.21.15.1:0/3584058805 conn(0x560e69a50300 msgr2=0x560e69dfc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.456+0000 7f5450db1740 1 -- 172.21.15.1:0/3584058805 shutdown_connections 2024-05-04T03:19:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.456+0000 7f5450db1740 1 -- 172.21.15.1:0/3584058805 wait complete. 2024-05-04T03:19:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.456+0000 7f5450db1740 20 client.15261 trim_cache size 0 max 16384 2024-05-04T03:19:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:19:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:19:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:19:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:19:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.457+0000 7f5449f00640 1 -- 172.21.15.1:0/1444890131 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f542c0b2920 msgr2=0x7f542c0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.457+0000 7f5449f00640 1 --2- 172.21.15.1:0/1444890131 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f542c0b2920 0x7f542c0b4e30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5434000c00 tx=0x7f54340314a0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.457+0000 7f5449f00640 1 -- 172.21.15.1:0/1444890131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5444109a00 msgr2=0x7f5444102770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.457+0000 7f5449f00640 1 --2- 172.21.15.1:0/1444890131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5444109a00 0x7f5444102770 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f543c00dac0 tx=0x7f543c00df80 comp rx=0 tx=0).stop 2024-05-04T03:19:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.457+0000 7f5449f00640 1 -- 172.21.15.1:0/1444890131 shutdown_connections 2024-05-04T03:19:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.457+0000 7f5449f00640 1 --2- 172.21.15.1:0/1444890131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5444109a00 0x7f5444102770 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.457+0000 7f5449f00640 1 --2- 172.21.15.1:0/1444890131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54441070a0 0x7f5444102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.457+0000 7f5449f00640 1 --2- 172.21.15.1:0/1444890131 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f542c0b2920 0x7f542c0b4e30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.457+0000 7f5449f00640 1 --2- 172.21.15.1:0/1444890131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5444104740 0x7f5444103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.457+0000 7f5449f00640 1 -- 172.21.15.1:0/1444890131 >> 172.21.15.1:0/1444890131 conn(0x7f54440fdd60 msgr2=0x7f5444105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.457+0000 7f5449f00640 1 -- 172.21.15.1:0/1444890131 shutdown_connections 2024-05-04T03:19:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:25.457+0000 7f5449f00640 1 -- 172.21.15.1:0/1444890131 wait complete. 2024-05-04T03:19:26.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:26 smithi001 ceph-mon[29731]: pgmap v399: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-04T03:19:26.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:26 smithi077 ceph-mon[33531]: pgmap v399: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-04T03:19:26.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:26 smithi072 ceph-mon[33332]: pgmap v399: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-04T03:19:28.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:28 smithi001 ceph-mon[29731]: pgmap v400: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-04T03:19:28.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:28 smithi077 ceph-mon[33531]: pgmap v400: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-04T03:19:28.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:28 smithi072 ceph-mon[33332]: pgmap v400: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-04T03:19:30.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:30 smithi001 ceph-mon[29731]: pgmap v401: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-04T03:19:30.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:30 smithi077 ceph-mon[33531]: pgmap v401: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-04T03:19:30.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:30 smithi072 ceph-mon[33332]: pgmap v401: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-04T03:19:31.146 INFO:tasks.quiescer.fs.[cephfs]:Total duration (58.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-04T03:19:31.146 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.25 2024-05-04T03:19:31.147 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:19:31.413 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:19:32.684 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:32 smithi001 ceph-mon[29731]: pgmap v402: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-04T03:19:32.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.922+0000 7f63798e2700 1 -- 172.21.15.1:0/2251209518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6374108f00 msgr2=0x7f6374109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:32.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.922+0000 7f63798e2700 1 --2- 172.21.15.1:0/2251209518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6374108f00 0x7f6374109310 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f63640099c0 tx=0x7f6364009cd0 comp rx=0 tx=0).stop 2024-05-04T03:19:32.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.922+0000 7f63798e2700 1 -- 172.21.15.1:0/2251209518 shutdown_connections 2024-05-04T03:19:32.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.923+0000 7f63798e2700 1 --2- 172.21.15.1:0/2251209518 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6374109850 0x7f63741111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:32.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.923+0000 7f63798e2700 1 --2- 172.21.15.1:0/2251209518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63741045d0 0x7f63741049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:32.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.923+0000 7f63798e2700 1 --2- 172.21.15.1:0/2251209518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6374108f00 0x7f6374109310 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:32.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.923+0000 7f63798e2700 1 -- 172.21.15.1:0/2251209518 >> 172.21.15.1:0/2251209518 conn(0x7f63740ffea0 msgr2=0x7f63741022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:32.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.923+0000 7f63798e2700 1 -- 172.21.15.1:0/2251209518 shutdown_connections 2024-05-04T03:19:32.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.923+0000 7f63798e2700 1 -- 172.21.15.1:0/2251209518 wait complete. 2024-05-04T03:19:32.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.923+0000 7f63798e2700 1 Processor -- start 2024-05-04T03:19:32.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.923+0000 7f63798e2700 1 -- start start 2024-05-04T03:19:32.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.924+0000 7f63798e2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f63741045d0 0x7f6374076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:32.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.924+0000 7f63798e2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6374108f00 0x7f6374077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:32.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.924+0000 7f63798e2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6374109850 0x7f6374077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:32.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.924+0000 7f63798e2700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6374114460 con 0x7f6374108f00 2024-05-04T03:19:32.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.924+0000 7f63798e2700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f63741142e0 con 0x7f63741045d0 2024-05-04T03:19:32.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.924+0000 7f63798e2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f63741145e0 con 0x7f6374109850 2024-05-04T03:19:32.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.924+0000 7f63727fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6374108f00 0x7f6374077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:32.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.924+0000 7f6372ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f63741045d0 0x7f6374076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:32.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.924+0000 7f63727fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6374108f00 0x7f6374077110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41854/0 (socket says 172.21.15.1:41854) 2024-05-04T03:19:32.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.924+0000 7f63727fc700 1 -- 172.21.15.1:0/1168741379 learned_addr learned my addr 172.21.15.1:0/1168741379 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:32.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.924+0000 7f63737fe700 1 --2- 172.21.15.1:0/1168741379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6374109850 0x7f6374077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:32.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.924+0000 7f63727fc700 1 -- 172.21.15.1:0/1168741379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6374109850 msgr2=0x7f6374077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:32.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.924+0000 7f63727fc700 1 --2- 172.21.15.1:0/1168741379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6374109850 0x7f6374077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:32.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.924+0000 7f63727fc700 1 -- 172.21.15.1:0/1168741379 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f63741045d0 msgr2=0x7f6374076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:32.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.924+0000 7f63727fc700 1 --2- 172.21.15.1:0/1168741379 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f63741045d0 0x7f6374076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:32.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.924+0000 7f63727fc700 1 -- 172.21.15.1:0/1168741379 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63741a90c0 con 0x7f6374108f00 2024-05-04T03:19:32.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.925+0000 7f63727fc700 1 --2- 172.21.15.1:0/1168741379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6374108f00 0x7f6374077110 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6364006010 tx=0x7f6364005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:32.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.925+0000 7f63788e0700 1 -- 172.21.15.1:0/1168741379 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364027070 con 0x7f6374108f00 2024-05-04T03:19:32.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.925+0000 7f63788e0700 1 -- 172.21.15.1:0/1168741379 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6364003f20 con 0x7f6374108f00 2024-05-04T03:19:32.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.925+0000 7f63798e2700 1 -- 172.21.15.1:0/1168741379 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63741a9350 con 0x7f6374108f00 2024-05-04T03:19:32.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.925+0000 7f63788e0700 1 -- 172.21.15.1:0/1168741379 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364022bd0 con 0x7f6374108f00 2024-05-04T03:19:32.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.925+0000 7f63798e2700 1 -- 172.21.15.1:0/1168741379 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63741a9800 con 0x7f6374108f00 2024-05-04T03:19:32.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.926+0000 7f63798e2700 1 -- 172.21.15.1:0/1168741379 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63740512c0 con 0x7f6374108f00 2024-05-04T03:19:32.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:32 smithi077 ceph-mon[33531]: pgmap v402: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-04T03:19:32.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.928+0000 7f63788e0700 1 -- 172.21.15.1:0/1168741379 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7f636402a070 con 0x7f6374108f00 2024-05-04T03:19:32.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.929+0000 7f63788e0700 1 --2- 172.21.15.1:0/1168741379 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f63600b2990 0x7f63600b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:32.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.929+0000 7f63788e0700 1 -- 172.21.15.1:0/1168741379 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f63640e3610 con 0x7f6374108f00 2024-05-04T03:19:32.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.929+0000 7f6372ffd700 1 --2- 172.21.15.1:0/1168741379 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f63600b2990 0x7f63600b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:32.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.930+0000 7f6372ffd700 1 --2- 172.21.15.1:0/1168741379 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f63600b2990 0x7f63600b4ea0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f635c00ac50 tx=0x7f635c00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:32.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:32.933+0000 7f63788e0700 1 -- 172.21.15.1:0/1168741379 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6364031080 con 0x7f6374108f00 2024-05-04T03:19:32.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:32 smithi072 ceph-mon[33332]: pgmap v402: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-04T03:19:33.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:33.191+0000 7f63798e2700 1 -- 172.21.15.1:0/1168741379 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f637402e5f0 con 0x7f6374108f00 2024-05-04T03:19:33.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:33.192+0000 7f63788e0700 1 -- 172.21.15.1:0/1168741379 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11844 (secure 0 0 0) 0x7f6364013070 con 0x7f6374108f00 2024-05-04T03:19:33.194 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:19:33.194 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2186681910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2186681910},{"type":"v1","addr":"172.21.15.1:6835","nonce":2186681910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2747191047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2747191047},{"type":"v1","addr":"172.21.15.1:6837","nonce":2747191047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4115120540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4115120540},{"type":"v1","addr":"172.21.15.1:6839","nonce":4115120540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3405709780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3405709780},{"type":"v1","addr":"172.21.15.1:6841","nonce":3405709780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/3816098761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":3816098761},{"type":"v1","addr":"172.21.15.72:6833","nonce":3816098761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/379026128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":379026128},{"type":"v1","addr":"172.21.15.77:6833","nonce":379026128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:15:49.823471+0000","modified":"2024-05-04T03:18:45.058419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24472,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/3930984976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3930984976},{"type":"v1","addr":"172.21.15.72:6835","nonce":3930984976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/4279951305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4279951305},{"type":"v1","addr":"172.21.15.77:6835","nonce":4279951305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2003521417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2003521417},{"type":"v1","addr":"172.21.15.77:6837","nonce":2003521417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/1417614203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1417614203},{"type":"v1","addr":"172.21.15.72:6837","nonce":1417614203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.77:6839/922723189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":922723189},{"type":"v1","addr":"172.21.15.77:6839","nonce":922723189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2976050650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2976050650},{"type":"v1","addr":"172.21.15.72:6839","nonce":2976050650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24467,24472,24494]},"id":1}]} 2024-05-04T03:19:33.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:33.194+0000 7f63798e2700 1 -- 172.21.15.1:0/1168741379 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f63600b2990 msgr2=0x7f63600b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:33.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:33.194+0000 7f63798e2700 1 --2- 172.21.15.1:0/1168741379 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f63600b2990 0x7f63600b4ea0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f635c00ac50 tx=0x7f635c00a380 comp rx=0 tx=0).stop 2024-05-04T03:19:33.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:33.194+0000 7f63798e2700 1 -- 172.21.15.1:0/1168741379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6374108f00 msgr2=0x7f6374077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:33.194+0000 7f63798e2700 1 --2- 172.21.15.1:0/1168741379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6374108f00 0x7f6374077110 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6364006010 tx=0x7f6364005f10 comp rx=0 tx=0).stop 2024-05-04T03:19:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:33.194+0000 7f63798e2700 1 -- 172.21.15.1:0/1168741379 shutdown_connections 2024-05-04T03:19:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:33.194+0000 7f63798e2700 1 --2- 172.21.15.1:0/1168741379 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f63741045d0 0x7f6374076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:33.194+0000 7f63798e2700 1 --2- 172.21.15.1:0/1168741379 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f63600b2990 0x7f63600b4ea0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:33.194+0000 7f63798e2700 1 --2- 172.21.15.1:0/1168741379 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6374109850 0x7f6374077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:33.194+0000 7f63798e2700 1 --2- 172.21.15.1:0/1168741379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6374108f00 0x7f6374077110 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:33.194+0000 7f63798e2700 1 -- 172.21.15.1:0/1168741379 >> 172.21.15.1:0/1168741379 conn(0x7f63740ffea0 msgr2=0x7f637410a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:33.195+0000 7f63798e2700 1 -- 172.21.15.1:0/1168741379 shutdown_connections 2024-05-04T03:19:33.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:33.195+0000 7f63798e2700 1 -- 172.21.15.1:0/1168741379 wait complete. 2024-05-04T03:19:33.198 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 26 2024-05-04T03:19:33.796 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:33 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/1168741379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:19:33.824 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 60 --expiration 89.25 --await-for 42.0' 2024-05-04T03:19:33.824 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24494 quiesce db / --timeout 60 --expiration 89.25 --await-for 42.0 2024-05-04T03:19:33.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:33 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/1168741379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:19:33.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:33 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/1168741379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:19:34.081 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:19:34.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:34 smithi001 ceph-mon[29731]: pgmap v403: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-04T03:19:34.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:34 smithi077 ceph-mon[33531]: pgmap v403: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-04T03:19:34.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:34 smithi072 ceph-mon[33332]: pgmap v403: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-04T03:19:35.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.715+0000 7fd8122c9700 1 -- 172.21.15.1:0/1229242208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd80c108740 msgr2=0x7fd80c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:35.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.715+0000 7fd8122c9700 1 --2- 172.21.15.1:0/1229242208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd80c108740 0x7fd80c1100a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd80400b0d0 tx=0x7fd80400b3e0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.715+0000 7fd8122c9700 1 -- 172.21.15.1:0/1229242208 shutdown_connections 2024-05-04T03:19:35.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.715+0000 7fd8122c9700 1 --2- 172.21.15.1:0/1229242208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd80c108740 0x7fd80c1100a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.715+0000 7fd8122c9700 1 --2- 172.21.15.1:0/1229242208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd80c1034c0 0x7fd80c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.715+0000 7fd8122c9700 1 --2- 172.21.15.1:0/1229242208 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd80c107df0 0x7fd80c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.715+0000 7fd8122c9700 1 -- 172.21.15.1:0/1229242208 >> 172.21.15.1:0/1229242208 conn(0x7fd80c0fedc0 msgr2=0x7fd80c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:35.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.715+0000 7fd8122c9700 1 -- 172.21.15.1:0/1229242208 shutdown_connections 2024-05-04T03:19:35.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.715+0000 7fd8122c9700 1 -- 172.21.15.1:0/1229242208 wait complete. 2024-05-04T03:19:35.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.716+0000 7fd8122c9700 1 Processor -- start 2024-05-04T03:19:35.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.716+0000 7fd8122c9700 1 -- start start 2024-05-04T03:19:35.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.716+0000 7fd8122c9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd80c1034c0 0x7fd80c19f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:35.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.716+0000 7fd8122c9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd80c107df0 0x7fd80c19fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:35.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.716+0000 7fd8122c9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd80c108740 0x7fd80c199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:35.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.716+0000 7fd8122c9700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c113370 con 0x7fd80c107df0 2024-05-04T03:19:35.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.716+0000 7fd8122c9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c1131f0 con 0x7fd80c1034c0 2024-05-04T03:19:35.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.716+0000 7fd8122c9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c1134f0 con 0x7fd80c108740 2024-05-04T03:19:35.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.717+0000 7fd80b7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd80c107df0 0x7fd80c19fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:35.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.717+0000 7fd80b7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd80c107df0 0x7fd80c19fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41874/0 (socket says 172.21.15.1:41874) 2024-05-04T03:19:35.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.717+0000 7fd80b7fe700 1 -- 172.21.15.1:0/3884164125 learned_addr learned my addr 172.21.15.1:0/3884164125 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:35.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.717+0000 7fd810866700 1 --2- 172.21.15.1:0/3884164125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd80c108740 0x7fd80c199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:35.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.717+0000 7fd80bfff700 1 --2- 172.21.15.1:0/3884164125 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd80c1034c0 0x7fd80c19f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:35.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.717+0000 7fd80b7fe700 1 -- 172.21.15.1:0/3884164125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd80c108740 msgr2=0x7fd80c199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:35.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.717+0000 7fd80b7fe700 1 --2- 172.21.15.1:0/3884164125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd80c108740 0x7fd80c199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.717+0000 7fd80b7fe700 1 -- 172.21.15.1:0/3884164125 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd80c1034c0 msgr2=0x7fd80c19f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:35.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.717+0000 7fd80b7fe700 1 --2- 172.21.15.1:0/3884164125 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd80c1034c0 0x7fd80c19f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.717+0000 7fd80b7fe700 1 -- 172.21.15.1:0/3884164125 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80c19a0f0 con 0x7fd80c107df0 2024-05-04T03:19:35.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.717+0000 7fd80b7fe700 1 --2- 172.21.15.1:0/3884164125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd80c107df0 0x7fd80c19fdd0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd7f400e970 tx=0x7fd7f400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:35.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.717+0000 7fd8097fa700 1 -- 172.21.15.1:0/3884164125 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4004830 con 0x7fd80c107df0 2024-05-04T03:19:35.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.717+0000 7fd8097fa700 1 -- 172.21.15.1:0/3884164125 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7f400d7f0 con 0x7fd80c107df0 2024-05-04T03:19:35.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.717+0000 7fd8122c9700 1 -- 172.21.15.1:0/3884164125 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd80c19a3e0 con 0x7fd80c107df0 2024-05-04T03:19:35.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.717+0000 7fd8097fa700 1 -- 172.21.15.1:0/3884164125 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4005330 con 0x7fd80c107df0 2024-05-04T03:19:35.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.717+0000 7fd8122c9700 1 -- 172.21.15.1:0/3884164125 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd80c06b610 con 0x7fd80c107df0 2024-05-04T03:19:35.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.720+0000 7fd8097fa700 1 -- 172.21.15.1:0/3884164125 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7fd7f4005510 con 0x7fd80c107df0 2024-05-04T03:19:35.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.721+0000 7fd8097fa700 1 --2- 172.21.15.1:0/3884164125 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd7f80b2910 0x7fd7f80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:35.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.721+0000 7fd80bfff700 1 --2- 172.21.15.1:0/3884164125 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd7f80b2910 0x7fd7f80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:35.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.721+0000 7fd820cddb80 1 Processor -- start 2024-05-04T03:19:35.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.721+0000 7fd8097fa700 1 -- 172.21.15.1:0/3884164125 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fd7f40e1910 con 0x7fd80c107df0 2024-05-04T03:19:35.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.721+0000 7fd820cddb80 1 -- start start 2024-05-04T03:19:35.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.721+0000 7fd820cddb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562865c77b60 0x562865c77f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:35.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.721+0000 7fd820cddb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562865c784b0 0x562865c82790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:35.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.721+0000 7fd820cddb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562865c82cd0 0x562865c850f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:35.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.721+0000 7fd820cddb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562865b9aad0 con 0x562865c82cd0 2024-05-04T03:19:35.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.721+0000 7fd820cddb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x562865afff30 con 0x562865c77b60 2024-05-04T03:19:35.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.722+0000 7fd820cddb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562865b96a90 con 0x562865c784b0 2024-05-04T03:19:35.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.722+0000 7fd80bfff700 1 --2- 172.21.15.1:0/3884164125 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd7f80b2910 0x7fd7f80b4e20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd80c19abf0 tx=0x7fd7fc00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:35.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.722+0000 7fd810866700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562865c82cd0 0x562865c850f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:35.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.722+0000 7fd80bfff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562865c77b60 0x562865c77f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:35.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.722+0000 7fd810866700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562865c82cd0 0x562865c850f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41890/0 (socket says 172.21.15.1:41890) 2024-05-04T03:19:35.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.722+0000 7fd810866700 1 -- 172.21.15.1:0/2734442304 learned_addr learned my addr 172.21.15.1:0/2734442304 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:35.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.722+0000 7fd80b7fe700 1 --2- 172.21.15.1:0/2734442304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562865c784b0 0x562865c82790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:35.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.722+0000 7fd810866700 1 -- 172.21.15.1:0/2734442304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562865c784b0 msgr2=0x562865c82790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:35.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.722+0000 7fd810866700 1 --2- 172.21.15.1:0/2734442304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562865c784b0 0x562865c82790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.722+0000 7fd810866700 1 -- 172.21.15.1:0/2734442304 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562865c77b60 msgr2=0x562865c77f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:35.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.722+0000 7fd810866700 1 --2- 172.21.15.1:0/2734442304 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562865c77b60 0x562865c77f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.722+0000 7fd810866700 1 -- 172.21.15.1:0/2734442304 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562865ae7a60 con 0x562865c82cd0 2024-05-04T03:19:35.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.722+0000 7fd80bfff700 1 --2- 172.21.15.1:0/2734442304 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562865c77b60 0x562865c77f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:19:35.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.722+0000 7fd810866700 1 --2- 172.21.15.1:0/2734442304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562865c82cd0 0x562865c850f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd804009cb0 tx=0x7fd80400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91733fa104e22847 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:35.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.722+0000 7fd8027fc700 1 -- 172.21.15.1:0/2734442304 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80402b070 con 0x562865c82cd0 2024-05-04T03:19:35.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.722+0000 7fd8027fc700 1 -- 172.21.15.1:0/2734442304 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd804003a00 con 0x562865c82cd0 2024-05-04T03:19:35.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.722+0000 7fd8027fc700 1 -- 172.21.15.1:0/2734442304 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80401cc00 con 0x562865c82cd0 2024-05-04T03:19:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.723+0000 7fd820cddb80 1 -- 172.21.15.1:0/2734442304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562865c82cd0 msgr2=0x562865c850f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.723+0000 7fd820cddb80 1 --2- 172.21.15.1:0/2734442304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562865c82cd0 0x562865c850f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd804009cb0 tx=0x7fd80400b040 comp rx=0 tx=0).stop 2024-05-04T03:19:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.723+0000 7fd820cddb80 1 -- 172.21.15.1:0/2734442304 shutdown_connections 2024-05-04T03:19:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.723+0000 7fd820cddb80 1 --2- 172.21.15.1:0/2734442304 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562865c77b60 0x562865c77f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.723+0000 7fd820cddb80 1 --2- 172.21.15.1:0/2734442304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562865c82cd0 0x562865c850f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.723+0000 7fd820cddb80 1 --2- 172.21.15.1:0/2734442304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562865c784b0 0x562865c82790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.723+0000 7fd820cddb80 1 -- 172.21.15.1:0/2734442304 >> 172.21.15.1:0/2734442304 conn(0x562865afafd0 msgr2=0x562865c73a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.723+0000 7fd820cddb80 1 -- 172.21.15.1:0/2734442304 shutdown_connections 2024-05-04T03:19:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.723+0000 7fd820cddb80 1 -- 172.21.15.1:0/2734442304 wait complete. 2024-05-04T03:19:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.724+0000 7fd820cddb80 1 -- start start 2024-05-04T03:19:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.724+0000 7fd820cddb80 1 Processor -- start 2024-05-04T03:19:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.724+0000 7fd820cddb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562865c77b60 0x562865c7fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.724+0000 7fd820cddb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562865c784b0 0x562865c80320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.724+0000 7fd820cddb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562865c82cd0 0x562865c80860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.724+0000 7fd820cddb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x562865c85a40 con 0x562865c82cd0 2024-05-04T03:19:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.724+0000 7fd820cddb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x562865b54060 con 0x562865c784b0 2024-05-04T03:19:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.724+0000 7fd820cddb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562865c85bc0 con 0x562865c77b60 2024-05-04T03:19:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.724+0000 7fd820cddb80 10 client.0 _ll_register_callbacks cb 0x562865b32ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.725+0000 7fd810866700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562865c82cd0 0x562865c80860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.725+0000 7fd80bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562865c77b60 0x562865c7fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.725+0000 7fd80b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562865c784b0 0x562865c80320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.725+0000 7fd80bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562865c77b60 0x562865c7fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.1:33922/0 (socket says 172.21.15.1:33922) 2024-05-04T03:19:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.725+0000 7fd810866700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562865c82cd0 0x562865c80860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41892/0 (socket says 172.21.15.1:41892) 2024-05-04T03:19:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.725+0000 7fd80bfff700 1 -- 172.21.15.1:0/3492749769 learned_addr learned my addr 172.21.15.1:0/3492749769 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.725+0000 7fd810866700 1 -- 172.21.15.1:0/3492749769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562865c77b60 msgr2=0x562865c7fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.725+0000 7fd810866700 1 --2- 172.21.15.1:0/3492749769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562865c77b60 0x562865c7fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.725+0000 7fd810866700 1 -- 172.21.15.1:0/3492749769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562865c784b0 msgr2=0x562865c80320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.725+0000 7fd810866700 1 --2- 172.21.15.1:0/3492749769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562865c784b0 0x562865c80320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.725+0000 7fd810866700 1 -- 172.21.15.1:0/3492749769 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562865b518f0 con 0x562865c82cd0 2024-05-04T03:19:35.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.725+0000 7fd810866700 1 --2- 172.21.15.1:0/3492749769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562865c82cd0 0x562865c80860 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd804029040 tx=0x7fd804009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1efcd6b85731d77 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:35.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.725+0000 7fd7eb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:19:35.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.725+0000 7fd7eb7fe700 1 -- 172.21.15.1:0/3492749769 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804033020 con 0x562865c82cd0 2024-05-04T03:19:35.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.725+0000 7fd7eb7fe700 1 -- 172.21.15.1:0/3492749769 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fd804003bb0 con 0x562865c82cd0 2024-05-04T03:19:35.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.725+0000 7fd820cddb80 1 -- 172.21.15.1:0/3492749769 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd804003a00 con 0x562865c82cd0 2024-05-04T03:19:35.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.726+0000 7fd7eb7fe700 1 -- 172.21.15.1:0/3492749769 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd80402b040 con 0x562865c82cd0 2024-05-04T03:19:35.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.726+0000 7fd820cddb80 10 client.15279 fetch_fsmap learned FSMap version 26 2024-05-04T03:19:35.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.726+0000 7fd820cddb80 1 -- 172.21.15.1:0/3492749769 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562865c90760 con 0x562865c82cd0 2024-05-04T03:19:35.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.726+0000 7fd7eb7fe700 1 -- 172.21.15.1:0/3492749769 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd804003a00 con 0x562865c82cd0 2024-05-04T03:19:35.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.726+0000 7fd7eb7fe700 1 -- 172.21.15.1:0/3492749769 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd80402e070 con 0x562865c82cd0 2024-05-04T03:19:35.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.726+0000 7fd820cddb80 10 client.15279 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:19:35.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.726+0000 7fd820cddb80 10 client.15279 resolve_mds: validated gid 24494 aka daemon mds.b 2024-05-04T03:19:35.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.726+0000 7fd820cddb80 20 client.15279 populate_metadata read hostname 'smithi001' 2024-05-04T03:19:35.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.726+0000 7fd820cddb80 1 --2- 172.21.15.1:0/3492749769 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x562865c92c30 0x562865c95050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:35.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.726+0000 7fd820cddb80 4 client.15279 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:19:35.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.726+0000 7fd820cddb80 1 -- 172.21.15.1:0/3492749769 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562865ba84b0 con 0x562865c92c30 2024-05-04T03:19:35.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.726+0000 7fd80bfff700 1 --2- 172.21.15.1:0/3492749769 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x562865c92c30 0x562865c95050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:35.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.727+0000 7fd7eb7fe700 10 client.15279 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:19:35.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.727+0000 7fd80bfff700 1 --2- 172.21.15.1:0/3492749769 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x562865c92c30 0x562865c95050 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cafbae5976fb9312 server_cookie=a71a9ec6ecf4731f in_seq=0 out_seq=0 2024-05-04T03:19:35.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.728+0000 7fd7eb7fe700 1 -- 172.21.15.1:0/3492749769 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562865ba84b0 con 0x562865c92c30 2024-05-04T03:19:35.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.728+0000 7fd7eb7fe700 10 client.15279 handle_command_reply: tid=0 2024-05-04T03:19:35.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.728+0000 7fd820cddb80 1 client.15279 shutdown 2024-05-04T03:19:35.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.728+0000 7fd820cddb80 1 -- 172.21.15.1:0/3492749769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562865c82cd0 msgr2=0x562865c80860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:35.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.728+0000 7fd820cddb80 1 --2- 172.21.15.1:0/3492749769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562865c82cd0 0x562865c80860 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd804029040 tx=0x7fd804009570 comp rx=0 tx=0).stop 2024-05-04T03:19:35.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.728+0000 7fd820cddb80 1 -- 172.21.15.1:0/3492749769 shutdown_connections 2024-05-04T03:19:35.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.728+0000 7fd820cddb80 1 --2- 172.21.15.1:0/3492749769 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x562865c92c30 0x562865c95050 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.728+0000 7fd820cddb80 1 --2- 172.21.15.1:0/3492749769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562865c82cd0 0x562865c80860 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.728+0000 7fd7eb7fe700 0 client.15279 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:19:35.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.728+0000 7fd820cddb80 1 --2- 172.21.15.1:0/3492749769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562865c77b60 0x562865c7fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.728+0000 7fd820cddb80 1 --2- 172.21.15.1:0/3492749769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562865c784b0 0x562865c80320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.728+0000 7fd820cddb80 1 -- 172.21.15.1:0/3492749769 >> 172.21.15.1:0/3492749769 conn(0x562865afafd0 msgr2=0x562865c79600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:35.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.728+0000 7fd820cddb80 1 -- 172.21.15.1:0/3492749769 shutdown_connections 2024-05-04T03:19:35.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.728+0000 7fd820cddb80 1 -- 172.21.15.1:0/3492749769 wait complete. 2024-05-04T03:19:35.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.728+0000 7fd820cddb80 20 client.15279 trim_cache size 0 max 16384 2024-05-04T03:19:35.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.748+0000 7fd820cddb80 1 Processor -- start 2024-05-04T03:19:35.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.748+0000 7fd820cddb80 1 -- start start 2024-05-04T03:19:35.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.748+0000 7fd820cddb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562865c77b60 0x562865c79c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:35.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.748+0000 7fd820cddb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562865c784b0 0x562865c807d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:35.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.748+0000 7fd820cddb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562865c82cd0 0x562865c7f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:35.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.749+0000 7fd820cddb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c1aef90 con 0x562865c77b60 2024-05-04T03:19:35.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.749+0000 7fd820cddb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c1ac820 con 0x562865c784b0 2024-05-04T03:19:35.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.749+0000 7fd820cddb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c1aa690 con 0x562865c82cd0 2024-05-04T03:19:35.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.749+0000 7fd80bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562865c77b60 0x562865c79c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:35.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.749+0000 7fd80bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562865c77b60 0x562865c79c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41894/0 (socket says 172.21.15.1:41894) 2024-05-04T03:19:35.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.749+0000 7fd80bfff700 1 -- 172.21.15.1:0/300614008 learned_addr learned my addr 172.21.15.1:0/300614008 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:35.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.749+0000 7fd80b7fe700 1 --2- 172.21.15.1:0/300614008 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562865c784b0 0x562865c807d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:35.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.749+0000 7fd810866700 1 --2- 172.21.15.1:0/300614008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562865c82cd0 0x562865c7f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:35.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.749+0000 7fd80bfff700 1 -- 172.21.15.1:0/300614008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562865c82cd0 msgr2=0x562865c7f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:35.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.749+0000 7fd80bfff700 1 --2- 172.21.15.1:0/300614008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562865c82cd0 0x562865c7f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.749+0000 7fd80bfff700 1 -- 172.21.15.1:0/300614008 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562865c784b0 msgr2=0x562865c807d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:35.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.749+0000 7fd80bfff700 1 --2- 172.21.15.1:0/300614008 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562865c784b0 0x562865c807d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.749+0000 7fd80bfff700 1 -- 172.21.15.1:0/300614008 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562865c7e350 con 0x562865c77b60 2024-05-04T03:19:35.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.749+0000 7fd810866700 1 --2- 172.21.15.1:0/300614008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562865c82cd0 0x562865c7f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:19:35.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.749+0000 7fd80b7fe700 1 --2- 172.21.15.1:0/300614008 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562865c784b0 0x562865c807d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:19:35.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.749+0000 7fd80bfff700 1 --2- 172.21.15.1:0/300614008 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562865c77b60 0x562865c79c80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00aa10 tx=0x7fd7fc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc59fb10fdacfb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:35.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.749+0000 7fd801ffb700 1 -- 172.21.15.1:0/300614008 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc016070 con 0x562865c77b60 2024-05-04T03:19:35.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.749+0000 7fd801ffb700 1 -- 172.21.15.1:0/300614008 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7fc02e070 con 0x562865c77b60 2024-05-04T03:19:35.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.749+0000 7fd801ffb700 1 -- 172.21.15.1:0/300614008 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc010a20 con 0x562865c77b60 2024-05-04T03:19:35.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.750+0000 7fd820cddb80 1 -- 172.21.15.1:0/300614008 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562865c77b60 msgr2=0x562865c79c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:35.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.750+0000 7fd820cddb80 1 --2- 172.21.15.1:0/300614008 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562865c77b60 0x562865c79c80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00aa10 tx=0x7fd7fc012000 comp rx=0 tx=0).stop 2024-05-04T03:19:35.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.750+0000 7fd820cddb80 1 -- 172.21.15.1:0/300614008 shutdown_connections 2024-05-04T03:19:35.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.750+0000 7fd820cddb80 1 --2- 172.21.15.1:0/300614008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562865c82cd0 0x562865c7f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.750+0000 7fd820cddb80 1 --2- 172.21.15.1:0/300614008 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562865c77b60 0x562865c79c80 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.750+0000 7fd820cddb80 1 --2- 172.21.15.1:0/300614008 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562865c784b0 0x562865c807d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.750+0000 7fd820cddb80 1 -- 172.21.15.1:0/300614008 >> 172.21.15.1:0/300614008 conn(0x562865afafd0 msgr2=0x562865c81440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:35.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.750+0000 7fd820cddb80 1 -- 172.21.15.1:0/300614008 shutdown_connections 2024-05-04T03:19:35.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.750+0000 7fd820cddb80 1 -- 172.21.15.1:0/300614008 wait complete. 2024-05-04T03:19:35.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.751+0000 7fd820cddb80 1 -- start start 2024-05-04T03:19:35.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.751+0000 7fd820cddb80 1 Processor -- start 2024-05-04T03:19:35.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.751+0000 7fd820cddb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562865c77b60 0x562865c98050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:35.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.751+0000 7fd820cddb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562865c784b0 0x562865c98590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:35.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.751+0000 7fd820cddb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562865c82cd0 0x562865c96470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:35.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.751+0000 7fd820cddb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c1aace0 con 0x562865c77b60 2024-05-04T03:19:35.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.751+0000 7fd820cddb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c1ac150 con 0x562865c82cd0 2024-05-04T03:19:35.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.751+0000 7fd820cddb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c1ab3b0 con 0x562865c784b0 2024-05-04T03:19:35.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.751+0000 7fd820cddb80 10 client.0 _ll_register_callbacks cb 0x562865b32ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:35.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.751+0000 7fd80bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562865c77b60 0x562865c98050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:35.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.751+0000 7fd810866700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562865c82cd0 0x562865c96470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:35.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.751+0000 7fd80b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562865c784b0 0x562865c98590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:35.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.751+0000 7fd80bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562865c77b60 0x562865c98050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41904/0 (socket says 172.21.15.1:41904) 2024-05-04T03:19:35.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.751+0000 7fd80bfff700 1 -- 172.21.15.1:0/1945026515 learned_addr learned my addr 172.21.15.1:0/1945026515 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:35.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.752+0000 7fd810866700 1 -- 172.21.15.1:0/1945026515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562865c784b0 msgr2=0x562865c98590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:35.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.752+0000 7fd810866700 1 --2- 172.21.15.1:0/1945026515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562865c784b0 0x562865c98590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.752+0000 7fd810866700 1 -- 172.21.15.1:0/1945026515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562865c77b60 msgr2=0x562865c98050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:35.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.752+0000 7fd810866700 1 --2- 172.21.15.1:0/1945026515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562865c77b60 0x562865c98050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.752+0000 7fd810866700 1 -- 172.21.15.1:0/1945026515 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562865c96b80 con 0x562865c82cd0 2024-05-04T03:19:35.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.752+0000 7fd80bfff700 1 --2- 172.21.15.1:0/1945026515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562865c77b60 0x562865c98050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:19:35.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.752+0000 7fd810866700 1 --2- 172.21.15.1:0/1945026515 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562865c82cd0 0x562865c96470 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd804005bd0 tx=0x7fd8040021d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af9d35cab5faed5c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:35.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.752+0000 7fd7eb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-04T03:19:35.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.752+0000 7fd7eb7fe700 1 -- 172.21.15.1:0/1945026515 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804033070 con 0x562865c82cd0 2024-05-04T03:19:35.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.752+0000 7fd7eb7fe700 1 -- 172.21.15.1:0/1945026515 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fd8040057c0 con 0x562865c82cd0 2024-05-04T03:19:35.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.752+0000 7fd820cddb80 1 -- 172.21.15.1:0/1945026515 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7fc02e070 con 0x562865c82cd0 2024-05-04T03:19:35.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.752+0000 7fd7eb7fe700 1 -- 172.21.15.1:0/1945026515 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd804003a00 con 0x562865c82cd0 2024-05-04T03:19:35.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.752+0000 7fd820cddb80 10 client.25037 fetch_fsmap learned FSMap version 26 2024-05-04T03:19:35.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.752+0000 7fd820cddb80 1 -- 172.21.15.1:0/1945026515 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562865cfda40 con 0x562865c82cd0 2024-05-04T03:19:35.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.753+0000 7fd7eb7fe700 1 -- 172.21.15.1:0/1945026515 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7fc02e070 con 0x562865c82cd0 2024-05-04T03:19:35.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.753+0000 7fd7eb7fe700 1 -- 172.21.15.1:0/1945026515 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd80402e040 con 0x562865c82cd0 2024-05-04T03:19:35.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.753+0000 7fd820cddb80 10 client.25037 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:19:35.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.753+0000 7fd820cddb80 10 client.25037 resolve_mds: validated gid 24494 aka daemon mds.b 2024-05-04T03:19:35.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.753+0000 7fd820cddb80 20 client.25037 populate_metadata read hostname 'smithi001' 2024-05-04T03:19:35.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.753+0000 7fd820cddb80 1 --2- 172.21.15.1:0/1945026515 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x562865c92c30 0x562865d00050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:35.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.753+0000 7fd820cddb80 4 client.25037 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.25, "await_for": 42.0}] 2024-05-04T03:19:35.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.753+0000 7fd820cddb80 1 -- 172.21.15.1:0/1945026515 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.25, "await_for": 42.0}) -- 0x562865ba2550 con 0x562865c92c30 2024-05-04T03:19:35.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.753+0000 7fd80bfff700 1 --2- 172.21.15.1:0/1945026515 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x562865c92c30 0x562865d00050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:35.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.754+0000 7fd80bfff700 1 --2- 172.21.15.1:0/1945026515 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x562865c92c30 0x562865d00050 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ea3d4c4b13db79f server_cookie=a9a1bedd4d6efe12 in_seq=0 out_seq=0 2024-05-04T03:19:35.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.754+0000 7fd7eb7fe700 10 client.25037 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:19:35.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.766+0000 7fd7eb7fe700 1 -- 172.21.15.1:0/1945026515 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x562865ba2550 con 0x562865c92c30 2024-05-04T03:19:35.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.766+0000 7fd7eb7fe700 10 client.25037 handle_command_reply: tid=0 2024-05-04T03:19:35.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.766+0000 7fd820cddb80 1 client.25037 shutdown 2024-05-04T03:19:35.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.767+0000 7fd820cddb80 1 -- 172.21.15.1:0/1945026515 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562865c82cd0 msgr2=0x562865c96470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:35.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.767+0000 7fd820cddb80 1 --2- 172.21.15.1:0/1945026515 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562865c82cd0 0x562865c96470 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd804005bd0 tx=0x7fd8040021d0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.767+0000 7fd820cddb80 1 -- 172.21.15.1:0/1945026515 shutdown_connections 2024-05-04T03:19:35.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.767+0000 7fd820cddb80 1 --2- 172.21.15.1:0/1945026515 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x562865c92c30 0x562865d00050 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.767+0000 7fd820cddb80 1 --2- 172.21.15.1:0/1945026515 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562865c82cd0 0x562865c96470 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.767+0000 7fd820cddb80 1 --2- 172.21.15.1:0/1945026515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562865c77b60 0x562865c98050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.767+0000 7fd7eb7fe700 0 client.25037 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:19:35.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.767+0000 7fd820cddb80 1 --2- 172.21.15.1:0/1945026515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562865c784b0 0x562865c98590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.767+0000 7fd820cddb80 1 -- 172.21.15.1:0/1945026515 >> 172.21.15.1:0/1945026515 conn(0x562865afafd0 msgr2=0x562865c80530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:35.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.767+0000 7fd820cddb80 1 -- 172.21.15.1:0/1945026515 shutdown_connections 2024-05-04T03:19:35.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.767+0000 7fd820cddb80 1 -- 172.21.15.1:0/1945026515 wait complete. 2024-05-04T03:19:35.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.767+0000 7fd820cddb80 20 client.25037 trim_cache size 0 max 16384 2024-05-04T03:19:35.770 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-04T03:19:35.770 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 26, 2024-05-04T03:19:35.770 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24494, 2024-05-04T03:19:35.770 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 2, 2024-05-04T03:19:35.770 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-05-04T03:19:35.770 INFO:teuthology.orchestra.run.smithi001.stdout: "27b22eee": { 2024-05-04T03:19:35.770 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 2, 2024-05-04T03:19:35.771 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-05-04T03:19:35.771 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-05-04T03:19:35.771 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-05-04T03:19:35.771 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-05-04T03:19:35.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:19:35.771 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 60.0, 2024-05-04T03:19:35.771 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 89.2, 2024-05-04T03:19:35.771 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-05-04T03:19:35.771 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-05-04T03:19:35.771 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-05-04T03:19:35.771 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-05-04T03:19:35.771 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-05-04T03:19:35.771 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-05-04T03:19:35.771 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:19:35.772 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:19:35.772 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:19:35.772 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:19:35.772 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:19:35.772 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-04T03:19:35.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.767+0000 7fd8122c9700 1 -- 172.21.15.1:0/3884164125 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd7f80b2910 msgr2=0x7fd7f80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:35.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.767+0000 7fd8122c9700 1 --2- 172.21.15.1:0/3884164125 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd7f80b2910 0x7fd7f80b4e20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd80c19abf0 tx=0x7fd7fc00a380 comp rx=0 tx=0).stop 2024-05-04T03:19:35.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.767+0000 7fd8122c9700 1 -- 172.21.15.1:0/3884164125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd80c107df0 msgr2=0x7fd80c19fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:35.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.767+0000 7fd8122c9700 1 --2- 172.21.15.1:0/3884164125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd80c107df0 0x7fd80c19fdd0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd7f400e970 tx=0x7fd7f400ec80 comp rx=0 tx=0).stop 2024-05-04T03:19:35.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.768+0000 7fd8122c9700 1 -- 172.21.15.1:0/3884164125 shutdown_connections 2024-05-04T03:19:35.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.768+0000 7fd8122c9700 1 --2- 172.21.15.1:0/3884164125 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd80c1034c0 0x7fd80c19f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.768+0000 7fd8122c9700 1 --2- 172.21.15.1:0/3884164125 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd7f80b2910 0x7fd7f80b4e20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.768+0000 7fd8122c9700 1 --2- 172.21.15.1:0/3884164125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd80c108740 0x7fd80c199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.768+0000 7fd8122c9700 1 --2- 172.21.15.1:0/3884164125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd80c107df0 0x7fd80c19fdd0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:35.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.768+0000 7fd8122c9700 1 -- 172.21.15.1:0/3884164125 >> 172.21.15.1:0/3884164125 conn(0x7fd80c0fedc0 msgr2=0x7fd80c107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:35.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.768+0000 7fd8122c9700 1 -- 172.21.15.1:0/3884164125 shutdown_connections 2024-05-04T03:19:35.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:19:35.768+0000 7fd8122c9700 1 -- 172.21.15.1:0/3884164125 wait complete. 2024-05-04T03:19:36.484 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '27b22eee', quiesce took 5.3 seconds. Will release after: 29.25 2024-05-04T03:19:36.882 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:36 smithi001 ceph-mon[29731]: pgmap v404: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-04T03:19:36.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:36 smithi077 ceph-mon[33531]: pgmap v404: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-04T03:19:36.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:36 smithi072 ceph-mon[33332]: pgmap v404: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-04T03:19:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.570+0000 7f2d0cd9d640 1 -- 172.21.15.1:0/1154074837 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d08103c60 msgr2=0x7f2d08104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.570+0000 7f2d0cd9d640 1 --2- 172.21.15.1:0/1154074837 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d08103c60 0x7f2d08104960 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8009900 tx=0x7f2cf802a410 comp rx=0 tx=0).stop 2024-05-04T03:19:38.572 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.570+0000 7f2d0cd9d640 1 -- 172.21.15.1:0/1154074837 shutdown_connections 2024-05-04T03:19:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.570+0000 7f2d0cd9d640 1 --2- 172.21.15.1:0/1154074837 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d08109ba0 0x7f2d0810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.570+0000 7f2d0cd9d640 1 --2- 172.21.15.1:0/1154074837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d08104ea0 0x7f2d081052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.570+0000 7f2d0cd9d640 1 --2- 172.21.15.1:0/1154074837 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d08103c60 0x7f2d08104960 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.570+0000 7f2d0cd9d640 1 -- 172.21.15.1:0/1154074837 >> 172.21.15.1:0/1154074837 conn(0x7f2d080fee70 msgr2=0x7f2d08101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.570+0000 7f2d0cd9d640 1 -- 172.21.15.1:0/1154074837 shutdown_connections 2024-05-04T03:19:38.573 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.570+0000 7f2d0cd9d640 1 -- 172.21.15.1:0/1154074837 wait complete. 2024-05-04T03:19:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.571+0000 7f2d0cd9d640 1 Processor -- start 2024-05-04T03:19:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.571+0000 7f2d0cd9d640 1 -- start start 2024-05-04T03:19:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.571+0000 7f2d0cd9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d08103c60 0x7f2d0819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.571+0000 7f2d0cd9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d08104ea0 0x7f2d0819a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.571+0000 7f2d0cd9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d08109ba0 0x7f2d081a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.571+0000 7f2d0cd9d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0810f760 con 0x7f2d08104ea0 2024-05-04T03:19:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.571+0000 7f2d0cd9d640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0810f5e0 con 0x7f2d08103c60 2024-05-04T03:19:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.571+0000 7f2d0cd9d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0810f8e0 con 0x7f2d08109ba0 2024-05-04T03:19:38.575 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.571+0000 7f2d05d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d08104ea0 0x7f2d0819a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:38.576 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.572+0000 7f2d05d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d08104ea0 0x7f2d0819a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41922/0 (socket says 172.21.15.1:41922) 2024-05-04T03:19:38.576 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.572+0000 7f2d05d9b640 1 -- 172.21.15.1:0/3301444328 learned_addr learned my addr 172.21.15.1:0/3301444328 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:38.576 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.572+0000 7f2d06d9d640 1 --2- 172.21.15.1:0/3301444328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d08109ba0 0x7f2d081a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:38.576 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.572+0000 7f2d0659c640 1 --2- 172.21.15.1:0/3301444328 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d08103c60 0x7f2d0819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:38.576 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.572+0000 7f2d05d9b640 1 -- 172.21.15.1:0/3301444328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d08109ba0 msgr2=0x7f2d081a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:38.576 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.572+0000 7f2d05d9b640 1 --2- 172.21.15.1:0/3301444328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d08109ba0 0x7f2d081a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.576 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.572+0000 7f2d05d9b640 1 -- 172.21.15.1:0/3301444328 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d08103c60 msgr2=0x7f2d0819a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:38.576 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.572+0000 7f2d05d9b640 1 --2- 172.21.15.1:0/3301444328 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d08103c60 0x7f2d0819a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.576 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.572+0000 7f2d05d9b640 1 -- 172.21.15.1:0/3301444328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d081a1e10 con 0x7f2d08104ea0 2024-05-04T03:19:38.576 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.572+0000 7f2d05d9b640 1 --2- 172.21.15.1:0/3301444328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d08104ea0 0x7f2d0819a810 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00d770 tx=0x7f2cfc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:38.576 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.572+0000 7f2cf77fe640 1 -- 172.21.15.1:0/3301444328 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc017070 con 0x7f2d08104ea0 2024-05-04T03:19:38.576 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.572+0000 7f2cf77fe640 1 -- 172.21.15.1:0/3301444328 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cfc0054d0 con 0x7f2d08104ea0 2024-05-04T03:19:38.576 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.572+0000 7f2d0cd9d640 1 -- 172.21.15.1:0/3301444328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d081a2100 con 0x7f2d08104ea0 2024-05-04T03:19:38.576 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.572+0000 7f2cf77fe640 1 -- 172.21.15.1:0/3301444328 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc004c30 con 0x7f2d08104ea0 2024-05-04T03:19:38.576 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.572+0000 7f2d0cd9d640 1 -- 172.21.15.1:0/3301444328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d081a2600 con 0x7f2d08104ea0 2024-05-04T03:19:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.575+0000 7f2d13c52740 1 Processor -- start 2024-05-04T03:19:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.575+0000 7f2d13c52740 1 -- start start 2024-05-04T03:19:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.575+0000 7f2d13c52740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592c4d201b0 0x5592c50c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.575+0000 7f2cf77fe640 1 -- 172.21.15.1:0/3301444328 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7f2cfc010460 con 0x7f2d08104ea0 2024-05-04T03:19:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.575+0000 7f2d13c52740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c4cf72f0 0x5592c50c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.575+0000 7f2d13c52740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c4cff4d0 0x5592c50cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.575+0000 7f2d13c52740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5592c50265d0 con 0x5592c4d201b0 2024-05-04T03:19:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.575+0000 7f2d13c52740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5592c4fdf890 con 0x5592c4cff4d0 2024-05-04T03:19:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.575+0000 7f2d13c52740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5592c5022590 con 0x5592c4cf72f0 2024-05-04T03:19:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.576+0000 7f2d06d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592c4d201b0 0x5592c50c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.576+0000 7f2d06d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592c4d201b0 0x5592c50c5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41926/0 (socket says 172.21.15.1:41926) 2024-05-04T03:19:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.576+0000 7f2d06d9d640 1 -- 172.21.15.1:0/2672517860 learned_addr learned my addr 172.21.15.1:0/2672517860 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.576+0000 7f2d0659c640 1 --2- 172.21.15.1:0/2672517860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c4cf72f0 0x5592c50c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:38.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.576+0000 7f2d05d9b640 1 --2- 172.21.15.1:0/2672517860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c4cff4d0 0x5592c50cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.576+0000 7f2d06d9d640 1 -- 172.21.15.1:0/2672517860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c4cf72f0 msgr2=0x5592c50c63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.576+0000 7f2d06d9d640 1 --2- 172.21.15.1:0/2672517860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c4cf72f0 0x5592c50c63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.576+0000 7f2d06d9d640 1 -- 172.21.15.1:0/2672517860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c4cff4d0 msgr2=0x5592c50cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.576+0000 7f2d06d9d640 1 --2- 172.21.15.1:0/2672517860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c4cff4d0 0x5592c50cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.576+0000 7f2d06d9d640 1 -- 172.21.15.1:0/2672517860 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592c4d38210 con 0x5592c4d201b0 2024-05-04T03:19:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.576+0000 7f2d06d9d640 1 --2- 172.21.15.1:0/2672517860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592c4d201b0 0x5592c50c5e70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009fd0 tx=0x7f2d000118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2699e26c6759a3c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.576+0000 7f2cf77fe640 1 --2- 172.21.15.1:0/3301444328 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f2cf00b2920 0x7f2cf00b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.576+0000 7f2cf4ff9640 1 -- 172.21.15.1:0/2672517860 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00004d20 con 0x5592c4d201b0 2024-05-04T03:19:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.576+0000 7f2cf4ff9640 1 -- 172.21.15.1:0/2672517860 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d00004050 con 0x5592c4d201b0 2024-05-04T03:19:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.577+0000 7f2cf4ff9640 1 -- 172.21.15.1:0/2672517860 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00005220 con 0x5592c4d201b0 2024-05-04T03:19:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.577+0000 7f2d0659c640 1 --2- 172.21.15.1:0/3301444328 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f2cf00b2920 0x7f2cf00b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:38.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.577+0000 7f2cf77fe640 1 -- 172.21.15.1:0/3301444328 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f2cfc0e1c90 con 0x7f2d08104ea0 2024-05-04T03:19:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.577+0000 7f2d13c52740 1 -- 172.21.15.1:0/2672517860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592c4d201b0 msgr2=0x5592c50c5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:38.584 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.577+0000 7f2d13c52740 1 --2- 172.21.15.1:0/2672517860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592c4d201b0 0x5592c50c5e70 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009fd0 tx=0x7f2d000118d0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.577+0000 7f2d0659c640 1 --2- 172.21.15.1:0/3301444328 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f2cf00b2920 0x7f2cf00b4e30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8005ab0 tx=0x7f2cf8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.577+0000 7f2d13c52740 1 -- 172.21.15.1:0/2672517860 shutdown_connections 2024-05-04T03:19:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.577+0000 7f2d13c52740 1 --2- 172.21.15.1:0/2672517860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c4cff4d0 0x5592c50cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.577+0000 7f2d13c52740 1 --2- 172.21.15.1:0/2672517860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c4cf72f0 0x5592c50c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.577+0000 7f2d13c52740 1 --2- 172.21.15.1:0/2672517860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592c4d201b0 0x5592c50c5e70 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.577+0000 7f2d13c52740 1 -- 172.21.15.1:0/2672517860 >> 172.21.15.1:0/2672517860 conn(0x5592c4d14300 msgr2=0x5592c50bd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.578+0000 7f2d13c52740 1 -- 172.21.15.1:0/2672517860 shutdown_connections 2024-05-04T03:19:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.578+0000 7f2d13c52740 1 -- 172.21.15.1:0/2672517860 wait complete. 2024-05-04T03:19:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.578+0000 7f2d13c52740 1 -- start start 2024-05-04T03:19:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.578+0000 7f2d13c52740 1 Processor -- start 2024-05-04T03:19:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.578+0000 7f2d13c52740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592c4cf72f0 0x5592c50cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.578+0000 7f2d13c52740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c4cff4d0 0x5592c50d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.578+0000 7f2d13c52740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c4d201b0 0x5592c50d6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:38.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.578+0000 7f2d13c52740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5592c50cdd40 con 0x5592c4cf72f0 2024-05-04T03:19:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.578+0000 7f2d13c52740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5592c4fe3370 con 0x5592c4d201b0 2024-05-04T03:19:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.578+0000 7f2d13c52740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5592c50cdec0 con 0x5592c4cff4d0 2024-05-04T03:19:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.578+0000 7f2d13c52740 10 client.0 _ll_register_callbacks cb 0x5592c4d37a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.578+0000 7f2d0659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592c4cf72f0 0x5592c50cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.578+0000 7f2d0659c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592c4cf72f0 0x5592c50cfae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41928/0 (socket says 172.21.15.1:41928) 2024-05-04T03:19:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.578+0000 7f2d0659c640 1 -- 172.21.15.1:0/833316967 learned_addr learned my addr 172.21.15.1:0/833316967 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.579+0000 7f2d05d9b640 1 --2- 172.21.15.1:0/833316967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c4cff4d0 0x5592c50d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.579+0000 7f2d06d9d640 1 --2- 172.21.15.1:0/833316967 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c4d201b0 0x5592c50d6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.579+0000 7f2d0659c640 1 -- 172.21.15.1:0/833316967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c4cff4d0 msgr2=0x5592c50d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.579+0000 7f2d0659c640 1 --2- 172.21.15.1:0/833316967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c4cff4d0 0x5592c50d0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.579+0000 7f2d0659c640 1 -- 172.21.15.1:0/833316967 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c4d201b0 msgr2=0x5592c50d6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.579+0000 7f2d0659c640 1 --2- 172.21.15.1:0/833316967 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c4d201b0 0x5592c50d6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.579+0000 7f2d0659c640 1 -- 172.21.15.1:0/833316967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592c4e04620 con 0x5592c4cf72f0 2024-05-04T03:19:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.579+0000 7f2d0659c640 1 --2- 172.21.15.1:0/833316967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592c4cf72f0 0x5592c50cfae0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8009900 tx=0x7f2cf802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2536a763b974f6bb server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.579+0000 7f2cedffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:19:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.579+0000 7f2cedffb640 1 -- 172.21.15.1:0/833316967 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8035040 con 0x5592c4cf72f0 2024-05-04T03:19:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.579+0000 7f2cedffb640 1 -- 172.21.15.1:0/833316967 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f2cf8004300 con 0x5592c4cf72f0 2024-05-04T03:19:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.579+0000 7f2d13c52740 1 -- 172.21.15.1:0/833316967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d00004050 con 0x5592c4cf72f0 2024-05-04T03:19:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.579+0000 7f2cedffb640 1 -- 172.21.15.1:0/833316967 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2cf8056070 con 0x5592c4cf72f0 2024-05-04T03:19:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.579+0000 7f2d13c52740 10 client.15297 fetch_fsmap learned FSMap version 26 2024-05-04T03:19:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.579+0000 7f2d13c52740 1 -- 172.21.15.1:0/833316967 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5592c50d9f20 con 0x5592c4cf72f0 2024-05-04T03:19:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.580+0000 7f2cedffb640 1 -- 172.21.15.1:0/833316967 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d00004050 con 0x5592c4cf72f0 2024-05-04T03:19:38.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.580+0000 7f2cedffb640 1 -- 172.21.15.1:0/833316967 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f2cf8044070 con 0x5592c4cf72f0 2024-05-04T03:19:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.580+0000 7f2d13c52740 10 client.15297 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:19:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.580+0000 7f2d13c52740 10 client.15297 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:19:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.580+0000 7f2d13c52740 20 client.15297 populate_metadata read hostname 'smithi001' 2024-05-04T03:19:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.580+0000 7f2d13c52740 1 --2- 172.21.15.1:0/833316967 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5592c4cded80 0x5592c50dd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.580+0000 7f2d13c52740 4 client.15297 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:19:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.580+0000 7f2d13c52740 1 -- 172.21.15.1:0/833316967 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592c50341e0 con 0x5592c4cded80 2024-05-04T03:19:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.580+0000 7f2d05d9b640 1 --2- 172.21.15.1:0/833316967 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5592c4cded80 0x5592c50dd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.581+0000 7f2d05d9b640 1 --2- 172.21.15.1:0/833316967 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5592c4cded80 0x5592c50dd680 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76870274594a1500 server_cookie=495db230274563ba in_seq=0 out_seq=0 2024-05-04T03:19:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.581+0000 7f2cedffb640 10 client.15297 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:19:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.581+0000 7f2cedffb640 1 -- 172.21.15.1:0/833316967 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5592c50341e0 con 0x5592c4cded80 2024-05-04T03:19:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.581+0000 7f2cedffb640 10 client.15297 handle_command_reply: tid=0 2024-05-04T03:19:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.581+0000 7f2d13c52740 1 client.15297 shutdown 2024-05-04T03:19:38.588 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.581+0000 7f2d13c52740 1 -- 172.21.15.1:0/833316967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592c4cf72f0 msgr2=0x5592c50cfae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.581+0000 7f2d13c52740 1 --2- 172.21.15.1:0/833316967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592c4cf72f0 0x5592c50cfae0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8009900 tx=0x7f2cf802a410 comp rx=0 tx=0).stop 2024-05-04T03:19:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.582+0000 7f2d13c52740 1 -- 172.21.15.1:0/833316967 shutdown_connections 2024-05-04T03:19:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.582+0000 7f2d13c52740 1 --2- 172.21.15.1:0/833316967 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5592c4cded80 0x5592c50dd680 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.582+0000 7f2d13c52740 1 --2- 172.21.15.1:0/833316967 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c4d201b0 0x5592c50d6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.582+0000 7f2d13c52740 1 --2- 172.21.15.1:0/833316967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c4cff4d0 0x5592c50d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.582+0000 7f2d13c52740 1 --2- 172.21.15.1:0/833316967 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592c4cf72f0 0x5592c50cfae0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.582+0000 7f2cedffb640 0 client.15297 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:19:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.582+0000 7f2d13c52740 1 -- 172.21.15.1:0/833316967 >> 172.21.15.1:0/833316967 conn(0x5592c4d14300 msgr2=0x5592c50c0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.582+0000 7f2d13c52740 1 -- 172.21.15.1:0/833316967 shutdown_connections 2024-05-04T03:19:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.582+0000 7f2d13c52740 1 -- 172.21.15.1:0/833316967 wait complete. 2024-05-04T03:19:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.582+0000 7f2d13c52740 20 client.15297 trim_cache size 0 max 16384 2024-05-04T03:19:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.597+0000 7f2d13c52740 1 Processor -- start 2024-05-04T03:19:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.597+0000 7f2d13c52740 1 -- start start 2024-05-04T03:19:38.600 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.598+0000 7f2d13c52740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c4cf72f0 0x5592c50c0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.598+0000 7f2d13c52740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592c4cff4d0 0x5592c50d6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.598+0000 7f2d13c52740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c4d201b0 0x5592c50d7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.598+0000 7f2d13c52740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2d081ab840 con 0x5592c4cff4d0 2024-05-04T03:19:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.598+0000 7f2d13c52740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2d081a9110 con 0x5592c4cf72f0 2024-05-04T03:19:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.598+0000 7f2d13c52740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2d081a6fa0 con 0x5592c4d201b0 2024-05-04T03:19:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.598+0000 7f2d05d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592c4cff4d0 0x5592c50d6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.598+0000 7f2d05d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592c4cff4d0 0x5592c50d6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41936/0 (socket says 172.21.15.1:41936) 2024-05-04T03:19:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.598+0000 7f2d05d9b640 1 -- 172.21.15.1:0/1160016931 learned_addr learned my addr 172.21.15.1:0/1160016931 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.598+0000 7f2d0659c640 1 --2- 172.21.15.1:0/1160016931 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c4cf72f0 0x5592c50c0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.598+0000 7f2d06d9d640 1 --2- 172.21.15.1:0/1160016931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c4d201b0 0x5592c50d7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.598+0000 7f2d05d9b640 1 -- 172.21.15.1:0/1160016931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c4d201b0 msgr2=0x5592c50d7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.598+0000 7f2d05d9b640 1 --2- 172.21.15.1:0/1160016931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c4d201b0 0x5592c50d7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.598+0000 7f2d05d9b640 1 -- 172.21.15.1:0/1160016931 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c4cf72f0 msgr2=0x5592c50c0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.598+0000 7f2d05d9b640 1 --2- 172.21.15.1:0/1160016931 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c4cf72f0 0x5592c50c0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.598+0000 7f2d05d9b640 1 -- 172.21.15.1:0/1160016931 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592c50bdd00 con 0x5592c4cff4d0 2024-05-04T03:19:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.598+0000 7f2d0659c640 1 --2- 172.21.15.1:0/1160016931 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c4cf72f0 0x5592c50c0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:19:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.598+0000 7f2d05d9b640 1 --2- 172.21.15.1:0/1160016931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592c4cff4d0 0x5592c50d6f10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009610 tx=0x7f2cfc091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f46d8500b311ab server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.598+0000 7f2ceffff640 1 -- 172.21.15.1:0/1160016931 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc020040 con 0x5592c4cff4d0 2024-05-04T03:19:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.598+0000 7f2ceffff640 1 -- 172.21.15.1:0/1160016931 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cfc0a8e30 con 0x5592c4cff4d0 2024-05-04T03:19:38.602 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.598+0000 7f2ceffff640 1 -- 172.21.15.1:0/1160016931 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc0b20d0 con 0x5592c4cff4d0 2024-05-04T03:19:38.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.599+0000 7f2d13c52740 1 -- 172.21.15.1:0/1160016931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592c4cff4d0 msgr2=0x5592c50d6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.599+0000 7f2d13c52740 1 --2- 172.21.15.1:0/1160016931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592c4cff4d0 0x5592c50d6f10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009610 tx=0x7f2cfc091040 comp rx=0 tx=0).stop 2024-05-04T03:19:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.599+0000 7f2d13c52740 1 -- 172.21.15.1:0/1160016931 shutdown_connections 2024-05-04T03:19:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.599+0000 7f2d13c52740 1 --2- 172.21.15.1:0/1160016931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c4d201b0 0x5592c50d7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.599+0000 7f2d13c52740 1 --2- 172.21.15.1:0/1160016931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592c4cff4d0 0x5592c50d6f10 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.599+0000 7f2d13c52740 1 --2- 172.21.15.1:0/1160016931 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c4cf72f0 0x5592c50c0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.599+0000 7f2d13c52740 1 -- 172.21.15.1:0/1160016931 >> 172.21.15.1:0/1160016931 conn(0x5592c4d14300 msgr2=0x5592c50c01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.599+0000 7f2d13c52740 1 -- 172.21.15.1:0/1160016931 shutdown_connections 2024-05-04T03:19:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.599+0000 7f2d13c52740 1 -- 172.21.15.1:0/1160016931 wait complete. 2024-05-04T03:19:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.600+0000 7f2d13c52740 1 -- start start 2024-05-04T03:19:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.600+0000 7f2d13c52740 1 Processor -- start 2024-05-04T03:19:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.600+0000 7f2d13c52740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c4cf72f0 0x5592c50d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.600+0000 7f2d13c52740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c4cff4d0 0x5592c50d48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.600+0000 7f2d13c52740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592c4d201b0 0x5592c5143a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.600+0000 7f2d13c52740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f2d081a75f0 con 0x5592c4d201b0 2024-05-04T03:19:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.600+0000 7f2d13c52740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2d081a8a40 con 0x5592c4cff4d0 2024-05-04T03:19:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.600+0000 7f2d13c52740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2d081a7ca0 con 0x5592c4cf72f0 2024-05-04T03:19:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.600+0000 7f2d13c52740 10 client.0 _ll_register_callbacks cb 0x7f2cf8005e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.600+0000 7f2d06d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592c4d201b0 0x5592c5143a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.600+0000 7f2d0659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c4cf72f0 0x5592c50d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.600+0000 7f2d06d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592c4d201b0 0x5592c5143a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41948/0 (socket says 172.21.15.1:41948) 2024-05-04T03:19:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.600+0000 7f2d06d9d640 1 -- 172.21.15.1:0/1619715579 learned_addr learned my addr 172.21.15.1:0/1619715579 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.600+0000 7f2d05d9b640 1 --2- 172.21.15.1:0/1619715579 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c4cff4d0 0x5592c50d48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.601+0000 7f2d0659c640 1 -- 172.21.15.1:0/1619715579 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c4cff4d0 msgr2=0x5592c50d48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.601+0000 7f2d0659c640 1 --2- 172.21.15.1:0/1619715579 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c4cff4d0 0x5592c50d48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.601+0000 7f2d0659c640 1 -- 172.21.15.1:0/1619715579 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592c4d201b0 msgr2=0x5592c5143a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.601+0000 7f2d0659c640 1 --2- 172.21.15.1:0/1619715579 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592c4d201b0 0x5592c5143a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.601+0000 7f2d0659c640 1 -- 172.21.15.1:0/1619715579 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592c50d7880 con 0x5592c4cf72f0 2024-05-04T03:19:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.601+0000 7f2d06d9d640 1 --2- 172.21.15.1:0/1619715579 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592c4d201b0 0x5592c5143a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:19:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.601+0000 7f2d0659c640 1 --2- 172.21.15.1:0/1619715579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c4cf72f0 0x5592c50d4380 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2cf802a930 tx=0x7f2cf803dae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbf4e4c725e196b server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.601+0000 7f2cedffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-04T03:19:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.601+0000 7f2cedffb640 1 -- 172.21.15.1:0/1619715579 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf805f070 con 0x5592c4cf72f0 2024-05-04T03:19:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.601+0000 7f2d13c52740 1 -- 172.21.15.1:0/1619715579 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cfc0a8e30 con 0x5592c4cf72f0 2024-05-04T03:19:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.601+0000 7f2cedffb640 1 -- 172.21.15.1:0/1619715579 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f2cf80035f0 con 0x5592c4cf72f0 2024-05-04T03:19:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.601+0000 7f2cedffb640 1 -- 172.21.15.1:0/1619715579 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2cf806b050 con 0x5592c4cf72f0 2024-05-04T03:19:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.601+0000 7f2d13c52740 10 client.25036 fetch_fsmap learned FSMap version 26 2024-05-04T03:19:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.601+0000 7f2d13c52740 1 -- 172.21.15.1:0/1619715579 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5592c5144e50 con 0x5592c4cf72f0 2024-05-04T03:19:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.601+0000 7f2cedffb640 1 -- 172.21.15.1:0/1619715579 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cfc0a8e30 con 0x5592c4cf72f0 2024-05-04T03:19:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.602+0000 7f2cedffb640 1 -- 172.21.15.1:0/1619715579 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f2cf8044050 con 0x5592c4cf72f0 2024-05-04T03:19:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.602+0000 7f2d13c52740 10 client.25036 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:19:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.602+0000 7f2d13c52740 10 client.25036 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:19:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.602+0000 7f2d13c52740 20 client.25036 populate_metadata read hostname 'smithi001' 2024-05-04T03:19:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.602+0000 7f2d13c52740 1 --2- 172.21.15.1:0/1619715579 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5592c4cb41c0 0x5592c5147810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:38.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.602+0000 7f2d13c52740 4 client.25036 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:19:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.602+0000 7f2d13c52740 1 -- 172.21.15.1:0/1619715579 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5592c502e050 con 0x5592c4cb41c0 2024-05-04T03:19:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.602+0000 7f2d05d9b640 1 --2- 172.21.15.1:0/1619715579 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5592c4cb41c0 0x5592c5147810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.602+0000 7f2d05d9b640 1 --2- 172.21.15.1:0/1619715579 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5592c4cb41c0 0x5592c5147810 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c98ffa36f0ceb86 server_cookie=fd5331074c26d8a7 in_seq=0 out_seq=0 2024-05-04T03:19:38.609 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.602+0000 7f2cedffb640 10 client.25036 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:19:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.609+0000 7f2cedffb640 1 -- 172.21.15.1:0/1619715579 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5592c502e050 con 0x5592c4cb41c0 2024-05-04T03:19:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.609+0000 7f2cedffb640 10 client.25036 handle_command_reply: tid=0 2024-05-04T03:19:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.609+0000 7f2d13c52740 1 client.25036 shutdown 2024-05-04T03:19:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.609+0000 7f2d13c52740 1 -- 172.21.15.1:0/1619715579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c4cf72f0 msgr2=0x5592c50d4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.609+0000 7f2d13c52740 1 --2- 172.21.15.1:0/1619715579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c4cf72f0 0x5592c50d4380 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2cf802a930 tx=0x7f2cf803dae0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.609+0000 7f2d13c52740 1 -- 172.21.15.1:0/1619715579 shutdown_connections 2024-05-04T03:19:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.609+0000 7f2d13c52740 1 --2- 172.21.15.1:0/1619715579 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5592c4cb41c0 0x5592c5147810 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.609+0000 7f2d13c52740 1 --2- 172.21.15.1:0/1619715579 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592c4cff4d0 0x5592c50d48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.609+0000 7f2d13c52740 1 --2- 172.21.15.1:0/1619715579 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5592c4d201b0 0x5592c5143a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.609+0000 7f2cedffb640 0 client.25036 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:19:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.609+0000 7f2d13c52740 1 --2- 172.21.15.1:0/1619715579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5592c4cf72f0 0x5592c50d4380 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.609+0000 7f2d13c52740 1 -- 172.21.15.1:0/1619715579 >> 172.21.15.1:0/1619715579 conn(0x5592c4d14300 msgr2=0x5592c50cf650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.609+0000 7f2d13c52740 1 -- 172.21.15.1:0/1619715579 shutdown_connections 2024-05-04T03:19:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.609+0000 7f2d13c52740 1 -- 172.21.15.1:0/1619715579 wait complete. 2024-05-04T03:19:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.610+0000 7f2d13c52740 20 client.25036 trim_cache size 0 max 16384 2024-05-04T03:19:38.613 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:19:38.613 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:19:38.613 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:19:38.613 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:19:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.610+0000 7f2d0cd9d640 1 -- 172.21.15.1:0/3301444328 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f2cf00b2920 msgr2=0x7f2cf00b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.610+0000 7f2d0cd9d640 1 --2- 172.21.15.1:0/3301444328 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f2cf00b2920 0x7f2cf00b4e30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8005ab0 tx=0x7f2cf8031400 comp rx=0 tx=0).stop 2024-05-04T03:19:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.610+0000 7f2d0cd9d640 1 -- 172.21.15.1:0/3301444328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d08104ea0 msgr2=0x7f2d0819a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.610+0000 7f2d0cd9d640 1 --2- 172.21.15.1:0/3301444328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d08104ea0 0x7f2d0819a810 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00d770 tx=0x7f2cfc00dc30 comp rx=0 tx=0).stop 2024-05-04T03:19:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.610+0000 7f2d0cd9d640 1 -- 172.21.15.1:0/3301444328 shutdown_connections 2024-05-04T03:19:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.610+0000 7f2d0cd9d640 1 --2- 172.21.15.1:0/3301444328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d08109ba0 0x7f2d081a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.610+0000 7f2d0cd9d640 1 --2- 172.21.15.1:0/3301444328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2d08104ea0 0x7f2d0819a810 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.610+0000 7f2d0cd9d640 1 --2- 172.21.15.1:0/3301444328 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f2cf00b2920 0x7f2cf00b4e30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.610+0000 7f2d0cd9d640 1 --2- 172.21.15.1:0/3301444328 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d08103c60 0x7f2d0819a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.610+0000 7f2d0cd9d640 1 -- 172.21.15.1:0/3301444328 >> 172.21.15.1:0/3301444328 conn(0x7f2d080fee70 msgr2=0x7f2d08105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.610+0000 7f2d0cd9d640 1 -- 172.21.15.1:0/3301444328 shutdown_connections 2024-05-04T03:19:38.614 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:38.610+0000 7f2d0cd9d640 1 -- 172.21.15.1:0/3301444328 wait complete. 2024-05-04T03:19:38.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:38 smithi001 ceph-mon[29731]: pgmap v405: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-04T03:19:38.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:38 smithi077 ceph-mon[33531]: pgmap v405: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-04T03:19:38.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:38 smithi072 ceph-mon[33332]: pgmap v405: 97 pgs: 97 active+clean; 666 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-04T03:19:40.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:40 smithi001 ceph-mon[29731]: pgmap v406: 97 pgs: 97 active+clean; 672 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-04T03:19:40.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:40 smithi077 ceph-mon[33531]: pgmap v406: 97 pgs: 97 active+clean; 672 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-04T03:19:40.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:40 smithi072 ceph-mon[33332]: pgmap v406: 97 pgs: 97 active+clean; 672 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-04T03:19:42.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:42 smithi001 ceph-mon[29731]: pgmap v407: 97 pgs: 97 active+clean; 672 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-04T03:19:42.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:42 smithi077 ceph-mon[33531]: pgmap v407: 97 pgs: 97 active+clean; 672 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-04T03:19:42.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:42 smithi072 ceph-mon[33332]: pgmap v407: 97 pgs: 97 active+clean; 672 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-04T03:19:44.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:44 smithi001 ceph-mon[29731]: pgmap v408: 97 pgs: 97 active+clean; 672 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-04T03:19:44.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:44 smithi077 ceph-mon[33531]: pgmap v408: 97 pgs: 97 active+clean; 672 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-04T03:19:44.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:44 smithi072 ceph-mon[33332]: pgmap v408: 97 pgs: 97 active+clean; 672 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-04T03:19:46.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:46 smithi077 ceph-mon[33531]: pgmap v409: 97 pgs: 97 active+clean; 672 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-04T03:19:46.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:46 smithi072 ceph-mon[33332]: pgmap v409: 97 pgs: 97 active+clean; 672 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-04T03:19:47.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:46 smithi001 ceph-mon[29731]: pgmap v409: 97 pgs: 97 active+clean; 672 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-05-04T03:19:48.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:48 smithi077 ceph-mon[33531]: pgmap v410: 97 pgs: 97 active+clean; 672 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-04T03:19:48.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:48 smithi072 ceph-mon[33332]: pgmap v410: 97 pgs: 97 active+clean; 672 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-04T03:19:49.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:48 smithi001 ceph-mon[29731]: pgmap v410: 97 pgs: 97 active+clean; 672 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-04T03:19:50.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:50 smithi077 ceph-mon[33531]: pgmap v411: 97 pgs: 97 active+clean; 672 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-05-04T03:19:50.930 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:50 smithi001 ceph-mon[29731]: pgmap v411: 97 pgs: 97 active+clean; 672 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-05-04T03:19:50.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:50 smithi072 ceph-mon[33332]: pgmap v411: 97 pgs: 97 active+clean; 672 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-05-04T03:19:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.719+0000 7fe8d80f2640 1 -- 172.21.15.1:0/3626852991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8d0103d90 msgr2=0x7fe8d01041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.719+0000 7fe8d80f2640 1 --2- 172.21.15.1:0/3626852991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8d0103d90 0x7fe8d01041a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0009900 tx=0x7fe8c002a410 comp rx=0 tx=0).stop 2024-05-04T03:19:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.719+0000 7fe8d80f2640 1 -- 172.21.15.1:0/3626852991 shutdown_connections 2024-05-04T03:19:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.719+0000 7fe8d80f2640 1 --2- 172.21.15.1:0/3626852991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8d0108a90 0x7fe8d010aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.719+0000 7fe8d80f2640 1 --2- 172.21.15.1:0/3626852991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8d0103d90 0x7fe8d01041a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.719+0000 7fe8d80f2640 1 --2- 172.21.15.1:0/3626852991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe8d0102b50 0x7fe8d0103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.719+0000 7fe8d80f2640 1 -- 172.21.15.1:0/3626852991 >> 172.21.15.1:0/3626852991 conn(0x7fe8d00fdd40 msgr2=0x7fe8d01001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.719+0000 7fe8d80f2640 1 -- 172.21.15.1:0/3626852991 shutdown_connections 2024-05-04T03:19:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.719+0000 7fe8d80f2640 1 -- 172.21.15.1:0/3626852991 wait complete. 2024-05-04T03:19:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.720+0000 7fe8d80f2640 1 Processor -- start 2024-05-04T03:19:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.720+0000 7fe8d80f2640 1 -- start start 2024-05-04T03:19:51.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.720+0000 7fe8d80f2640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8d0102b50 0x7fe8d010d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.720+0000 7fe8d80f2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8d0103d90 0x7fe8d010bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.720+0000 7fe8d80f2640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe8d0108a90 0x7fe8d010c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.720+0000 7fe8d80f2640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d010f750 con 0x7fe8d0102b50 2024-05-04T03:19:51.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.720+0000 7fe8d80f2640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d010f5d0 con 0x7fe8d0108a90 2024-05-04T03:19:51.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.720+0000 7fe8d80f2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d010f8d0 con 0x7fe8d0103d90 2024-05-04T03:19:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.721+0000 7fe8d5e8e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8d0102b50 0x7fe8d010d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.721+0000 7fe8d5e8e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8d0102b50 0x7fe8d010d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45912/0 (socket says 172.21.15.1:45912) 2024-05-04T03:19:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.721+0000 7fe8d5e8e640 1 -- 172.21.15.1:0/3857210612 learned_addr learned my addr 172.21.15.1:0/3857210612 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.721+0000 7fe8d568d640 1 --2- 172.21.15.1:0/3857210612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8d0103d90 0x7fe8d010bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.721+0000 7fe8d668f640 1 --2- 172.21.15.1:0/3857210612 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe8d0108a90 0x7fe8d010c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.721+0000 7fe8d5e8e640 1 -- 172.21.15.1:0/3857210612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8d0103d90 msgr2=0x7fe8d010bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.721+0000 7fe8d5e8e640 1 --2- 172.21.15.1:0/3857210612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8d0103d90 0x7fe8d010bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.721+0000 7fe8d5e8e640 1 -- 172.21.15.1:0/3857210612 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe8d0108a90 msgr2=0x7fe8d010c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.721+0000 7fe8d5e8e640 1 --2- 172.21.15.1:0/3857210612 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe8d0108a90 0x7fe8d010c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.721+0000 7fe8d5e8e640 1 -- 172.21.15.1:0/3857210612 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d010c930 con 0x7fe8d0102b50 2024-05-04T03:19:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.721+0000 7fe8d5e8e640 1 --2- 172.21.15.1:0/3857210612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8d0102b50 0x7fe8d010d6b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00c970 tx=0x7fe8cc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.721+0000 7fe8c6ffd640 1 -- 172.21.15.1:0/3857210612 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc007c70 con 0x7fe8d0102b50 2024-05-04T03:19:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.721+0000 7fe8c6ffd640 1 -- 172.21.15.1:0/3857210612 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8cc007e10 con 0x7fe8d0102b50 2024-05-04T03:19:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.721+0000 7fe8c6ffd640 1 -- 172.21.15.1:0/3857210612 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc0187b0 con 0x7fe8d0102b50 2024-05-04T03:19:51.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.721+0000 7fe8d80f2640 1 -- 172.21.15.1:0/3857210612 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d01a4320 con 0x7fe8d0102b50 2024-05-04T03:19:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.721+0000 7fe8d80f2640 1 -- 172.21.15.1:0/3857210612 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d01a4730 con 0x7fe8d0102b50 2024-05-04T03:19:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.724+0000 7fe8defa5740 1 Processor -- start 2024-05-04T03:19:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.724+0000 7fe8defa5740 1 -- start start 2024-05-04T03:19:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.724+0000 7fe8c6ffd640 1 -- 172.21.15.1:0/3857210612 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 19) ==== 149702+0+0 (secure 0 0 0) 0x7fe8cc00fab0 con 0x7fe8d0102b50 2024-05-04T03:19:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.725+0000 7fe8defa5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56106c8911b0 0x56106cc32a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.725+0000 7fe8defa5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56106c8682f0 0x56106cc32fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.725+0000 7fe8defa5740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56106c8704d0 0x56106cc39ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.725+0000 7fe8defa5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56106cb975d0 con 0x56106c8682f0 2024-05-04T03:19:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.725+0000 7fe8defa5740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x56106cb50890 con 0x56106c8704d0 2024-05-04T03:19:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.725+0000 7fe8defa5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56106cb93590 con 0x56106c8911b0 2024-05-04T03:19:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.725+0000 7fe8d5e8e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56106c8682f0 0x56106cc32fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.725+0000 7fe8d5e8e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56106c8682f0 0x56106cc32fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45924/0 (socket says 172.21.15.1:45924) 2024-05-04T03:19:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.725+0000 7fe8d5e8e640 1 -- 172.21.15.1:0/3697268299 learned_addr learned my addr 172.21.15.1:0/3697268299 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.725+0000 7fe8d568d640 1 --2- 172.21.15.1:0/3697268299 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56106c8704d0 0x56106cc39ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.725+0000 7fe8d668f640 1 --2- 172.21.15.1:0/3697268299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56106c8911b0 0x56106cc32a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.725+0000 7fe8d5e8e640 1 -- 172.21.15.1:0/3697268299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56106c8911b0 msgr2=0x56106cc32a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.725+0000 7fe8d5e8e640 1 --2- 172.21.15.1:0/3697268299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56106c8911b0 0x56106cc32a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.725+0000 7fe8d5e8e640 1 -- 172.21.15.1:0/3697268299 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56106c8704d0 msgr2=0x56106cc39ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.725+0000 7fe8d5e8e640 1 --2- 172.21.15.1:0/3697268299 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56106c8704d0 0x56106cc39ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.725+0000 7fe8d5e8e640 1 -- 172.21.15.1:0/3697268299 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56106c8a9210 con 0x56106c8682f0 2024-05-04T03:19:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.725+0000 7fe8d568d640 1 --2- 172.21.15.1:0/3697268299 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56106c8704d0 0x56106cc39ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:19:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.726+0000 7fe8c6ffd640 1 --2- 172.21.15.1:0/3857210612 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe8bc0b2920 0x7fe8bc0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.726+0000 7fe8d5e8e640 1 --2- 172.21.15.1:0/3697268299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56106c8682f0 0x56106cc32fe0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc0dfc60 tx=0x7fe8cc0ea600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3def8b017b8b891 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.726+0000 7fe8d668f640 1 --2- 172.21.15.1:0/3857210612 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe8bc0b2920 0x7fe8bc0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.726+0000 7fe8bbfff640 1 -- 172.21.15.1:0/3697268299 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc0a56f0 con 0x56106c8682f0 2024-05-04T03:19:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.726+0000 7fe8c6ffd640 1 -- 172.21.15.1:0/3857210612 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fe8cc0e2250 con 0x7fe8d0102b50 2024-05-04T03:19:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.726+0000 7fe8bbfff640 1 -- 172.21.15.1:0/3697268299 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8cc0eae30 con 0x56106c8682f0 2024-05-04T03:19:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.726+0000 7fe8bbfff640 1 -- 172.21.15.1:0/3697268299 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc0a8fa0 con 0x56106c8682f0 2024-05-04T03:19:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.727+0000 7fe8defa5740 1 -- 172.21.15.1:0/3697268299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56106c8682f0 msgr2=0x56106cc32fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.727+0000 7fe8d668f640 1 --2- 172.21.15.1:0/3857210612 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe8bc0b2920 0x7fe8bc0b4e30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x56106cc33c30 tx=0x7fe8c8009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.727+0000 7fe8defa5740 1 --2- 172.21.15.1:0/3697268299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56106c8682f0 0x56106cc32fe0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc0dfc60 tx=0x7fe8cc0ea600 comp rx=0 tx=0).stop 2024-05-04T03:19:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.727+0000 7fe8defa5740 1 -- 172.21.15.1:0/3697268299 shutdown_connections 2024-05-04T03:19:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.727+0000 7fe8defa5740 1 --2- 172.21.15.1:0/3697268299 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56106c8704d0 0x56106cc39ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.727+0000 7fe8defa5740 1 --2- 172.21.15.1:0/3697268299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56106c8682f0 0x56106cc32fe0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.727+0000 7fe8defa5740 1 --2- 172.21.15.1:0/3697268299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56106c8911b0 0x56106cc32a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.727+0000 7fe8defa5740 1 -- 172.21.15.1:0/3697268299 >> 172.21.15.1:0/3697268299 conn(0x56106c885300 msgr2=0x56106cc2e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.727+0000 7fe8defa5740 1 -- 172.21.15.1:0/3697268299 shutdown_connections 2024-05-04T03:19:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.727+0000 7fe8defa5740 1 -- 172.21.15.1:0/3697268299 wait complete. 2024-05-04T03:19:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.727+0000 7fe8defa5740 1 -- start start 2024-05-04T03:19:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.728+0000 7fe8defa5740 1 Processor -- start 2024-05-04T03:19:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.728+0000 7fe8defa5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56106c8682f0 0x56106cc3c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.728+0000 7fe8defa5740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56106c8704d0 0x56106cc3cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.728+0000 7fe8defa5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56106c8911b0 0x56106cc43b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.728+0000 7fe8defa5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56106cc3a8d0 con 0x56106c8682f0 2024-05-04T03:19:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.728+0000 7fe8defa5740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x56106cb54370 con 0x56106c8704d0 2024-05-04T03:19:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.728+0000 7fe8defa5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56106cc3aa50 con 0x56106c8911b0 2024-05-04T03:19:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.728+0000 7fe8d5e8e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56106c8682f0 0x56106cc3c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.728+0000 7fe8defa5740 10 client.0 _ll_register_callbacks cb 0x56106c8a8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.728+0000 7fe8d5e8e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56106c8682f0 0x56106cc3c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45928/0 (socket says 172.21.15.1:45928) 2024-05-04T03:19:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.728+0000 7fe8d5e8e640 1 -- 172.21.15.1:0/2796340861 learned_addr learned my addr 172.21.15.1:0/2796340861 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.728+0000 7fe8d668f640 1 --2- 172.21.15.1:0/2796340861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56106c8911b0 0x56106cc43b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.728+0000 7fe8d568d640 1 --2- 172.21.15.1:0/2796340861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56106c8704d0 0x56106cc3cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.728+0000 7fe8d5e8e640 1 -- 172.21.15.1:0/2796340861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56106c8911b0 msgr2=0x56106cc43b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.728+0000 7fe8d5e8e640 1 --2- 172.21.15.1:0/2796340861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56106c8911b0 0x56106cc43b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.728+0000 7fe8d5e8e640 1 -- 172.21.15.1:0/2796340861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56106c8704d0 msgr2=0x56106cc3cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.728+0000 7fe8d5e8e640 1 --2- 172.21.15.1:0/2796340861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56106c8704d0 0x56106cc3cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.729+0000 7fe8d5e8e640 1 -- 172.21.15.1:0/2796340861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56106c975620 con 0x56106c8682f0 2024-05-04T03:19:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.729+0000 7fe8d568d640 1 --2- 172.21.15.1:0/2796340861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56106c8704d0 0x56106cc3cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:19:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.729+0000 7fe8d5e8e640 1 --2- 172.21.15.1:0/2796340861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56106c8682f0 0x56106cc3c680 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc0dfda0 tx=0x7fe8cc00af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1195fceb55c7f244 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.729+0000 7fe8b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:19:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.729+0000 7fe8b97fa640 1 -- 172.21.15.1:0/2796340861 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc0a9180 con 0x56106c8682f0 2024-05-04T03:19:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.729+0000 7fe8b97fa640 1 -- 172.21.15.1:0/2796340861 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fe8cc0e1f90 con 0x56106c8682f0 2024-05-04T03:19:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.729+0000 7fe8defa5740 1 -- 172.21.15.1:0/2796340861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8cc0eae30 con 0x56106c8682f0 2024-05-04T03:19:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.729+0000 7fe8b97fa640 1 -- 172.21.15.1:0/2796340861 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe8cc0c3050 con 0x56106c8682f0 2024-05-04T03:19:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.729+0000 7fe8defa5740 10 client.15315 fetch_fsmap learned FSMap version 26 2024-05-04T03:19:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.729+0000 7fe8defa5740 1 -- 172.21.15.1:0/2796340861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56106cc3a8d0 con 0x56106c8682f0 2024-05-04T03:19:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.729+0000 7fe8b97fa640 1 -- 172.21.15.1:0/2796340861 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8cc0eae30 con 0x56106c8682f0 2024-05-04T03:19:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.729+0000 7fe8b97fa640 1 -- 172.21.15.1:0/2796340861 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe8cc0caae0 con 0x56106c8682f0 2024-05-04T03:19:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.730+0000 7fe8defa5740 10 client.15315 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:19:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.730+0000 7fe8defa5740 10 client.15315 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:19:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.730+0000 7fe8defa5740 20 client.15315 populate_metadata read hostname 'smithi001' 2024-05-04T03:19:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.730+0000 7fe8defa5740 1 --2- 172.21.15.1:0/2796340861 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x56106c84fd80 0x56106cc49be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.730+0000 7fe8defa5740 4 client.15315 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:19:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.730+0000 7fe8defa5740 1 -- 172.21.15.1:0/2796340861 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56106cba51e0 con 0x56106c84fd80 2024-05-04T03:19:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.730+0000 7fe8d568d640 1 --2- 172.21.15.1:0/2796340861 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x56106c84fd80 0x56106cc49be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.730+0000 7fe8d568d640 1 --2- 172.21.15.1:0/2796340861 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x56106c84fd80 0x56106cc49be0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f469e17c5501b055 server_cookie=102e37d4d3ee74d2 in_seq=0 out_seq=0 2024-05-04T03:19:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.730+0000 7fe8b97fa640 10 client.15315 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:19:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.731+0000 7fe8b97fa640 1 -- 172.21.15.1:0/2796340861 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56106cba51e0 con 0x56106c84fd80 2024-05-04T03:19:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.731+0000 7fe8b97fa640 10 client.15315 handle_command_reply: tid=0 2024-05-04T03:19:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.731+0000 7fe8defa5740 1 client.15315 shutdown 2024-05-04T03:19:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.731+0000 7fe8defa5740 1 -- 172.21.15.1:0/2796340861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56106c8682f0 msgr2=0x56106cc3c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.731+0000 7fe8defa5740 1 --2- 172.21.15.1:0/2796340861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56106c8682f0 0x56106cc3c680 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc0dfda0 tx=0x7fe8cc00af70 comp rx=0 tx=0).stop 2024-05-04T03:19:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.731+0000 7fe8defa5740 1 -- 172.21.15.1:0/2796340861 shutdown_connections 2024-05-04T03:19:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.731+0000 7fe8defa5740 1 --2- 172.21.15.1:0/2796340861 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x56106c84fd80 0x56106cc49be0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.731+0000 7fe8defa5740 1 --2- 172.21.15.1:0/2796340861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56106c8704d0 0x56106cc3cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.731+0000 7fe8defa5740 1 --2- 172.21.15.1:0/2796340861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56106c8911b0 0x56106cc43b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.731+0000 7fe8b97fa640 0 client.15315 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:19:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.731+0000 7fe8defa5740 1 --2- 172.21.15.1:0/2796340861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56106c8682f0 0x56106cc3c680 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.731+0000 7fe8defa5740 1 -- 172.21.15.1:0/2796340861 >> 172.21.15.1:0/2796340861 conn(0x56106c885300 msgr2=0x56106cc31300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.731+0000 7fe8defa5740 1 -- 172.21.15.1:0/2796340861 shutdown_connections 2024-05-04T03:19:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.731+0000 7fe8defa5740 1 -- 172.21.15.1:0/2796340861 wait complete. 2024-05-04T03:19:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.732+0000 7fe8defa5740 20 client.15315 trim_cache size 0 max 16384 2024-05-04T03:19:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.747+0000 7fe8defa5740 1 Processor -- start 2024-05-04T03:19:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.747+0000 7fe8defa5740 1 -- start start 2024-05-04T03:19:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.747+0000 7fe8defa5740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56106c8682f0 0x56106cc2e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.747+0000 7fe8defa5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56106c8704d0 0x56106cc43ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.747+0000 7fe8defa5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56106c8911b0 0x56106cc43ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.747+0000 7fe8defa5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d01afb40 con 0x56106c8911b0 2024-05-04T03:19:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.747+0000 7fe8defa5740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d01ad410 con 0x56106c8682f0 2024-05-04T03:19:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.747+0000 7fe8defa5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d01ab2a0 con 0x56106c8704d0 2024-05-04T03:19:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.747+0000 7fe8d668f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56106c8911b0 0x56106cc43ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.747+0000 7fe8d668f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56106c8911b0 0x56106cc43ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45930/0 (socket says 172.21.15.1:45930) 2024-05-04T03:19:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.747+0000 7fe8d668f640 1 -- 172.21.15.1:0/425629222 learned_addr learned my addr 172.21.15.1:0/425629222 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.747+0000 7fe8d568d640 1 --2- 172.21.15.1:0/425629222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56106c8704d0 0x56106cc43ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.747+0000 7fe8d5e8e640 1 --2- 172.21.15.1:0/425629222 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56106c8682f0 0x56106cc2e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.748+0000 7fe8d668f640 1 -- 172.21.15.1:0/425629222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56106c8704d0 msgr2=0x56106cc43ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.748+0000 7fe8d668f640 1 --2- 172.21.15.1:0/425629222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56106c8704d0 0x56106cc43ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.748+0000 7fe8d668f640 1 -- 172.21.15.1:0/425629222 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56106c8682f0 msgr2=0x56106cc2e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.748+0000 7fe8d668f640 1 --2- 172.21.15.1:0/425629222 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56106c8682f0 0x56106cc2e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.748+0000 7fe8d668f640 1 -- 172.21.15.1:0/425629222 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56106cc49f50 con 0x56106c8911b0 2024-05-04T03:19:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.748+0000 7fe8d668f640 1 --2- 172.21.15.1:0/425629222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56106c8911b0 0x56106cc43ff0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8003c00 tx=0x7fe8c8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8a6acca8ac346d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.748+0000 7fe8bb7fe640 1 -- 172.21.15.1:0/425629222 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c80115e0 con 0x56106c8911b0 2024-05-04T03:19:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.748+0000 7fe8bb7fe640 1 -- 172.21.15.1:0/425629222 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8c8011c30 con 0x56106c8911b0 2024-05-04T03:19:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.748+0000 7fe8bb7fe640 1 -- 172.21.15.1:0/425629222 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8021ca0 con 0x56106c8911b0 2024-05-04T03:19:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.749+0000 7fe8defa5740 1 -- 172.21.15.1:0/425629222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56106c8911b0 msgr2=0x56106cc43ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.749+0000 7fe8defa5740 1 --2- 172.21.15.1:0/425629222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56106c8911b0 0x56106cc43ff0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8003c00 tx=0x7fe8c8016000 comp rx=0 tx=0).stop 2024-05-04T03:19:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.749+0000 7fe8defa5740 1 -- 172.21.15.1:0/425629222 shutdown_connections 2024-05-04T03:19:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.749+0000 7fe8defa5740 1 --2- 172.21.15.1:0/425629222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56106c8911b0 0x56106cc43ff0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.749+0000 7fe8defa5740 1 --2- 172.21.15.1:0/425629222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56106c8704d0 0x56106cc43ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.749+0000 7fe8defa5740 1 --2- 172.21.15.1:0/425629222 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56106c8682f0 0x56106cc2e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.749+0000 7fe8defa5740 1 -- 172.21.15.1:0/425629222 >> 172.21.15.1:0/425629222 conn(0x56106c885300 msgr2=0x56106cc2dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.749+0000 7fe8defa5740 1 -- 172.21.15.1:0/425629222 shutdown_connections 2024-05-04T03:19:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.749+0000 7fe8defa5740 1 -- 172.21.15.1:0/425629222 wait complete. 2024-05-04T03:19:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.749+0000 7fe8defa5740 1 -- start start 2024-05-04T03:19:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.750+0000 7fe8defa5740 1 Processor -- start 2024-05-04T03:19:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.750+0000 7fe8defa5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56106c8682f0 0x56106cc41480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.750+0000 7fe8defa5740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56106c8704d0 0x56106cc419c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.750+0000 7fe8defa5740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56106c8911b0 0x56106cc428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.750+0000 7fe8defa5740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d01ab8f0 con 0x56106c8704d0 2024-05-04T03:19:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.750+0000 7fe8defa5740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d01acd40 con 0x56106c8911b0 2024-05-04T03:19:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.750+0000 7fe8defa5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d01abfa0 con 0x56106c8682f0 2024-05-04T03:19:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.750+0000 7fe8defa5740 10 client.0 _ll_register_callbacks cb 0x7fe8cc0a3f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:19:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.750+0000 7fe8d568d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56106c8704d0 0x56106cc419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.750+0000 7fe8d568d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56106c8704d0 0x56106cc419c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45934/0 (socket says 172.21.15.1:45934) 2024-05-04T03:19:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.750+0000 7fe8d568d640 1 -- 172.21.15.1:0/1865901531 learned_addr learned my addr 172.21.15.1:0/1865901531 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:19:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.750+0000 7fe8d5e8e640 1 --2- 172.21.15.1:0/1865901531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56106c8682f0 0x56106cc41480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.750+0000 7fe8d668f640 1 --2- 172.21.15.1:0/1865901531 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56106c8911b0 0x56106cc428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.751+0000 7fe8d568d640 1 -- 172.21.15.1:0/1865901531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56106c8682f0 msgr2=0x56106cc41480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.751+0000 7fe8d568d640 1 --2- 172.21.15.1:0/1865901531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56106c8682f0 0x56106cc41480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.751+0000 7fe8d568d640 1 -- 172.21.15.1:0/1865901531 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56106c8911b0 msgr2=0x56106cc428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:19:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.751+0000 7fe8d568d640 1 --2- 172.21.15.1:0/1865901531 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56106c8911b0 0x56106cc428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.751+0000 7fe8d568d640 1 -- 172.21.15.1:0/1865901531 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56106cc42eb0 con 0x56106c8704d0 2024-05-04T03:19:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.751+0000 7fe8d668f640 1 --2- 172.21.15.1:0/1865901531 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56106c8911b0 0x56106cc428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:19:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.751+0000 7fe8d568d640 1 --2- 172.21.15.1:0/1865901531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56106c8704d0 0x56106cc419c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe8c002a930 tx=0x7fe8c00319b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a61cb480b7017 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:19:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.751+0000 7fe8b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:19:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.751+0000 7fe8b97fa640 1 -- 172.21.15.1:0/1865901531 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0035040 con 0x56106c8704d0 2024-05-04T03:19:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.751+0000 7fe8b97fa640 1 -- 172.21.15.1:0/1865901531 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fe8c0046720 con 0x56106c8704d0 2024-05-04T03:19:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.751+0000 7fe8defa5740 1 -- 172.21.15.1:0/1865901531 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8c8011c30 con 0x56106c8704d0 2024-05-04T03:19:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.751+0000 7fe8b97fa640 1 -- 172.21.15.1:0/1865901531 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe8c0056050 con 0x56106c8704d0 2024-05-04T03:19:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.751+0000 7fe8defa5740 10 client.15321 fetch_fsmap learned FSMap version 26 2024-05-04T03:19:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.751+0000 7fe8defa5740 1 -- 172.21.15.1:0/1865901531 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56106cc44240 con 0x56106c8704d0 2024-05-04T03:19:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.751+0000 7fe8b97fa640 1 -- 172.21.15.1:0/1865901531 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8c8011c30 con 0x56106c8704d0 2024-05-04T03:19:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.752+0000 7fe8b97fa640 1 -- 172.21.15.1:0/1865901531 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe8c003b440 con 0x56106c8704d0 2024-05-04T03:19:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.752+0000 7fe8defa5740 10 client.15321 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:19:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.752+0000 7fe8defa5740 10 client.15321 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:19:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.752+0000 7fe8defa5740 20 client.15321 populate_metadata read hostname 'smithi001' 2024-05-04T03:19:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.752+0000 7fe8defa5740 1 --2- 172.21.15.1:0/1865901531 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x56106c8251c0 0x56106ccb4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:19:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.752+0000 7fe8defa5740 4 client.15321 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:19:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.752+0000 7fe8defa5740 1 -- 172.21.15.1:0/1865901531 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56106cb9f050 con 0x56106c8251c0 2024-05-04T03:19:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.752+0000 7fe8d5e8e640 1 --2- 172.21.15.1:0/1865901531 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x56106c8251c0 0x56106ccb4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:19:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.752+0000 7fe8b97fa640 10 client.15321 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:19:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.752+0000 7fe8d5e8e640 1 --2- 172.21.15.1:0/1865901531 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x56106c8251c0 0x56106ccb4670 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e186b2559e3506af server_cookie=efde4f7118ed91d8 in_seq=0 out_seq=0 2024-05-04T03:19:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.756+0000 7fe8b97fa640 1 -- 172.21.15.1:0/1865901531 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56106cb9f050 con 0x56106c8251c0 2024-05-04T03:19:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.756+0000 7fe8b97fa640 10 client.15321 handle_command_reply: tid=0 2024-05-04T03:19:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.756+0000 7fe8defa5740 1 client.15321 shutdown 2024-05-04T03:19:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.756+0000 7fe8defa5740 1 -- 172.21.15.1:0/1865901531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56106c8704d0 msgr2=0x56106cc419c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.756+0000 7fe8defa5740 1 --2- 172.21.15.1:0/1865901531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56106c8704d0 0x56106cc419c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe8c002a930 tx=0x7fe8c00319b0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.756+0000 7fe8defa5740 1 -- 172.21.15.1:0/1865901531 shutdown_connections 2024-05-04T03:19:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.756+0000 7fe8defa5740 1 --2- 172.21.15.1:0/1865901531 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x56106c8251c0 0x56106ccb4670 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.756+0000 7fe8defa5740 1 --2- 172.21.15.1:0/1865901531 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56106c8911b0 0x56106cc428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.756+0000 7fe8defa5740 1 --2- 172.21.15.1:0/1865901531 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56106c8704d0 0x56106cc419c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.756+0000 7fe8defa5740 1 --2- 172.21.15.1:0/1865901531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56106c8682f0 0x56106cc41480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.756+0000 7fe8b97fa640 0 client.15321 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:19:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.756+0000 7fe8defa5740 1 -- 172.21.15.1:0/1865901531 >> 172.21.15.1:0/1865901531 conn(0x56106c885300 msgr2=0x56106cc4e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.757+0000 7fe8defa5740 1 -- 172.21.15.1:0/1865901531 shutdown_connections 2024-05-04T03:19:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.757+0000 7fe8defa5740 1 -- 172.21.15.1:0/1865901531 wait complete. 2024-05-04T03:19:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.757+0000 7fe8defa5740 20 client.15321 trim_cache size 0 max 16384 2024-05-04T03:19:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:19:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:19:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:19:51.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:19:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.757+0000 7fe8d80f2640 1 -- 172.21.15.1:0/3857210612 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe8bc0b2920 msgr2=0x7fe8bc0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.757+0000 7fe8d80f2640 1 --2- 172.21.15.1:0/3857210612 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe8bc0b2920 0x7fe8bc0b4e30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x56106cc33c30 tx=0x7fe8c8009600 comp rx=0 tx=0).stop 2024-05-04T03:19:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.757+0000 7fe8d80f2640 1 -- 172.21.15.1:0/3857210612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8d0102b50 msgr2=0x7fe8d010d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:19:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.757+0000 7fe8d80f2640 1 --2- 172.21.15.1:0/3857210612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8d0102b50 0x7fe8d010d6b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00c970 tx=0x7fe8cc00ce30 comp rx=0 tx=0).stop 2024-05-04T03:19:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.757+0000 7fe8d80f2640 1 -- 172.21.15.1:0/3857210612 shutdown_connections 2024-05-04T03:19:51.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.757+0000 7fe8d80f2640 1 --2- 172.21.15.1:0/3857210612 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe8bc0b2920 0x7fe8bc0b4e30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.757+0000 7fe8d80f2640 1 --2- 172.21.15.1:0/3857210612 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe8d0108a90 0x7fe8d010c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.757+0000 7fe8d80f2640 1 --2- 172.21.15.1:0/3857210612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8d0103d90 0x7fe8d010bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.757+0000 7fe8d80f2640 1 --2- 172.21.15.1:0/3857210612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe8d0102b50 0x7fe8d010d6b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:19:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.757+0000 7fe8d80f2640 1 -- 172.21.15.1:0/3857210612 >> 172.21.15.1:0/3857210612 conn(0x7fe8d00fdd40 msgr2=0x7fe8d0104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:19:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.757+0000 7fe8d80f2640 1 -- 172.21.15.1:0/3857210612 shutdown_connections 2024-05-04T03:19:51.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:19:51.757+0000 7fe8d80f2640 1 -- 172.21.15.1:0/3857210612 wait complete. 2024-05-04T03:19:52.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:52 smithi077 ceph-mon[33531]: pgmap v412: 97 pgs: 97 active+clean; 672 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-04T03:19:52.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:52 smithi072 ceph-mon[33332]: pgmap v412: 97 pgs: 97 active+clean; 672 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-04T03:19:53.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:52 smithi001 ceph-mon[29731]: pgmap v412: 97 pgs: 97 active+clean; 672 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-04T03:19:54.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:54 smithi077 ceph-mon[33531]: pgmap v413: 97 pgs: 97 active+clean; 673 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-04T03:19:54.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:54 smithi072 ceph-mon[33332]: pgmap v413: 97 pgs: 97 active+clean; 673 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-04T03:19:55.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:54 smithi001 ceph-mon[29731]: pgmap v413: 97 pgs: 97 active+clean; 673 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 938 B/s wr, 8 op/s 2024-05-04T03:19:56.930 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:56 smithi077 ceph-mon[33531]: pgmap v414: 97 pgs: 97 active+clean; 673 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-04T03:19:56.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:56 smithi072 ceph-mon[33332]: pgmap v414: 97 pgs: 97 active+clean; 673 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-04T03:19:57.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:56 smithi001 ceph-mon[29731]: pgmap v414: 97 pgs: 97 active+clean; 673 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-04T03:19:58.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:19:58 smithi077 ceph-mon[33531]: pgmap v415: 97 pgs: 97 active+clean; 673 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-04T03:19:58.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:19:58 smithi072 ceph-mon[33332]: pgmap v415: 97 pgs: 97 active+clean; 673 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-04T03:19:59.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:19:58 smithi001 ceph-mon[29731]: pgmap v415: 97 pgs: 97 active+clean; 673 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-04T03:20:00.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:00 smithi077 ceph-mon[33531]: pgmap v416: 97 pgs: 97 active+clean; 673 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-04T03:20:00.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:00 smithi077 ceph-mon[33531]: overall HEALTH_OK 2024-05-04T03:20:00.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:00 smithi072 ceph-mon[33332]: pgmap v416: 97 pgs: 97 active+clean; 673 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-04T03:20:00.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:00 smithi072 ceph-mon[33332]: overall HEALTH_OK 2024-05-04T03:20:01.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:00 smithi001 ceph-mon[29731]: pgmap v416: 97 pgs: 97 active+clean; 673 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-04T03:20:01.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:00 smithi001 ceph-mon[29731]: overall HEALTH_OK 2024-05-04T03:20:02.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:02 smithi077 ceph-mon[33531]: pgmap v417: 97 pgs: 97 active+clean; 673 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-04T03:20:02.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:02 smithi072 ceph-mon[33332]: pgmap v417: 97 pgs: 97 active+clean; 673 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-04T03:20:03.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:02 smithi001 ceph-mon[29731]: pgmap v417: 97 pgs: 97 active+clean; 673 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-04T03:20:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.873+0000 7fef16fe3640 1 -- 172.21.15.1:0/1253447308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef10102b50 msgr2=0x7fef10103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.873+0000 7fef16fe3640 1 --2- 172.21.15.1:0/1253447308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef10102b50 0x7fef10103850 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fef04009900 tx=0x7fef0402a410 comp rx=0 tx=0).stop 2024-05-04T03:20:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.873+0000 7fef16fe3640 1 -- 172.21.15.1:0/1253447308 shutdown_connections 2024-05-04T03:20:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.873+0000 7fef16fe3640 1 --2- 172.21.15.1:0/1253447308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef10108a90 0x7fef1010aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.873+0000 7fef16fe3640 1 --2- 172.21.15.1:0/1253447308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef10103d90 0x7fef101041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.873+0000 7fef16fe3640 1 --2- 172.21.15.1:0/1253447308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef10102b50 0x7fef10103850 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.873+0000 7fef16fe3640 1 -- 172.21.15.1:0/1253447308 >> 172.21.15.1:0/1253447308 conn(0x7fef100fdd40 msgr2=0x7fef101001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.873+0000 7fef16fe3640 1 -- 172.21.15.1:0/1253447308 shutdown_connections 2024-05-04T03:20:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.873+0000 7fef16fe3640 1 -- 172.21.15.1:0/1253447308 wait complete. 2024-05-04T03:20:04.879 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.874+0000 7fef16fe3640 1 Processor -- start 2024-05-04T03:20:04.879 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.874+0000 7fef16fe3640 1 -- start start 2024-05-04T03:20:04.879 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.875+0000 7fef16fe3640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef10102b50 0x7fef1019a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:04.879 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.875+0000 7fef16fe3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef10103d90 0x7fef1019a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.875+0000 7fef16fe3640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef10108a90 0x7fef101a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.875+0000 7fef14d7f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef10102b50 0x7fef1019a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.875+0000 7fef16fe3640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fef1010f740 con 0x7fef10102b50 2024-05-04T03:20:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.875+0000 7fef14d7f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef10102b50 0x7fef1019a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52512/0 (socket says 172.21.15.1:52512) 2024-05-04T03:20:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.875+0000 7fef14d7f640 1 -- 172.21.15.1:0/3134861908 learned_addr learned my addr 172.21.15.1:0/3134861908 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.875+0000 7fef16fe3640 1 -- 172.21.15.1:0/3134861908 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fef1010f5c0 con 0x7fef10108a90 2024-05-04T03:20:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.875+0000 7fef16fe3640 1 -- 172.21.15.1:0/3134861908 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fef1010f8c0 con 0x7fef10103d90 2024-05-04T03:20:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.875+0000 7fef15580640 1 --2- 172.21.15.1:0/3134861908 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef10108a90 0x7fef101a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.875+0000 7fef0ffff640 1 --2- 172.21.15.1:0/3134861908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef10103d90 0x7fef1019a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.875+0000 7fef14d7f640 1 -- 172.21.15.1:0/3134861908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef10103d90 msgr2=0x7fef1019a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.875+0000 7fef14d7f640 1 --2- 172.21.15.1:0/3134861908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef10103d90 0x7fef1019a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.875+0000 7fef14d7f640 1 -- 172.21.15.1:0/3134861908 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef10108a90 msgr2=0x7fef101a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.875+0000 7fef14d7f640 1 --2- 172.21.15.1:0/3134861908 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef10108a90 0x7fef101a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.875+0000 7fef14d7f640 1 -- 172.21.15.1:0/3134861908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef101a1e20 con 0x7fef10102b50 2024-05-04T03:20:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.875+0000 7fef14d7f640 1 --2- 172.21.15.1:0/3134861908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef10102b50 0x7fef1019a2e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fef04006010 tx=0x7fef04004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.876+0000 7fef0dffb640 1 -- 172.21.15.1:0/3134861908 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef04035070 con 0x7fef10102b50 2024-05-04T03:20:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.876+0000 7fef16fe3640 1 -- 172.21.15.1:0/3134861908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef1010bc30 con 0x7fef10102b50 2024-05-04T03:20:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.876+0000 7fef0dffb640 1 -- 172.21.15.1:0/3134861908 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef0403dc10 con 0x7fef10102b50 2024-05-04T03:20:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.876+0000 7fef0dffb640 1 -- 172.21.15.1:0/3134861908 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef04003e20 con 0x7fef10102b50 2024-05-04T03:20:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.876+0000 7fef16fe3640 1 -- 172.21.15.1:0/3134861908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef1010c170 con 0x7fef10102b50 2024-05-04T03:20:04.883 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.879+0000 7fef1df6f740 1 Processor -- start 2024-05-04T03:20:04.883 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.879+0000 7fef1df6f740 1 -- start start 2024-05-04T03:20:04.883 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.879+0000 7fef1df6f740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555aabf9f1b0 0x555aac340a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:04.883 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.879+0000 7fef1df6f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555aabf762f0 0x555aac340fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:04.883 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.879+0000 7fef1df6f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aabf7e4d0 0x555aac347ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:04.883 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.879+0000 7fef1df6f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555aac2a55d0 con 0x555aabf762f0 2024-05-04T03:20:04.883 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.879+0000 7fef1df6f740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x555aac25e890 con 0x555aabf9f1b0 2024-05-04T03:20:04.883 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.879+0000 7fef1df6f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555aac2a1590 con 0x555aabf7e4d0 2024-05-04T03:20:04.883 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.879+0000 7fef0ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aabf7e4d0 0x555aac347ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.879+0000 7fef15580640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555aabf9f1b0 0x555aac340a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.879+0000 7fef0ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aabf7e4d0 0x555aac347ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.1:35500/0 (socket says 172.21.15.1:35500) 2024-05-04T03:20:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.879+0000 7fef15580640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555aabf9f1b0 0x555aac340a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:44376/0 (socket says 172.21.15.1:44376) 2024-05-04T03:20:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.879+0000 7fef0ffff640 1 -- 172.21.15.1:0/2628264563 learned_addr learned my addr 172.21.15.1:0/2628264563 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.880+0000 7fef15580640 1 -- 172.21.15.1:0/2628264563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aabf7e4d0 msgr2=0x555aac347ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.880+0000 7fef15580640 1 --2- 172.21.15.1:0/2628264563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aabf7e4d0 0x555aac347ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.880+0000 7fef15580640 1 -- 172.21.15.1:0/2628264563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555aabf762f0 msgr2=0x555aac340fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-04T03:20:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.880+0000 7fef15580640 1 --2- 172.21.15.1:0/2628264563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555aabf762f0 0x555aac340fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.880+0000 7fef15580640 1 -- 172.21.15.1:0/2628264563 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aabfb7210 con 0x555aabf9f1b0 2024-05-04T03:20:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.880+0000 7fef0ffff640 1 --2- 172.21.15.1:0/2628264563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aabf7e4d0 0x555aac347ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:20:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.880+0000 7fef15580640 1 --2- 172.21.15.1:0/2628264563 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555aabf9f1b0 0x555aac340a60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fef08009f00 tx=0x7fef08011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f02cce985ab6b84 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.880+0000 7fef0dffb640 1 -- 172.21.15.1:0/3134861908 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7fef040035f0 con 0x7fef10102b50 2024-05-04T03:20:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.880+0000 7feefaffd640 1 -- 172.21.15.1:0/2628264563 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef08004d20 con 0x555aabf9f1b0 2024-05-04T03:20:04.884 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.880+0000 7feefaffd640 1 -- 172.21.15.1:0/2628264563 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef08004030 con 0x555aabf9f1b0 2024-05-04T03:20:04.885 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.880+0000 7feefaffd640 1 -- 172.21.15.1:0/2628264563 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef08005240 con 0x555aabf9f1b0 2024-05-04T03:20:04.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.881+0000 7fef0dffb640 1 --2- 172.21.15.1:0/3134861908 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fef000b28a0 0x7fef000b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:04.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.881+0000 7fef14d7f640 1 --2- 172.21.15.1:0/3134861908 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fef000b28a0 0x7fef000b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.881+0000 7fef0dffb640 1 -- 172.21.15.1:0/3134861908 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fef04106d30 con 0x7fef10102b50 2024-05-04T03:20:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.881+0000 7fef1df6f740 1 -- 172.21.15.1:0/2628264563 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555aabf9f1b0 msgr2=0x555aac340a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.881+0000 7fef1df6f740 1 --2- 172.21.15.1:0/2628264563 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555aabf9f1b0 0x555aac340a60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fef08009f00 tx=0x7fef08011810 comp rx=0 tx=0).stop 2024-05-04T03:20:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.881+0000 7fef1df6f740 1 -- 172.21.15.1:0/2628264563 shutdown_connections 2024-05-04T03:20:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.881+0000 7fef1df6f740 1 --2- 172.21.15.1:0/2628264563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aabf7e4d0 0x555aac347ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.881+0000 7fef1df6f740 1 --2- 172.21.15.1:0/2628264563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555aabf762f0 0x555aac340fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.881+0000 7fef1df6f740 1 --2- 172.21.15.1:0/2628264563 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555aabf9f1b0 0x555aac340a60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.881+0000 7fef1df6f740 1 -- 172.21.15.1:0/2628264563 >> 172.21.15.1:0/2628264563 conn(0x555aabf93300 msgr2=0x555aac33c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.881+0000 7fef1df6f740 1 -- 172.21.15.1:0/2628264563 shutdown_connections 2024-05-04T03:20:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.881+0000 7fef1df6f740 1 -- 172.21.15.1:0/2628264563 wait complete. 2024-05-04T03:20:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.881+0000 7fef14d7f640 1 --2- 172.21.15.1:0/3134861908 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fef000b28a0 0x7fef000b4db0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fef040ce110 tx=0x7fef04097d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:04.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.882+0000 7fef1df6f740 1 -- start start 2024-05-04T03:20:04.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.882+0000 7fef1df6f740 1 Processor -- start 2024-05-04T03:20:04.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.883+0000 7fef1df6f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aabf762f0 0x555aac34a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:04.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.883+0000 7fef1df6f740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555aabf7e4d0 0x555aac34ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:04.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.883+0000 7fef1df6f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555aabf9f1b0 0x555aac351b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:04.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.883+0000 7fef1df6f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555aac348930 con 0x555aabf9f1b0 2024-05-04T03:20:04.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.883+0000 7fef1df6f740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x555aac262370 con 0x555aabf7e4d0 2024-05-04T03:20:04.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.883+0000 7fef1df6f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555aac348ab0 con 0x555aabf762f0 2024-05-04T03:20:04.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.883+0000 7fef15580640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555aabf9f1b0 0x555aac351b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:04.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.883+0000 7fef15580640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555aabf9f1b0 0x555aac351b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52518/0 (socket says 172.21.15.1:52518) 2024-05-04T03:20:04.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.883+0000 7fef15580640 1 -- 172.21.15.1:0/3591853751 learned_addr learned my addr 172.21.15.1:0/3591853751 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:04.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.883+0000 7fef0ffff640 1 --2- 172.21.15.1:0/3591853751 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555aabf7e4d0 0x555aac34ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:04.890 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.883+0000 7fef15580640 1 --2- 172.21.15.1:0/3591853751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aabf762f0 0x555aac34a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.883+0000 7fef1df6f740 10 client.0 _ll_register_callbacks cb 0x555aabfb6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:20:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.883+0000 7fef15580640 1 -- 172.21.15.1:0/3591853751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aabf762f0 msgr2=0x555aac34a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.883+0000 7fef15580640 1 --2- 172.21.15.1:0/3591853751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aabf762f0 0x555aac34a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.883+0000 7fef15580640 1 -- 172.21.15.1:0/3591853751 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555aabf7e4d0 msgr2=0x555aac34ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.883+0000 7fef15580640 1 --2- 172.21.15.1:0/3591853751 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555aabf7e4d0 0x555aac34ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.883+0000 7fef15580640 1 -- 172.21.15.1:0/3591853751 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aac083620 con 0x555aabf9f1b0 2024-05-04T03:20:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.883+0000 7fef15580640 1 --2- 172.21.15.1:0/3591853751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555aabf9f1b0 0x555aac351b00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fef08005fd0 tx=0x7fef08004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c6064bb879799 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.884+0000 7feeeffff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:20:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.884+0000 7feeeffff640 1 -- 172.21.15.1:0/3591853751 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef080042c0 con 0x555aabf9f1b0 2024-05-04T03:20:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.884+0000 7feeeffff640 1 -- 172.21.15.1:0/3591853751 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fef08004c40 con 0x555aabf9f1b0 2024-05-04T03:20:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.884+0000 7fef1df6f740 1 -- 172.21.15.1:0/3591853751 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef08004030 con 0x555aabf9f1b0 2024-05-04T03:20:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.884+0000 7feeeffff640 1 -- 172.21.15.1:0/3591853751 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fef08030070 con 0x555aabf9f1b0 2024-05-04T03:20:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.884+0000 7fef1df6f740 10 client.15330 fetch_fsmap learned FSMap version 26 2024-05-04T03:20:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.884+0000 7fef1df6f740 1 -- 172.21.15.1:0/3591853751 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555aac3549f0 con 0x555aabf9f1b0 2024-05-04T03:20:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.884+0000 7feeeffff640 1 -- 172.21.15.1:0/3591853751 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef0800f480 con 0x555aabf9f1b0 2024-05-04T03:20:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.885+0000 7feeeffff640 1 -- 172.21.15.1:0/3591853751 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fef08018070 con 0x555aabf9f1b0 2024-05-04T03:20:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.885+0000 7fef1df6f740 10 client.15330 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:20:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.885+0000 7fef1df6f740 10 client.15330 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:20:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.885+0000 7fef1df6f740 20 client.15330 populate_metadata read hostname 'smithi001' 2024-05-04T03:20:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.885+0000 7fef1df6f740 1 --2- 172.21.15.1:0/3591853751 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x555aabf5dd80 0x555aac357f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.885+0000 7fef1df6f740 4 client.15330 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:20:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.885+0000 7fef1df6f740 1 -- 172.21.15.1:0/3591853751 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fef08004030 con 0x555aabf5dd80 2024-05-04T03:20:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.885+0000 7fef0ffff640 1 --2- 172.21.15.1:0/3591853751 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x555aabf5dd80 0x555aac357f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.886+0000 7feeeffff640 10 client.15330 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.886+0000 7fef0ffff640 1 --2- 172.21.15.1:0/3591853751 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x555aabf5dd80 0x555aac357f80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e27d8b6fb6ca742 server_cookie=dce3e61f78a616ac in_seq=0 out_seq=0 2024-05-04T03:20:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.886+0000 7feeeffff640 1 -- 172.21.15.1:0/3591853751 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fef08004030 con 0x555aabf5dd80 2024-05-04T03:20:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.886+0000 7feeeffff640 10 client.15330 handle_command_reply: tid=0 2024-05-04T03:20:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.886+0000 7fef1df6f740 1 client.15330 shutdown 2024-05-04T03:20:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.886+0000 7fef1df6f740 1 -- 172.21.15.1:0/3591853751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555aabf9f1b0 msgr2=0x555aac351b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.886+0000 7fef1df6f740 1 --2- 172.21.15.1:0/3591853751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555aabf9f1b0 0x555aac351b00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fef08005fd0 tx=0x7fef08004790 comp rx=0 tx=0).stop 2024-05-04T03:20:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.887+0000 7fef1df6f740 1 -- 172.21.15.1:0/3591853751 shutdown_connections 2024-05-04T03:20:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.887+0000 7fef1df6f740 1 --2- 172.21.15.1:0/3591853751 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x555aabf5dd80 0x555aac357f80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.887+0000 7fef1df6f740 1 --2- 172.21.15.1:0/3591853751 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555aabf7e4d0 0x555aac34ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.887+0000 7fef1df6f740 1 --2- 172.21.15.1:0/3591853751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555aabf9f1b0 0x555aac351b00 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.887+0000 7fef1df6f740 1 --2- 172.21.15.1:0/3591853751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aabf762f0 0x555aac34a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.887+0000 7feeeffff640 0 client.15330 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.887+0000 7fef1df6f740 1 -- 172.21.15.1:0/3591853751 >> 172.21.15.1:0/3591853751 conn(0x555aabf93300 msgr2=0x555aac33b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.887+0000 7fef1df6f740 1 -- 172.21.15.1:0/3591853751 shutdown_connections 2024-05-04T03:20:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.887+0000 7fef1df6f740 1 -- 172.21.15.1:0/3591853751 wait complete. 2024-05-04T03:20:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.887+0000 7fef1df6f740 20 client.15330 trim_cache size 0 max 16384 2024-05-04T03:20:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.902+0000 7fef1df6f740 1 Processor -- start 2024-05-04T03:20:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.902+0000 7fef1df6f740 1 -- start start 2024-05-04T03:20:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.902+0000 7fef1df6f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555aabf5dd80 0x555aac33bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.902+0000 7fef1df6f740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555aabf762f0 0x555aac33c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.902+0000 7fef1df6f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aabf7e4d0 0x555aac351a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.902+0000 7fef1df6f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fef101ab880 con 0x555aabf5dd80 2024-05-04T03:20:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.902+0000 7fef1df6f740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fef101a9150 con 0x555aabf762f0 2024-05-04T03:20:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.902+0000 7fef1df6f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fef101a6fc0 con 0x555aabf7e4d0 2024-05-04T03:20:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.903+0000 7fef15580640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555aabf5dd80 0x555aac33bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.903+0000 7fef15580640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555aabf5dd80 0x555aac33bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52522/0 (socket says 172.21.15.1:52522) 2024-05-04T03:20:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.903+0000 7fef15580640 1 -- 172.21.15.1:0/3248190128 learned_addr learned my addr 172.21.15.1:0/3248190128 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.903+0000 7fef0ffff640 1 --2- 172.21.15.1:0/3248190128 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555aabf762f0 0x555aac33c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.903+0000 7fef15580640 1 -- 172.21.15.1:0/3248190128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aabf7e4d0 msgr2=0x555aac351a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.903+0000 7fef15580640 1 --2- 172.21.15.1:0/3248190128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aabf7e4d0 0x555aac351a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.903+0000 7fef15580640 1 -- 172.21.15.1:0/3248190128 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555aabf762f0 msgr2=0x555aac33c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.903+0000 7fef15580640 1 --2- 172.21.15.1:0/3248190128 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555aabf762f0 0x555aac33c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.903+0000 7fef15580640 1 -- 172.21.15.1:0/3248190128 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aac33cd00 con 0x555aabf5dd80 2024-05-04T03:20:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.903+0000 7fef0ffff640 1 --2- 172.21.15.1:0/3248190128 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555aabf762f0 0x555aac33c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:20:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.903+0000 7fef15580640 1 --2- 172.21.15.1:0/3248190128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555aabf5dd80 0x555aac33bd60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fef08014020 tx=0x7fef08009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56f2396d15955f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.903+0000 7feefa7fc640 1 -- 172.21.15.1:0/3248190128 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0802a4e0 con 0x555aabf5dd80 2024-05-04T03:20:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.903+0000 7feefa7fc640 1 -- 172.21.15.1:0/3248190128 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef08003f80 con 0x555aabf5dd80 2024-05-04T03:20:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.903+0000 7feefa7fc640 1 -- 172.21.15.1:0/3248190128 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0800f6e0 con 0x555aabf5dd80 2024-05-04T03:20:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.904+0000 7fef1df6f740 1 -- 172.21.15.1:0/3248190128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555aabf5dd80 msgr2=0x555aac33bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.904+0000 7fef1df6f740 1 --2- 172.21.15.1:0/3248190128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555aabf5dd80 0x555aac33bd60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fef08014020 tx=0x7fef08009b90 comp rx=0 tx=0).stop 2024-05-04T03:20:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.904+0000 7fef1df6f740 1 -- 172.21.15.1:0/3248190128 shutdown_connections 2024-05-04T03:20:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.904+0000 7fef1df6f740 1 --2- 172.21.15.1:0/3248190128 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555aabf762f0 0x555aac33c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.904+0000 7fef1df6f740 1 --2- 172.21.15.1:0/3248190128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aabf7e4d0 0x555aac351a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.904+0000 7fef1df6f740 1 --2- 172.21.15.1:0/3248190128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555aabf5dd80 0x555aac33bd60 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.904+0000 7fef1df6f740 1 -- 172.21.15.1:0/3248190128 >> 172.21.15.1:0/3248190128 conn(0x555aabf93300 msgr2=0x555aac344ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.904+0000 7fef1df6f740 1 -- 172.21.15.1:0/3248190128 shutdown_connections 2024-05-04T03:20:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.904+0000 7fef1df6f740 1 -- 172.21.15.1:0/3248190128 wait complete. 2024-05-04T03:20:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.905+0000 7fef1df6f740 1 -- start start 2024-05-04T03:20:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.905+0000 7fef1df6f740 1 Processor -- start 2024-05-04T03:20:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.905+0000 7fef1df6f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555aabf5dd80 0x555aac34d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.905+0000 7fef1df6f740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555aabf762f0 0x555aac34d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.905+0000 7fef1df6f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aabf7e4d0 0x555aac3be320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.905+0000 7fef1df6f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fef101a7610 con 0x555aabf5dd80 2024-05-04T03:20:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.905+0000 7fef1df6f740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fef101a8a80 con 0x555aabf762f0 2024-05-04T03:20:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.905+0000 7fef1df6f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fef101a7ce0 con 0x555aabf7e4d0 2024-05-04T03:20:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.906+0000 7fef1df6f740 10 client.0 _ll_register_callbacks cb 0x7feefc00b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:20:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.906+0000 7fef15580640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555aabf5dd80 0x555aac34d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.906+0000 7fef15580640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555aabf5dd80 0x555aac34d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52526/0 (socket says 172.21.15.1:52526) 2024-05-04T03:20:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.906+0000 7fef15580640 1 -- 172.21.15.1:0/3812350628 learned_addr learned my addr 172.21.15.1:0/3812350628 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.906+0000 7fef0ffff640 1 --2- 172.21.15.1:0/3812350628 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555aabf762f0 0x555aac34d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.906+0000 7fef15580640 1 -- 172.21.15.1:0/3812350628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aabf7e4d0 msgr2=0x555aac3be320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:04.913 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.906+0000 7fef15580640 1 --2- 172.21.15.1:0/3812350628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aabf7e4d0 0x555aac3be320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.906+0000 7fef15580640 1 -- 172.21.15.1:0/3812350628 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555aabf762f0 msgr2=0x555aac34d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.906+0000 7fef15580640 1 --2- 172.21.15.1:0/3812350628 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555aabf762f0 0x555aac34d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.906+0000 7fef15580640 1 -- 172.21.15.1:0/3812350628 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aac3be860 con 0x555aabf5dd80 2024-05-04T03:20:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.906+0000 7fef15580640 1 --2- 172.21.15.1:0/3812350628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555aabf5dd80 0x555aac34d300 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fef08011850 tx=0x7fef080047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d481d62dde24ddc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.906+0000 7feeeffff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:20:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.906+0000 7feeeffff640 1 -- 172.21.15.1:0/3812350628 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef08043070 con 0x555aabf5dd80 2024-05-04T03:20:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.906+0000 7feeeffff640 1 -- 172.21.15.1:0/3812350628 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fef0802a910 con 0x555aabf5dd80 2024-05-04T03:20:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.906+0000 7fef1df6f740 1 -- 172.21.15.1:0/3812350628 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef08003f80 con 0x555aabf5dd80 2024-05-04T03:20:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.907+0000 7feeeffff640 1 -- 172.21.15.1:0/3812350628 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fef0802ab10 con 0x555aabf5dd80 2024-05-04T03:20:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.907+0000 7fef1df6f740 10 client.15336 fetch_fsmap learned FSMap version 26 2024-05-04T03:20:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.907+0000 7fef1df6f740 1 -- 172.21.15.1:0/3812350628 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555aac3bfa90 con 0x555aabf5dd80 2024-05-04T03:20:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.907+0000 7feeeffff640 1 -- 172.21.15.1:0/3812350628 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef0800f930 con 0x555aabf5dd80 2024-05-04T03:20:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.907+0000 7feeeffff640 1 -- 172.21.15.1:0/3812350628 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fef08018070 con 0x555aabf5dd80 2024-05-04T03:20:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.907+0000 7fef1df6f740 10 client.15336 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:20:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.907+0000 7fef1df6f740 10 client.15336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:20:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.907+0000 7fef1df6f740 20 client.15336 populate_metadata read hostname 'smithi001' 2024-05-04T03:20:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.907+0000 7fef1df6f740 1 --2- 172.21.15.1:0/3812350628 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x555aabf9f1b0 0x555aac3c2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.907+0000 7fef1df6f740 4 client.15336 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:20:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.907+0000 7fef1df6f740 1 -- 172.21.15.1:0/3812350628 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fef08003f80 con 0x555aabf9f1b0 2024-05-04T03:20:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.907+0000 7fef0ffff640 1 --2- 172.21.15.1:0/3812350628 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x555aabf9f1b0 0x555aac3c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.908+0000 7feeeffff640 10 client.15336 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.908+0000 7fef0ffff640 1 --2- 172.21.15.1:0/3812350628 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x555aabf9f1b0 0x555aac3c2380 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d87d3b84a9abb200 server_cookie=b6cdffa3415f5877 in_seq=0 out_seq=0 2024-05-04T03:20:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.914+0000 7feeeffff640 1 -- 172.21.15.1:0/3812350628 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fef08003f80 con 0x555aabf9f1b0 2024-05-04T03:20:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.914+0000 7feeeffff640 10 client.15336 handle_command_reply: tid=0 2024-05-04T03:20:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.914+0000 7fef1df6f740 1 client.15336 shutdown 2024-05-04T03:20:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.914+0000 7fef1df6f740 1 -- 172.21.15.1:0/3812350628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555aabf5dd80 msgr2=0x555aac34d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.914+0000 7fef1df6f740 1 --2- 172.21.15.1:0/3812350628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555aabf5dd80 0x555aac34d300 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fef08011850 tx=0x7fef080047c0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.915+0000 7fef1df6f740 1 -- 172.21.15.1:0/3812350628 shutdown_connections 2024-05-04T03:20:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.915+0000 7fef1df6f740 1 --2- 172.21.15.1:0/3812350628 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x555aabf9f1b0 0x555aac3c2380 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.915+0000 7fef1df6f740 1 --2- 172.21.15.1:0/3812350628 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555aabf762f0 0x555aac34d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.915+0000 7fef1df6f740 1 --2- 172.21.15.1:0/3812350628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aabf7e4d0 0x555aac3be320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.915+0000 7feeeffff640 0 client.15336 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.915+0000 7fef1df6f740 1 --2- 172.21.15.1:0/3812350628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555aabf5dd80 0x555aac34d300 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.915+0000 7fef1df6f740 1 -- 172.21.15.1:0/3812350628 >> 172.21.15.1:0/3812350628 conn(0x555aabf93300 msgr2=0x555aac33bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.915+0000 7fef1df6f740 1 -- 172.21.15.1:0/3812350628 shutdown_connections 2024-05-04T03:20:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.915+0000 7fef1df6f740 1 -- 172.21.15.1:0/3812350628 wait complete. 2024-05-04T03:20:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.915+0000 7fef1df6f740 20 client.15336 trim_cache size 0 max 16384 2024-05-04T03:20:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:20:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:20:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:20:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:20:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.916+0000 7fef16fe3640 1 -- 172.21.15.1:0/3134861908 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fef000b28a0 msgr2=0x7fef000b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.916+0000 7fef16fe3640 1 --2- 172.21.15.1:0/3134861908 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fef000b28a0 0x7fef000b4db0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fef040ce110 tx=0x7fef04097d00 comp rx=0 tx=0).stop 2024-05-04T03:20:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.916+0000 7fef16fe3640 1 -- 172.21.15.1:0/3134861908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef10102b50 msgr2=0x7fef1019a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.916+0000 7fef16fe3640 1 --2- 172.21.15.1:0/3134861908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef10102b50 0x7fef1019a2e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fef04006010 tx=0x7fef04004270 comp rx=0 tx=0).stop 2024-05-04T03:20:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.916+0000 7fef16fe3640 1 -- 172.21.15.1:0/3134861908 shutdown_connections 2024-05-04T03:20:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.916+0000 7fef16fe3640 1 --2- 172.21.15.1:0/3134861908 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fef000b28a0 0x7fef000b4db0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.916+0000 7fef16fe3640 1 --2- 172.21.15.1:0/3134861908 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fef10108a90 0x7fef101a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.916+0000 7fef16fe3640 1 --2- 172.21.15.1:0/3134861908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef10103d90 0x7fef1019a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.916+0000 7fef16fe3640 1 --2- 172.21.15.1:0/3134861908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fef10102b50 0x7fef1019a2e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.916+0000 7fef16fe3640 1 -- 172.21.15.1:0/3134861908 >> 172.21.15.1:0/3134861908 conn(0x7fef100fdd40 msgr2=0x7fef10104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.916+0000 7fef16fe3640 1 -- 172.21.15.1:0/3134861908 shutdown_connections 2024-05-04T03:20:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:04.916+0000 7fef16fe3640 1 -- 172.21.15.1:0/3134861908 wait complete. 2024-05-04T03:20:04.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:04 smithi077 ceph-mon[33531]: pgmap v418: 97 pgs: 97 active+clean; 673 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-04T03:20:04.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:04 smithi077 ceph-mon[33531]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-05-04T03:20:04.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:04 smithi072 ceph-mon[33332]: pgmap v418: 97 pgs: 97 active+clean; 673 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-04T03:20:04.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:04 smithi072 ceph-mon[33332]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-05-04T03:20:05.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:04 smithi001 ceph-mon[29731]: pgmap v418: 97 pgs: 97 active+clean; 673 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-04T03:20:05.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:04 smithi001 ceph-mon[29731]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-05-04T03:20:05.735 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:20:06.086 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:20:06.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:06 smithi001 ceph-mon[29731]: pgmap v419: 97 pgs: 97 active+clean; 673 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-04T03:20:06.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:06 smithi077 ceph-mon[33531]: pgmap v419: 97 pgs: 97 active+clean; 673 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-04T03:20:06.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:06 smithi072 ceph-mon[33332]: pgmap v419: 97 pgs: 97 active+clean; 673 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-04T03:20:08.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.007+0000 7f6b75eab700 1 -- 172.21.15.1:0/100902985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b701034c0 msgr2=0x7f6b701038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:08.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.007+0000 7f6b75eab700 1 --2- 172.21.15.1:0/100902985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b701034c0 0x7f6b701038d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009990 tx=0x7f6b60009ca0 comp rx=0 tx=0).stop 2024-05-04T03:20:08.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.007+0000 7f6b75eab700 1 -- 172.21.15.1:0/100902985 shutdown_connections 2024-05-04T03:20:08.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.007+0000 7f6b75eab700 1 --2- 172.21.15.1:0/100902985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b70108740 0x7f6b701100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:08.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.007+0000 7f6b75eab700 1 --2- 172.21.15.1:0/100902985 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b701034c0 0x7f6b701038d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:08.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.007+0000 7f6b75eab700 1 --2- 172.21.15.1:0/100902985 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b70107df0 0x7f6b70108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:08.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.007+0000 7f6b75eab700 1 -- 172.21.15.1:0/100902985 >> 172.21.15.1:0/100902985 conn(0x7f6b700fedc0 msgr2=0x7f6b701011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:08.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.008+0000 7f6b75eab700 1 -- 172.21.15.1:0/100902985 shutdown_connections 2024-05-04T03:20:08.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.008+0000 7f6b75eab700 1 -- 172.21.15.1:0/100902985 wait complete. 2024-05-04T03:20:08.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.009+0000 7f6b75eab700 1 Processor -- start 2024-05-04T03:20:08.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.009+0000 7f6b75eab700 1 -- start start 2024-05-04T03:20:08.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.009+0000 7f6b75eab700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b701034c0 0x7f6b7019d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:08.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.009+0000 7f6b75eab700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b70107df0 0x7f6b7019d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:08.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.009+0000 7f6b75eab700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b70108740 0x7f6b701a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:08.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.009+0000 7f6b75eab700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f6b70114510 con 0x7f6b70107df0 2024-05-04T03:20:08.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.009+0000 7f6b75eab700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6b70114390 con 0x7f6b701034c0 2024-05-04T03:20:08.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.009+0000 7f6b75eab700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6b70114690 con 0x7f6b70108740 2024-05-04T03:20:08.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.009+0000 7f6b6effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b70107df0 0x7f6b7019d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:08.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.009+0000 7f6b6effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b70107df0 0x7f6b7019d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52548/0 (socket says 172.21.15.1:52548) 2024-05-04T03:20:08.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.009+0000 7f6b6effd700 1 -- 172.21.15.1:0/3679367482 learned_addr learned my addr 172.21.15.1:0/3679367482 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:08.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.009+0000 7f6b6f7fe700 1 --2- 172.21.15.1:0/3679367482 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b701034c0 0x7f6b7019d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:08.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.010+0000 7f6b6ffff700 1 --2- 172.21.15.1:0/3679367482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b70108740 0x7f6b701a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:08.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.010+0000 7f6b6effd700 1 -- 172.21.15.1:0/3679367482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b70108740 msgr2=0x7f6b701a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:08.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.010+0000 7f6b6effd700 1 --2- 172.21.15.1:0/3679367482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b70108740 0x7f6b701a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:08.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.010+0000 7f6b6effd700 1 -- 172.21.15.1:0/3679367482 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b701034c0 msgr2=0x7f6b7019d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:08.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.010+0000 7f6b6effd700 1 --2- 172.21.15.1:0/3679367482 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b701034c0 0x7f6b7019d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:08.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.010+0000 7f6b6effd700 1 -- 172.21.15.1:0/3679367482 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b701a1e60 con 0x7f6b70107df0 2024-05-04T03:20:08.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.010+0000 7f6b6effd700 1 --2- 172.21.15.1:0/3679367482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b70107df0 0x7f6b7019d5e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6b6400e970 tx=0x7f6b6400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:08.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.010+0000 7f6b6cff9700 1 -- 172.21.15.1:0/3679367482 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6400cbd0 con 0x7f6b70107df0 2024-05-04T03:20:08.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.011+0000 7f6b75eab700 1 -- 172.21.15.1:0/3679367482 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b701a2150 con 0x7f6b70107df0 2024-05-04T03:20:08.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.010+0000 7f6b6cff9700 1 -- 172.21.15.1:0/3679367482 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b6400cd70 con 0x7f6b70107df0 2024-05-04T03:20:08.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.011+0000 7f6b6cff9700 1 -- 172.21.15.1:0/3679367482 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b64005680 con 0x7f6b70107df0 2024-05-04T03:20:08.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.011+0000 7f6b75eab700 1 -- 172.21.15.1:0/3679367482 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b701a2680 con 0x7f6b70107df0 2024-05-04T03:20:08.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.012+0000 7f6b75eab700 1 -- 172.21.15.1:0/3679367482 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b700512c0 con 0x7f6b70107df0 2024-05-04T03:20:08.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.015+0000 7f6b6cff9700 1 -- 172.21.15.1:0/3679367482 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f6b64004750 con 0x7f6b70107df0 2024-05-04T03:20:08.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.016+0000 7f6b6cff9700 1 --2- 172.21.15.1:0/3679367482 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6b580b2900 0x7f6b580b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:08.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.016+0000 7f6b6cff9700 1 -- 172.21.15.1:0/3679367482 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f6b640e2540 con 0x7f6b70107df0 2024-05-04T03:20:08.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.016+0000 7f6b6f7fe700 1 --2- 172.21.15.1:0/3679367482 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6b580b2900 0x7f6b580b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:08.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.017+0000 7f6b6f7fe700 1 --2- 172.21.15.1:0/3679367482 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6b580b2900 0x7f6b580b4e10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6b60006010 tx=0x7f6b6000b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:08.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.024+0000 7f6b6cff9700 1 -- 172.21.15.1:0/3679367482 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6b64054df0 con 0x7f6b70107df0 2024-05-04T03:20:08.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.552+0000 7f6b75eab700 1 -- 172.21.15.1:0/3679367482 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b7002e5f0 con 0x7f6b70107df0 2024-05-04T03:20:08.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.553+0000 7f6b6cff9700 1 -- 172.21.15.1:0/3679367482 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11844 (secure 0 0 0) 0x7f6b640aa320 con 0x7f6b70107df0 2024-05-04T03:20:08.556 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:20:08.556 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2186681910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2186681910},{"type":"v1","addr":"172.21.15.1:6835","nonce":2186681910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2747191047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2747191047},{"type":"v1","addr":"172.21.15.1:6837","nonce":2747191047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4115120540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4115120540},{"type":"v1","addr":"172.21.15.1:6839","nonce":4115120540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3405709780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3405709780},{"type":"v1","addr":"172.21.15.1:6841","nonce":3405709780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/3816098761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":3816098761},{"type":"v1","addr":"172.21.15.72:6833","nonce":3816098761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/379026128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":379026128},{"type":"v1","addr":"172.21.15.77:6833","nonce":379026128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:15:49.823471+0000","modified":"2024-05-04T03:18:45.058419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24472,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/3930984976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3930984976},{"type":"v1","addr":"172.21.15.72:6835","nonce":3930984976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/4279951305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4279951305},{"type":"v1","addr":"172.21.15.77:6835","nonce":4279951305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2003521417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2003521417},{"type":"v1","addr":"172.21.15.77:6837","nonce":2003521417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/1417614203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1417614203},{"type":"v1","addr":"172.21.15.72:6837","nonce":1417614203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.77:6839/922723189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":922723189},{"type":"v1","addr":"172.21.15.77:6839","nonce":922723189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2976050650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2976050650},{"type":"v1","addr":"172.21.15.72:6839","nonce":2976050650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24467,24472,24494]},"id":1}]} 2024-05-04T03:20:08.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.556+0000 7f6b75eab700 1 -- 172.21.15.1:0/3679367482 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6b580b2900 msgr2=0x7f6b580b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:08.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.556+0000 7f6b75eab700 1 --2- 172.21.15.1:0/3679367482 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6b580b2900 0x7f6b580b4e10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6b60006010 tx=0x7f6b6000b260 comp rx=0 tx=0).stop 2024-05-04T03:20:08.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.556+0000 7f6b75eab700 1 -- 172.21.15.1:0/3679367482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b70107df0 msgr2=0x7f6b7019d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:08.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.556+0000 7f6b75eab700 1 --2- 172.21.15.1:0/3679367482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b70107df0 0x7f6b7019d5e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6b6400e970 tx=0x7f6b6400ec80 comp rx=0 tx=0).stop 2024-05-04T03:20:08.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.556+0000 7f6b75eab700 1 -- 172.21.15.1:0/3679367482 shutdown_connections 2024-05-04T03:20:08.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.556+0000 7f6b75eab700 1 --2- 172.21.15.1:0/3679367482 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6b701034c0 0x7f6b7019d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:08.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.556+0000 7f6b75eab700 1 --2- 172.21.15.1:0/3679367482 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f6b580b2900 0x7f6b580b4e10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:08.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.556+0000 7f6b75eab700 1 --2- 172.21.15.1:0/3679367482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b70108740 0x7f6b701a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:08.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.556+0000 7f6b75eab700 1 --2- 172.21.15.1:0/3679367482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6b70107df0 0x7f6b7019d5e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:08.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.556+0000 7f6b75eab700 1 -- 172.21.15.1:0/3679367482 >> 172.21.15.1:0/3679367482 conn(0x7f6b700fedc0 msgr2=0x7f6b701011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:08.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.557+0000 7f6b75eab700 1 -- 172.21.15.1:0/3679367482 shutdown_connections 2024-05-04T03:20:08.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:08.557+0000 7f6b75eab700 1 -- 172.21.15.1:0/3679367482 wait complete. 2024-05-04T03:20:08.560 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 26 2024-05-04T03:20:08.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:08 smithi001 ceph-mon[29731]: pgmap v420: 97 pgs: 97 active+clean; 673 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-04T03:20:08.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:08 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:20:08.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:08 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3679367482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:20:08.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:08 smithi077 ceph-mon[33531]: pgmap v420: 97 pgs: 97 active+clean; 673 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-04T03:20:08.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:08 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:20:08.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:08 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/3679367482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:20:08.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:08 smithi072 ceph-mon[33332]: pgmap v420: 97 pgs: 97 active+clean; 673 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-04T03:20:08.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:08 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:20:08.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:08 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/3679367482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:20:09.307 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 27b22eee --release --await' 2024-05-04T03:20:09.308 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24494 quiesce db --set-id 27b22eee --release --await 2024-05-04T03:20:09.698 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:20:09.957 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:09 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:20:09.957 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:09 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:20:09.958 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:09 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:20:09.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:09 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:20:09.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:09 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:20:09.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:09 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:20:10.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:09 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:20:10.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:09 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:20:10.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:09 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:20:10.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:10 smithi072 ceph-mon[33332]: pgmap v421: 97 pgs: 97 active+clean; 674 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-04T03:20:11.124 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:10 smithi001 ceph-mon[29731]: pgmap v421: 97 pgs: 97 active+clean; 674 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-04T03:20:11.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:10 smithi077 ceph-mon[33531]: pgmap v421: 97 pgs: 97 active+clean; 674 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-04T03:20:11.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.589+0000 7f088f278700 1 -- 172.21.15.1:0/1372683074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0888107e00 msgr2=0x7f0888108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.589+0000 7f088f278700 1 --2- 172.21.15.1:0/1372683074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0888107e00 0x7f0888108210 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f08780099c0 tx=0x7f0878009cd0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.589+0000 7f088f278700 1 -- 172.21.15.1:0/1372683074 shutdown_connections 2024-05-04T03:20:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.589+0000 7f088f278700 1 --2- 172.21.15.1:0/1372683074 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0888103520 0x7f0888103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.589+0000 7f088f278700 1 --2- 172.21.15.1:0/1372683074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0888108750 0x7f08881100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.589+0000 7f088f278700 1 --2- 172.21.15.1:0/1372683074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0888107e00 0x7f0888108210 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.589+0000 7f088f278700 1 -- 172.21.15.1:0/1372683074 >> 172.21.15.1:0/1372683074 conn(0x7f08880fef90 msgr2=0x7f08881013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.589+0000 7f088f278700 1 -- 172.21.15.1:0/1372683074 shutdown_connections 2024-05-04T03:20:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.589+0000 7f088f278700 1 -- 172.21.15.1:0/1372683074 wait complete. 2024-05-04T03:20:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.590+0000 7f088f278700 1 Processor -- start 2024-05-04T03:20:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.590+0000 7f088f278700 1 -- start start 2024-05-04T03:20:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.590+0000 7f088f278700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0888103520 0x7f088810c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.590+0000 7f088f278700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0888107e00 0x7f088810c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:11.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.590+0000 7f088f278700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0888108750 0x7f088810ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:11.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.590+0000 7f088f278700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0888113360 con 0x7f0888107e00 2024-05-04T03:20:11.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.590+0000 7f088f278700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f08881131e0 con 0x7f0888103520 2024-05-04T03:20:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.590+0000 7f088f278700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f08881134e0 con 0x7f0888108750 2024-05-04T03:20:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.590+0000 7f088c813700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0888107e00 0x7f088810c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.590+0000 7f088c813700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0888107e00 0x7f088810c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52564/0 (socket says 172.21.15.1:52564) 2024-05-04T03:20:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.590+0000 7f088c813700 1 -- 172.21.15.1:0/399250912 learned_addr learned my addr 172.21.15.1:0/399250912 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.590+0000 7f088d014700 1 --2- 172.21.15.1:0/399250912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0888103520 0x7f088810c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.590+0000 7f088d815700 1 --2- 172.21.15.1:0/399250912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0888108750 0x7f088810ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.591+0000 7f088c813700 1 -- 172.21.15.1:0/399250912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0888108750 msgr2=0x7f088810ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.591+0000 7f088c813700 1 --2- 172.21.15.1:0/399250912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0888108750 0x7f088810ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.591+0000 7f088c813700 1 -- 172.21.15.1:0/399250912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0888103520 msgr2=0x7f088810c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.591+0000 7f088c813700 1 --2- 172.21.15.1:0/399250912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0888103520 0x7f088810c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.591+0000 7f088c813700 1 -- 172.21.15.1:0/399250912 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08881ad550 con 0x7f0888107e00 2024-05-04T03:20:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.591+0000 7f088c813700 1 --2- 172.21.15.1:0/399250912 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0888107e00 0x7f088810c900 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f08780045e0 tx=0x7f0878004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.591+0000 7f087e7fc700 1 -- 172.21.15.1:0/399250912 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0878026070 con 0x7f0888107e00 2024-05-04T03:20:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.591+0000 7f088f278700 1 -- 172.21.15.1:0/399250912 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08881ad7e0 con 0x7f0888107e00 2024-05-04T03:20:11.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.591+0000 7f087e7fc700 1 -- 172.21.15.1:0/399250912 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f087800f460 con 0x7f0888107e00 2024-05-04T03:20:11.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.591+0000 7f087e7fc700 1 -- 172.21.15.1:0/399250912 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0878009e60 con 0x7f0888107e00 2024-05-04T03:20:11.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.591+0000 7f088f278700 1 -- 172.21.15.1:0/399250912 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08881adc90 con 0x7f0888107e00 2024-05-04T03:20:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.595+0000 7f087e7fc700 1 -- 172.21.15.1:0/399250912 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f0878029070 con 0x7f0888107e00 2024-05-04T03:20:11.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.597+0000 7f087e7fc700 1 --2- 172.21.15.1:0/399250912 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f08740b2900 0x7f08740b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:11.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.597+0000 7f088d014700 1 --2- 172.21.15.1:0/399250912 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f08740b2900 0x7f08740b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:11.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.597+0000 7f089dc8cb80 1 Processor -- start 2024-05-04T03:20:11.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.598+0000 7f089dc8cb80 1 -- start start 2024-05-04T03:20:11.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.598+0000 7f087e7fc700 1 -- 172.21.15.1:0/399250912 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f08780e2880 con 0x7f0888107e00 2024-05-04T03:20:11.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.598+0000 7f088d014700 1 --2- 172.21.15.1:0/399250912 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f08740b2900 0x7f08740b4e10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0884005950 tx=0x7f088400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:11.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.598+0000 7f089dc8cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca846ca5d0 0x55ca846ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:11.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.598+0000 7f089dc8cb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ca846caf20 0x55ca846d53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:11.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.598+0000 7f089dc8cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ca846d58e0 0x55ca846d7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:11.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.598+0000 7f088d014700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca846ca5d0 0x55ca846ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:11.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.598+0000 7f088d014700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca846ca5d0 0x55ca846ca9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52576/0 (socket says 172.21.15.1:52576) 2024-05-04T03:20:11.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.599+0000 7f088c813700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ca846caf20 0x55ca846d53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:11.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.599+0000 7f088d014700 1 -- 172.21.15.1:0/236623677 learned_addr learned my addr 172.21.15.1:0/236623677 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:11.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.599+0000 7f088d815700 1 --2- 172.21.15.1:0/236623677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ca846d58e0 0x55ca846d7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:11.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.599+0000 7f089dc8cb80 1 -- 172.21.15.1:0/236623677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ca845ed960 con 0x55ca846ca5d0 2024-05-04T03:20:11.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.599+0000 7f089dc8cb80 1 -- 172.21.15.1:0/236623677 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ca8458c8c0 con 0x55ca846caf20 2024-05-04T03:20:11.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.599+0000 7f089dc8cb80 1 -- 172.21.15.1:0/236623677 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ca845e9920 con 0x55ca846d58e0 2024-05-04T03:20:11.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.599+0000 7f088d815700 1 -- 172.21.15.1:0/236623677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ca846caf20 msgr2=0x55ca846d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:11.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.599+0000 7f088d815700 1 --2- 172.21.15.1:0/236623677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ca846caf20 0x55ca846d53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.599+0000 7f088d815700 1 -- 172.21.15.1:0/236623677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca846ca5d0 msgr2=0x55ca846ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:11.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.599+0000 7f088d815700 1 --2- 172.21.15.1:0/236623677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca846ca5d0 0x55ca846ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.600+0000 7f088d815700 1 -- 172.21.15.1:0/236623677 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca8453a9d0 con 0x55ca846d58e0 2024-05-04T03:20:11.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.600+0000 7f088d815700 1 --2- 172.21.15.1:0/236623677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ca846d58e0 0x55ca846d7d00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0880011580 tx=0x7f0880009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5343ab67b2712e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:11.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.600+0000 7f08737fe700 1 -- 172.21.15.1:0/236623677 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0880011e80 con 0x55ca846d58e0 2024-05-04T03:20:11.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.600+0000 7f08737fe700 1 -- 172.21.15.1:0/236623677 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0880004bb0 con 0x55ca846d58e0 2024-05-04T03:20:11.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.600+0000 7f08737fe700 1 -- 172.21.15.1:0/236623677 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0880004690 con 0x55ca846d58e0 2024-05-04T03:20:11.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.602+0000 7f089dc8cb80 1 -- 172.21.15.1:0/236623677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ca846d58e0 msgr2=0x55ca846d7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:11.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.602+0000 7f089dc8cb80 1 --2- 172.21.15.1:0/236623677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ca846d58e0 0x55ca846d7d00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0880011580 tx=0x7f0880009920 comp rx=0 tx=0).stop 2024-05-04T03:20:11.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.602+0000 7f089dc8cb80 1 -- 172.21.15.1:0/236623677 shutdown_connections 2024-05-04T03:20:11.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.602+0000 7f089dc8cb80 1 --2- 172.21.15.1:0/236623677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ca846d58e0 0x55ca846d7d00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.602+0000 7f089dc8cb80 1 --2- 172.21.15.1:0/236623677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca846ca5d0 0x55ca846ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.602+0000 7f089dc8cb80 1 --2- 172.21.15.1:0/236623677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ca846caf20 0x55ca846d53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.602+0000 7f089dc8cb80 1 -- 172.21.15.1:0/236623677 >> 172.21.15.1:0/236623677 conn(0x55ca8454df40 msgr2=0x55ca846c6600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:11.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.602+0000 7f089dc8cb80 1 -- 172.21.15.1:0/236623677 shutdown_connections 2024-05-04T03:20:11.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.602+0000 7f089dc8cb80 1 -- 172.21.15.1:0/236623677 wait complete. 2024-05-04T03:20:11.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.603+0000 7f089dc8cb80 1 -- start start 2024-05-04T03:20:11.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.603+0000 7f089dc8cb80 1 Processor -- start 2024-05-04T03:20:11.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.604+0000 7f089dc8cb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ca846ca5d0 0x55ca846d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:11.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.604+0000 7f089dc8cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca846caf20 0x55ca846d2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:11.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.604+0000 7f088d014700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ca846ca5d0 0x55ca846d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:11.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.604+0000 7f088d014700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ca846ca5d0 0x55ca846d27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:44444/0 (socket says 172.21.15.1:44444) 2024-05-04T03:20:11.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.604+0000 7f088c813700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca846caf20 0x55ca846d2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:11.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.604+0000 7f088d014700 1 -- 172.21.15.1:0/3231954356 learned_addr learned my addr 172.21.15.1:0/3231954356 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:11.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.604+0000 7f088c813700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca846caf20 0x55ca846d2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52582/0 (socket says 172.21.15.1:52582) 2024-05-04T03:20:11.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.604+0000 7f089dc8cb80 1 --2- 172.21.15.1:0/3231954356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ca846d58e0 0x55ca846d3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:11.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.605+0000 7f089dc8cb80 1 -- 172.21.15.1:0/3231954356 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55ca846d8670 con 0x55ca846caf20 2024-05-04T03:20:11.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.605+0000 7f089dc8cb80 1 -- 172.21.15.1:0/3231954356 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ca84553d30 con 0x55ca846ca5d0 2024-05-04T03:20:11.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.605+0000 7f089dc8cb80 1 -- 172.21.15.1:0/3231954356 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ca846d87f0 con 0x55ca846d58e0 2024-05-04T03:20:11.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.605+0000 7f089dc8cb80 10 client.0 _ll_register_callbacks cb 0x55ca84585a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:20:11.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.605+0000 7f088d815700 1 --2- 172.21.15.1:0/3231954356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ca846d58e0 0x55ca846d3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:11.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.605+0000 7f088d014700 1 -- 172.21.15.1:0/3231954356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ca846d58e0 msgr2=0x55ca846d3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:11.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.605+0000 7f088d014700 1 --2- 172.21.15.1:0/3231954356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ca846d58e0 0x55ca846d3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.605+0000 7f088d014700 1 -- 172.21.15.1:0/3231954356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca846caf20 msgr2=0x55ca846d2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:11.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.605+0000 7f088d014700 1 --2- 172.21.15.1:0/3231954356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca846caf20 0x55ca846d2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.605+0000 7f088d014700 1 -- 172.21.15.1:0/3231954356 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca84589790 con 0x55ca846ca5d0 2024-05-04T03:20:11.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.606+0000 7f088d014700 1 --2- 172.21.15.1:0/3231954356 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ca846ca5d0 0x55ca846d27c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f08840116f0 tx=0x7f08840047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9e89050bf15f64d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:11.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.606+0000 7f0870ff9700 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-04T03:20:11.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.606+0000 7f0870ff9700 1 -- 172.21.15.1:0/3231954356 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0884028470 con 0x55ca846ca5d0 2024-05-04T03:20:11.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.606+0000 7f089dc8cb80 1 -- 172.21.15.1:0/3231954356 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0880004bb0 con 0x55ca846ca5d0 2024-05-04T03:20:11.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.606+0000 7f0870ff9700 1 -- 172.21.15.1:0/3231954356 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f088401f430 con 0x55ca846ca5d0 2024-05-04T03:20:11.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.606+0000 7f0870ff9700 1 -- 172.21.15.1:0/3231954356 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f088401f630 con 0x55ca846ca5d0 2024-05-04T03:20:11.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.606+0000 7f089dc8cb80 10 client.25106 fetch_fsmap learned FSMap version 26 2024-05-04T03:20:11.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.606+0000 7f089dc8cb80 1 -- 172.21.15.1:0/3231954356 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ca846e3210 con 0x55ca846ca5d0 2024-05-04T03:20:11.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.606+0000 7f0870ff9700 1 -- 172.21.15.1:0/3231954356 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0880004bb0 con 0x55ca846ca5d0 2024-05-04T03:20:11.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.607+0000 7f0870ff9700 1 -- 172.21.15.1:0/3231954356 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0884028a20 con 0x55ca846ca5d0 2024-05-04T03:20:11.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.607+0000 7f089dc8cb80 10 client.25106 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:20:11.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.607+0000 7f089dc8cb80 10 client.25106 resolve_mds: validated gid 24494 aka daemon mds.b 2024-05-04T03:20:11.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.607+0000 7f089dc8cb80 20 client.25106 populate_metadata read hostname 'smithi001' 2024-05-04T03:20:11.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.607+0000 7f089dc8cb80 1 --2- 172.21.15.1:0/3231954356 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55ca846e5850 0x55ca846e7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:11.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.607+0000 7f089dc8cb80 4 client.25106 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:20:11.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.607+0000 7f089dc8cb80 1 -- 172.21.15.1:0/3231954356 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca845fb340 con 0x55ca846e5850 2024-05-04T03:20:11.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.607+0000 7f088c813700 1 --2- 172.21.15.1:0/3231954356 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55ca846e5850 0x55ca846e7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:11.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.608+0000 7f0870ff9700 10 client.25106 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:11.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.608+0000 7f088c813700 1 --2- 172.21.15.1:0/3231954356 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55ca846e5850 0x55ca846e7cd0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49482b619d10c073 server_cookie=373f9016ff1c279e in_seq=0 out_seq=0 2024-05-04T03:20:11.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.609+0000 7f0870ff9700 1 -- 172.21.15.1:0/3231954356 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ca845fb340 con 0x55ca846e5850 2024-05-04T03:20:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.609+0000 7f0870ff9700 10 client.25106 handle_command_reply: tid=0 2024-05-04T03:20:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.609+0000 7f089dc8cb80 1 client.25106 shutdown 2024-05-04T03:20:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.609+0000 7f089dc8cb80 1 -- 172.21.15.1:0/3231954356 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ca846ca5d0 msgr2=0x55ca846d27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.609+0000 7f089dc8cb80 1 --2- 172.21.15.1:0/3231954356 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ca846ca5d0 0x55ca846d27c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f08840116f0 tx=0x7f08840047c0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.610+0000 7f089dc8cb80 1 -- 172.21.15.1:0/3231954356 shutdown_connections 2024-05-04T03:20:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.610+0000 7f089dc8cb80 1 --2- 172.21.15.1:0/3231954356 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55ca846e5850 0x55ca846e7cd0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.610+0000 7f089dc8cb80 1 --2- 172.21.15.1:0/3231954356 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ca846ca5d0 0x55ca846d27c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.610+0000 7f089dc8cb80 1 --2- 172.21.15.1:0/3231954356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ca846d58e0 0x55ca846d3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.610+0000 7f089dc8cb80 1 --2- 172.21.15.1:0/3231954356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca846caf20 0x55ca846d2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.610+0000 7f089dc8cb80 1 -- 172.21.15.1:0/3231954356 >> 172.21.15.1:0/3231954356 conn(0x55ca8454df40 msgr2=0x55ca846cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.610+0000 7f0870ff9700 0 client.25106 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.610+0000 7f089dc8cb80 1 -- 172.21.15.1:0/3231954356 shutdown_connections 2024-05-04T03:20:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.610+0000 7f089dc8cb80 1 -- 172.21.15.1:0/3231954356 wait complete. 2024-05-04T03:20:11.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.610+0000 7f089dc8cb80 20 client.25106 trim_cache size 0 max 16384 2024-05-04T03:20:11.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.650+0000 7f089dc8cb80 1 Processor -- start 2024-05-04T03:20:11.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.650+0000 7f089dc8cb80 1 -- start start 2024-05-04T03:20:11.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.651+0000 7f089dc8cb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ca846ca5d0 0x55ca846d3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:11.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.651+0000 7f089dc8cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca846caf20 0x55ca846e1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:11.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.651+0000 7f089dc8cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ca846d58e0 0x55ca846ed490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:11.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.651+0000 7f089dc8cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f08881b7810 con 0x55ca846caf20 2024-05-04T03:20:11.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.651+0000 7f088d014700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ca846ca5d0 0x55ca846d3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:11.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.651+0000 7f089dc8cb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f08881b50a0 con 0x55ca846ca5d0 2024-05-04T03:20:11.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.651+0000 7f089dc8cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f08881b2f70 con 0x55ca846d58e0 2024-05-04T03:20:11.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.651+0000 7f088d014700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ca846ca5d0 0x55ca846d3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:44460/0 (socket says 172.21.15.1:44460) 2024-05-04T03:20:11.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.651+0000 7f088d014700 1 -- 172.21.15.1:0/3044610534 learned_addr learned my addr 172.21.15.1:0/3044610534 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:11.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.651+0000 7f088c813700 1 --2- 172.21.15.1:0/3044610534 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca846caf20 0x55ca846e1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:11.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.652+0000 7f088d014700 1 -- 172.21.15.1:0/3044610534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ca846d58e0 msgr2=0x55ca846ed490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-04T03:20:11.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.652+0000 7f088d014700 1 --2- 172.21.15.1:0/3044610534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ca846d58e0 0x55ca846ed490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.652+0000 7f088d014700 1 -- 172.21.15.1:0/3044610534 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca846caf20 msgr2=0x55ca846e1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:11.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.652+0000 7f088d014700 1 --2- 172.21.15.1:0/3044610534 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca846caf20 0x55ca846e1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.652+0000 7f088d014700 1 -- 172.21.15.1:0/3044610534 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca846d0d30 con 0x55ca846ca5d0 2024-05-04T03:20:11.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.652+0000 7f088d014700 1 --2- 172.21.15.1:0/3044610534 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ca846ca5d0 0x55ca846d3080 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0884000c00 tx=0x7f0884010f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f46d2bf2050f56cf server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:11.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.652+0000 7f0872ffd700 1 -- 172.21.15.1:0/3044610534 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088401e4b0 con 0x55ca846ca5d0 2024-05-04T03:20:11.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.653+0000 7f0872ffd700 1 -- 172.21.15.1:0/3044610534 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0884023070 con 0x55ca846ca5d0 2024-05-04T03:20:11.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.653+0000 7f0872ffd700 1 -- 172.21.15.1:0/3044610534 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0884028470 con 0x55ca846ca5d0 2024-05-04T03:20:11.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.654+0000 7f089dc8cb80 1 -- 172.21.15.1:0/3044610534 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ca846ca5d0 msgr2=0x55ca846d3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:11.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.654+0000 7f089dc8cb80 1 --2- 172.21.15.1:0/3044610534 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ca846ca5d0 0x55ca846d3080 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0884000c00 tx=0x7f0884010f00 comp rx=0 tx=0).stop 2024-05-04T03:20:11.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.654+0000 7f089dc8cb80 1 -- 172.21.15.1:0/3044610534 shutdown_connections 2024-05-04T03:20:11.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.654+0000 7f089dc8cb80 1 --2- 172.21.15.1:0/3044610534 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ca846ca5d0 0x55ca846d3080 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.654+0000 7f089dc8cb80 1 --2- 172.21.15.1:0/3044610534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ca846d58e0 0x55ca846ed490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.654+0000 7f089dc8cb80 1 --2- 172.21.15.1:0/3044610534 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca846caf20 0x55ca846e1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.654+0000 7f089dc8cb80 1 -- 172.21.15.1:0/3044610534 >> 172.21.15.1:0/3044610534 conn(0x55ca8454df40 msgr2=0x55ca846d4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:11.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.654+0000 7f089dc8cb80 1 -- 172.21.15.1:0/3044610534 shutdown_connections 2024-05-04T03:20:11.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.654+0000 7f089dc8cb80 1 -- 172.21.15.1:0/3044610534 wait complete. 2024-05-04T03:20:11.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.655+0000 7f089dc8cb80 1 -- start start 2024-05-04T03:20:11.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.655+0000 7f089dc8cb80 1 Processor -- start 2024-05-04T03:20:11.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.655+0000 7f089dc8cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca846ca5d0 0x55ca846dda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:11.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.656+0000 7f089dc8cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ca846caf20 0x55ca846ddfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:11.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.656+0000 7f089dc8cb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ca846d58e0 0x55ca846daa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:11.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.656+0000 7f089dc8cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f08881b35c0 con 0x55ca846ca5d0 2024-05-04T03:20:11.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.656+0000 7f089dc8cb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f08881b49d0 con 0x55ca846d58e0 2024-05-04T03:20:11.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.656+0000 7f089dc8cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f08881b3c30 con 0x55ca846caf20 2024-05-04T03:20:11.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.656+0000 7f089dc8cb80 10 client.0 _ll_register_callbacks cb 0x55ca84585a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:20:11.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.656+0000 7f088d014700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca846ca5d0 0x55ca846dda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:11.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.656+0000 7f088d014700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca846ca5d0 0x55ca846dda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57232/0 (socket says 172.21.15.1:57232) 2024-05-04T03:20:11.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.656+0000 7f088d014700 1 -- 172.21.15.1:0/3031768047 learned_addr learned my addr 172.21.15.1:0/3031768047 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:11.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.656+0000 7f088d815700 1 --2- 172.21.15.1:0/3031768047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ca846d58e0 0x55ca846daa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:11.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.656+0000 7f088d014700 1 -- 172.21.15.1:0/3031768047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ca846caf20 msgr2=0x55ca846ddfc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-04T03:20:11.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.656+0000 7f088d014700 1 --2- 172.21.15.1:0/3031768047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ca846caf20 0x55ca846ddfc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.656+0000 7f088d014700 1 -- 172.21.15.1:0/3031768047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ca846d58e0 msgr2=0x55ca846daa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:11.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.656+0000 7f088d014700 1 --2- 172.21.15.1:0/3031768047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ca846d58e0 0x55ca846daa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.656+0000 7f088d014700 1 -- 172.21.15.1:0/3031768047 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca846db010 con 0x55ca846ca5d0 2024-05-04T03:20:11.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.657+0000 7f088d014700 1 --2- 172.21.15.1:0/3031768047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca846ca5d0 0x55ca846dda80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f08840116f0 tx=0x7f088401e840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0d14dd0d76e65f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:11.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.657+0000 7f0870ff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:20:11.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.657+0000 7f0870ff9700 1 -- 172.21.15.1:0/3031768047 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088401eb20 con 0x55ca846ca5d0 2024-05-04T03:20:11.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.658+0000 7f0870ff9700 1 -- 172.21.15.1:0/3031768047 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f088401ecc0 con 0x55ca846ca5d0 2024-05-04T03:20:11.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.658+0000 7f089dc8cb80 1 -- 172.21.15.1:0/3031768047 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0884023070 con 0x55ca846ca5d0 2024-05-04T03:20:11.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.658+0000 7f0870ff9700 1 -- 172.21.15.1:0/3031768047 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f088404e050 con 0x55ca846ca5d0 2024-05-04T03:20:11.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.658+0000 7f089dc8cb80 10 client.15360 fetch_fsmap learned FSMap version 26 2024-05-04T03:20:11.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.658+0000 7f089dc8cb80 1 -- 172.21.15.1:0/3031768047 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ca84750890 con 0x55ca846ca5d0 2024-05-04T03:20:11.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.659+0000 7f0870ff9700 1 -- 172.21.15.1:0/3031768047 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0884041070 con 0x55ca846ca5d0 2024-05-04T03:20:11.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.659+0000 7f0870ff9700 1 -- 172.21.15.1:0/3031768047 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f088403c030 con 0x55ca846ca5d0 2024-05-04T03:20:11.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.659+0000 7f089dc8cb80 10 client.15360 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:20:11.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.659+0000 7f089dc8cb80 10 client.15360 resolve_mds: validated gid 24494 aka daemon mds.b 2024-05-04T03:20:11.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.659+0000 7f089dc8cb80 20 client.15360 populate_metadata read hostname 'smithi001' 2024-05-04T03:20:11.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.659+0000 7f089dc8cb80 1 --2- 172.21.15.1:0/3031768047 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55ca84750eb0 0x55ca847532d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:11.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.660+0000 7f089dc8cb80 4 client.15360 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "27b22eee", "release": true, "await": true}] 2024-05-04T03:20:11.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.660+0000 7f089dc8cb80 1 -- 172.21.15.1:0/3031768047 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "quiesce db", "set_id": "27b22eee", "release": true, "await": true}) -- 0x7f0884023070 con 0x55ca84750eb0 2024-05-04T03:20:11.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.660+0000 7f088c813700 1 --2- 172.21.15.1:0/3031768047 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55ca84750eb0 0x55ca847532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:11.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.661+0000 7f088c813700 1 --2- 172.21.15.1:0/3031768047 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55ca84750eb0 0x55ca847532d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebb556d5edde5fc6 server_cookie=b49837e0189a8d2f in_seq=0 out_seq=0 2024-05-04T03:20:11.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.661+0000 7f0870ff9700 10 client.15360 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:11.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.663+0000 7f0870ff9700 1 -- 172.21.15.1:0/3031768047 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x7f0884023070 con 0x55ca84750eb0 2024-05-04T03:20:11.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.663+0000 7f0870ff9700 10 client.15360 handle_command_reply: tid=0 2024-05-04T03:20:11.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.663+0000 7f089dc8cb80 1 client.15360 shutdown 2024-05-04T03:20:11.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.663+0000 7f089dc8cb80 1 -- 172.21.15.1:0/3031768047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca846ca5d0 msgr2=0x55ca846dda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:11.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.663+0000 7f089dc8cb80 1 --2- 172.21.15.1:0/3031768047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca846ca5d0 0x55ca846dda80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f08840116f0 tx=0x7f088401e840 comp rx=0 tx=0).stop 2024-05-04T03:20:11.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.663+0000 7f089dc8cb80 1 -- 172.21.15.1:0/3031768047 shutdown_connections 2024-05-04T03:20:11.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.664+0000 7f089dc8cb80 1 --2- 172.21.15.1:0/3031768047 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55ca84750eb0 0x55ca847532d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.664+0000 7f089dc8cb80 1 --2- 172.21.15.1:0/3031768047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ca846d58e0 0x55ca846daa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.664+0000 7f089dc8cb80 1 --2- 172.21.15.1:0/3031768047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca846ca5d0 0x55ca846dda80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.664+0000 7f089dc8cb80 1 --2- 172.21.15.1:0/3031768047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ca846caf20 0x55ca846ddfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.664+0000 7f089dc8cb80 1 -- 172.21.15.1:0/3031768047 >> 172.21.15.1:0/3031768047 conn(0x55ca8454df40 msgr2=0x55ca846d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:11.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.664+0000 7f0870ff9700 0 client.15360 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:11.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.664+0000 7f089dc8cb80 1 -- 172.21.15.1:0/3031768047 shutdown_connections 2024-05-04T03:20:11.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.664+0000 7f089dc8cb80 1 -- 172.21.15.1:0/3031768047 wait complete. 2024-05-04T03:20:11.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.664+0000 7f089dc8cb80 20 client.15360 trim_cache size 0 max 16384 2024-05-04T03:20:11.669 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-04T03:20:11.669 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 26, 2024-05-04T03:20:11.669 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24494, 2024-05-04T03:20:11.669 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 4, 2024-05-04T03:20:11.669 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-05-04T03:20:11.669 INFO:teuthology.orchestra.run.smithi001.stdout: "27b22eee": { 2024-05-04T03:20:11.669 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 4, 2024-05-04T03:20:11.670 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-05-04T03:20:11.670 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-05-04T03:20:11.670 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-05-04T03:20:11.670 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-05-04T03:20:11.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:20:11.670 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 60.0, 2024-05-04T03:20:11.670 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 89.2, 2024-05-04T03:20:11.670 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-05-04T03:20:11.670 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-05-04T03:20:11.670 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-05-04T03:20:11.670 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-05-04T03:20:11.670 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-05-04T03:20:11.670 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-05-04T03:20:11.670 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:20:11.671 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:20:11.671 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:20:11.671 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:20:11.671 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:20:11.671 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-04T03:20:11.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.665+0000 7f088f278700 1 -- 172.21.15.1:0/399250912 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f08740b2900 msgr2=0x7f08740b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:11.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.665+0000 7f088f278700 1 --2- 172.21.15.1:0/399250912 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f08740b2900 0x7f08740b4e10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0884005950 tx=0x7f088400b410 comp rx=0 tx=0).stop 2024-05-04T03:20:11.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.665+0000 7f088f278700 1 -- 172.21.15.1:0/399250912 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0888107e00 msgr2=0x7f088810c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:11.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.665+0000 7f088f278700 1 --2- 172.21.15.1:0/399250912 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0888107e00 0x7f088810c900 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f08780045e0 tx=0x7f0878004610 comp rx=0 tx=0).stop 2024-05-04T03:20:11.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.665+0000 7f088f278700 1 -- 172.21.15.1:0/399250912 shutdown_connections 2024-05-04T03:20:11.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.665+0000 7f088f278700 1 --2- 172.21.15.1:0/399250912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0888103520 0x7f088810c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.665+0000 7f088f278700 1 --2- 172.21.15.1:0/399250912 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f08740b2900 0x7f08740b4e10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.665+0000 7f088f278700 1 --2- 172.21.15.1:0/399250912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0888108750 0x7f088810ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.665+0000 7f088f278700 1 --2- 172.21.15.1:0/399250912 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0888107e00 0x7f088810c900 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:11.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.665+0000 7f088f278700 1 -- 172.21.15.1:0/399250912 >> 172.21.15.1:0/399250912 conn(0x7f08880fef90 msgr2=0x7f08881013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:11.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.666+0000 7f088f278700 1 -- 172.21.15.1:0/399250912 shutdown_connections 2024-05-04T03:20:11.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:11.666+0000 7f088f278700 1 -- 172.21.15.1:0/399250912 wait complete. 2024-05-04T03:20:12.325 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '27b22eee', seconds elapsed: 41.2 2024-05-04T03:20:12.325 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.25 2024-05-04T03:20:12.326 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:20:12.607 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:20:12.984 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:12 smithi001 ceph-mon[29731]: pgmap v422: 97 pgs: 97 active+clean; 674 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-04T03:20:12.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:12 smithi072 ceph-mon[33332]: pgmap v422: 97 pgs: 97 active+clean; 674 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-04T03:20:13.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:12 smithi077 ceph-mon[33531]: pgmap v422: 97 pgs: 97 active+clean; 674 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-04T03:20:14.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.460+0000 7f9f69aa4700 1 -- 172.21.15.1:0/765454020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f64108f20 msgr2=0x7f9f64109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:14.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.460+0000 7f9f69aa4700 1 --2- 172.21.15.1:0/765454020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f64108f20 0x7f9f64109330 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9f580099c0 tx=0x7f9f58009cd0 comp rx=0 tx=0).stop 2024-05-04T03:20:14.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.460+0000 7f9f69aa4700 1 -- 172.21.15.1:0/765454020 shutdown_connections 2024-05-04T03:20:14.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.460+0000 7f9f69aa4700 1 --2- 172.21.15.1:0/765454020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9f64109870 0x7f9f641111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:14.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.460+0000 7f9f69aa4700 1 --2- 172.21.15.1:0/765454020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f641045f0 0x7f9f64104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:14.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.460+0000 7f9f69aa4700 1 --2- 172.21.15.1:0/765454020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f64108f20 0x7f9f64109330 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:14.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.460+0000 7f9f69aa4700 1 -- 172.21.15.1:0/765454020 >> 172.21.15.1:0/765454020 conn(0x7f9f64100040 msgr2=0x7f9f64102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:14.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.461+0000 7f9f69aa4700 1 -- 172.21.15.1:0/765454020 shutdown_connections 2024-05-04T03:20:14.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.461+0000 7f9f69aa4700 1 -- 172.21.15.1:0/765454020 wait complete. 2024-05-04T03:20:14.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.462+0000 7f9f69aa4700 1 Processor -- start 2024-05-04T03:20:14.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.462+0000 7f9f69aa4700 1 -- start start 2024-05-04T03:20:14.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.463+0000 7f9f69aa4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f641045f0 0x7f9f6419b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:14.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.463+0000 7f9f69aa4700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f64109870 0x7f9f6419b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:14.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.463+0000 7f9f69aa4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9f6419f840 0x7f9f6419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:14.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.463+0000 7f9f69aa4700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9f641144d0 con 0x7f9f64109870 2024-05-04T03:20:14.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.463+0000 7f9f69aa4700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9f64114350 con 0x7f9f6419f840 2024-05-04T03:20:14.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.463+0000 7f9f69aa4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9f64114650 con 0x7f9f641045f0 2024-05-04T03:20:14.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.463+0000 7f9f62ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f64109870 0x7f9f6419b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:14.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.463+0000 7f9f63fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9f6419f840 0x7f9f6419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:14.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.463+0000 7f9f63fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9f6419f840 0x7f9f6419fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:34376/0 (socket says 172.21.15.1:34376) 2024-05-04T03:20:14.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.463+0000 7f9f62ffd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f64109870 0x7f9f6419b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57238/0 (socket says 172.21.15.1:57238) 2024-05-04T03:20:14.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.463+0000 7f9f63fff700 1 -- 172.21.15.1:0/587910056 learned_addr learned my addr 172.21.15.1:0/587910056 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:14.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.463+0000 7f9f62ffd700 1 -- 172.21.15.1:0/587910056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f641045f0 msgr2=0x7f9f6419b120 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-04T03:20:14.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.463+0000 7f9f62ffd700 1 --2- 172.21.15.1:0/587910056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f641045f0 0x7f9f6419b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:14.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.463+0000 7f9f62ffd700 1 -- 172.21.15.1:0/587910056 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9f6419f840 msgr2=0x7f9f6419fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:14.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.463+0000 7f9f62ffd700 1 --2- 172.21.15.1:0/587910056 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9f6419f840 0x7f9f6419fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:14.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.463+0000 7f9f62ffd700 1 -- 172.21.15.1:0/587910056 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f641a0390 con 0x7f9f64109870 2024-05-04T03:20:14.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.463+0000 7f9f63fff700 1 --2- 172.21.15.1:0/587910056 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9f6419f840 0x7f9f6419fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:20:14.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.464+0000 7f9f62ffd700 1 --2- 172.21.15.1:0/587910056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f64109870 0x7f9f6419b6c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9f64105420 tx=0x7f9f58009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:14.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.464+0000 7f9f60ff9700 1 -- 172.21.15.1:0/587910056 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f58003a70 con 0x7f9f64109870 2024-05-04T03:20:14.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.464+0000 7f9f69aa4700 1 -- 172.21.15.1:0/587910056 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f641a0620 con 0x7f9f64109870 2024-05-04T03:20:14.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.464+0000 7f9f60ff9700 1 -- 172.21.15.1:0/587910056 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f580040c0 con 0x7f9f64109870 2024-05-04T03:20:14.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.464+0000 7f9f60ff9700 1 -- 172.21.15.1:0/587910056 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5800fd00 con 0x7f9f64109870 2024-05-04T03:20:14.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.464+0000 7f9f69aa4700 1 -- 172.21.15.1:0/587910056 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f641a7d10 con 0x7f9f64109870 2024-05-04T03:20:14.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.465+0000 7f9f4a7fc700 1 -- 172.21.15.1:0/587910056 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f44005330 con 0x7f9f64109870 2024-05-04T03:20:14.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.468+0000 7f9f60ff9700 1 -- 172.21.15.1:0/587910056 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f9f5800f460 con 0x7f9f64109870 2024-05-04T03:20:14.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.469+0000 7f9f60ff9700 1 --2- 172.21.15.1:0/587910056 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9f4c0b2900 0x7f9f4c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:14.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.470+0000 7f9f637fe700 1 --2- 172.21.15.1:0/587910056 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9f4c0b2900 0x7f9f4c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:14.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.470+0000 7f9f60ff9700 1 -- 172.21.15.1:0/587910056 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f9f580e2d20 con 0x7f9f64109870 2024-05-04T03:20:14.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.471+0000 7f9f637fe700 1 --2- 172.21.15.1:0/587910056 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9f4c0b2900 0x7f9f4c0b4e10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400ba10 tx=0x7f9f5400b3f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:14.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:14.477+0000 7f9f60ff9700 1 -- 172.21.15.1:0/587910056 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9f58013070 con 0x7f9f64109870 2024-05-04T03:20:14.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:14 smithi072 ceph-mon[33332]: pgmap v423: 97 pgs: 97 active+clean; 674 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-04T03:20:15.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:15.005+0000 7f9f4a7fc700 1 -- 172.21.15.1:0/587910056 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f44005620 con 0x7f9f64109870 2024-05-04T03:20:15.009 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:14 smithi001 ceph-mon[29731]: pgmap v423: 97 pgs: 97 active+clean; 674 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-04T03:20:15.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:15.006+0000 7f9f60ff9700 1 -- 172.21.15.1:0/587910056 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11844 (secure 0 0 0) 0x7f9f580aac20 con 0x7f9f64109870 2024-05-04T03:20:15.009 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:20:15.010 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2186681910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2186681910},{"type":"v1","addr":"172.21.15.1:6835","nonce":2186681910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2747191047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2747191047},{"type":"v1","addr":"172.21.15.1:6837","nonce":2747191047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4115120540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4115120540},{"type":"v1","addr":"172.21.15.1:6839","nonce":4115120540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3405709780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3405709780},{"type":"v1","addr":"172.21.15.1:6841","nonce":3405709780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/3816098761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":3816098761},{"type":"v1","addr":"172.21.15.72:6833","nonce":3816098761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/379026128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":379026128},{"type":"v1","addr":"172.21.15.77:6833","nonce":379026128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:15:49.823471+0000","modified":"2024-05-04T03:18:45.058419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24472,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/3930984976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3930984976},{"type":"v1","addr":"172.21.15.72:6835","nonce":3930984976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/4279951305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4279951305},{"type":"v1","addr":"172.21.15.77:6835","nonce":4279951305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2003521417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2003521417},{"type":"v1","addr":"172.21.15.77:6837","nonce":2003521417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/1417614203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1417614203},{"type":"v1","addr":"172.21.15.72:6837","nonce":1417614203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.77:6839/922723189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":922723189},{"type":"v1","addr":"172.21.15.77:6839","nonce":922723189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2976050650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2976050650},{"type":"v1","addr":"172.21.15.72:6839","nonce":2976050650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24467,24472,24494]},"id":1}]} 2024-05-04T03:20:15.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:15.009+0000 7f9f4a7fc700 1 -- 172.21.15.1:0/587910056 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9f4c0b2900 msgr2=0x7f9f4c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:15.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:15.009+0000 7f9f4a7fc700 1 --2- 172.21.15.1:0/587910056 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9f4c0b2900 0x7f9f4c0b4e10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400ba10 tx=0x7f9f5400b3f0 comp rx=0 tx=0).stop 2024-05-04T03:20:15.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:15.009+0000 7f9f4a7fc700 1 -- 172.21.15.1:0/587910056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f64109870 msgr2=0x7f9f6419b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:15.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:15.009+0000 7f9f4a7fc700 1 --2- 172.21.15.1:0/587910056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f64109870 0x7f9f6419b6c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9f64105420 tx=0x7f9f58009580 comp rx=0 tx=0).stop 2024-05-04T03:20:15.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:15.010+0000 7f9f4a7fc700 1 -- 172.21.15.1:0/587910056 shutdown_connections 2024-05-04T03:20:15.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:15.010+0000 7f9f4a7fc700 1 --2- 172.21.15.1:0/587910056 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9f6419f840 0x7f9f6419fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:15.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:15.010+0000 7f9f4a7fc700 1 --2- 172.21.15.1:0/587910056 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9f4c0b2900 0x7f9f4c0b4e10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:15.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:15.010+0000 7f9f4a7fc700 1 --2- 172.21.15.1:0/587910056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f641045f0 0x7f9f6419b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:15.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:15.010+0000 7f9f4a7fc700 1 --2- 172.21.15.1:0/587910056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f64109870 0x7f9f6419b6c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:15.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:15.010+0000 7f9f4a7fc700 1 -- 172.21.15.1:0/587910056 >> 172.21.15.1:0/587910056 conn(0x7f9f64100040 msgr2=0x7f9f64102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:15.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:15.010+0000 7f9f4a7fc700 1 -- 172.21.15.1:0/587910056 shutdown_connections 2024-05-04T03:20:15.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:15.010+0000 7f9f4a7fc700 1 -- 172.21.15.1:0/587910056 wait complete. 2024-05-04T03:20:15.013 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 26 2024-05-04T03:20:15.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:14 smithi077 ceph-mon[33531]: pgmap v423: 97 pgs: 97 active+clean; 674 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-04T03:20:15.767 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 60 --expiration 89.25 --await-for 42.0' 2024-05-04T03:20:16.226 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24494 quiesce db / --timeout 60 --expiration 89.25 --await-for 42.0 2024-05-04T03:20:16.227 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:15 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/587910056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:20:16.228 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:15 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/587910056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:20:16.228 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:15 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/587910056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:20:16.611 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:20:16.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:16 smithi072 ceph-mon[33332]: pgmap v424: 97 pgs: 97 active+clean; 674 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-04T03:20:16.999 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:16 smithi001 ceph-mon[29731]: pgmap v424: 97 pgs: 97 active+clean; 674 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-04T03:20:17.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:16 smithi077 ceph-mon[33531]: pgmap v424: 97 pgs: 97 active+clean; 674 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-04T03:20:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.071+0000 7f17cd10d640 1 -- 172.21.15.1:0/1568580910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17c8109b90 msgr2=0x7f17c810bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.071+0000 7f17cd10d640 1 --2- 172.21.15.1:0/1568580910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17c8109b90 0x7f17c810bfb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009be0 tx=0x7f17c002a6c0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.071+0000 7f17cd10d640 1 -- 172.21.15.1:0/1568580910 shutdown_connections 2024-05-04T03:20:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.071+0000 7f17cd10d640 1 --2- 172.21.15.1:0/1568580910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17c8109b90 0x7f17c810bfb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.071+0000 7f17cd10d640 1 --2- 172.21.15.1:0/1568580910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17c8104e90 0x7f17c81052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.071+0000 7f17cd10d640 1 --2- 172.21.15.1:0/1568580910 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f17c8103c50 0x7f17c8104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.071+0000 7f17cd10d640 1 -- 172.21.15.1:0/1568580910 >> 172.21.15.1:0/1568580910 conn(0x7f17c80fee60 msgr2=0x7f17c81012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.071+0000 7f17cd10d640 1 -- 172.21.15.1:0/1568580910 shutdown_connections 2024-05-04T03:20:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.071+0000 7f17cd10d640 1 -- 172.21.15.1:0/1568580910 wait complete. 2024-05-04T03:20:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.072+0000 7f17cd10d640 1 Processor -- start 2024-05-04T03:20:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.073+0000 7f17cd10d640 1 -- start start 2024-05-04T03:20:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.073+0000 7f17cd10d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17c8103c50 0x7f17c8076980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.073+0000 7f17cd10d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f17c8104e90 0x7f17c8075020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.073+0000 7f17cd10d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17c8109b90 0x7f17c8075560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.073+0000 7f17c6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17c8103c50 0x7f17c8076980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.073+0000 7f17c6d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17c8103c50 0x7f17c8076980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57262/0 (socket says 172.21.15.1:57262) 2024-05-04T03:20:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.073+0000 7f17c6d9d640 1 -- 172.21.15.1:0/764281884 learned_addr learned my addr 172.21.15.1:0/764281884 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.073+0000 7f17c759e640 1 --2- 172.21.15.1:0/764281884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17c8109b90 0x7f17c8075560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.073+0000 7f17cd10d640 1 -- 172.21.15.1:0/764281884 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f17c810f760 con 0x7f17c8103c50 2024-05-04T03:20:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.073+0000 7f17cd10d640 1 -- 172.21.15.1:0/764281884 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f17c810f5e0 con 0x7f17c8104e90 2024-05-04T03:20:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.073+0000 7f17cd10d640 1 -- 172.21.15.1:0/764281884 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f17c810f8e0 con 0x7f17c8109b90 2024-05-04T03:20:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.073+0000 7f17c659c640 1 --2- 172.21.15.1:0/764281884 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f17c8104e90 0x7f17c8075020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.074+0000 7f17c6d9d640 1 -- 172.21.15.1:0/764281884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17c8109b90 msgr2=0x7f17c8075560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.074+0000 7f17c6d9d640 1 --2- 172.21.15.1:0/764281884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17c8109b90 0x7f17c8075560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.074+0000 7f17c6d9d640 1 -- 172.21.15.1:0/764281884 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f17c8104e90 msgr2=0x7f17c8075020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.074+0000 7f17c6d9d640 1 --2- 172.21.15.1:0/764281884 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f17c8104e90 0x7f17c8075020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.074+0000 7f17c6d9d640 1 -- 172.21.15.1:0/764281884 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c8075c90 con 0x7f17c8103c50 2024-05-04T03:20:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.074+0000 7f17c6d9d640 1 --2- 172.21.15.1:0/764281884 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17c8103c50 0x7f17c8076980 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00e9f0 tx=0x7f17bc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.074+0000 7f17affff640 1 -- 172.21.15.1:0/764281884 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc00ccc0 con 0x7f17c8103c50 2024-05-04T03:20:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.074+0000 7f17affff640 1 -- 172.21.15.1:0/764281884 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17bc0035f0 con 0x7f17c8103c50 2024-05-04T03:20:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.075+0000 7f17affff640 1 -- 172.21.15.1:0/764281884 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc00ce60 con 0x7f17c8103c50 2024-05-04T03:20:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.075+0000 7f17cd10d640 1 -- 172.21.15.1:0/764281884 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17c81a4360 con 0x7f17c8103c50 2024-05-04T03:20:18.079 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.075+0000 7f17cd10d640 1 -- 172.21.15.1:0/764281884 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17c81a47d0 con 0x7f17c8103c50 2024-05-04T03:20:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.079+0000 7f17d3fc4740 1 Processor -- start 2024-05-04T03:20:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.079+0000 7f17d3fc4740 1 -- start start 2024-05-04T03:20:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.079+0000 7f17d3fc4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f916b911b0 0x55f916f32a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.079+0000 7f17d3fc4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f916b682f0 0x55f916f32fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.079+0000 7f17d3fc4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f916b704d0 0x55f916f39ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.079+0000 7f17d3fc4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f916e975d0 con 0x55f916b911b0 2024-05-04T03:20:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.079+0000 7f17d3fc4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55f916e50890 con 0x55f916b682f0 2024-05-04T03:20:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.079+0000 7f17d3fc4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f916e93590 con 0x55f916b704d0 2024-05-04T03:20:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.079+0000 7f17affff640 1 -- 172.21.15.1:0/764281884 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f17bc004000 con 0x7f17c8103c50 2024-05-04T03:20:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.080+0000 7f17c759e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f916b911b0 0x55f916f32a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.080+0000 7f17c759e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f916b911b0 0x55f916f32a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57272/0 (socket says 172.21.15.1:57272) 2024-05-04T03:20:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.080+0000 7f17c759e640 1 -- 172.21.15.1:0/2715443288 learned_addr learned my addr 172.21.15.1:0/2715443288 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.080+0000 7f17c659c640 1 --2- 172.21.15.1:0/2715443288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f916b704d0 0x55f916f39ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.080+0000 7f17c6d9d640 1 --2- 172.21.15.1:0/2715443288 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f916b682f0 0x55f916f32fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.080+0000 7f17c759e640 1 -- 172.21.15.1:0/2715443288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f916b704d0 msgr2=0x55f916f39ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.080+0000 7f17c759e640 1 --2- 172.21.15.1:0/2715443288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f916b704d0 0x55f916f39ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.080+0000 7f17c759e640 1 -- 172.21.15.1:0/2715443288 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f916b682f0 msgr2=0x55f916f32fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.080+0000 7f17c759e640 1 --2- 172.21.15.1:0/2715443288 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f916b682f0 0x55f916f32fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.080+0000 7f17c759e640 1 -- 172.21.15.1:0/2715443288 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f916ba9210 con 0x55f916b911b0 2024-05-04T03:20:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.080+0000 7f17c659c640 1 --2- 172.21.15.1:0/2715443288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f916b704d0 0x55f916f39ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:20:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.081+0000 7f17c759e640 1 --2- 172.21.15.1:0/2715443288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f916b911b0 0x55f916f32a60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009be0 tx=0x7f17c00303a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8438b776be48feec server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:18.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.081+0000 7f17affff640 1 --2- 172.21.15.1:0/764281884 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f17b00b2920 0x7f17b00b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.081+0000 7f17ad7fa640 1 -- 172.21.15.1:0/2715443288 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0040030 con 0x55f916b911b0 2024-05-04T03:20:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.081+0000 7f17c6d9d640 1 --2- 172.21.15.1:0/764281884 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f17b00b2920 0x7f17b00b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.081+0000 7f17ad7fa640 1 -- 172.21.15.1:0/2715443288 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17c0030590 con 0x55f916b911b0 2024-05-04T03:20:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.081+0000 7f17ad7fa640 1 -- 172.21.15.1:0/2715443288 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0036040 con 0x55f916b911b0 2024-05-04T03:20:18.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.081+0000 7f17affff640 1 -- 172.21.15.1:0/764281884 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f17bc0e2620 con 0x7f17c8103c50 2024-05-04T03:20:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.082+0000 7f17d3fc4740 1 -- 172.21.15.1:0/2715443288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f916b911b0 msgr2=0x55f916f32a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.082+0000 7f17d3fc4740 1 --2- 172.21.15.1:0/2715443288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f916b911b0 0x55f916f32a60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009be0 tx=0x7f17c00303a0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.082+0000 7f17c6d9d640 1 --2- 172.21.15.1:0/764281884 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f17b00b2920 0x7f17b00b4e30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f17bc0a6ed0 tx=0x7f17bc0e80e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.083+0000 7f17d3fc4740 1 -- 172.21.15.1:0/2715443288 shutdown_connections 2024-05-04T03:20:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.083+0000 7f17d3fc4740 1 --2- 172.21.15.1:0/2715443288 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f916b682f0 0x55f916f32fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.083+0000 7f17d3fc4740 1 --2- 172.21.15.1:0/2715443288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f916b704d0 0x55f916f39ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.083+0000 7f17d3fc4740 1 --2- 172.21.15.1:0/2715443288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f916b911b0 0x55f916f32a60 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.083+0000 7f17d3fc4740 1 -- 172.21.15.1:0/2715443288 >> 172.21.15.1:0/2715443288 conn(0x55f916b85300 msgr2=0x55f916f2e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.083+0000 7f17d3fc4740 1 -- 172.21.15.1:0/2715443288 shutdown_connections 2024-05-04T03:20:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.083+0000 7f17d3fc4740 1 -- 172.21.15.1:0/2715443288 wait complete. 2024-05-04T03:20:18.085 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.083+0000 7f17d3fc4740 1 -- start start 2024-05-04T03:20:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.083+0000 7f17d3fc4740 1 Processor -- start 2024-05-04T03:20:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.084+0000 7f17d3fc4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f916b682f0 0x55f916f3c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.084+0000 7f17c759e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f916b682f0 0x55f916f3c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.084+0000 7f17c759e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f916b682f0 0x55f916f3c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57284/0 (socket says 172.21.15.1:57284) 2024-05-04T03:20:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.084+0000 7f17c759e640 1 -- 172.21.15.1:0/568577304 learned_addr learned my addr 172.21.15.1:0/568577304 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.084+0000 7f17d3fc4740 1 --2- 172.21.15.1:0/568577304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f916b704d0 0x55f916f3cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.084+0000 7f17d3fc4740 1 --2- 172.21.15.1:0/568577304 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f916b911b0 0x55f916f43ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.084+0000 7f17d3fc4740 1 -- 172.21.15.1:0/568577304 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f916f3a8d0 con 0x55f916b682f0 2024-05-04T03:20:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.084+0000 7f17d3fc4740 1 -- 172.21.15.1:0/568577304 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55f916e54370 con 0x55f916b911b0 2024-05-04T03:20:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.084+0000 7f17d3fc4740 1 -- 172.21.15.1:0/568577304 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f916f3aa50 con 0x55f916b704d0 2024-05-04T03:20:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.085+0000 7f17c659c640 1 --2- 172.21.15.1:0/568577304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f916b704d0 0x55f916f3cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.085+0000 7f17c759e640 1 -- 172.21.15.1:0/568577304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f916b704d0 msgr2=0x55f916f3cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.085+0000 7f17c759e640 1 --2- 172.21.15.1:0/568577304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f916b704d0 0x55f916f3cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.085+0000 7f17c759e640 1 -- 172.21.15.1:0/568577304 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f916b911b0 msgr2=0x55f916f43ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-04T03:20:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.085+0000 7f17c759e640 1 --2- 172.21.15.1:0/568577304 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f916b911b0 0x55f916f43ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.089 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.085+0000 7f17c759e640 1 -- 172.21.15.1:0/568577304 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f916c75620 con 0x55f916b682f0 2024-05-04T03:20:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.085+0000 7f17d3fc4740 10 client.0 _ll_register_callbacks cb 0x55f916ba8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:20:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.085+0000 7f17d3fc4740 1 -- 172.21.15.1:0/568577304 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17c0030590 con 0x55f916b682f0 2024-05-04T03:20:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.085+0000 7f17c759e640 1 --2- 172.21.15.1:0/568577304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f916b682f0 0x55f916f3c640 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f17c0034040 tx=0x7f17c0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ab3f61f0efde91 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.085+0000 7f17aaff5640 10 client.15378 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:20:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.086+0000 7f17aaff5640 1 -- 172.21.15.1:0/568577304 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0053050 con 0x55f916b682f0 2024-05-04T03:20:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.086+0000 7f17aaff5640 1 -- 172.21.15.1:0/568577304 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f17c0030c70 con 0x55f916b682f0 2024-05-04T03:20:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.086+0000 7f17aaff5640 1 -- 172.21.15.1:0/568577304 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f17c0049070 con 0x55f916b682f0 2024-05-04T03:20:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.086+0000 7f17d3fc4740 10 client.15378 fetch_fsmap learned FSMap version 26 2024-05-04T03:20:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.086+0000 7f17d3fc4740 1 -- 172.21.15.1:0/568577304 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55f916f464f0 con 0x55f916b682f0 2024-05-04T03:20:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.086+0000 7f17aaff5640 1 -- 172.21.15.1:0/568577304 <== mon.0 v2:172.21.15.1:3300/0 4 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f17c0003ea0 con 0x55f916b682f0 2024-05-04T03:20:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.087+0000 7f17d3fc4740 10 client.15378 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:20:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.087+0000 7f17d3fc4740 10 client.15378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:20:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.087+0000 7f17d3fc4740 20 client.15378 populate_metadata read hostname 'smithi001' 2024-05-04T03:20:18.090 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.087+0000 7f17d3fc4740 1 --2- 172.21.15.1:0/568577304 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55f916b4fd80 0x55f916f49850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.087+0000 7f17d3fc4740 4 client.15378 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:20:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.087+0000 7f17d3fc4740 1 -- 172.21.15.1:0/568577304 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f916ea51e0 con 0x55f916b4fd80 2024-05-04T03:20:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.087+0000 7f17c659c640 1 --2- 172.21.15.1:0/568577304 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55f916b4fd80 0x55f916f49850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.087+0000 7f17aaff5640 10 client.15378 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.087+0000 7f17c659c640 1 --2- 172.21.15.1:0/568577304 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55f916b4fd80 0x55f916f49850 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63d29ad9a021cb4 server_cookie=5705dfaba9290b95 in_seq=0 out_seq=0 2024-05-04T03:20:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.088+0000 7f17aaff5640 1 -- 172.21.15.1:0/568577304 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f916ea51e0 con 0x55f916b4fd80 2024-05-04T03:20:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.088+0000 7f17aaff5640 10 client.15378 handle_command_reply: tid=0 2024-05-04T03:20:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.088+0000 7f17d3fc4740 1 client.15378 shutdown 2024-05-04T03:20:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.088+0000 7f17d3fc4740 1 -- 172.21.15.1:0/568577304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f916b682f0 msgr2=0x55f916f3c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.088+0000 7f17d3fc4740 1 --2- 172.21.15.1:0/568577304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f916b682f0 0x55f916f3c640 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f17c0034040 tx=0x7f17c0003ba0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.089+0000 7f17d3fc4740 1 -- 172.21.15.1:0/568577304 shutdown_connections 2024-05-04T03:20:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.089+0000 7f17d3fc4740 1 --2- 172.21.15.1:0/568577304 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55f916b4fd80 0x55f916f49850 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.089+0000 7f17d3fc4740 1 --2- 172.21.15.1:0/568577304 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f916b911b0 0x55f916f43ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.089+0000 7f17aaff5640 0 client.15378 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.089+0000 7f17d3fc4740 1 --2- 172.21.15.1:0/568577304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f916b704d0 0x55f916f3cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.089+0000 7f17d3fc4740 1 --2- 172.21.15.1:0/568577304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f916b682f0 0x55f916f3c640 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.089+0000 7f17d3fc4740 1 -- 172.21.15.1:0/568577304 >> 172.21.15.1:0/568577304 conn(0x55f916b85300 msgr2=0x55f916f31300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.089+0000 7f17d3fc4740 1 -- 172.21.15.1:0/568577304 shutdown_connections 2024-05-04T03:20:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.089+0000 7f17d3fc4740 1 -- 172.21.15.1:0/568577304 wait complete. 2024-05-04T03:20:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.089+0000 7f17d3fc4740 20 client.15378 trim_cache size 0 max 16384 2024-05-04T03:20:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.119+0000 7f17d3fc4740 1 Processor -- start 2024-05-04T03:20:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.120+0000 7f17d3fc4740 1 -- start start 2024-05-04T03:20:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.120+0000 7f17d3fc4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f916b4fd80 0x55f916f31270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.120+0000 7f17d3fc4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f916b682f0 0x55f916f317b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.120+0000 7f17d3fc4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f916b704d0 0x55f916f43a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.120+0000 7f17d3fc4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f17c81afce0 con 0x55f916b4fd80 2024-05-04T03:20:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.120+0000 7f17d3fc4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f17c81ad5b0 con 0x55f916b704d0 2024-05-04T03:20:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.120+0000 7f17d3fc4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f17c81ab450 con 0x55f916b682f0 2024-05-04T03:20:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.120+0000 7f17c759e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f916b4fd80 0x55f916f31270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.120+0000 7f17c759e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f916b4fd80 0x55f916f31270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57300/0 (socket says 172.21.15.1:57300) 2024-05-04T03:20:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.120+0000 7f17c759e640 1 -- 172.21.15.1:0/2584876133 learned_addr learned my addr 172.21.15.1:0/2584876133 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.120+0000 7f17c659c640 1 --2- 172.21.15.1:0/2584876133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f916b682f0 0x55f916f317b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.123 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.121+0000 7f17c759e640 1 -- 172.21.15.1:0/2584876133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f916b682f0 msgr2=0x55f916f317b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.121+0000 7f17c759e640 1 --2- 172.21.15.1:0/2584876133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f916b682f0 0x55f916f317b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.121+0000 7f17c759e640 1 -- 172.21.15.1:0/2584876133 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f916b704d0 msgr2=0x55f916f43a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.121+0000 7f17c759e640 1 --2- 172.21.15.1:0/2584876133 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f916b704d0 0x55f916f43a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.121+0000 7f17c759e640 1 -- 172.21.15.1:0/2584876133 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f916f2ed00 con 0x55f916b4fd80 2024-05-04T03:20:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.121+0000 7f17c759e640 1 --2- 172.21.15.1:0/2584876133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f916b4fd80 0x55f916f31270 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f17c00309d0 tx=0x7f17c0046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe1c4ca56f94ff2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.121+0000 7f17acff9640 1 -- 172.21.15.1:0/2584876133 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0053050 con 0x55f916b4fd80 2024-05-04T03:20:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.121+0000 7f17acff9640 1 -- 172.21.15.1:0/2584876133 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17c005c020 con 0x55f916b4fd80 2024-05-04T03:20:18.124 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.121+0000 7f17acff9640 1 -- 172.21.15.1:0/2584876133 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0036070 con 0x55f916b4fd80 2024-05-04T03:20:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.123+0000 7f17d3fc4740 1 -- 172.21.15.1:0/2584876133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f916b4fd80 msgr2=0x55f916f31270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.123+0000 7f17d3fc4740 1 --2- 172.21.15.1:0/2584876133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f916b4fd80 0x55f916f31270 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f17c00309d0 tx=0x7f17c0046000 comp rx=0 tx=0).stop 2024-05-04T03:20:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.123+0000 7f17d3fc4740 1 -- 172.21.15.1:0/2584876133 shutdown_connections 2024-05-04T03:20:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.123+0000 7f17d3fc4740 1 --2- 172.21.15.1:0/2584876133 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f916b704d0 0x55f916f43a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.123+0000 7f17d3fc4740 1 --2- 172.21.15.1:0/2584876133 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f916b682f0 0x55f916f317b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.126 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.123+0000 7f17d3fc4740 1 --2- 172.21.15.1:0/2584876133 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f916b4fd80 0x55f916f31270 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.123+0000 7f17d3fc4740 1 -- 172.21.15.1:0/2584876133 >> 172.21.15.1:0/2584876133 conn(0x55f916b85300 msgr2=0x55f916f36a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.123+0000 7f17d3fc4740 1 -- 172.21.15.1:0/2584876133 shutdown_connections 2024-05-04T03:20:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.123+0000 7f17d3fc4740 1 -- 172.21.15.1:0/2584876133 wait complete. 2024-05-04T03:20:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.123+0000 7f17d3fc4740 1 -- start start 2024-05-04T03:20:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.124+0000 7f17d3fc4740 1 Processor -- start 2024-05-04T03:20:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.124+0000 7f17d3fc4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f916b4fd80 0x55f916f41610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.124+0000 7f17d3fc4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f916b682f0 0x55f916f41b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.124+0000 7f17d3fc4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f916b704d0 0x55f916f42a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.124+0000 7f17d3fc4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f17c81abac0 con 0x55f916b704d0 2024-05-04T03:20:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.124+0000 7f17d3fc4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f17c81acee0 con 0x55f916b4fd80 2024-05-04T03:20:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.124+0000 7f17d3fc4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f17c81ac140 con 0x55f916b682f0 2024-05-04T03:20:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.124+0000 7f17c759e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f916b704d0 0x55f916f42a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.124+0000 7f17c759e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f916b704d0 0x55f916f42a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57308/0 (socket says 172.21.15.1:57308) 2024-05-04T03:20:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.124+0000 7f17c659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f916b682f0 0x55f916f41b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.127 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.124+0000 7f17c759e640 1 -- 172.21.15.1:0/3765963344 learned_addr learned my addr 172.21.15.1:0/3765963344 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.125+0000 7f17c759e640 1 --2- 172.21.15.1:0/3765963344 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f916b4fd80 0x55f916f41610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.125+0000 7f17d3fc4740 10 client.0 _ll_register_callbacks cb 0x7f17bc0e5f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:20:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.125+0000 7f17c659c640 1 -- 172.21.15.1:0/3765963344 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f916b4fd80 msgr2=0x55f916f41610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:18.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.125+0000 7f17c659c640 1 --2- 172.21.15.1:0/3765963344 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f916b4fd80 0x55f916f41610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.125+0000 7f17c659c640 1 -- 172.21.15.1:0/3765963344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f916b704d0 msgr2=0x55f916f42a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.125+0000 7f17c659c640 1 --2- 172.21.15.1:0/3765963344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f916b704d0 0x55f916f42a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.125+0000 7f17c659c640 1 -- 172.21.15.1:0/3765963344 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f916f43040 con 0x55f916b682f0 2024-05-04T03:20:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.125+0000 7f17d3fc4740 1 -- 172.21.15.1:0/3765963344 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17c005c020 con 0x55f916b682f0 2024-05-04T03:20:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.125+0000 7f17c659c640 1 --2- 172.21.15.1:0/3765963344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f916b682f0 0x55f916f41b50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55f916f49f70 tx=0x7f17b80042b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea689330c5902019 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.125+0000 7f17aaff5640 10 client.25099 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-04T03:20:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.125+0000 7f17aaff5640 1 -- 172.21.15.1:0/3765963344 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b80035f0 con 0x55f916b682f0 2024-05-04T03:20:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.125+0000 7f17aaff5640 1 -- 172.21.15.1:0/3765963344 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f17b8003790 con 0x55f916b682f0 2024-05-04T03:20:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.125+0000 7f17aaff5640 1 -- 172.21.15.1:0/3765963344 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f17b80039b0 con 0x55f916b682f0 2024-05-04T03:20:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.126+0000 7f17d3fc4740 10 client.25099 fetch_fsmap learned FSMap version 26 2024-05-04T03:20:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.126+0000 7f17d3fc4740 1 -- 172.21.15.1:0/3765963344 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55f916f44280 con 0x55f916b682f0 2024-05-04T03:20:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.126+0000 7f17aaff5640 1 -- 172.21.15.1:0/3765963344 <== mon.1 v2:172.21.15.77:3300/0 4 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f17b8003b50 con 0x55f916b682f0 2024-05-04T03:20:18.130 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.126+0000 7f17d3fc4740 10 client.25099 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:20:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.126+0000 7f17d3fc4740 10 client.25099 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:20:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.126+0000 7f17d3fc4740 20 client.25099 populate_metadata read hostname 'smithi001' 2024-05-04T03:20:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.126+0000 7f17d3fc4740 1 --2- 172.21.15.1:0/3765963344 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55f916b911b0 0x55f916fb41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.126+0000 7f17d3fc4740 4 client.25099 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:20:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.126+0000 7f17d3fc4740 1 -- 172.21.15.1:0/3765963344 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f916e9f050 con 0x55f916b911b0 2024-05-04T03:20:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.126+0000 7f17c759e640 1 --2- 172.21.15.1:0/3765963344 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55f916b911b0 0x55f916fb41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.127+0000 7f17aaff5640 10 client.25099 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.127+0000 7f17c759e640 1 --2- 172.21.15.1:0/3765963344 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55f916b911b0 0x55f916fb41d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50810a494dacdd server_cookie=dd66ca23192a0c5b in_seq=0 out_seq=0 2024-05-04T03:20:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.131+0000 7f17aaff5640 1 -- 172.21.15.1:0/3765963344 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f916e9f050 con 0x55f916b911b0 2024-05-04T03:20:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.131+0000 7f17aaff5640 10 client.25099 handle_command_reply: tid=0 2024-05-04T03:20:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.131+0000 7f17d3fc4740 1 client.25099 shutdown 2024-05-04T03:20:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.131+0000 7f17d3fc4740 1 -- 172.21.15.1:0/3765963344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f916b682f0 msgr2=0x55f916f41b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.131+0000 7f17d3fc4740 1 --2- 172.21.15.1:0/3765963344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f916b682f0 0x55f916f41b50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55f916f49f70 tx=0x7f17b80042b0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.131+0000 7f17d3fc4740 1 -- 172.21.15.1:0/3765963344 shutdown_connections 2024-05-04T03:20:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.131+0000 7f17d3fc4740 1 --2- 172.21.15.1:0/3765963344 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55f916b911b0 0x55f916fb41d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.131+0000 7f17d3fc4740 1 --2- 172.21.15.1:0/3765963344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f916b704d0 0x55f916f42a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.131+0000 7f17aaff5640 0 client.25099 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.131+0000 7f17d3fc4740 1 --2- 172.21.15.1:0/3765963344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f916b682f0 0x55f916f41b50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.131+0000 7f17d3fc4740 1 --2- 172.21.15.1:0/3765963344 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f916b4fd80 0x55f916f41610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.131+0000 7f17d3fc4740 1 -- 172.21.15.1:0/3765963344 >> 172.21.15.1:0/3765963344 conn(0x55f916b85300 msgr2=0x55f916f31270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.132+0000 7f17d3fc4740 1 -- 172.21.15.1:0/3765963344 shutdown_connections 2024-05-04T03:20:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.132+0000 7f17d3fc4740 1 -- 172.21.15.1:0/3765963344 wait complete. 2024-05-04T03:20:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.132+0000 7f17d3fc4740 20 client.25099 trim_cache size 0 max 16384 2024-05-04T03:20:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:20:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:20:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:20:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:20:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.132+0000 7f17cd10d640 1 -- 172.21.15.1:0/764281884 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f17b00b2920 msgr2=0x7f17b00b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.132+0000 7f17cd10d640 1 --2- 172.21.15.1:0/764281884 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f17b00b2920 0x7f17b00b4e30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f17bc0a6ed0 tx=0x7f17bc0e80e0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.132+0000 7f17cd10d640 1 -- 172.21.15.1:0/764281884 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17c8103c50 msgr2=0x7f17c8076980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.132+0000 7f17cd10d640 1 --2- 172.21.15.1:0/764281884 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17c8103c50 0x7f17c8076980 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00e9f0 tx=0x7f17bc00edf0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.132+0000 7f17cd10d640 1 -- 172.21.15.1:0/764281884 shutdown_connections 2024-05-04T03:20:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.132+0000 7f17cd10d640 1 --2- 172.21.15.1:0/764281884 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f17b00b2920 0x7f17b00b4e30 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.132+0000 7f17cd10d640 1 --2- 172.21.15.1:0/764281884 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f17c8104e90 0x7f17c8075020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.132+0000 7f17cd10d640 1 --2- 172.21.15.1:0/764281884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17c8109b90 0x7f17c8075560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.132+0000 7f17cd10d640 1 --2- 172.21.15.1:0/764281884 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17c8103c50 0x7f17c8076980 secure :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00e9f0 tx=0x7f17bc00edf0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.132+0000 7f17cd10d640 1 -- 172.21.15.1:0/764281884 >> 172.21.15.1:0/764281884 conn(0x7f17c80fee60 msgr2=0x7f17c8105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.133+0000 7f17cd10d640 1 -- 172.21.15.1:0/764281884 shutdown_connections 2024-05-04T03:20:18.136 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:18.133+0000 7f17cd10d640 1 -- 172.21.15.1:0/764281884 wait complete. 2024-05-04T03:20:18.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.506+0000 7f5e720d5700 1 -- 172.21.15.1:0/3632822437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e6c103310 msgr2=0x7f5e6c10f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:18.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.506+0000 7f5e720d5700 1 --2- 172.21.15.1:0/3632822437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e6c103310 0x7f5e6c10f890 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5e6400b0a0 tx=0x7f5e6400b3b0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.506+0000 7f5e720d5700 1 -- 172.21.15.1:0/3632822437 shutdown_connections 2024-05-04T03:20:18.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.506+0000 7f5e720d5700 1 --2- 172.21.15.1:0/3632822437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e6c103310 0x7f5e6c10f890 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.506+0000 7f5e720d5700 1 --2- 172.21.15.1:0/3632822437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e6c1089f0 0x7f5e6c108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.506+0000 7f5e720d5700 1 --2- 172.21.15.1:0/3632822437 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e6c1029c0 0x7f5e6c102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.506+0000 7f5e720d5700 1 -- 172.21.15.1:0/3632822437 >> 172.21.15.1:0/3632822437 conn(0x7f5e6c0fe6f0 msgr2=0x7f5e6c100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:18.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.507+0000 7f5e720d5700 1 -- 172.21.15.1:0/3632822437 shutdown_connections 2024-05-04T03:20:18.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.507+0000 7f5e720d5700 1 -- 172.21.15.1:0/3632822437 wait complete. 2024-05-04T03:20:18.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.508+0000 7f5e720d5700 1 Processor -- start 2024-05-04T03:20:18.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.508+0000 7f5e720d5700 1 -- start start 2024-05-04T03:20:18.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.508+0000 7f5e720d5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e6c1029c0 0x7f5e6c100770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.508+0000 7f5e720d5700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e6c103310 0x7f5e6c0febd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.508+0000 7f5e720d5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e6c1089f0 0x7f5e6c0ff1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.508+0000 7f5e6affd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e6c103310 0x7f5e6c0febd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.508+0000 7f5e6affd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e6c103310 0x7f5e6c0febd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57328/0 (socket says 172.21.15.1:57328) 2024-05-04T03:20:18.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.509+0000 7f5e6affd700 1 -- 172.21.15.1:0/1497855983 learned_addr learned my addr 172.21.15.1:0/1497855983 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:18.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.509+0000 7f5e720d5700 1 -- 172.21.15.1:0/1497855983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c112b60 con 0x7f5e6c103310 2024-05-04T03:20:18.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.509+0000 7f5e720d5700 1 -- 172.21.15.1:0/1497855983 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c1129e0 con 0x7f5e6c1029c0 2024-05-04T03:20:18.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.509+0000 7f5e720d5700 1 -- 172.21.15.1:0/1497855983 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c112ce0 con 0x7f5e6c1089f0 2024-05-04T03:20:18.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.509+0000 7f5e6b7fe700 1 --2- 172.21.15.1:0/1497855983 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e6c1029c0 0x7f5e6c100770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.509+0000 7f5e6bfff700 1 --2- 172.21.15.1:0/1497855983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e6c1089f0 0x7f5e6c0ff1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.509+0000 7f5e6affd700 1 -- 172.21.15.1:0/1497855983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e6c1089f0 msgr2=0x7f5e6c0ff1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:18.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.509+0000 7f5e6affd700 1 --2- 172.21.15.1:0/1497855983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e6c1089f0 0x7f5e6c0ff1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.509+0000 7f5e6affd700 1 -- 172.21.15.1:0/1497855983 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e6c1029c0 msgr2=0x7f5e6c100770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:18.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.509+0000 7f5e6affd700 1 --2- 172.21.15.1:0/1497855983 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e6c1029c0 0x7f5e6c100770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.509+0000 7f5e6affd700 1 -- 172.21.15.1:0/1497855983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e6c0ff9e0 con 0x7f5e6c103310 2024-05-04T03:20:18.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.509+0000 7f5e6affd700 1 --2- 172.21.15.1:0/1497855983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e6c103310 0x7f5e6c0febd0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000d950 tx=0x7f5e6000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:18.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.510+0000 7f5e68ff9700 1 -- 172.21.15.1:0/1497855983 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e60016070 con 0x7f5e6c103310 2024-05-04T03:20:18.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.510+0000 7f5e720d5700 1 -- 172.21.15.1:0/1497855983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e6c1ad700 con 0x7f5e6c103310 2024-05-04T03:20:18.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.510+0000 7f5e68ff9700 1 -- 172.21.15.1:0/1497855983 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e60004830 con 0x7f5e6c103310 2024-05-04T03:20:18.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.510+0000 7f5e68ff9700 1 -- 172.21.15.1:0/1497855983 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e60005520 con 0x7f5e6c103310 2024-05-04T03:20:18.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.510+0000 7f5e720d5700 1 -- 172.21.15.1:0/1497855983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e6c1adc60 con 0x7f5e6c103310 2024-05-04T03:20:18.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.515+0000 7f5e80ae9b80 1 Processor -- start 2024-05-04T03:20:18.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.515+0000 7f5e80ae9b80 1 -- start start 2024-05-04T03:20:18.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.515+0000 7f5e68ff9700 1 -- 172.21.15.1:0/1497855983 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f5e60004220 con 0x7f5e6c103310 2024-05-04T03:20:18.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.515+0000 7f5e80ae9b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e644ea5b60 0x55e644ea5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.515+0000 7f5e80ae9b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e644ea64b0 0x55e644eb0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.515+0000 7f5e80ae9b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e644eb0cd0 0x55e644eb30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.515+0000 7f5e80ae9b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e644dc8ad0 con 0x55e644eb0cd0 2024-05-04T03:20:18.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.515+0000 7f5e80ae9b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55e644d2df30 con 0x55e644ea5b60 2024-05-04T03:20:18.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.515+0000 7f5e80ae9b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e644dc4a90 con 0x55e644ea64b0 2024-05-04T03:20:18.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.516+0000 7f5e6bfff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e644ea5b60 0x55e644ea5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.516+0000 7f5e6bfff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e644ea5b60 0x55e644ea5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:34442/0 (socket says 172.21.15.1:34442) 2024-05-04T03:20:18.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.516+0000 7f5e6bfff700 1 -- 172.21.15.1:0/4091639774 learned_addr learned my addr 172.21.15.1:0/4091639774 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:18.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.516+0000 7f5e6b7fe700 1 --2- 172.21.15.1:0/4091639774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e644ea64b0 0x55e644eb0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.516+0000 7f5e6affd700 1 --2- 172.21.15.1:0/4091639774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e644eb0cd0 0x55e644eb30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.516+0000 7f5e6bfff700 1 -- 172.21.15.1:0/4091639774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e644ea64b0 msgr2=0x55e644eb0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:18.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.516+0000 7f5e6bfff700 1 --2- 172.21.15.1:0/4091639774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e644ea64b0 0x55e644eb0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.516+0000 7f5e6bfff700 1 -- 172.21.15.1:0/4091639774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e644eb0cd0 msgr2=0x55e644eb30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:18.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.516+0000 7f5e6bfff700 1 --2- 172.21.15.1:0/4091639774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e644eb0cd0 0x55e644eb30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.516+0000 7f5e6bfff700 1 -- 172.21.15.1:0/4091639774 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e644d15a60 con 0x55e644ea5b60 2024-05-04T03:20:18.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.516+0000 7f5e68ff9700 1 --2- 172.21.15.1:0/1497855983 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5e540b2980 0x7f5e540b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.516+0000 7f5e6affd700 1 --2- 172.21.15.1:0/4091639774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e644eb0cd0 0x55e644eb30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:20:18.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.517+0000 7f5e6b7fe700 1 --2- 172.21.15.1:0/1497855983 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5e540b2980 0x7f5e540b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.517+0000 7f5e68ff9700 1 -- 172.21.15.1:0/1497855983 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f5e600e1e30 con 0x7f5e6c103310 2024-05-04T03:20:18.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.517+0000 7f5e6bfff700 1 --2- 172.21.15.1:0/4091639774 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e644ea5b60 0x55e644ea5f70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5e64015000 tx=0x7f5e640090d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=323036ce2dfbf853 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:18.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.517+0000 7f5e6b7fe700 1 --2- 172.21.15.1:0/1497855983 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5e540b2980 0x7f5e540b4e90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55e644ea2e50 tx=0x7f5e5c0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:18.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.517+0000 7f5e51ffb700 1 -- 172.21.15.1:0/4091639774 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e64020070 con 0x55e644ea5b60 2024-05-04T03:20:18.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.517+0000 7f5e51ffb700 1 -- 172.21.15.1:0/4091639774 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e64003c00 con 0x55e644ea5b60 2024-05-04T03:20:18.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.519+0000 7f5e51ffb700 1 -- 172.21.15.1:0/4091639774 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e64005240 con 0x55e644ea5b60 2024-05-04T03:20:18.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.519+0000 7f5e80ae9b80 1 -- 172.21.15.1:0/4091639774 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e644ea5b60 msgr2=0x55e644ea5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:18.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.519+0000 7f5e80ae9b80 1 --2- 172.21.15.1:0/4091639774 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e644ea5b60 0x55e644ea5f70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5e64015000 tx=0x7f5e640090d0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.519+0000 7f5e80ae9b80 1 -- 172.21.15.1:0/4091639774 shutdown_connections 2024-05-04T03:20:18.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.519+0000 7f5e80ae9b80 1 --2- 172.21.15.1:0/4091639774 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e644ea5b60 0x55e644ea5f70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.519+0000 7f5e80ae9b80 1 --2- 172.21.15.1:0/4091639774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e644eb0cd0 0x55e644eb30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.519+0000 7f5e80ae9b80 1 --2- 172.21.15.1:0/4091639774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e644ea64b0 0x55e644eb0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.519+0000 7f5e80ae9b80 1 -- 172.21.15.1:0/4091639774 >> 172.21.15.1:0/4091639774 conn(0x55e644d28fd0 msgr2=0x55e644ea1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:18.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.519+0000 7f5e80ae9b80 1 -- 172.21.15.1:0/4091639774 shutdown_connections 2024-05-04T03:20:18.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.519+0000 7f5e80ae9b80 1 -- 172.21.15.1:0/4091639774 wait complete. 2024-05-04T03:20:18.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.519+0000 7f5e80ae9b80 1 -- start start 2024-05-04T03:20:18.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.520+0000 7f5e80ae9b80 1 Processor -- start 2024-05-04T03:20:18.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.520+0000 7f5e80ae9b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e644ea5b60 0x55e644eadc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.520+0000 7f5e80ae9b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e644ea64b0 0x55e644eae140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.520+0000 7f5e80ae9b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e644eb0cd0 0x55e644eae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.520+0000 7f5e80ae9b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e644eb3a90 con 0x55e644ea5b60 2024-05-04T03:20:18.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.520+0000 7f5e80ae9b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55e644d82060 con 0x55e644eb0cd0 2024-05-04T03:20:18.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.520+0000 7f5e80ae9b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e644eb3c10 con 0x55e644ea64b0 2024-05-04T03:20:18.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.520+0000 7f5e80ae9b80 10 client.0 _ll_register_callbacks cb 0x55e644d60ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:20:18.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.521+0000 7f5e6b7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e644ea5b60 0x55e644eadc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.521+0000 7f5e6b7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e644ea5b60 0x55e644eadc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57346/0 (socket says 172.21.15.1:57346) 2024-05-04T03:20:18.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.521+0000 7f5e6b7fe700 1 -- 172.21.15.1:0/3065800042 learned_addr learned my addr 172.21.15.1:0/3065800042 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:18.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.521+0000 7f5e6bfff700 1 --2- 172.21.15.1:0/3065800042 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e644eb0cd0 0x55e644eae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.521+0000 7f5e6affd700 1 --2- 172.21.15.1:0/3065800042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e644ea64b0 0x55e644eae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.521+0000 7f5e6b7fe700 1 -- 172.21.15.1:0/3065800042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e644ea64b0 msgr2=0x55e644eae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:18.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.521+0000 7f5e6b7fe700 1 --2- 172.21.15.1:0/3065800042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e644ea64b0 0x55e644eae140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.521+0000 7f5e6b7fe700 1 -- 172.21.15.1:0/3065800042 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e644eb0cd0 msgr2=0x55e644eae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:18.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.521+0000 7f5e6b7fe700 1 --2- 172.21.15.1:0/3065800042 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e644eb0cd0 0x55e644eae680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.521+0000 7f5e6b7fe700 1 -- 172.21.15.1:0/3065800042 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e644d7f8f0 con 0x55e644ea5b60 2024-05-04T03:20:18.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.521+0000 7f5e6b7fe700 1 --2- 172.21.15.1:0/3065800042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e644ea5b60 0x55e644eadc00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c011f10 tx=0x7f5e5c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a62805f4835c03 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:18.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.521+0000 7f5e4affd700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:20:18.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.522+0000 7f5e4affd700 1 -- 172.21.15.1:0/3065800042 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c01fde0 con 0x55e644ea5b60 2024-05-04T03:20:18.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.522+0000 7f5e4affd700 1 -- 172.21.15.1:0/3065800042 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f5e5c017030 con 0x55e644ea5b60 2024-05-04T03:20:18.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.522+0000 7f5e80ae9b80 1 -- 172.21.15.1:0/3065800042 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e64003c00 con 0x55e644ea5b60 2024-05-04T03:20:18.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.522+0000 7f5e4affd700 1 -- 172.21.15.1:0/3065800042 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5e5c004020 con 0x55e644ea5b60 2024-05-04T03:20:18.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.522+0000 7f5e80ae9b80 10 client.15396 fetch_fsmap learned FSMap version 26 2024-05-04T03:20:18.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.522+0000 7f5e80ae9b80 1 -- 172.21.15.1:0/3065800042 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e644ebe700 con 0x55e644ea5b60 2024-05-04T03:20:18.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.523+0000 7f5e4affd700 1 -- 172.21.15.1:0/3065800042 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e64003c00 con 0x55e644ea5b60 2024-05-04T03:20:18.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.523+0000 7f5e4affd700 1 -- 172.21.15.1:0/3065800042 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f5e5c027410 con 0x55e644ea5b60 2024-05-04T03:20:18.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.523+0000 7f5e80ae9b80 10 client.15396 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:20:18.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.523+0000 7f5e80ae9b80 10 client.15396 resolve_mds: validated gid 24494 aka daemon mds.b 2024-05-04T03:20:18.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.523+0000 7f5e80ae9b80 20 client.15396 populate_metadata read hostname 'smithi001' 2024-05-04T03:20:18.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.523+0000 7f5e80ae9b80 1 --2- 172.21.15.1:0/3065800042 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55e644ec0d40 0x55e644ec3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.523+0000 7f5e80ae9b80 4 client.15396 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:20:18.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.523+0000 7f5e80ae9b80 1 -- 172.21.15.1:0/3065800042 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e644dd64b0 con 0x55e644ec0d40 2024-05-04T03:20:18.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.523+0000 7f5e6affd700 1 --2- 172.21.15.1:0/3065800042 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55e644ec0d40 0x55e644ec3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.524+0000 7f5e4affd700 10 client.15396 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:18.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.524+0000 7f5e6affd700 1 --2- 172.21.15.1:0/3065800042 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55e644ec0d40 0x55e644ec3180 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55be464626f780c8 server_cookie=9632ddddf23f21c8 in_seq=0 out_seq=0 2024-05-04T03:20:18.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.525+0000 7f5e4affd700 1 -- 172.21.15.1:0/3065800042 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e644dd64b0 con 0x55e644ec0d40 2024-05-04T03:20:18.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.525+0000 7f5e4affd700 10 client.15396 handle_command_reply: tid=0 2024-05-04T03:20:18.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.525+0000 7f5e80ae9b80 1 client.15396 shutdown 2024-05-04T03:20:18.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.525+0000 7f5e80ae9b80 1 -- 172.21.15.1:0/3065800042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e644ea5b60 msgr2=0x55e644eadc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:18.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.525+0000 7f5e80ae9b80 1 --2- 172.21.15.1:0/3065800042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e644ea5b60 0x55e644eadc00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c011f10 tx=0x7f5e5c0047c0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.525+0000 7f5e80ae9b80 1 -- 172.21.15.1:0/3065800042 shutdown_connections 2024-05-04T03:20:18.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.525+0000 7f5e80ae9b80 1 --2- 172.21.15.1:0/3065800042 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55e644ec0d40 0x55e644ec3180 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.525+0000 7f5e80ae9b80 1 --2- 172.21.15.1:0/3065800042 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e644eb0cd0 0x55e644eae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.525+0000 7f5e80ae9b80 1 --2- 172.21.15.1:0/3065800042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e644ea5b60 0x55e644eadc00 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.525+0000 7f5e80ae9b80 1 --2- 172.21.15.1:0/3065800042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e644ea64b0 0x55e644eae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.525+0000 7f5e80ae9b80 1 -- 172.21.15.1:0/3065800042 >> 172.21.15.1:0/3065800042 conn(0x55e644d28fd0 msgr2=0x55e644ea7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:18.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.525+0000 7f5e4affd700 0 client.15396 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:18.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.525+0000 7f5e80ae9b80 1 -- 172.21.15.1:0/3065800042 shutdown_connections 2024-05-04T03:20:18.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.525+0000 7f5e80ae9b80 1 -- 172.21.15.1:0/3065800042 wait complete. 2024-05-04T03:20:18.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.525+0000 7f5e80ae9b80 20 client.15396 trim_cache size 0 max 16384 2024-05-04T03:20:18.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.547+0000 7f5e80ae9b80 1 Processor -- start 2024-05-04T03:20:18.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.547+0000 7f5e80ae9b80 1 -- start start 2024-05-04T03:20:18.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.548+0000 7f5e80ae9b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e644d28fd0 0x55e644ebcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.548+0000 7f5e80ae9b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e644ea64b0 0x55e644ea7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.548+0000 7f5e80ae9b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e644eb0cd0 0x55e644ec8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.548+0000 7f5e80ae9b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c1b7880 con 0x55e644d28fd0 2024-05-04T03:20:18.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.548+0000 7f5e80ae9b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c1b5110 con 0x55e644ea64b0 2024-05-04T03:20:18.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.548+0000 7f5e80ae9b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c1b2fc0 con 0x55e644eb0cd0 2024-05-04T03:20:18.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.548+0000 7f5e6b7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e644d28fd0 0x55e644ebcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.548+0000 7f5e6affd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e644ea64b0 0x55e644ea7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.548+0000 7f5e6b7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e644d28fd0 0x55e644ebcba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57356/0 (socket says 172.21.15.1:57356) 2024-05-04T03:20:18.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.548+0000 7f5e6b7fe700 1 -- 172.21.15.1:0/2693246086 learned_addr learned my addr 172.21.15.1:0/2693246086 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:18.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.548+0000 7f5e6bfff700 1 --2- 172.21.15.1:0/2693246086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e644eb0cd0 0x55e644ec8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.548+0000 7f5e6affd700 1 -- 172.21.15.1:0/2693246086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e644eb0cd0 msgr2=0x55e644ec8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:18.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.549+0000 7f5e6affd700 1 --2- 172.21.15.1:0/2693246086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e644eb0cd0 0x55e644ec8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.549+0000 7f5e6affd700 1 -- 172.21.15.1:0/2693246086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e644d28fd0 msgr2=0x55e644ebcba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:18.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.549+0000 7f5e6affd700 1 --2- 172.21.15.1:0/2693246086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e644d28fd0 0x55e644ebcba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.549+0000 7f5e6affd700 1 -- 172.21.15.1:0/2693246086 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e644eac170 con 0x55e644ea64b0 2024-05-04T03:20:18.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.549+0000 7f5e6b7fe700 1 --2- 172.21.15.1:0/2693246086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e644d28fd0 0x55e644ebcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:20:18.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.549+0000 7f5e6affd700 1 --2- 172.21.15.1:0/2693246086 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e644ea64b0 0x55e644ea7ba0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5e600a54f0 tx=0x7f5e60091040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cf66f167a1605f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:18.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.549+0000 7f5e517fa700 1 -- 172.21.15.1:0/2693246086 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e60004510 con 0x55e644ea64b0 2024-05-04T03:20:18.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.549+0000 7f5e517fa700 1 -- 172.21.15.1:0/2693246086 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e600a8f80 con 0x55e644ea64b0 2024-05-04T03:20:18.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.549+0000 7f5e517fa700 1 -- 172.21.15.1:0/2693246086 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e600056c0 con 0x55e644ea64b0 2024-05-04T03:20:18.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.550+0000 7f5e80ae9b80 1 -- 172.21.15.1:0/2693246086 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e644ea64b0 msgr2=0x55e644ea7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:18.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.550+0000 7f5e80ae9b80 1 --2- 172.21.15.1:0/2693246086 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e644ea64b0 0x55e644ea7ba0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5e600a54f0 tx=0x7f5e60091040 comp rx=0 tx=0).stop 2024-05-04T03:20:18.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.550+0000 7f5e80ae9b80 1 -- 172.21.15.1:0/2693246086 shutdown_connections 2024-05-04T03:20:18.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.550+0000 7f5e80ae9b80 1 --2- 172.21.15.1:0/2693246086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e644eb0cd0 0x55e644ec8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.550+0000 7f5e80ae9b80 1 --2- 172.21.15.1:0/2693246086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e644d28fd0 0x55e644ebcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.550+0000 7f5e80ae9b80 1 --2- 172.21.15.1:0/2693246086 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e644ea64b0 0x55e644ea7ba0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.550+0000 7f5e80ae9b80 1 -- 172.21.15.1:0/2693246086 >> 172.21.15.1:0/2693246086 conn(0x55e644ea1a30 msgr2=0x55e644ea7490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:18.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.551+0000 7f5e80ae9b80 1 -- 172.21.15.1:0/2693246086 shutdown_connections 2024-05-04T03:20:18.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.551+0000 7f5e80ae9b80 1 -- 172.21.15.1:0/2693246086 wait complete. 2024-05-04T03:20:18.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.551+0000 7f5e80ae9b80 1 -- start start 2024-05-04T03:20:18.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.551+0000 7f5e80ae9b80 1 Processor -- start 2024-05-04T03:20:18.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.552+0000 7f5e80ae9b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e644d28fd0 0x55e644eb95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.552+0000 7f5e80ae9b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e644ea64b0 0x55e644eb5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.552+0000 7f5e80ae9b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e644eb0cd0 0x55e644eb6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.552+0000 7f5e80ae9b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c1b3610 con 0x55e644eb0cd0 2024-05-04T03:20:18.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.552+0000 7f5e80ae9b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c1b4a40 con 0x55e644d28fd0 2024-05-04T03:20:18.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.552+0000 7f5e80ae9b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c1b3ca0 con 0x55e644ea64b0 2024-05-04T03:20:18.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.552+0000 7f5e6bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e644eb0cd0 0x55e644eb6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.552+0000 7f5e6bfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e644eb0cd0 0x55e644eb6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57362/0 (socket says 172.21.15.1:57362) 2024-05-04T03:20:18.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.552+0000 7f5e6bfff700 1 -- 172.21.15.1:0/2113269204 learned_addr learned my addr 172.21.15.1:0/2113269204 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:18.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.552+0000 7f5e80ae9b80 10 client.0 _ll_register_callbacks cb 0x55e644d60ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:20:18.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.552+0000 7f5e6affd700 1 --2- 172.21.15.1:0/2113269204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e644ea64b0 0x55e644eb5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.552+0000 7f5e6b7fe700 1 --2- 172.21.15.1:0/2113269204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e644d28fd0 0x55e644eb95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.552+0000 7f5e6bfff700 1 -- 172.21.15.1:0/2113269204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e644ea64b0 msgr2=0x55e644eb5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:18.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.552+0000 7f5e6bfff700 1 --2- 172.21.15.1:0/2113269204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e644ea64b0 0x55e644eb5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.553+0000 7f5e6bfff700 1 -- 172.21.15.1:0/2113269204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e644d28fd0 msgr2=0x55e644eb95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:18.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.553+0000 7f5e6bfff700 1 --2- 172.21.15.1:0/2113269204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e644d28fd0 0x55e644eb95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.553+0000 7f5e6bfff700 1 -- 172.21.15.1:0/2113269204 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e644eb6ca0 con 0x55e644eb0cd0 2024-05-04T03:20:18.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.553+0000 7f5e6affd700 1 --2- 172.21.15.1:0/2113269204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e644ea64b0 0x55e644eb5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:20:18.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.553+0000 7f5e6b7fe700 1 --2- 172.21.15.1:0/2113269204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e644d28fd0 0x55e644eb95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:20:18.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.553+0000 7f5e6bfff700 1 --2- 172.21.15.1:0/2113269204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e644eb0cd0 0x55e644eb6520 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5e64009140 tx=0x7f5e64003e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b304957517ec066 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:18.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.553+0000 7f5e4affd700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:20:18.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.553+0000 7f5e4affd700 1 -- 172.21.15.1:0/2113269204 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e64020070 con 0x55e644eb0cd0 2024-05-04T03:20:18.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.553+0000 7f5e80ae9b80 1 -- 172.21.15.1:0/2113269204 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e600a8f80 con 0x55e644eb0cd0 2024-05-04T03:20:18.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.553+0000 7f5e4affd700 1 -- 172.21.15.1:0/2113269204 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f5e6400f070 con 0x55e644eb0cd0 2024-05-04T03:20:18.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.553+0000 7f5e4affd700 1 -- 172.21.15.1:0/2113269204 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5e640077d0 con 0x55e644eb0cd0 2024-05-04T03:20:18.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.554+0000 7f5e80ae9b80 10 client.15402 fetch_fsmap learned FSMap version 26 2024-05-04T03:20:18.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.554+0000 7f5e80ae9b80 1 -- 172.21.15.1:0/2113269204 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e644f2c4d0 con 0x55e644eb0cd0 2024-05-04T03:20:18.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.554+0000 7f5e4affd700 1 -- 172.21.15.1:0/2113269204 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e64007e20 con 0x55e644eb0cd0 2024-05-04T03:20:18.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.554+0000 7f5e4affd700 1 -- 172.21.15.1:0/2113269204 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f5e64011070 con 0x55e644eb0cd0 2024-05-04T03:20:18.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.554+0000 7f5e80ae9b80 10 client.15402 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:20:18.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.554+0000 7f5e80ae9b80 10 client.15402 resolve_mds: validated gid 24494 aka daemon mds.b 2024-05-04T03:20:18.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.554+0000 7f5e80ae9b80 20 client.15402 populate_metadata read hostname 'smithi001' 2024-05-04T03:20:18.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.554+0000 7f5e80ae9b80 1 --2- 172.21.15.1:0/2113269204 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55e644f2c750 0x55e644f2eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:18.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.554+0000 7f5e80ae9b80 4 client.15402 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.25, "await_for": 42.0}] 2024-05-04T03:20:18.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.554+0000 7f5e80ae9b80 1 -- 172.21.15.1:0/2113269204 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.25, "await_for": 42.0}) -- 0x7f5e600a8f80 con 0x55e644f2c750 2024-05-04T03:20:18.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.554+0000 7f5e6b7fe700 1 --2- 172.21.15.1:0/2113269204 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55e644f2c750 0x55e644f2eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:18.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.555+0000 7f5e4affd700 10 client.15402 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:18.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.555+0000 7f5e6b7fe700 1 --2- 172.21.15.1:0/2113269204 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55e644f2c750 0x55e644f2eb70 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81908637e87782fc server_cookie=5be1c418969cdf13 in_seq=0 out_seq=0 2024-05-04T03:20:18.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.563+0000 7f5e4affd700 1 -- 172.21.15.1:0/2113269204 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x7f5e600a8f80 con 0x55e644f2c750 2024-05-04T03:20:18.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.563+0000 7f5e4affd700 10 client.15402 handle_command_reply: tid=0 2024-05-04T03:20:18.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.563+0000 7f5e80ae9b80 1 client.15402 shutdown 2024-05-04T03:20:18.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.563+0000 7f5e80ae9b80 1 -- 172.21.15.1:0/2113269204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e644eb0cd0 msgr2=0x55e644eb6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:18.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.563+0000 7f5e80ae9b80 1 --2- 172.21.15.1:0/2113269204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e644eb0cd0 0x55e644eb6520 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5e64009140 tx=0x7f5e64003e50 comp rx=0 tx=0).stop 2024-05-04T03:20:18.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.563+0000 7f5e80ae9b80 1 -- 172.21.15.1:0/2113269204 shutdown_connections 2024-05-04T03:20:18.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.563+0000 7f5e80ae9b80 1 --2- 172.21.15.1:0/2113269204 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55e644f2c750 0x55e644f2eb70 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.563+0000 7f5e80ae9b80 1 --2- 172.21.15.1:0/2113269204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e644d28fd0 0x55e644eb95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.563+0000 7f5e80ae9b80 1 --2- 172.21.15.1:0/2113269204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e644eb0cd0 0x55e644eb6520 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.563+0000 7f5e80ae9b80 1 --2- 172.21.15.1:0/2113269204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e644ea64b0 0x55e644eb5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.563+0000 7f5e80ae9b80 1 -- 172.21.15.1:0/2113269204 >> 172.21.15.1:0/2113269204 conn(0x55e644ea1a30 msgr2=0x55e644ea7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:18.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.563+0000 7f5e4affd700 0 client.15402 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:18.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.563+0000 7f5e80ae9b80 1 -- 172.21.15.1:0/2113269204 shutdown_connections 2024-05-04T03:20:18.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.564+0000 7f5e80ae9b80 1 -- 172.21.15.1:0/2113269204 wait complete. 2024-05-04T03:20:18.566 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-04T03:20:18.566 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 26, 2024-05-04T03:20:18.566 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24494, 2024-05-04T03:20:18.566 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 6, 2024-05-04T03:20:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-05-04T03:20:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: "661992bc": { 2024-05-04T03:20:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 6, 2024-05-04T03:20:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-05-04T03:20:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-05-04T03:20:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-05-04T03:20:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-05-04T03:20:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:20:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 60.0, 2024-05-04T03:20:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 89.2, 2024-05-04T03:20:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-05-04T03:20:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-05-04T03:20:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-05-04T03:20:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-05-04T03:20:18.567 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-05-04T03:20:18.568 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-05-04T03:20:18.568 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:20:18.568 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:20:18.568 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:20:18.568 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:20:18.568 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:20:18.568 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-04T03:20:18.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.564+0000 7f5e80ae9b80 20 client.15402 trim_cache size 0 max 16384 2024-05-04T03:20:18.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.565+0000 7f5e720d5700 1 -- 172.21.15.1:0/1497855983 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5e540b2980 msgr2=0x7f5e540b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:18.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.565+0000 7f5e720d5700 1 --2- 172.21.15.1:0/1497855983 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5e540b2980 0x7f5e540b4e90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55e644ea2e50 tx=0x7f5e5c0073d0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.565+0000 7f5e720d5700 1 -- 172.21.15.1:0/1497855983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e6c103310 msgr2=0x7f5e6c0febd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:18.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.565+0000 7f5e720d5700 1 --2- 172.21.15.1:0/1497855983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e6c103310 0x7f5e6c0febd0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000d950 tx=0x7f5e6000dc60 comp rx=0 tx=0).stop 2024-05-04T03:20:18.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.565+0000 7f5e720d5700 1 -- 172.21.15.1:0/1497855983 shutdown_connections 2024-05-04T03:20:18.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.565+0000 7f5e720d5700 1 --2- 172.21.15.1:0/1497855983 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5e6c1029c0 0x7f5e6c100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.565+0000 7f5e720d5700 1 --2- 172.21.15.1:0/1497855983 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5e540b2980 0x7f5e540b4e90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.565+0000 7f5e720d5700 1 --2- 172.21.15.1:0/1497855983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e6c1089f0 0x7f5e6c0ff1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.565+0000 7f5e720d5700 1 --2- 172.21.15.1:0/1497855983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e6c103310 0x7f5e6c0febd0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:18.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.565+0000 7f5e720d5700 1 -- 172.21.15.1:0/1497855983 >> 172.21.15.1:0/1497855983 conn(0x7f5e6c0fe6f0 msgr2=0x7f5e6c109820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:18.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.565+0000 7f5e720d5700 1 -- 172.21.15.1:0/1497855983 shutdown_connections 2024-05-04T03:20:18.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:18.565+0000 7f5e720d5700 1 -- 172.21.15.1:0/1497855983 wait complete. 2024-05-04T03:20:18.858 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:18 smithi001 ceph-mon[29731]: pgmap v425: 97 pgs: 97 active+clean; 674 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-04T03:20:18.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:18 smithi072 ceph-mon[33332]: pgmap v425: 97 pgs: 97 active+clean; 674 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-04T03:20:19.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:18 smithi077 ceph-mon[33531]: pgmap v425: 97 pgs: 97 active+clean; 674 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-04T03:20:19.347 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '661992bc', quiesce took 7.0 seconds. Will release after: 29.25 2024-05-04T03:20:20.698 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:20 smithi001 ceph-mon[29731]: pgmap v426: 97 pgs: 97 active+clean; 674 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-04T03:20:20.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:20 smithi072 ceph-mon[33332]: pgmap v426: 97 pgs: 97 active+clean; 674 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-04T03:20:21.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:20 smithi077 ceph-mon[33531]: pgmap v426: 97 pgs: 97 active+clean; 674 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-04T03:20:22.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:22 smithi072 ceph-mon[33332]: pgmap v427: 97 pgs: 97 active+clean; 674 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:20:23.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:22 smithi001 ceph-mon[29731]: pgmap v427: 97 pgs: 97 active+clean; 674 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:20:23.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:22 smithi077 ceph-mon[33531]: pgmap v427: 97 pgs: 97 active+clean; 674 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:20:24.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:24 smithi072 ceph-mon[33332]: pgmap v428: 97 pgs: 97 active+clean; 675 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-04T03:20:25.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:24 smithi001 ceph-mon[29731]: pgmap v428: 97 pgs: 97 active+clean; 675 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-04T03:20:25.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:24 smithi077 ceph-mon[33531]: pgmap v428: 97 pgs: 97 active+clean; 675 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-04T03:20:26.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:26 smithi072 ceph-mon[33332]: pgmap v429: 97 pgs: 97 active+clean; 675 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-04T03:20:27.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:26 smithi001 ceph-mon[29731]: pgmap v429: 97 pgs: 97 active+clean; 675 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-04T03:20:27.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:26 smithi077 ceph-mon[33531]: pgmap v429: 97 pgs: 97 active+clean; 675 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-04T03:20:28.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:28 smithi072 ceph-mon[33332]: pgmap v430: 97 pgs: 97 active+clean; 675 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-04T03:20:29.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:28 smithi001 ceph-mon[29731]: pgmap v430: 97 pgs: 97 active+clean; 675 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-04T03:20:29.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:28 smithi077 ceph-mon[33531]: pgmap v430: 97 pgs: 97 active+clean; 675 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-04T03:20:29.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:29 smithi072 ceph-mon[33332]: pgmap v431: 97 pgs: 97 active+clean; 675 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-04T03:20:30.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:29 smithi001 ceph-mon[29731]: pgmap v431: 97 pgs: 97 active+clean; 675 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-04T03:20:30.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:29 smithi077 ceph-mon[33531]: pgmap v431: 97 pgs: 97 active+clean; 675 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-04T03:20:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.240+0000 7fd60de14640 1 -- 172.21.15.1:0/1309020825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd608102bf0 msgr2=0x7fd6081038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.240+0000 7fd60de14640 1 --2- 172.21.15.1:0/1309020825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd608102bf0 0x7fd6081038f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8009900 tx=0x7fd5f802a410 comp rx=0 tx=0).stop 2024-05-04T03:20:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.240+0000 7fd60de14640 1 -- 172.21.15.1:0/1309020825 shutdown_connections 2024-05-04T03:20:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.240+0000 7fd60de14640 1 --2- 172.21.15.1:0/1309020825 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd608108b30 0x7fd60810af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.240+0000 7fd60de14640 1 --2- 172.21.15.1:0/1309020825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd608103e30 0x7fd608104240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.240+0000 7fd60de14640 1 --2- 172.21.15.1:0/1309020825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd608102bf0 0x7fd6081038f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.240+0000 7fd60de14640 1 -- 172.21.15.1:0/1309020825 >> 172.21.15.1:0/1309020825 conn(0x7fd6080fddc0 msgr2=0x7fd608100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.240+0000 7fd60de14640 1 -- 172.21.15.1:0/1309020825 shutdown_connections 2024-05-04T03:20:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.240+0000 7fd60de14640 1 -- 172.21.15.1:0/1309020825 wait complete. 2024-05-04T03:20:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.241+0000 7fd60de14640 1 Processor -- start 2024-05-04T03:20:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.241+0000 7fd60de14640 1 -- start start 2024-05-04T03:20:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.241+0000 7fd60de14640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd608102bf0 0x7fd6081992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.241+0000 7fd60de14640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd608103e30 0x7fd6081997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:31.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.241+0000 7fd60de14640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd608108b30 0x7fd6081a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.241+0000 7fd60de14640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd60810e6d0 con 0x7fd608102bf0 2024-05-04T03:20:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.241+0000 7fd60de14640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd60810e550 con 0x7fd608108b30 2024-05-04T03:20:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.241+0000 7fd60de14640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd60810e850 con 0x7fd608103e30 2024-05-04T03:20:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.241+0000 7fd6077fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd608102bf0 0x7fd6081992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.242+0000 7fd6077fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd608102bf0 0x7fd6081992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42458/0 (socket says 172.21.15.1:42458) 2024-05-04T03:20:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.242+0000 7fd6077fe640 1 -- 172.21.15.1:0/1408016340 learned_addr learned my addr 172.21.15.1:0/1408016340 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.242+0000 7fd607fff640 1 --2- 172.21.15.1:0/1408016340 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd608108b30 0x7fd6081a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.242+0000 7fd606ffd640 1 --2- 172.21.15.1:0/1408016340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd608103e30 0x7fd6081997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.242+0000 7fd6077fe640 1 -- 172.21.15.1:0/1408016340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd608103e30 msgr2=0x7fd6081997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:31.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.242+0000 7fd6077fe640 1 --2- 172.21.15.1:0/1408016340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd608103e30 0x7fd6081997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.242+0000 7fd6077fe640 1 -- 172.21.15.1:0/1408016340 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd608108b30 msgr2=0x7fd6081a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.242+0000 7fd6077fe640 1 --2- 172.21.15.1:0/1408016340 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd608108b30 0x7fd6081a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.242+0000 7fd6077fe640 1 -- 172.21.15.1:0/1408016340 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6081a0d30 con 0x7fd608102bf0 2024-05-04T03:20:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.242+0000 7fd606ffd640 1 --2- 172.21.15.1:0/1408016340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd608103e30 0x7fd6081997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:20:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.242+0000 7fd6077fe640 1 --2- 172.21.15.1:0/1408016340 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd608102bf0 0x7fd6081992a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8006010 tx=0x7fd5f8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.242+0000 7fd604ff9640 1 -- 172.21.15.1:0/1408016340 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f803f030 con 0x7fd608102bf0 2024-05-04T03:20:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.242+0000 7fd604ff9640 1 -- 172.21.15.1:0/1408016340 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5f8003aa0 con 0x7fd608102bf0 2024-05-04T03:20:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.242+0000 7fd60de14640 1 -- 172.21.15.1:0/1408016340 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6081a0fc0 con 0x7fd608102bf0 2024-05-04T03:20:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.242+0000 7fd604ff9640 1 -- 172.21.15.1:0/1408016340 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8035070 con 0x7fd608102bf0 2024-05-04T03:20:31.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.242+0000 7fd60de14640 1 -- 172.21.15.1:0/1408016340 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6081a13c0 con 0x7fd608102bf0 2024-05-04T03:20:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.245+0000 7fd614d9c740 1 Processor -- start 2024-05-04T03:20:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.245+0000 7fd614d9c740 1 -- start start 2024-05-04T03:20:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.245+0000 7fd604ff9640 1 -- 172.21.15.1:0/1408016340 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7fd5f8042030 con 0x7fd608102bf0 2024-05-04T03:20:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.245+0000 7fd614d9c740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55656e5fd1b0 0x55656e9a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.245+0000 7fd614d9c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55656e5d42f0 0x55656e9a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.245+0000 7fd614d9c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55656e5dc4d0 0x55656e9a80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.245+0000 7fd614d9c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55656e9035d0 con 0x55656e5d42f0 2024-05-04T03:20:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.245+0000 7fd614d9c740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55656e8bc890 con 0x55656e5fd1b0 2024-05-04T03:20:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.245+0000 7fd614d9c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55656e8ff590 con 0x55656e5dc4d0 2024-05-04T03:20:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.246+0000 7fd6077fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55656e5d42f0 0x55656e9a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.246+0000 7fd6077fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55656e5d42f0 0x55656e9a11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42460/0 (socket says 172.21.15.1:42460) 2024-05-04T03:20:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.246+0000 7fd6077fe640 1 -- 172.21.15.1:0/251828080 learned_addr learned my addr 172.21.15.1:0/251828080 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:31.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.246+0000 7fd607fff640 1 --2- 172.21.15.1:0/251828080 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55656e5fd1b0 0x55656e9a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.246+0000 7fd606ffd640 1 --2- 172.21.15.1:0/251828080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55656e5dc4d0 0x55656e9a80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.246+0000 7fd6077fe640 1 -- 172.21.15.1:0/251828080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55656e5dc4d0 msgr2=0x55656e9a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.246+0000 7fd6077fe640 1 --2- 172.21.15.1:0/251828080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55656e5dc4d0 0x55656e9a80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.246+0000 7fd6077fe640 1 -- 172.21.15.1:0/251828080 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55656e5fd1b0 msgr2=0x55656e9a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.246+0000 7fd6077fe640 1 --2- 172.21.15.1:0/251828080 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55656e5fd1b0 0x55656e9a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.246+0000 7fd6077fe640 1 -- 172.21.15.1:0/251828080 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55656e615210 con 0x55656e5d42f0 2024-05-04T03:20:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.246+0000 7fd607fff640 1 --2- 172.21.15.1:0/251828080 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55656e5fd1b0 0x55656e9a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:20:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.246+0000 7fd604ff9640 1 --2- 172.21.15.1:0/1408016340 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd5f00b28a0 0x7fd5f00b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.246+0000 7fd6077fe640 1 --2- 172.21.15.1:0/251828080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55656e5d42f0 0x55656e9a11f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8108bf0 tx=0x7fd5f8108ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24fd2171b949688c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.246+0000 7fd607fff640 1 --2- 172.21.15.1:0/1408016340 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd5f00b28a0 0x7fd5f00b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:31.259 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.246+0000 7fd5edffb640 1 -- 172.21.15.1:0/251828080 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f80c5d50 con 0x55656e5d42f0 2024-05-04T03:20:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.247+0000 7fd604ff9640 1 -- 172.21.15.1:0/1408016340 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fd5f81027b0 con 0x7fd608102bf0 2024-05-04T03:20:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.247+0000 7fd5edffb640 1 -- 172.21.15.1:0/251828080 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5f80c99a0 con 0x55656e5d42f0 2024-05-04T03:20:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.247+0000 7fd5edffb640 1 -- 172.21.15.1:0/251828080 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f80c9600 con 0x55656e5d42f0 2024-05-04T03:20:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.247+0000 7fd614d9c740 1 -- 172.21.15.1:0/251828080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55656e5d42f0 msgr2=0x55656e9a11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.248+0000 7fd607fff640 1 --2- 172.21.15.1:0/1408016340 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd5f00b28a0 0x7fd5f00b4db0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55656e9a1e40 tx=0x7fd600009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.248+0000 7fd614d9c740 1 --2- 172.21.15.1:0/251828080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55656e5d42f0 0x55656e9a11f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8108bf0 tx=0x7fd5f8108ce0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.248+0000 7fd614d9c740 1 -- 172.21.15.1:0/251828080 shutdown_connections 2024-05-04T03:20:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.248+0000 7fd614d9c740 1 --2- 172.21.15.1:0/251828080 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55656e5dc4d0 0x55656e9a80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.248+0000 7fd614d9c740 1 --2- 172.21.15.1:0/251828080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55656e5d42f0 0x55656e9a11f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.248+0000 7fd614d9c740 1 --2- 172.21.15.1:0/251828080 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55656e5fd1b0 0x55656e9a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.248+0000 7fd614d9c740 1 -- 172.21.15.1:0/251828080 >> 172.21.15.1:0/251828080 conn(0x55656e5f1300 msgr2=0x55656e99c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.248+0000 7fd614d9c740 1 -- 172.21.15.1:0/251828080 shutdown_connections 2024-05-04T03:20:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.248+0000 7fd614d9c740 1 -- 172.21.15.1:0/251828080 wait complete. 2024-05-04T03:20:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.248+0000 7fd614d9c740 1 -- start start 2024-05-04T03:20:31.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.248+0000 7fd614d9c740 1 Processor -- start 2024-05-04T03:20:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.249+0000 7fd614d9c740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55656e5d42f0 0x55656e9aa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.249+0000 7fd614d9c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55656e5dc4d0 0x55656e9aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.249+0000 7fd614d9c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55656e5fd1b0 0x55656e9b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.249+0000 7fd614d9c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55656e9a8900 con 0x55656e5fd1b0 2024-05-04T03:20:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.249+0000 7fd614d9c740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55656e8c0370 con 0x55656e5d42f0 2024-05-04T03:20:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.249+0000 7fd614d9c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55656e9a8a80 con 0x55656e5dc4d0 2024-05-04T03:20:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.249+0000 7fd614d9c740 10 client.0 _ll_register_callbacks cb 0x55656e614a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:20:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.249+0000 7fd607fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55656e5fd1b0 0x55656e9b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.249+0000 7fd607fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55656e5fd1b0 0x55656e9b1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42470/0 (socket says 172.21.15.1:42470) 2024-05-04T03:20:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.249+0000 7fd607fff640 1 -- 172.21.15.1:0/2035131810 learned_addr learned my addr 172.21.15.1:0/2035131810 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.249+0000 7fd6077fe640 1 --2- 172.21.15.1:0/2035131810 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55656e5d42f0 0x55656e9aa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.249+0000 7fd606ffd640 1 --2- 172.21.15.1:0/2035131810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55656e5dc4d0 0x55656e9aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:31.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.249+0000 7fd607fff640 1 -- 172.21.15.1:0/2035131810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55656e5dc4d0 msgr2=0x55656e9aaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.249+0000 7fd607fff640 1 --2- 172.21.15.1:0/2035131810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55656e5dc4d0 0x55656e9aaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.249+0000 7fd607fff640 1 -- 172.21.15.1:0/2035131810 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55656e5d42f0 msgr2=0x55656e9aa5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.249+0000 7fd607fff640 1 --2- 172.21.15.1:0/2035131810 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55656e5d42f0 0x55656e9aa5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.249+0000 7fd607fff640 1 -- 172.21.15.1:0/2035131810 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55656e6e1620 con 0x55656e5fd1b0 2024-05-04T03:20:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.249+0000 7fd606ffd640 1 --2- 172.21.15.1:0/2035131810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55656e5dc4d0 0x55656e9aaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:20:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.249+0000 7fd6077fe640 1 --2- 172.21.15.1:0/2035131810 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55656e5d42f0 0x55656e9aa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:20:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.249+0000 7fd607fff640 1 --2- 172.21.15.1:0/2035131810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55656e5fd1b0 0x55656e9b1a40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd600009fd0 tx=0x7fd600004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dfd7d96028b453 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.249+0000 7fd5e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:20:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.250+0000 7fd5e6ffd640 1 -- 172.21.15.1:0/2035131810 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600004b20 con 0x55656e5fd1b0 2024-05-04T03:20:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.250+0000 7fd5e6ffd640 1 -- 172.21.15.1:0/2035131810 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fd600004cc0 con 0x55656e5fd1b0 2024-05-04T03:20:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.250+0000 7fd614d9c740 1 -- 172.21.15.1:0/2035131810 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5f80c99a0 con 0x55656e5fd1b0 2024-05-04T03:20:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.250+0000 7fd5e6ffd640 1 -- 172.21.15.1:0/2035131810 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd60002e030 con 0x55656e5fd1b0 2024-05-04T03:20:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.250+0000 7fd614d9c740 10 client.15414 fetch_fsmap learned FSMap version 26 2024-05-04T03:20:31.262 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.250+0000 7fd614d9c740 1 -- 172.21.15.1:0/2035131810 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55656e9b4a40 con 0x55656e5fd1b0 2024-05-04T03:20:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.250+0000 7fd5e6ffd640 1 -- 172.21.15.1:0/2035131810 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5f80c99a0 con 0x55656e5fd1b0 2024-05-04T03:20:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.250+0000 7fd5e6ffd640 1 -- 172.21.15.1:0/2035131810 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd600027070 con 0x55656e5fd1b0 2024-05-04T03:20:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.250+0000 7fd614d9c740 10 client.15414 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:20:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.250+0000 7fd614d9c740 10 client.15414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:20:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.250+0000 7fd614d9c740 20 client.15414 populate_metadata read hostname 'smithi001' 2024-05-04T03:20:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.250+0000 7fd614d9c740 1 --2- 172.21.15.1:0/2035131810 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55656e5bbd80 0x55656e9b8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.250+0000 7fd614d9c740 4 client.15414 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:20:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.250+0000 7fd614d9c740 1 -- 172.21.15.1:0/2035131810 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55656e9111e0 con 0x55656e5bbd80 2024-05-04T03:20:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.251+0000 7fd6077fe640 1 --2- 172.21.15.1:0/2035131810 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55656e5bbd80 0x55656e9b8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.251+0000 7fd5e6ffd640 10 client.15414 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.251+0000 7fd6077fe640 1 --2- 172.21.15.1:0/2035131810 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55656e5bbd80 0x55656e9b8220 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b05fcd7401f754b server_cookie=66db8ace137f3b4c in_seq=0 out_seq=0 2024-05-04T03:20:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.252+0000 7fd5e6ffd640 1 -- 172.21.15.1:0/2035131810 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55656e9111e0 con 0x55656e5bbd80 2024-05-04T03:20:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.252+0000 7fd5e6ffd640 10 client.15414 handle_command_reply: tid=0 2024-05-04T03:20:31.263 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.252+0000 7fd614d9c740 1 client.15414 shutdown 2024-05-04T03:20:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.252+0000 7fd614d9c740 1 -- 172.21.15.1:0/2035131810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55656e5fd1b0 msgr2=0x55656e9b1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.252+0000 7fd614d9c740 1 --2- 172.21.15.1:0/2035131810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55656e5fd1b0 0x55656e9b1a40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd600009fd0 tx=0x7fd600004ab0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.252+0000 7fd614d9c740 1 -- 172.21.15.1:0/2035131810 shutdown_connections 2024-05-04T03:20:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.252+0000 7fd614d9c740 1 --2- 172.21.15.1:0/2035131810 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55656e5bbd80 0x55656e9b8220 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.252+0000 7fd614d9c740 1 --2- 172.21.15.1:0/2035131810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55656e5fd1b0 0x55656e9b1a40 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.252+0000 7fd614d9c740 1 --2- 172.21.15.1:0/2035131810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55656e5dc4d0 0x55656e9aaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.252+0000 7fd5e6ffd640 0 client.15414 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.252+0000 7fd614d9c740 1 --2- 172.21.15.1:0/2035131810 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55656e5d42f0 0x55656e9aa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.252+0000 7fd614d9c740 1 -- 172.21.15.1:0/2035131810 >> 172.21.15.1:0/2035131810 conn(0x55656e5f1300 msgr2=0x55656e99f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.252+0000 7fd614d9c740 1 -- 172.21.15.1:0/2035131810 shutdown_connections 2024-05-04T03:20:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.252+0000 7fd614d9c740 1 -- 172.21.15.1:0/2035131810 wait complete. 2024-05-04T03:20:31.264 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.252+0000 7fd614d9c740 20 client.15414 trim_cache size 0 max 16384 2024-05-04T03:20:31.271 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.268+0000 7fd614d9c740 1 Processor -- start 2024-05-04T03:20:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.268+0000 7fd614d9c740 1 -- start start 2024-05-04T03:20:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.268+0000 7fd614d9c740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55656e5d42f0 0x55656e99c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.268+0000 7fd614d9c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55656e5dc4d0 0x55656e9b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.268+0000 7fd614d9c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55656e5fd1b0 0x55656e9b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.268+0000 7fd614d9c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd6081ab7a0 con 0x55656e5fd1b0 2024-05-04T03:20:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.268+0000 7fd614d9c740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd6081a9070 con 0x55656e5d42f0 2024-05-04T03:20:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.268+0000 7fd614d9c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd6081a6ec0 con 0x55656e5dc4d0 2024-05-04T03:20:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.268+0000 7fd607fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55656e5fd1b0 0x55656e9b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.268+0000 7fd607fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55656e5fd1b0 0x55656e9b1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42482/0 (socket says 172.21.15.1:42482) 2024-05-04T03:20:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.268+0000 7fd607fff640 1 -- 172.21.15.1:0/3062175606 learned_addr learned my addr 172.21.15.1:0/3062175606 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.268+0000 7fd6077fe640 1 --2- 172.21.15.1:0/3062175606 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55656e5d42f0 0x55656e99c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.268+0000 7fd606ffd640 1 --2- 172.21.15.1:0/3062175606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55656e5dc4d0 0x55656e9b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.269+0000 7fd607fff640 1 -- 172.21.15.1:0/3062175606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55656e5dc4d0 msgr2=0x55656e9b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.269+0000 7fd607fff640 1 --2- 172.21.15.1:0/3062175606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55656e5dc4d0 0x55656e9b1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.269+0000 7fd607fff640 1 -- 172.21.15.1:0/3062175606 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55656e5d42f0 msgr2=0x55656e99c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.269+0000 7fd607fff640 1 --2- 172.21.15.1:0/3062175606 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55656e5d42f0 0x55656e99c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.269+0000 7fd607fff640 1 -- 172.21.15.1:0/3062175606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55656e99cf10 con 0x55656e5fd1b0 2024-05-04T03:20:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.269+0000 7fd6077fe640 1 --2- 172.21.15.1:0/3062175606 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55656e5d42f0 0x55656e99c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:20:31.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.269+0000 7fd607fff640 1 --2- 172.21.15.1:0/3062175606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55656e5fd1b0 0x55656e9b1ea0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd600012e30 tx=0x7fd600005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e131d8fd9a762ae8 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.270+0000 7fd5ed7fa640 1 -- 172.21.15.1:0/3062175606 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600037030 con 0x55656e5fd1b0 2024-05-04T03:20:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.270+0000 7fd5ed7fa640 1 -- 172.21.15.1:0/3062175606 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd60000fc60 con 0x55656e5fd1b0 2024-05-04T03:20:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.270+0000 7fd5ed7fa640 1 -- 172.21.15.1:0/3062175606 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600005050 con 0x55656e5fd1b0 2024-05-04T03:20:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.271+0000 7fd614d9c740 1 -- 172.21.15.1:0/3062175606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55656e5fd1b0 msgr2=0x55656e9b1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.271+0000 7fd614d9c740 1 --2- 172.21.15.1:0/3062175606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55656e5fd1b0 0x55656e9b1ea0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd600012e30 tx=0x7fd600005850 comp rx=0 tx=0).stop 2024-05-04T03:20:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.271+0000 7fd614d9c740 1 -- 172.21.15.1:0/3062175606 shutdown_connections 2024-05-04T03:20:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.271+0000 7fd614d9c740 1 --2- 172.21.15.1:0/3062175606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55656e5fd1b0 0x55656e9b1ea0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.271+0000 7fd614d9c740 1 --2- 172.21.15.1:0/3062175606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55656e5dc4d0 0x55656e9b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.271+0000 7fd614d9c740 1 --2- 172.21.15.1:0/3062175606 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55656e5d42f0 0x55656e99c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.271+0000 7fd614d9c740 1 -- 172.21.15.1:0/3062175606 >> 172.21.15.1:0/3062175606 conn(0x55656e5f1300 msgr2=0x55656e99bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.271+0000 7fd614d9c740 1 -- 172.21.15.1:0/3062175606 shutdown_connections 2024-05-04T03:20:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.271+0000 7fd614d9c740 1 -- 172.21.15.1:0/3062175606 wait complete. 2024-05-04T03:20:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.271+0000 7fd614d9c740 1 -- start start 2024-05-04T03:20:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.272+0000 7fd614d9c740 1 Processor -- start 2024-05-04T03:20:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.272+0000 7fd614d9c740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55656e5d42f0 0x55656e9af700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.272+0000 7fd614d9c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55656e5dc4d0 0x55656e9afc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.272+0000 7fd614d9c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55656e5fd1b0 0x55656e9b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.272+0000 7fd614d9c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd6081a7530 con 0x55656e5dc4d0 2024-05-04T03:20:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.272+0000 7fd614d9c740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd6081a89a0 con 0x55656e5d42f0 2024-05-04T03:20:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.272+0000 7fd614d9c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd6081a7c00 con 0x55656e5fd1b0 2024-05-04T03:20:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.272+0000 7fd614d9c740 10 client.0 _ll_register_callbacks cb 0x7fd600009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:20:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.272+0000 7fd606ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55656e5dc4d0 0x55656e9afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:31.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.272+0000 7fd606ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55656e5dc4d0 0x55656e9afc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42492/0 (socket says 172.21.15.1:42492) 2024-05-04T03:20:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.272+0000 7fd606ffd640 1 -- 172.21.15.1:0/2200091922 learned_addr learned my addr 172.21.15.1:0/2200091922 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.272+0000 7fd6077fe640 1 --2- 172.21.15.1:0/2200091922 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55656e5d42f0 0x55656e9af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.272+0000 7fd607fff640 1 --2- 172.21.15.1:0/2200091922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55656e5fd1b0 0x55656e9b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.272+0000 7fd606ffd640 1 -- 172.21.15.1:0/2200091922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55656e5fd1b0 msgr2=0x55656e9b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.272+0000 7fd606ffd640 1 --2- 172.21.15.1:0/2200091922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55656e5fd1b0 0x55656e9b0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.272+0000 7fd606ffd640 1 -- 172.21.15.1:0/2200091922 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55656e5d42f0 msgr2=0x55656e9af700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.272+0000 7fd606ffd640 1 --2- 172.21.15.1:0/2200091922 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55656e5d42f0 0x55656e9af700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.272+0000 7fd606ffd640 1 -- 172.21.15.1:0/2200091922 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55656e9b1130 con 0x55656e5dc4d0 2024-05-04T03:20:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.272+0000 7fd6077fe640 1 --2- 172.21.15.1:0/2200091922 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55656e5d42f0 0x55656e9af700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:20:31.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.273+0000 7fd606ffd640 1 --2- 172.21.15.1:0/2200091922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55656e5dc4d0 0x55656e9afc40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd60819a340 tx=0x7fd5fc012ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a8c59315e376fe0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.273+0000 7fd5e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:20:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.273+0000 7fd5e6ffd640 1 -- 172.21.15.1:0/2200091922 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc020060 con 0x55656e5dc4d0 2024-05-04T03:20:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.273+0000 7fd5e6ffd640 1 -- 172.21.15.1:0/2200091922 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fd5fc010ba0 con 0x55656e5dc4d0 2024-05-04T03:20:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.273+0000 7fd614d9c740 1 -- 172.21.15.1:0/2200091922 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd60000fc60 con 0x55656e5dc4d0 2024-05-04T03:20:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.273+0000 7fd5e6ffd640 1 -- 172.21.15.1:0/2200091922 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd5fc010de0 con 0x55656e5dc4d0 2024-05-04T03:20:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.273+0000 7fd614d9c740 10 client.15420 fetch_fsmap learned FSMap version 26 2024-05-04T03:20:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.273+0000 7fd614d9c740 1 -- 172.21.15.1:0/2200091922 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55656e9b24c0 con 0x55656e5dc4d0 2024-05-04T03:20:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.273+0000 7fd5e6ffd640 1 -- 172.21.15.1:0/2200091922 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd60000fc60 con 0x55656e5dc4d0 2024-05-04T03:20:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.273+0000 7fd5e6ffd640 1 -- 172.21.15.1:0/2200091922 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd5fc00f450 con 0x55656e5dc4d0 2024-05-04T03:20:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.273+0000 7fd614d9c740 10 client.15420 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:20:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.273+0000 7fd614d9c740 10 client.15420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:20:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.273+0000 7fd614d9c740 20 client.15420 populate_metadata read hostname 'smithi001' 2024-05-04T03:20:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.273+0000 7fd614d9c740 1 --2- 172.21.15.1:0/2200091922 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55656e5911c0 0x55656ea22d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.274+0000 7fd614d9c740 4 client.15420 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:20:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.274+0000 7fd614d9c740 1 -- 172.21.15.1:0/2200091922 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55656e90b050 con 0x55656e5911c0 2024-05-04T03:20:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.274+0000 7fd6077fe640 1 --2- 172.21.15.1:0/2200091922 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55656e5911c0 0x55656ea22d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.274+0000 7fd5e6ffd640 10 client.15420 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.274+0000 7fd6077fe640 1 --2- 172.21.15.1:0/2200091922 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55656e5911c0 0x55656ea22d90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84d4f141dd04953a server_cookie=ea2081984fba003a in_seq=0 out_seq=0 2024-05-04T03:20:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.279+0000 7fd5e6ffd640 1 -- 172.21.15.1:0/2200091922 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55656e90b050 con 0x55656e5911c0 2024-05-04T03:20:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.279+0000 7fd5e6ffd640 10 client.15420 handle_command_reply: tid=0 2024-05-04T03:20:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.279+0000 7fd614d9c740 1 client.15420 shutdown 2024-05-04T03:20:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.279+0000 7fd614d9c740 1 -- 172.21.15.1:0/2200091922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55656e5dc4d0 msgr2=0x55656e9afc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.279+0000 7fd614d9c740 1 --2- 172.21.15.1:0/2200091922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55656e5dc4d0 0x55656e9afc40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd60819a340 tx=0x7fd5fc012ed0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.279+0000 7fd614d9c740 1 -- 172.21.15.1:0/2200091922 shutdown_connections 2024-05-04T03:20:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.279+0000 7fd614d9c740 1 --2- 172.21.15.1:0/2200091922 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55656e5911c0 0x55656ea22d90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.279+0000 7fd614d9c740 1 --2- 172.21.15.1:0/2200091922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55656e5fd1b0 0x55656e9b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.279+0000 7fd614d9c740 1 --2- 172.21.15.1:0/2200091922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55656e5dc4d0 0x55656e9afc40 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.279+0000 7fd614d9c740 1 --2- 172.21.15.1:0/2200091922 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55656e5d42f0 0x55656e9af700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.279+0000 7fd5e6ffd640 0 client.15420 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.279+0000 7fd614d9c740 1 -- 172.21.15.1:0/2200091922 >> 172.21.15.1:0/2200091922 conn(0x55656e5f1300 msgr2=0x55656e998970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.279+0000 7fd614d9c740 1 -- 172.21.15.1:0/2200091922 shutdown_connections 2024-05-04T03:20:31.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.279+0000 7fd614d9c740 1 -- 172.21.15.1:0/2200091922 wait complete. 2024-05-04T03:20:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.279+0000 7fd614d9c740 20 client.15420 trim_cache size 0 max 16384 2024-05-04T03:20:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:20:31.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:20:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:20:31.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:20:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.280+0000 7fd60de14640 1 -- 172.21.15.1:0/1408016340 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd5f00b28a0 msgr2=0x7fd5f00b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:31.289 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.280+0000 7fd60de14640 1 --2- 172.21.15.1:0/1408016340 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd5f00b28a0 0x7fd5f00b4db0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55656e9a1e40 tx=0x7fd600009500 comp rx=0 tx=0).stop 2024-05-04T03:20:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.280+0000 7fd60de14640 1 -- 172.21.15.1:0/1408016340 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd608102bf0 msgr2=0x7fd6081992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.280+0000 7fd60de14640 1 --2- 172.21.15.1:0/1408016340 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd608102bf0 0x7fd6081992a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8006010 tx=0x7fd5f8004270 comp rx=0 tx=0).stop 2024-05-04T03:20:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.280+0000 7fd60de14640 1 -- 172.21.15.1:0/1408016340 shutdown_connections 2024-05-04T03:20:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.280+0000 7fd60de14640 1 --2- 172.21.15.1:0/1408016340 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd5f00b28a0 0x7fd5f00b4db0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.280+0000 7fd60de14640 1 --2- 172.21.15.1:0/1408016340 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd608108b30 0x7fd6081a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.280+0000 7fd60de14640 1 --2- 172.21.15.1:0/1408016340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd608103e30 0x7fd6081997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.280+0000 7fd60de14640 1 --2- 172.21.15.1:0/1408016340 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd608102bf0 0x7fd6081992a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:31.290 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.280+0000 7fd60de14640 1 -- 172.21.15.1:0/1408016340 >> 172.21.15.1:0/1408016340 conn(0x7fd6080fddc0 msgr2=0x7fd608104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.280+0000 7fd60de14640 1 -- 172.21.15.1:0/1408016340 shutdown_connections 2024-05-04T03:20:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:31.280+0000 7fd60de14640 1 -- 172.21.15.1:0/1408016340 wait complete. 2024-05-04T03:20:32.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:32 smithi001 ceph-mon[29731]: pgmap v432: 97 pgs: 97 active+clean; 675 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-04T03:20:32.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:32 smithi077 ceph-mon[33531]: pgmap v432: 97 pgs: 97 active+clean; 675 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-04T03:20:32.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:32 smithi072 ceph-mon[33332]: pgmap v432: 97 pgs: 97 active+clean; 675 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-04T03:20:32.866 INFO:tasks.workunit.client.0.smithi001.stderr:Updating files: 91% (11219/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-04T03:20:33.752 INFO:tasks.workunit.client.0.smithi001.stderr:Note: switching to 'bd4e00a39d766328b458e4c953f4d722fed96a31'. 2024-05-04T03:20:33.752 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-05-04T03:20:33.752 INFO:tasks.workunit.client.0.smithi001.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-04T03:20:33.753 INFO:tasks.workunit.client.0.smithi001.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-04T03:20:33.753 INFO:tasks.workunit.client.0.smithi001.stderr:state without impacting any branches by switching back to a branch. 2024-05-04T03:20:33.753 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-05-04T03:20:33.753 INFO:tasks.workunit.client.0.smithi001.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-04T03:20:33.753 INFO:tasks.workunit.client.0.smithi001.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-04T03:20:33.753 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-05-04T03:20:33.753 INFO:tasks.workunit.client.0.smithi001.stderr: git switch -c 2024-05-04T03:20:33.753 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-05-04T03:20:33.753 INFO:tasks.workunit.client.0.smithi001.stderr:Or undo this operation with: 2024-05-04T03:20:33.753 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-05-04T03:20:33.753 INFO:tasks.workunit.client.0.smithi001.stderr: git switch - 2024-05-04T03:20:33.753 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-05-04T03:20:33.753 INFO:tasks.workunit.client.0.smithi001.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-04T03:20:33.753 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-05-04T03:20:33.754 INFO:tasks.workunit.client.0.smithi001.stderr:HEAD is now at bd4e00a39d7 Merge PR #54433 into wip-pdonnell-testing-20240503.163550-debug 2024-05-04T03:20:33.759 DEBUG:teuthology.orchestra.run.smithi001:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-04T03:20:33.789 INFO:tasks.workunit.client.0.smithi001.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-04T03:20:33.791 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-04T03:20:33.792 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-04T03:20:34.394 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-04T03:20:34.450 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-04T03:20:34.491 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-04T03:20:34.493 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-04T03:20:34.493 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-04T03:20:34.532 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-04T03:20:34.539 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-04T03:20:34.539 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-04T03:20:34.595 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-05-04T03:20:34.596 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-05-04T03:20:34.597 DEBUG:teuthology.orchestra.run.smithi001:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bd4e00a39d766328b458e4c953f4d722fed96a31 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-05-04T03:20:34.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:34 smithi001 ceph-mon[29731]: pgmap v433: 97 pgs: 97 active+clean; 676 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-04T03:20:34.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:34 smithi077 ceph-mon[33531]: pgmap v433: 97 pgs: 97 active+clean; 676 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-04T03:20:34.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:34 smithi072 ceph-mon[33332]: pgmap v433: 97 pgs: 97 active+clean; 676 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2024-05-04T03:20:36.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:36 smithi001 ceph-mon[29731]: pgmap v434: 97 pgs: 97 active+clean; 676 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-04T03:20:36.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:36 smithi077 ceph-mon[33531]: pgmap v434: 97 pgs: 97 active+clean; 676 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-04T03:20:36.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:36 smithi072 ceph-mon[33332]: pgmap v434: 97 pgs: 97 active+clean; 676 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-04T03:20:38.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:38 smithi001 ceph-mon[29731]: pgmap v435: 97 pgs: 97 active+clean; 676 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-04T03:20:38.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:38 smithi077 ceph-mon[33531]: pgmap v435: 97 pgs: 97 active+clean; 676 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-04T03:20:38.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:38 smithi072 ceph-mon[33332]: pgmap v435: 97 pgs: 97 active+clean; 676 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-04T03:20:40.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:40 smithi001 ceph-mon[29731]: pgmap v436: 97 pgs: 97 active+clean; 676 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-04T03:20:40.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:40 smithi077 ceph-mon[33531]: pgmap v436: 97 pgs: 97 active+clean; 676 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-04T03:20:40.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:40 smithi072 ceph-mon[33332]: pgmap v436: 97 pgs: 97 active+clean; 676 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-04T03:20:42.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:42 smithi001 ceph-mon[29731]: pgmap v437: 97 pgs: 97 active+clean; 676 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-04T03:20:42.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:42 smithi077 ceph-mon[33531]: pgmap v437: 97 pgs: 97 active+clean; 676 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-04T03:20:42.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:42 smithi072 ceph-mon[33332]: pgmap v437: 97 pgs: 97 active+clean; 676 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-05-04T03:20:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.392+0000 7fdb95e27640 1 -- 172.21.15.1:0/505148406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb90102b30 msgr2=0x7fdb90103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:44.394 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.392+0000 7fdb95e27640 1 --2- 172.21.15.1:0/505148406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb90102b30 0x7fdb90103830 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fdb80009900 tx=0x7fdb8002a410 comp rx=0 tx=0).stop 2024-05-04T03:20:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.392+0000 7fdb95e27640 1 -- 172.21.15.1:0/505148406 shutdown_connections 2024-05-04T03:20:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.392+0000 7fdb95e27640 1 --2- 172.21.15.1:0/505148406 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb90108a70 0x7fdb9010ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.392+0000 7fdb95e27640 1 --2- 172.21.15.1:0/505148406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb90103d70 0x7fdb90104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.392+0000 7fdb95e27640 1 --2- 172.21.15.1:0/505148406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb90102b30 0x7fdb90103830 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.392+0000 7fdb95e27640 1 -- 172.21.15.1:0/505148406 >> 172.21.15.1:0/505148406 conn(0x7fdb900fdd20 msgr2=0x7fdb901001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.392+0000 7fdb95e27640 1 -- 172.21.15.1:0/505148406 shutdown_connections 2024-05-04T03:20:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.392+0000 7fdb95e27640 1 -- 172.21.15.1:0/505148406 wait complete. 2024-05-04T03:20:44.395 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.393+0000 7fdb95e27640 1 Processor -- start 2024-05-04T03:20:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.393+0000 7fdb95e27640 1 -- start start 2024-05-04T03:20:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.393+0000 7fdb95e27640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb90102b30 0x7fdb90075960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.393+0000 7fdb95e27640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb90103d70 0x7fdb9019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.393+0000 7fdb95e27640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb90108a70 0x7fdb901a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:44.397 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.393+0000 7fdb95e27640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9010e610 con 0x7fdb90108a70 2024-05-04T03:20:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.393+0000 7fdb95e27640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9010e490 con 0x7fdb90103d70 2024-05-04T03:20:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.393+0000 7fdb95e27640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9010e790 con 0x7fdb90102b30 2024-05-04T03:20:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.393+0000 7fdb8ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb90108a70 0x7fdb901a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.393+0000 7fdb8ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb90108a70 0x7fdb901a15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54356/0 (socket says 172.21.15.1:54356) 2024-05-04T03:20:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.393+0000 7fdb8ffff640 1 -- 172.21.15.1:0/3351635320 learned_addr learned my addr 172.21.15.1:0/3351635320 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.393+0000 7fdb8f7fe640 1 --2- 172.21.15.1:0/3351635320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb90102b30 0x7fdb90075960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.394+0000 7fdb8effd640 1 --2- 172.21.15.1:0/3351635320 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb90103d70 0x7fdb9019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.394+0000 7fdb8ffff640 1 -- 172.21.15.1:0/3351635320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb90102b30 msgr2=0x7fdb90075960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.394+0000 7fdb8ffff640 1 --2- 172.21.15.1:0/3351635320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb90102b30 0x7fdb90075960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.394+0000 7fdb8ffff640 1 -- 172.21.15.1:0/3351635320 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb90103d70 msgr2=0x7fdb9019f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.394+0000 7fdb8ffff640 1 --2- 172.21.15.1:0/3351635320 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb90103d70 0x7fdb9019f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.394+0000 7fdb8ffff640 1 -- 172.21.15.1:0/3351635320 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb901a1d70 con 0x7fdb90108a70 2024-05-04T03:20:44.398 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.394+0000 7fdb8effd640 1 --2- 172.21.15.1:0/3351635320 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb90103d70 0x7fdb9019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:20:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.394+0000 7fdb8ffff640 1 --2- 172.21.15.1:0/3351635320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb90108a70 0x7fdb901a15e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800eb10 tx=0x7fdb8800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.394+0000 7fdb8cff9640 1 -- 172.21.15.1:0/3351635320 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb88004b20 con 0x7fdb90108a70 2024-05-04T03:20:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.394+0000 7fdb8cff9640 1 -- 172.21.15.1:0/3351635320 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb88004cc0 con 0x7fdb90108a70 2024-05-04T03:20:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.394+0000 7fdb95e27640 1 -- 172.21.15.1:0/3351635320 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb901a2060 con 0x7fdb90108a70 2024-05-04T03:20:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.394+0000 7fdb8cff9640 1 -- 172.21.15.1:0/3351635320 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb88010670 con 0x7fdb90108a70 2024-05-04T03:20:44.399 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.394+0000 7fdb95e27640 1 -- 172.21.15.1:0/3351635320 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb901a25b0 con 0x7fdb90108a70 2024-05-04T03:20:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.397+0000 7fdb8cff9640 1 -- 172.21.15.1:0/3351635320 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7fdb88005020 con 0x7fdb90108a70 2024-05-04T03:20:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.397+0000 7fdb9cdbc740 1 Processor -- start 2024-05-04T03:20:44.404 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.397+0000 7fdb9cdbc740 1 -- start start 2024-05-04T03:20:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.397+0000 7fdb9cdbc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566b52521b0 0x5566b55f3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.397+0000 7fdb9cdbc740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5566b52292f0 0x5566b55f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.397+0000 7fdb9cdbc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566b52314d0 0x5566b55faee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.397+0000 7fdb9cdbc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5566b55585d0 con 0x5566b52314d0 2024-05-04T03:20:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.397+0000 7fdb9cdbc740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5566b5511890 con 0x5566b52292f0 2024-05-04T03:20:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.397+0000 7fdb9cdbc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5566b5554590 con 0x5566b52521b0 2024-05-04T03:20:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.397+0000 7fdb8effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566b52314d0 0x5566b55faee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.397+0000 7fdb8effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566b52314d0 0x5566b55faee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54370/0 (socket says 172.21.15.1:54370) 2024-05-04T03:20:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.397+0000 7fdb8effd640 1 -- 172.21.15.1:0/2128960044 learned_addr learned my addr 172.21.15.1:0/2128960044 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.398+0000 7fdb8effd640 1 --2- 172.21.15.1:0/2128960044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566b52521b0 0x5566b55f3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.398+0000 7fdb8f7fe640 1 --2- 172.21.15.1:0/2128960044 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5566b52292f0 0x5566b55f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.398+0000 7fdb8cff9640 1 --2- 172.21.15.1:0/3351635320 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fdb780b2920 0x7fdb780b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.398+0000 7fdb8effd640 1 -- 172.21.15.1:0/2128960044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566b52521b0 msgr2=0x5566b55f3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:44.405 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.398+0000 7fdb8effd640 1 --2- 172.21.15.1:0/2128960044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566b52521b0 0x5566b55f3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.398+0000 7fdb8effd640 1 -- 172.21.15.1:0/2128960044 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5566b52292f0 msgr2=0x5566b55f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.398+0000 7fdb8effd640 1 --2- 172.21.15.1:0/2128960044 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5566b52292f0 0x5566b55f3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.398+0000 7fdb8effd640 1 -- 172.21.15.1:0/2128960044 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566b526a210 con 0x5566b52314d0 2024-05-04T03:20:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.398+0000 7fdb8ffff640 1 --2- 172.21.15.1:0/3351635320 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fdb780b2920 0x7fdb780b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.398+0000 7fdb8cff9640 1 -- 172.21.15.1:0/3351635320 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fdb880e2700 con 0x7fdb90108a70 2024-05-04T03:20:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.398+0000 7fdb8effd640 1 --2- 172.21.15.1:0/2128960044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566b52314d0 0x5566b55faee0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdb840139d0 tx=0x7fdb8400b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a589883f3d1484 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.398+0000 7fdb75ffb640 1 -- 172.21.15.1:0/2128960044 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84004030 con 0x5566b52314d0 2024-05-04T03:20:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.398+0000 7fdb75ffb640 1 -- 172.21.15.1:0/2128960044 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb8400e960 con 0x5566b52314d0 2024-05-04T03:20:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.398+0000 7fdb75ffb640 1 -- 172.21.15.1:0/2128960044 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb840037f0 con 0x5566b52314d0 2024-05-04T03:20:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.399+0000 7fdb9cdbc740 1 -- 172.21.15.1:0/2128960044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566b52314d0 msgr2=0x5566b55faee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.399+0000 7fdb8ffff640 1 --2- 172.21.15.1:0/3351635320 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fdb780b2920 0x7fdb780b4e30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdb880a9d70 tx=0x7fdb8806f320 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.399+0000 7fdb9cdbc740 1 --2- 172.21.15.1:0/2128960044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566b52314d0 0x5566b55faee0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdb840139d0 tx=0x7fdb8400b520 comp rx=0 tx=0).stop 2024-05-04T03:20:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.399+0000 7fdb9cdbc740 1 -- 172.21.15.1:0/2128960044 shutdown_connections 2024-05-04T03:20:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.399+0000 7fdb9cdbc740 1 --2- 172.21.15.1:0/2128960044 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5566b52292f0 0x5566b55f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.399+0000 7fdb9cdbc740 1 --2- 172.21.15.1:0/2128960044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566b52314d0 0x5566b55faee0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.399+0000 7fdb9cdbc740 1 --2- 172.21.15.1:0/2128960044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566b52521b0 0x5566b55f3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.399+0000 7fdb9cdbc740 1 -- 172.21.15.1:0/2128960044 >> 172.21.15.1:0/2128960044 conn(0x5566b5246300 msgr2=0x5566b55ef3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.399+0000 7fdb9cdbc740 1 -- 172.21.15.1:0/2128960044 shutdown_connections 2024-05-04T03:20:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.399+0000 7fdb9cdbc740 1 -- 172.21.15.1:0/2128960044 wait complete. 2024-05-04T03:20:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.400+0000 7fdb9cdbc740 1 -- start start 2024-05-04T03:20:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.400+0000 7fdb9cdbc740 1 Processor -- start 2024-05-04T03:20:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.400+0000 7fdb9cdbc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566b52292f0 0x5566b55fd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.400+0000 7fdb9cdbc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566b52314d0 0x5566b55fdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.400+0000 7fdb9cdbc740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5566b52521b0 0x5566b5604b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.400+0000 7fdb9cdbc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5566b55fb930 con 0x5566b52292f0 2024-05-04T03:20:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.400+0000 7fdb9cdbc740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5566b5515370 con 0x5566b52521b0 2024-05-04T03:20:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.400+0000 7fdb9cdbc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5566b55fbab0 con 0x5566b52314d0 2024-05-04T03:20:44.407 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.400+0000 7fdb9cdbc740 10 client.0 _ll_register_callbacks cb 0x5566b5269a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:20:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.400+0000 7fdb8effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566b52292f0 0x5566b55fd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.400+0000 7fdb8effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566b52292f0 0x5566b55fd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54378/0 (socket says 172.21.15.1:54378) 2024-05-04T03:20:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.400+0000 7fdb8effd640 1 -- 172.21.15.1:0/1546523567 learned_addr learned my addr 172.21.15.1:0/1546523567 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.400+0000 7fdb8effd640 1 --2- 172.21.15.1:0/1546523567 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5566b52521b0 0x5566b5604b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.400+0000 7fdb8f7fe640 1 --2- 172.21.15.1:0/1546523567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566b52314d0 0x5566b55fdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.401+0000 7fdb8effd640 1 -- 172.21.15.1:0/1546523567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566b52314d0 msgr2=0x5566b55fdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.401+0000 7fdb8effd640 1 --2- 172.21.15.1:0/1546523567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566b52314d0 0x5566b55fdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.401+0000 7fdb8effd640 1 -- 172.21.15.1:0/1546523567 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5566b52521b0 msgr2=0x5566b5604b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.401+0000 7fdb8effd640 1 --2- 172.21.15.1:0/1546523567 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5566b52521b0 0x5566b5604b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.401+0000 7fdb8effd640 1 -- 172.21.15.1:0/1546523567 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566b5336620 con 0x5566b52292f0 2024-05-04T03:20:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.401+0000 7fdb8effd640 1 --2- 172.21.15.1:0/1546523567 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566b52292f0 0x5566b55fd680 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdb84013810 tx=0x7fdb84005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b53aef038ec1c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.401+0000 7fdb6effd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:20:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.401+0000 7fdb6effd640 1 -- 172.21.15.1:0/1546523567 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84005270 con 0x5566b52292f0 2024-05-04T03:20:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.401+0000 7fdb6effd640 1 -- 172.21.15.1:0/1546523567 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fdb84005410 con 0x5566b52292f0 2024-05-04T03:20:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.401+0000 7fdb9cdbc740 1 -- 172.21.15.1:0/1546523567 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb8400e960 con 0x5566b52292f0 2024-05-04T03:20:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.401+0000 7fdb6effd640 1 -- 172.21.15.1:0/1546523567 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdb84005610 con 0x5566b52292f0 2024-05-04T03:20:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.401+0000 7fdb9cdbc740 10 client.15432 fetch_fsmap learned FSMap version 26 2024-05-04T03:20:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.401+0000 7fdb9cdbc740 1 -- 172.21.15.1:0/1546523567 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5566b5607a50 con 0x5566b52292f0 2024-05-04T03:20:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.402+0000 7fdb6effd640 1 -- 172.21.15.1:0/1546523567 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb8400e960 con 0x5566b52292f0 2024-05-04T03:20:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.402+0000 7fdb6effd640 1 -- 172.21.15.1:0/1546523567 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fdb84011670 con 0x5566b52292f0 2024-05-04T03:20:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.402+0000 7fdb9cdbc740 10 client.15432 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:20:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.402+0000 7fdb9cdbc740 10 client.15432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:20:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.402+0000 7fdb9cdbc740 20 client.15432 populate_metadata read hostname 'smithi001' 2024-05-04T03:20:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.402+0000 7fdb9cdbc740 1 --2- 172.21.15.1:0/1546523567 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5566b5210d80 0x5566b560af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.402+0000 7fdb9cdbc740 4 client.15432 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:20:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.402+0000 7fdb9cdbc740 1 -- 172.21.15.1:0/1546523567 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566b55661e0 con 0x5566b5210d80 2024-05-04T03:20:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.402+0000 7fdb8f7fe640 1 --2- 172.21.15.1:0/1546523567 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5566b5210d80 0x5566b560af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:44.409 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.403+0000 7fdb8f7fe640 1 --2- 172.21.15.1:0/1546523567 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5566b5210d80 0x5566b560af20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84c8ecddb34c2446 server_cookie=7522885787c1f030 in_seq=0 out_seq=0 2024-05-04T03:20:44.410 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.403+0000 7fdb6effd640 10 client.15432 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.403+0000 7fdb6effd640 1 -- 172.21.15.1:0/1546523567 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5566b55661e0 con 0x5566b5210d80 2024-05-04T03:20:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.403+0000 7fdb6effd640 10 client.15432 handle_command_reply: tid=0 2024-05-04T03:20:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.403+0000 7fdb9cdbc740 1 client.15432 shutdown 2024-05-04T03:20:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.404+0000 7fdb9cdbc740 1 -- 172.21.15.1:0/1546523567 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566b52292f0 msgr2=0x5566b55fd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.404+0000 7fdb9cdbc740 1 --2- 172.21.15.1:0/1546523567 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566b52292f0 0x5566b55fd680 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdb84013810 tx=0x7fdb84005da0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.404+0000 7fdb9cdbc740 1 -- 172.21.15.1:0/1546523567 shutdown_connections 2024-05-04T03:20:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.404+0000 7fdb9cdbc740 1 --2- 172.21.15.1:0/1546523567 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5566b5210d80 0x5566b560af20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.404+0000 7fdb9cdbc740 1 --2- 172.21.15.1:0/1546523567 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5566b52521b0 0x5566b5604b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.404+0000 7fdb6effd640 0 client.15432 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.404+0000 7fdb9cdbc740 1 --2- 172.21.15.1:0/1546523567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566b52314d0 0x5566b55fdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.404+0000 7fdb9cdbc740 1 --2- 172.21.15.1:0/1546523567 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566b52292f0 0x5566b55fd680 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.404+0000 7fdb9cdbc740 1 -- 172.21.15.1:0/1546523567 >> 172.21.15.1:0/1546523567 conn(0x5566b5246300 msgr2=0x5566b55f2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:44.411 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.404+0000 7fdb9cdbc740 1 -- 172.21.15.1:0/1546523567 shutdown_connections 2024-05-04T03:20:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.404+0000 7fdb9cdbc740 1 -- 172.21.15.1:0/1546523567 wait complete. 2024-05-04T03:20:44.412 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.404+0000 7fdb9cdbc740 20 client.15432 trim_cache size 0 max 16384 2024-05-04T03:20:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.419+0000 7fdb9cdbc740 1 Processor -- start 2024-05-04T03:20:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.419+0000 7fdb9cdbc740 1 -- start start 2024-05-04T03:20:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.419+0000 7fdb9cdbc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566b52292f0 0x5566b55f8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.419+0000 7fdb9cdbc740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5566b52314d0 0x5566b55ef4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.419+0000 7fdb9cdbc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566b52521b0 0x5566b5604ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:44.430 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.419+0000 7fdb9cdbc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901ada20 con 0x5566b52292f0 2024-05-04T03:20:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.419+0000 7fdb9cdbc740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901ab2f0 con 0x5566b52314d0 2024-05-04T03:20:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.419+0000 7fdb9cdbc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901a9140 con 0x5566b52521b0 2024-05-04T03:20:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.420+0000 7fdb8effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566b52292f0 0x5566b55f8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.420+0000 7fdb8effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566b52292f0 0x5566b55f8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54380/0 (socket says 172.21.15.1:54380) 2024-05-04T03:20:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.420+0000 7fdb8effd640 1 -- 172.21.15.1:0/649351536 learned_addr learned my addr 172.21.15.1:0/649351536 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.420+0000 7fdb8f7fe640 1 --2- 172.21.15.1:0/649351536 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5566b52314d0 0x5566b55ef4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.420+0000 7fdb8effd640 1 -- 172.21.15.1:0/649351536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566b52521b0 msgr2=0x5566b5604ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.420+0000 7fdb8effd640 1 --2- 172.21.15.1:0/649351536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566b52521b0 0x5566b5604ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.420+0000 7fdb8effd640 1 -- 172.21.15.1:0/649351536 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5566b52314d0 msgr2=0x5566b55ef4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.420+0000 7fdb8effd640 1 --2- 172.21.15.1:0/649351536 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5566b52314d0 0x5566b55ef4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.420+0000 7fdb8effd640 1 -- 172.21.15.1:0/649351536 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566b55fbbb0 con 0x5566b52292f0 2024-05-04T03:20:44.432 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.420+0000 7fdb8f7fe640 1 --2- 172.21.15.1:0/649351536 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5566b52314d0 0x5566b55ef4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:20:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.420+0000 7fdb8effd640 1 --2- 172.21.15.1:0/649351536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566b52292f0 0x5566b55f8150 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fdb84005e10 tx=0x7fdb8400ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4ced4657aa0349 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.420+0000 7fdb757fa640 1 -- 172.21.15.1:0/649351536 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84022070 con 0x5566b52292f0 2024-05-04T03:20:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.420+0000 7fdb757fa640 1 -- 172.21.15.1:0/649351536 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb8400b9c0 con 0x5566b52292f0 2024-05-04T03:20:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.420+0000 7fdb757fa640 1 -- 172.21.15.1:0/649351536 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb840035f0 con 0x5566b52292f0 2024-05-04T03:20:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.421+0000 7fdb9cdbc740 1 -- 172.21.15.1:0/649351536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566b52292f0 msgr2=0x5566b55f8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.421+0000 7fdb9cdbc740 1 --2- 172.21.15.1:0/649351536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566b52292f0 0x5566b55f8150 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fdb84005e10 tx=0x7fdb8400ecf0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.433 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.421+0000 7fdb9cdbc740 1 -- 172.21.15.1:0/649351536 shutdown_connections 2024-05-04T03:20:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.421+0000 7fdb9cdbc740 1 --2- 172.21.15.1:0/649351536 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5566b52314d0 0x5566b55ef4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.421+0000 7fdb9cdbc740 1 --2- 172.21.15.1:0/649351536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566b52521b0 0x5566b5604ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.421+0000 7fdb9cdbc740 1 --2- 172.21.15.1:0/649351536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566b52292f0 0x5566b55f8150 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.421+0000 7fdb9cdbc740 1 -- 172.21.15.1:0/649351536 >> 172.21.15.1:0/649351536 conn(0x5566b5246300 msgr2=0x5566b55eedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.421+0000 7fdb9cdbc740 1 -- 172.21.15.1:0/649351536 shutdown_connections 2024-05-04T03:20:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.421+0000 7fdb9cdbc740 1 -- 172.21.15.1:0/649351536 wait complete. 2024-05-04T03:20:44.434 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.421+0000 7fdb9cdbc740 1 -- start start 2024-05-04T03:20:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.422+0000 7fdb9cdbc740 1 Processor -- start 2024-05-04T03:20:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.422+0000 7fdb9cdbc740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5566b52292f0 0x5566b56020d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.422+0000 7fdb9cdbc740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566b52314d0 0x5566b5602610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.422+0000 7fdb9cdbc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566b52521b0 0x5566b56712a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.422+0000 7fdb9cdbc740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901a97b0 con 0x5566b52314d0 2024-05-04T03:20:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.422+0000 7fdb9cdbc740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901aac20 con 0x5566b52292f0 2024-05-04T03:20:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.422+0000 7fdb9cdbc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901a9e80 con 0x5566b52521b0 2024-05-04T03:20:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.422+0000 7fdb9cdbc740 10 client.0 _ll_register_callbacks cb 0x7fdb80009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:20:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.422+0000 7fdb8f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566b52314d0 0x5566b5602610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.422+0000 7fdb8f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566b52314d0 0x5566b5602610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54382/0 (socket says 172.21.15.1:54382) 2024-05-04T03:20:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.422+0000 7fdb8f7fe640 1 -- 172.21.15.1:0/2192723608 learned_addr learned my addr 172.21.15.1:0/2192723608 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.422+0000 7fdb8effd640 1 --2- 172.21.15.1:0/2192723608 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566b52521b0 0x5566b56712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.422+0000 7fdb8effd640 1 --2- 172.21.15.1:0/2192723608 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5566b52292f0 0x5566b56020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.422+0000 7fdb8f7fe640 1 -- 172.21.15.1:0/2192723608 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566b52521b0 msgr2=0x5566b56712a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.422+0000 7fdb8f7fe640 1 --2- 172.21.15.1:0/2192723608 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566b52521b0 0x5566b56712a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.422+0000 7fdb8f7fe640 1 -- 172.21.15.1:0/2192723608 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5566b52292f0 msgr2=0x5566b56020d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.422+0000 7fdb8f7fe640 1 --2- 172.21.15.1:0/2192723608 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5566b52292f0 0x5566b56020d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.422+0000 7fdb8f7fe640 1 -- 172.21.15.1:0/2192723608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566b5605580 con 0x5566b52314d0 2024-05-04T03:20:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.422+0000 7fdb8effd640 1 --2- 172.21.15.1:0/2192723608 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566b52521b0 0x5566b56712a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:20:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.422+0000 7fdb8f7fe640 1 --2- 172.21.15.1:0/2192723608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566b52314d0 0x5566b5602610 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdb80005ab0 tx=0x7fdb80031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd09b9f35eface1b server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.423+0000 7fdb6effd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:20:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.423+0000 7fdb6effd640 1 -- 172.21.15.1:0/2192723608 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8004a020 con 0x5566b52314d0 2024-05-04T03:20:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.423+0000 7fdb6effd640 1 -- 172.21.15.1:0/2192723608 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fdb80004240 con 0x5566b52314d0 2024-05-04T03:20:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.423+0000 7fdb9cdbc740 1 -- 172.21.15.1:0/2192723608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb8400b9c0 con 0x5566b52314d0 2024-05-04T03:20:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.423+0000 7fdb6effd640 1 -- 172.21.15.1:0/2192723608 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdb80035040 con 0x5566b52314d0 2024-05-04T03:20:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.423+0000 7fdb9cdbc740 10 client.15438 fetch_fsmap learned FSMap version 26 2024-05-04T03:20:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.423+0000 7fdb9cdbc740 1 -- 172.21.15.1:0/2192723608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5566b56727d0 con 0x5566b52314d0 2024-05-04T03:20:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.423+0000 7fdb6effd640 1 -- 172.21.15.1:0/2192723608 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb8400b9c0 con 0x5566b52314d0 2024-05-04T03:20:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.423+0000 7fdb6effd640 1 -- 172.21.15.1:0/2192723608 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fdb800037b0 con 0x5566b52314d0 2024-05-04T03:20:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.423+0000 7fdb9cdbc740 10 client.15438 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:20:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.423+0000 7fdb9cdbc740 10 client.15438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:20:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.423+0000 7fdb9cdbc740 20 client.15438 populate_metadata read hostname 'smithi001' 2024-05-04T03:20:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.423+0000 7fdb9cdbc740 1 --2- 172.21.15.1:0/2192723608 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5566b51e61c0 0x5566b56751a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.423+0000 7fdb9cdbc740 4 client.15438 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:20:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.423+0000 7fdb9cdbc740 1 -- 172.21.15.1:0/2192723608 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5566b5560050 con 0x5566b51e61c0 2024-05-04T03:20:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.424+0000 7fdb8effd640 1 --2- 172.21.15.1:0/2192723608 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5566b51e61c0 0x5566b56751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.424+0000 7fdb8effd640 1 --2- 172.21.15.1:0/2192723608 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5566b51e61c0 0x5566b56751a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=542ea1b3b23b36fb server_cookie=eba081973ebb4a1 in_seq=0 out_seq=0 2024-05-04T03:20:44.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.424+0000 7fdb6effd640 10 client.15438 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.430+0000 7fdb6effd640 1 -- 172.21.15.1:0/2192723608 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5566b5560050 con 0x5566b51e61c0 2024-05-04T03:20:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.430+0000 7fdb6effd640 10 client.15438 handle_command_reply: tid=0 2024-05-04T03:20:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.430+0000 7fdb9cdbc740 1 client.15438 shutdown 2024-05-04T03:20:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.431+0000 7fdb9cdbc740 1 -- 172.21.15.1:0/2192723608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566b52314d0 msgr2=0x5566b5602610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.431+0000 7fdb9cdbc740 1 --2- 172.21.15.1:0/2192723608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566b52314d0 0x5566b5602610 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdb80005ab0 tx=0x7fdb80031700 comp rx=0 tx=0).stop 2024-05-04T03:20:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.431+0000 7fdb9cdbc740 1 -- 172.21.15.1:0/2192723608 shutdown_connections 2024-05-04T03:20:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.431+0000 7fdb9cdbc740 1 --2- 172.21.15.1:0/2192723608 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5566b51e61c0 0x5566b56751a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.431+0000 7fdb9cdbc740 1 --2- 172.21.15.1:0/2192723608 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5566b52521b0 0x5566b56712a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.431+0000 7fdb9cdbc740 1 --2- 172.21.15.1:0/2192723608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566b52314d0 0x5566b5602610 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.431+0000 7fdb6effd640 0 client.15438 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.431+0000 7fdb9cdbc740 1 --2- 172.21.15.1:0/2192723608 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5566b52292f0 0x5566b56020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.431+0000 7fdb9cdbc740 1 -- 172.21.15.1:0/2192723608 >> 172.21.15.1:0/2192723608 conn(0x5566b5246300 msgr2=0x5566b55f21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.431+0000 7fdb9cdbc740 1 -- 172.21.15.1:0/2192723608 shutdown_connections 2024-05-04T03:20:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.431+0000 7fdb9cdbc740 1 -- 172.21.15.1:0/2192723608 wait complete. 2024-05-04T03:20:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.431+0000 7fdb9cdbc740 20 client.15438 trim_cache size 0 max 16384 2024-05-04T03:20:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:20:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:20:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:20:44.441 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:20:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.431+0000 7fdb95e27640 1 -- 172.21.15.1:0/3351635320 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fdb780b2920 msgr2=0x7fdb780b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.431+0000 7fdb95e27640 1 --2- 172.21.15.1:0/3351635320 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fdb780b2920 0x7fdb780b4e30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdb880a9d70 tx=0x7fdb8806f320 comp rx=0 tx=0).stop 2024-05-04T03:20:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.431+0000 7fdb95e27640 1 -- 172.21.15.1:0/3351635320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb90108a70 msgr2=0x7fdb901a15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.431+0000 7fdb95e27640 1 --2- 172.21.15.1:0/3351635320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb90108a70 0x7fdb901a15e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800eb10 tx=0x7fdb8800c2d0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.431+0000 7fdb95e27640 1 -- 172.21.15.1:0/3351635320 shutdown_connections 2024-05-04T03:20:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.431+0000 7fdb95e27640 1 --2- 172.21.15.1:0/3351635320 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fdb780b2920 0x7fdb780b4e30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.442 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.431+0000 7fdb95e27640 1 --2- 172.21.15.1:0/3351635320 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb90103d70 0x7fdb9019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.431+0000 7fdb95e27640 1 --2- 172.21.15.1:0/3351635320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb90108a70 0x7fdb901a15e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.432+0000 7fdb95e27640 1 --2- 172.21.15.1:0/3351635320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb90102b30 0x7fdb90075960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.432+0000 7fdb95e27640 1 -- 172.21.15.1:0/3351635320 >> 172.21.15.1:0/3351635320 conn(0x7fdb900fdd20 msgr2=0x7fdb90104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.432+0000 7fdb95e27640 1 -- 172.21.15.1:0/3351635320 shutdown_connections 2024-05-04T03:20:44.443 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:44.432+0000 7fdb95e27640 1 -- 172.21.15.1:0/3351635320 wait complete. 2024-05-04T03:20:44.548 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:44 smithi072 ceph-mon[33332]: pgmap v438: 97 pgs: 97 active+clean; 681 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-04T03:20:44.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:44 smithi001 ceph-mon[29731]: pgmap v438: 97 pgs: 97 active+clean; 681 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-04T03:20:44.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:44 smithi077 ceph-mon[33531]: pgmap v438: 97 pgs: 97 active+clean; 681 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-04T03:20:46.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:46 smithi001 ceph-mon[29731]: pgmap v439: 97 pgs: 97 active+clean; 681 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-04T03:20:46.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:46 smithi077 ceph-mon[33531]: pgmap v439: 97 pgs: 97 active+clean; 681 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-04T03:20:46.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:46 smithi072 ceph-mon[33332]: pgmap v439: 97 pgs: 97 active+clean; 681 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-04T03:20:48.598 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:20:48.626 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:48 smithi001 ceph-mon[29731]: pgmap v440: 97 pgs: 97 active+clean; 681 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-04T03:20:48.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:48 smithi077 ceph-mon[33531]: pgmap v440: 97 pgs: 97 active+clean; 681 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-04T03:20:48.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:48 smithi072 ceph-mon[33332]: pgmap v440: 97 pgs: 97 active+clean; 681 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 597 B/s wr, 5 op/s 2024-05-04T03:20:48.862 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:20:50.599 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:50 smithi001 ceph-mon[29731]: pgmap v441: 97 pgs: 97 active+clean; 682 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-05-04T03:20:50.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.614+0000 7fae27b4f700 1 -- 172.21.15.1:0/3688289687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae201042f0 msgr2=0x7fae20110070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:50.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.614+0000 7fae27b4f700 1 --2- 172.21.15.1:0/3688289687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae201042f0 0x7fae20110070 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fae1800b0a0 tx=0x7fae1800b3b0 comp rx=0 tx=0).stop 2024-05-04T03:20:50.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.615+0000 7fae27b4f700 1 -- 172.21.15.1:0/3688289687 shutdown_connections 2024-05-04T03:20:50.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.615+0000 7fae27b4f700 1 --2- 172.21.15.1:0/3688289687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae201042f0 0x7fae20110070 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:50.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.615+0000 7fae27b4f700 1 --2- 172.21.15.1:0/3688289687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae20108200 0x7fae20108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:50.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.615+0000 7fae27b4f700 1 --2- 172.21.15.1:0/3688289687 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae201039a0 0x7fae20103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:50.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.615+0000 7fae27b4f700 1 -- 172.21.15.1:0/3688289687 >> 172.21.15.1:0/3688289687 conn(0x7fae200fe4e0 msgr2=0x7fae20100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:50.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.615+0000 7fae27b4f700 1 -- 172.21.15.1:0/3688289687 shutdown_connections 2024-05-04T03:20:50.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.615+0000 7fae27b4f700 1 -- 172.21.15.1:0/3688289687 wait complete. 2024-05-04T03:20:50.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.616+0000 7fae27b4f700 1 Processor -- start 2024-05-04T03:20:50.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.616+0000 7fae27b4f700 1 -- start start 2024-05-04T03:20:50.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.616+0000 7fae27b4f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae201039a0 0x7fae2019bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:50.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.616+0000 7fae27b4f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae201042f0 0x7fae2019c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:50.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.616+0000 7fae27b4f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae20108200 0x7fae2019c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:50.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.616+0000 7fae27b4f700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fae20113390 con 0x7fae20108200 2024-05-04T03:20:50.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.616+0000 7fae27b4f700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fae20113210 con 0x7fae201042f0 2024-05-04T03:20:50.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.616+0000 7fae27b4f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fae20113510 con 0x7fae201039a0 2024-05-04T03:20:50.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.616+0000 7fae260ec700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae20108200 0x7fae2019c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:50.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.616+0000 7fae260ec700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae20108200 0x7fae2019c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54408/0 (socket says 172.21.15.1:54408) 2024-05-04T03:20:50.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.616+0000 7fae260ec700 1 -- 172.21.15.1:0/2061956724 learned_addr learned my addr 172.21.15.1:0/2061956724 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:50.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.616+0000 7fae258eb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae201039a0 0x7fae2019bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:50.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.616+0000 7fae250ea700 1 --2- 172.21.15.1:0/2061956724 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae201042f0 0x7fae2019c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:50.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.617+0000 7fae260ec700 1 -- 172.21.15.1:0/2061956724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae201039a0 msgr2=0x7fae2019bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:50.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.617+0000 7fae260ec700 1 --2- 172.21.15.1:0/2061956724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae201039a0 0x7fae2019bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:50.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.617+0000 7fae260ec700 1 -- 172.21.15.1:0/2061956724 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae201042f0 msgr2=0x7fae2019c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:50.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.617+0000 7fae260ec700 1 --2- 172.21.15.1:0/2061956724 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae201042f0 0x7fae2019c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:50.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.617+0000 7fae260ec700 1 -- 172.21.15.1:0/2061956724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae201a6f00 con 0x7fae20108200 2024-05-04T03:20:50.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.617+0000 7fae260ec700 1 --2- 172.21.15.1:0/2061956724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae20108200 0x7fae2019c650 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fae180062a0 tx=0x7fae18004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:50.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.617+0000 7fae16ffd700 1 -- 172.21.15.1:0/2061956724 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18021070 con 0x7fae20108200 2024-05-04T03:20:50.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.617+0000 7fae27b4f700 1 -- 172.21.15.1:0/2061956724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae201a7190 con 0x7fae20108200 2024-05-04T03:20:50.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.617+0000 7fae16ffd700 1 -- 172.21.15.1:0/2061956724 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae18007e30 con 0x7fae20108200 2024-05-04T03:20:50.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.617+0000 7fae16ffd700 1 -- 172.21.15.1:0/2061956724 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae180046d0 con 0x7fae20108200 2024-05-04T03:20:50.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.617+0000 7fae27b4f700 1 -- 172.21.15.1:0/2061956724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae201a7640 con 0x7fae20108200 2024-05-04T03:20:50.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.618+0000 7fae27b4f700 1 -- 172.21.15.1:0/2061956724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae200512c0 con 0x7fae20108200 2024-05-04T03:20:50.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.619+0000 7fae16ffd700 1 -- 172.21.15.1:0/2061956724 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7fae18007980 con 0x7fae20108200 2024-05-04T03:20:50.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.620+0000 7fae16ffd700 1 --2- 172.21.15.1:0/2061956724 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fae0c0b2880 0x7fae0c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:50.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.621+0000 7fae258eb700 1 --2- 172.21.15.1:0/2061956724 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fae0c0b2880 0x7fae0c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:50.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.621+0000 7fae16ffd700 1 -- 172.21.15.1:0/2061956724 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fae1800f020 con 0x7fae20108200 2024-05-04T03:20:50.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.621+0000 7fae258eb700 1 --2- 172.21.15.1:0/2061956724 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fae0c0b2880 0x7fae0c0b4d90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fae2006c310 tx=0x7fae1c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:50.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.624+0000 7fae16ffd700 1 -- 172.21.15.1:0/2061956724 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fae1802a020 con 0x7fae20108200 2024-05-04T03:20:50.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:50 smithi077 ceph-mon[33531]: pgmap v441: 97 pgs: 97 active+clean; 682 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-05-04T03:20:50.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:50 smithi072 ceph-mon[33332]: pgmap v441: 97 pgs: 97 active+clean; 682 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 7 op/s 2024-05-04T03:20:50.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.886+0000 7fae27b4f700 1 -- 172.21.15.1:0/2061956724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae2002e5f0 con 0x7fae20108200 2024-05-04T03:20:50.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.886+0000 7fae16ffd700 1 -- 172.21.15.1:0/2061956724 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11844 (secure 0 0 0) 0x7fae18004870 con 0x7fae20108200 2024-05-04T03:20:50.888 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:20:50.889 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2186681910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2186681910},{"type":"v1","addr":"172.21.15.1:6835","nonce":2186681910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2747191047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2747191047},{"type":"v1","addr":"172.21.15.1:6837","nonce":2747191047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4115120540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4115120540},{"type":"v1","addr":"172.21.15.1:6839","nonce":4115120540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3405709780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3405709780},{"type":"v1","addr":"172.21.15.1:6841","nonce":3405709780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/3816098761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":3816098761},{"type":"v1","addr":"172.21.15.72:6833","nonce":3816098761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/379026128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":379026128},{"type":"v1","addr":"172.21.15.77:6833","nonce":379026128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:15:49.823471+0000","modified":"2024-05-04T03:18:45.058419+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24472,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/3930984976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3930984976},{"type":"v1","addr":"172.21.15.72:6835","nonce":3930984976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/4279951305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4279951305},{"type":"v1","addr":"172.21.15.77:6835","nonce":4279951305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2003521417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2003521417},{"type":"v1","addr":"172.21.15.77:6837","nonce":2003521417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/1417614203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1417614203},{"type":"v1","addr":"172.21.15.72:6837","nonce":1417614203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.77:6839/922723189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":922723189},{"type":"v1","addr":"172.21.15.77:6839","nonce":922723189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2976050650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2976050650},{"type":"v1","addr":"172.21.15.72:6839","nonce":2976050650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24467,24472,24494]},"id":1}]} 2024-05-04T03:20:50.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.888+0000 7fae27b4f700 1 -- 172.21.15.1:0/2061956724 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fae0c0b2880 msgr2=0x7fae0c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:50.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.888+0000 7fae27b4f700 1 --2- 172.21.15.1:0/2061956724 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fae0c0b2880 0x7fae0c0b4d90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fae2006c310 tx=0x7fae1c00b410 comp rx=0 tx=0).stop 2024-05-04T03:20:50.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.888+0000 7fae27b4f700 1 -- 172.21.15.1:0/2061956724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae20108200 msgr2=0x7fae2019c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:50.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.888+0000 7fae27b4f700 1 --2- 172.21.15.1:0/2061956724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae20108200 0x7fae2019c650 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fae180062a0 tx=0x7fae18004c80 comp rx=0 tx=0).stop 2024-05-04T03:20:50.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.889+0000 7fae27b4f700 1 -- 172.21.15.1:0/2061956724 shutdown_connections 2024-05-04T03:20:50.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.889+0000 7fae27b4f700 1 --2- 172.21.15.1:0/2061956724 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fae0c0b2880 0x7fae0c0b4d90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:50.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.889+0000 7fae27b4f700 1 --2- 172.21.15.1:0/2061956724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fae20108200 0x7fae2019c650 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:50.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.889+0000 7fae27b4f700 1 --2- 172.21.15.1:0/2061956724 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae201039a0 0x7fae2019bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:50.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.889+0000 7fae27b4f700 1 --2- 172.21.15.1:0/2061956724 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae201042f0 0x7fae2019c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:50.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.889+0000 7fae27b4f700 1 -- 172.21.15.1:0/2061956724 >> 172.21.15.1:0/2061956724 conn(0x7fae200fe4e0 msgr2=0x7fae201075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:50.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.889+0000 7fae27b4f700 1 -- 172.21.15.1:0/2061956724 shutdown_connections 2024-05-04T03:20:50.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:50.889+0000 7fae27b4f700 1 -- 172.21.15.1:0/2061956724 wait complete. 2024-05-04T03:20:50.892 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 26 2024-05-04T03:20:51.603 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:51 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2061956724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:20:51.634 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 661992bc --release --await' 2024-05-04T03:20:51.634 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24494 quiesce db --set-id 661992bc --release --await 2024-05-04T03:20:51.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:51 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/2061956724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:20:51.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:51 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/2061956724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:20:51.934 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:20:52.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:52 smithi001 ceph-mon[29731]: pgmap v442: 97 pgs: 97 active+clean; 682 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-04T03:20:52.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:52 smithi077 ceph-mon[33531]: pgmap v442: 97 pgs: 97 active+clean; 682 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-04T03:20:52.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:52 smithi072 ceph-mon[33332]: pgmap v442: 97 pgs: 97 active+clean; 682 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-04T03:20:53.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.635+0000 7f369fdd7700 1 -- 172.21.15.1:0/728528927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3698102b70 msgr2=0x7f3698102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:53.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.635+0000 7f369fdd7700 1 --2- 172.21.15.1:0/728528927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3698102b70 0x7f3698102f80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3694009990 tx=0x7f3694009ca0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.635+0000 7f369fdd7700 1 -- 172.21.15.1:0/728528927 shutdown_connections 2024-05-04T03:20:53.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.635+0000 7f369fdd7700 1 --2- 172.21.15.1:0/728528927 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36981098e0 0x7f3698111220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.635+0000 7f369fdd7700 1 --2- 172.21.15.1:0/728528927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3698102b70 0x7f3698102f80 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.635+0000 7f369fdd7700 1 --2- 172.21.15.1:0/728528927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3698108f90 0x7f36981093a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.635+0000 7f369fdd7700 1 -- 172.21.15.1:0/728528927 >> 172.21.15.1:0/728528927 conn(0x7f36980fe510 msgr2=0x7f3698100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:53.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.635+0000 7f369fdd7700 1 -- 172.21.15.1:0/728528927 shutdown_connections 2024-05-04T03:20:53.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.635+0000 7f369fdd7700 1 -- 172.21.15.1:0/728528927 wait complete. 2024-05-04T03:20:53.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.636+0000 7f369fdd7700 1 Processor -- start 2024-05-04T03:20:53.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.636+0000 7f369fdd7700 1 -- start start 2024-05-04T03:20:53.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.636+0000 7f369fdd7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3698102b70 0x7f369819d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:53.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.636+0000 7f369fdd7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3698108f90 0x7f369819d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:53.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.636+0000 7f369fdd7700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36981098e0 0x7f36981a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:53.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.636+0000 7f369fdd7700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f36981144d0 con 0x7f36981098e0 2024-05-04T03:20:53.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.636+0000 7f369fdd7700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3698114350 con 0x7f3698102b70 2024-05-04T03:20:53.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.636+0000 7f369fdd7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3698114650 con 0x7f3698108f90 2024-05-04T03:20:53.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.637+0000 7f369e374700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36981098e0 0x7f36981a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:53.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.637+0000 7f369e374700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36981098e0 0x7f36981a1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54084/0 (socket says 172.21.15.1:54084) 2024-05-04T03:20:53.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.637+0000 7f369e374700 1 -- 172.21.15.1:0/649781952 learned_addr learned my addr 172.21.15.1:0/649781952 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:53.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.637+0000 7f369d372700 1 --2- 172.21.15.1:0/649781952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3698108f90 0x7f369819d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:53.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.637+0000 7f369db73700 1 --2- 172.21.15.1:0/649781952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3698102b70 0x7f369819d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:53.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.637+0000 7f369e374700 1 -- 172.21.15.1:0/649781952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3698108f90 msgr2=0x7f369819d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:53.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.637+0000 7f369e374700 1 --2- 172.21.15.1:0/649781952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3698108f90 0x7f369819d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.637+0000 7f369e374700 1 -- 172.21.15.1:0/649781952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3698102b70 msgr2=0x7f369819d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:53.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.637+0000 7f369e374700 1 --2- 172.21.15.1:0/649781952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3698102b70 0x7f369819d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.637+0000 7f369e374700 1 -- 172.21.15.1:0/649781952 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36981a1f30 con 0x7f36981098e0 2024-05-04T03:20:53.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.637+0000 7f369e374700 1 --2- 172.21.15.1:0/649781952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36981098e0 0x7f36981a1890 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3690007ee0 tx=0x7f369000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:53.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.637+0000 7f368effd700 1 -- 172.21.15.1:0/649781952 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369000dd20 con 0x7f36981098e0 2024-05-04T03:20:53.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.637+0000 7f368effd700 1 -- 172.21.15.1:0/649781952 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3690004020 con 0x7f36981098e0 2024-05-04T03:20:53.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.637+0000 7f369fdd7700 1 -- 172.21.15.1:0/649781952 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36981a2220 con 0x7f36981098e0 2024-05-04T03:20:53.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.637+0000 7f368effd700 1 -- 172.21.15.1:0/649781952 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690015660 con 0x7f36981098e0 2024-05-04T03:20:53.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.638+0000 7f369fdd7700 1 -- 172.21.15.1:0/649781952 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36981a9b40 con 0x7f36981098e0 2024-05-04T03:20:53.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.640+0000 7f368effd700 1 -- 172.21.15.1:0/649781952 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f36900041d0 con 0x7f36981098e0 2024-05-04T03:20:53.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.641+0000 7f368effd700 1 --2- 172.21.15.1:0/649781952 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f36840b2880 0x7f36840b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:53.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.641+0000 7f36ae7ebb80 1 Processor -- start 2024-05-04T03:20:53.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.641+0000 7f369db73700 1 --2- 172.21.15.1:0/649781952 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f36840b2880 0x7f36840b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:53.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.641+0000 7f368effd700 1 -- 172.21.15.1:0/649781952 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f36900e1bf0 con 0x7f36981098e0 2024-05-04T03:20:53.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.641+0000 7f36ae7ebb80 1 -- start start 2024-05-04T03:20:53.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.642+0000 7f369db73700 1 --2- 172.21.15.1:0/649781952 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f36840b2880 0x7f36840b4d90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3694006010 tx=0x7f3694005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:53.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.642+0000 7f36ae7ebb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56036591e5d0 0x56036591e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:53.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.642+0000 7f36ae7ebb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56036591ef20 0x5603659293a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:53.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.642+0000 7f36ae7ebb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5603659298e0 0x56036592bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:53.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.642+0000 7f36ae7ebb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x560365841960 con 0x5603659298e0 2024-05-04T03:20:53.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.642+0000 7f36ae7ebb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5603657e08c0 con 0x56036591ef20 2024-05-04T03:20:53.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.642+0000 7f36ae7ebb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56036583d920 con 0x56036591e5d0 2024-05-04T03:20:53.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.642+0000 7f369e374700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5603659298e0 0x56036592bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:53.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.642+0000 7f369e374700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5603659298e0 0x56036592bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54094/0 (socket says 172.21.15.1:54094) 2024-05-04T03:20:53.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.642+0000 7f369e374700 1 -- 172.21.15.1:0/2727921577 learned_addr learned my addr 172.21.15.1:0/2727921577 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:53.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.642+0000 7f369d372700 1 --2- 172.21.15.1:0/2727921577 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56036591ef20 0x5603659293a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:53.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.642+0000 7f369db73700 1 --2- 172.21.15.1:0/2727921577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56036591e5d0 0x56036591e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:53.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.642+0000 7f369e374700 1 -- 172.21.15.1:0/2727921577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56036591e5d0 msgr2=0x56036591e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:53.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.642+0000 7f369e374700 1 --2- 172.21.15.1:0/2727921577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56036591e5d0 0x56036591e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.642+0000 7f369e374700 1 -- 172.21.15.1:0/2727921577 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56036591ef20 msgr2=0x5603659293a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:53.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.642+0000 7f369e374700 1 --2- 172.21.15.1:0/2727921577 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56036591ef20 0x5603659293a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.642+0000 7f369e374700 1 -- 172.21.15.1:0/2727921577 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56036578e9d0 con 0x5603659298e0 2024-05-04T03:20:53.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.642+0000 7f369e374700 1 --2- 172.21.15.1:0/2727921577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5603659298e0 0x56036592bd00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f36900a91c0 tx=0x7f3690009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6bf52fdbac8c011 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:53.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.643+0000 7f3683fff700 1 -- 172.21.15.1:0/2727921577 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36900a9230 con 0x5603659298e0 2024-05-04T03:20:53.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.643+0000 7f3683fff700 1 -- 172.21.15.1:0/2727921577 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36900ad5e0 con 0x5603659298e0 2024-05-04T03:20:53.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.643+0000 7f3683fff700 1 -- 172.21.15.1:0/2727921577 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690004e30 con 0x5603659298e0 2024-05-04T03:20:53.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.644+0000 7f36ae7ebb80 1 -- 172.21.15.1:0/2727921577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5603659298e0 msgr2=0x56036592bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:53.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.644+0000 7f36ae7ebb80 1 --2- 172.21.15.1:0/2727921577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5603659298e0 0x56036592bd00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f36900a91c0 tx=0x7f3690009590 comp rx=0 tx=0).stop 2024-05-04T03:20:53.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.644+0000 7f36ae7ebb80 1 -- 172.21.15.1:0/2727921577 shutdown_connections 2024-05-04T03:20:53.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.644+0000 7f36ae7ebb80 1 --2- 172.21.15.1:0/2727921577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5603659298e0 0x56036592bd00 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.644+0000 7f36ae7ebb80 1 --2- 172.21.15.1:0/2727921577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56036591e5d0 0x56036591e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.644+0000 7f36ae7ebb80 1 --2- 172.21.15.1:0/2727921577 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56036591ef20 0x5603659293a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.644+0000 7f36ae7ebb80 1 -- 172.21.15.1:0/2727921577 >> 172.21.15.1:0/2727921577 conn(0x5603657a1f40 msgr2=0x56036591a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:53.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.644+0000 7f36ae7ebb80 1 -- 172.21.15.1:0/2727921577 shutdown_connections 2024-05-04T03:20:53.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.644+0000 7f36ae7ebb80 1 -- 172.21.15.1:0/2727921577 wait complete. 2024-05-04T03:20:53.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.644+0000 7f36ae7ebb80 1 -- start start 2024-05-04T03:20:53.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.644+0000 7f36ae7ebb80 1 Processor -- start 2024-05-04T03:20:53.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.645+0000 7f36ae7ebb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56036591e5d0 0x560365926820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:53.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.645+0000 7f36ae7ebb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56036591ef20 0x560365926d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:53.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.645+0000 7f36ae7ebb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603659298e0 0x5603659272a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:53.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.645+0000 7f36ae7ebb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56036592c670 con 0x56036591ef20 2024-05-04T03:20:53.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.645+0000 7f36ae7ebb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5603657a7d30 con 0x5603659298e0 2024-05-04T03:20:53.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.645+0000 7f36ae7ebb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56036592c7f0 con 0x56036591e5d0 2024-05-04T03:20:53.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.645+0000 7f36ae7ebb80 10 client.0 _ll_register_callbacks cb 0x5603657d9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:20:53.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.645+0000 7f369d372700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56036591ef20 0x560365926d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:53.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.645+0000 7f369d372700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56036591ef20 0x560365926d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54104/0 (socket says 172.21.15.1:54104) 2024-05-04T03:20:53.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.645+0000 7f369d372700 1 -- 172.21.15.1:0/1670287691 learned_addr learned my addr 172.21.15.1:0/1670287691 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:53.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.645+0000 7f369db73700 1 --2- 172.21.15.1:0/1670287691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56036591e5d0 0x560365926820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:53.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.645+0000 7f369e374700 1 --2- 172.21.15.1:0/1670287691 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603659298e0 0x5603659272a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:53.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.645+0000 7f369d372700 1 -- 172.21.15.1:0/1670287691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56036591e5d0 msgr2=0x560365926820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:53.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.645+0000 7f369d372700 1 --2- 172.21.15.1:0/1670287691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56036591e5d0 0x560365926820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.645+0000 7f369d372700 1 -- 172.21.15.1:0/1670287691 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603659298e0 msgr2=0x5603659272a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:53.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.645+0000 7f369d372700 1 --2- 172.21.15.1:0/1670287691 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603659298e0 0x5603659272a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.645+0000 7f369d372700 1 -- 172.21.15.1:0/1670287691 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603657dd790 con 0x56036591ef20 2024-05-04T03:20:53.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.645+0000 7f369d372700 1 --2- 172.21.15.1:0/1670287691 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56036591ef20 0x560365926d60 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3688012740 tx=0x7f3688012a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c53b19f041820db server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:53.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.645+0000 7f36817fa700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:20:53.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.646+0000 7f36817fa700 1 -- 172.21.15.1:0/1670287691 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368800f480 con 0x56036591ef20 2024-05-04T03:20:53.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.646+0000 7f36817fa700 1 -- 172.21.15.1:0/1670287691 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f368800f620 con 0x56036591ef20 2024-05-04T03:20:53.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.646+0000 7f36ae7ebb80 1 -- 172.21.15.1:0/1670287691 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36900ad5e0 con 0x56036591ef20 2024-05-04T03:20:53.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.646+0000 7f36817fa700 1 -- 172.21.15.1:0/1670287691 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3688022030 con 0x56036591ef20 2024-05-04T03:20:53.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.646+0000 7f36ae7ebb80 10 client.15456 fetch_fsmap learned FSMap version 26 2024-05-04T03:20:53.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.646+0000 7f36ae7ebb80 1 -- 172.21.15.1:0/1670287691 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5603659372c0 con 0x56036591ef20 2024-05-04T03:20:53.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.646+0000 7f36817fa700 1 -- 172.21.15.1:0/1670287691 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36900ad5e0 con 0x56036591ef20 2024-05-04T03:20:53.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.646+0000 7f36817fa700 1 -- 172.21.15.1:0/1670287691 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f36880278e0 con 0x56036591ef20 2024-05-04T03:20:53.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.646+0000 7f36ae7ebb80 10 client.15456 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:20:53.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.646+0000 7f36ae7ebb80 10 client.15456 resolve_mds: validated gid 24494 aka daemon mds.b 2024-05-04T03:20:53.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.646+0000 7f36ae7ebb80 20 client.15456 populate_metadata read hostname 'smithi001' 2024-05-04T03:20:53.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.646+0000 7f36ae7ebb80 1 --2- 172.21.15.1:0/1670287691 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x560365939950 0x56036593bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:53.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.646+0000 7f36ae7ebb80 4 client.15456 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:20:53.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.646+0000 7f36ae7ebb80 1 -- 172.21.15.1:0/1670287691 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56036584f340 con 0x560365939950 2024-05-04T03:20:53.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.647+0000 7f369db73700 1 --2- 172.21.15.1:0/1670287691 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x560365939950 0x56036593bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:53.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.647+0000 7f369db73700 1 --2- 172.21.15.1:0/1670287691 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x560365939950 0x56036593bd90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87fb0b3dc5540f4d server_cookie=ced8fdf154af60f9 in_seq=0 out_seq=0 2024-05-04T03:20:53.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.647+0000 7f36817fa700 10 client.15456 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:53.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.648+0000 7f36817fa700 1 -- 172.21.15.1:0/1670287691 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56036584f340 con 0x560365939950 2024-05-04T03:20:53.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.648+0000 7f36817fa700 10 client.15456 handle_command_reply: tid=0 2024-05-04T03:20:53.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.648+0000 7f36ae7ebb80 1 client.15456 shutdown 2024-05-04T03:20:53.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.648+0000 7f36ae7ebb80 1 -- 172.21.15.1:0/1670287691 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56036591ef20 msgr2=0x560365926d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:53.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.648+0000 7f36ae7ebb80 1 --2- 172.21.15.1:0/1670287691 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56036591ef20 0x560365926d60 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3688012740 tx=0x7f3688012a50 comp rx=0 tx=0).stop 2024-05-04T03:20:53.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.648+0000 7f36ae7ebb80 1 -- 172.21.15.1:0/1670287691 shutdown_connections 2024-05-04T03:20:53.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.648+0000 7f36ae7ebb80 1 --2- 172.21.15.1:0/1670287691 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x560365939950 0x56036593bd90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.648+0000 7f36ae7ebb80 1 --2- 172.21.15.1:0/1670287691 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603659298e0 0x5603659272a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.648+0000 7f36ae7ebb80 1 --2- 172.21.15.1:0/1670287691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56036591e5d0 0x560365926820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.648+0000 7f36ae7ebb80 1 --2- 172.21.15.1:0/1670287691 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56036591ef20 0x560365926d60 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.648+0000 7f36817fa700 0 client.15456 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:53.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.648+0000 7f36ae7ebb80 1 -- 172.21.15.1:0/1670287691 >> 172.21.15.1:0/1670287691 conn(0x5603657a1f40 msgr2=0x5603659201c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:53.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.648+0000 7f36ae7ebb80 1 -- 172.21.15.1:0/1670287691 shutdown_connections 2024-05-04T03:20:53.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.648+0000 7f36ae7ebb80 1 -- 172.21.15.1:0/1670287691 wait complete. 2024-05-04T03:20:53.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.648+0000 7f36ae7ebb80 20 client.15456 trim_cache size 0 max 16384 2024-05-04T03:20:53.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.669+0000 7f36ae7ebb80 1 Processor -- start 2024-05-04T03:20:53.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.669+0000 7f36ae7ebb80 1 -- start start 2024-05-04T03:20:53.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.669+0000 7f36ae7ebb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56036591e5d0 0x5603659257e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:53.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.669+0000 7f36ae7ebb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56036591ef20 0x560365935700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:53.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.669+0000 7f36ae7ebb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603659298e0 0x560365931a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:53.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.669+0000 7f36ae7ebb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f36981b3580 con 0x56036591e5d0 2024-05-04T03:20:53.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.669+0000 7f36ae7ebb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f36981b0e10 con 0x56036591ef20 2024-05-04T03:20:53.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.669+0000 7f36ae7ebb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f36981aece0 con 0x5603659298e0 2024-05-04T03:20:53.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.669+0000 7f369db73700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56036591e5d0 0x5603659257e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:53.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.669+0000 7f369db73700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56036591e5d0 0x5603659257e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54118/0 (socket says 172.21.15.1:54118) 2024-05-04T03:20:53.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.669+0000 7f369db73700 1 -- 172.21.15.1:0/3398120727 learned_addr learned my addr 172.21.15.1:0/3398120727 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:53.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.669+0000 7f369d372700 1 --2- 172.21.15.1:0/3398120727 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56036591ef20 0x560365935700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:53.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.669+0000 7f369e374700 1 --2- 172.21.15.1:0/3398120727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603659298e0 0x560365931a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:53.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.669+0000 7f369db73700 1 -- 172.21.15.1:0/3398120727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603659298e0 msgr2=0x560365931a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:53.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.669+0000 7f369db73700 1 --2- 172.21.15.1:0/3398120727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603659298e0 0x560365931a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.669+0000 7f369db73700 1 -- 172.21.15.1:0/3398120727 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56036591ef20 msgr2=0x560365935700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:53.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.669+0000 7f369db73700 1 --2- 172.21.15.1:0/3398120727 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56036591ef20 0x560365935700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.669+0000 7f369db73700 1 -- 172.21.15.1:0/3398120727 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560365927fd0 con 0x56036591e5d0 2024-05-04T03:20:53.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.670+0000 7f369db73700 1 --2- 172.21.15.1:0/3398120727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56036591e5d0 0x5603659257e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f369806bc70 tx=0x7f3694021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef70cd95f0078a server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:53.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.670+0000 7f36837fe700 1 -- 172.21.15.1:0/3398120727 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3694039070 con 0x56036591e5d0 2024-05-04T03:20:53.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.670+0000 7f36837fe700 1 -- 172.21.15.1:0/3398120727 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3694003d30 con 0x56036591e5d0 2024-05-04T03:20:53.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.670+0000 7f36837fe700 1 -- 172.21.15.1:0/3398120727 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369400fd30 con 0x56036591e5d0 2024-05-04T03:20:53.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.671+0000 7f36ae7ebb80 1 -- 172.21.15.1:0/3398120727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56036591e5d0 msgr2=0x5603659257e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:53.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.671+0000 7f36ae7ebb80 1 --2- 172.21.15.1:0/3398120727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56036591e5d0 0x5603659257e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f369806bc70 tx=0x7f3694021000 comp rx=0 tx=0).stop 2024-05-04T03:20:53.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.671+0000 7f36ae7ebb80 1 -- 172.21.15.1:0/3398120727 shutdown_connections 2024-05-04T03:20:53.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.671+0000 7f36ae7ebb80 1 --2- 172.21.15.1:0/3398120727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603659298e0 0x560365931a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.671+0000 7f36ae7ebb80 1 --2- 172.21.15.1:0/3398120727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56036591e5d0 0x5603659257e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.671+0000 7f36ae7ebb80 1 --2- 172.21.15.1:0/3398120727 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56036591ef20 0x560365935700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.671+0000 7f36ae7ebb80 1 -- 172.21.15.1:0/3398120727 >> 172.21.15.1:0/3398120727 conn(0x5603657a1f40 msgr2=0x5603659260d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:53.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.671+0000 7f36ae7ebb80 1 -- 172.21.15.1:0/3398120727 shutdown_connections 2024-05-04T03:20:53.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.671+0000 7f36ae7ebb80 1 -- 172.21.15.1:0/3398120727 wait complete. 2024-05-04T03:20:53.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.671+0000 7f36ae7ebb80 1 -- start start 2024-05-04T03:20:53.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.671+0000 7f36ae7ebb80 1 Processor -- start 2024-05-04T03:20:53.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.672+0000 7f36ae7ebb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56036591e5d0 0x560365930cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:53.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.672+0000 7f36ae7ebb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56036591ef20 0x560365931230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:53.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.672+0000 7f36ae7ebb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603659298e0 0x5603659a4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:53.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.672+0000 7f36ae7ebb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f36981af330 con 0x56036591e5d0 2024-05-04T03:20:53.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.672+0000 7f36ae7ebb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f36981b0740 con 0x56036591ef20 2024-05-04T03:20:53.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.672+0000 7f36ae7ebb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f36981af9a0 con 0x5603659298e0 2024-05-04T03:20:53.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.672+0000 7f36ae7ebb80 10 client.0 _ll_register_callbacks cb 0x5603657d9a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:20:53.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.672+0000 7f369db73700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56036591e5d0 0x560365930cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.672+0000 7f369db73700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56036591e5d0 0x560365930cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54132/0 (socket says 172.21.15.1:54132) 2024-05-04T03:20:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.672+0000 7f369db73700 1 -- 172.21.15.1:0/3158512516 learned_addr learned my addr 172.21.15.1:0/3158512516 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.672+0000 7f369d372700 1 --2- 172.21.15.1:0/3158512516 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56036591ef20 0x560365931230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.672+0000 7f369e374700 1 --2- 172.21.15.1:0/3158512516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603659298e0 0x5603659a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.672+0000 7f369db73700 1 -- 172.21.15.1:0/3158512516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603659298e0 msgr2=0x5603659a4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.672+0000 7f369db73700 1 --2- 172.21.15.1:0/3158512516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603659298e0 0x5603659a4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.672+0000 7f369db73700 1 -- 172.21.15.1:0/3158512516 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56036591ef20 msgr2=0x560365931230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.672+0000 7f369db73700 1 --2- 172.21.15.1:0/3158512516 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56036591ef20 0x560365931230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.672+0000 7f369db73700 1 -- 172.21.15.1:0/3158512516 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603659a4660 con 0x56036591e5d0 2024-05-04T03:20:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.672+0000 7f369db73700 1 --2- 172.21.15.1:0/3158512516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56036591e5d0 0x560365930cf0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3694009ce0 tx=0x7f3694005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8f3badf8f1a731 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.672+0000 7f36817fa700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:20:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.673+0000 7f36817fa700 1 -- 172.21.15.1:0/3158512516 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3694042060 con 0x56036591e5d0 2024-05-04T03:20:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.673+0000 7f36ae7ebb80 1 -- 172.21.15.1:0/3158512516 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3694003d30 con 0x56036591e5d0 2024-05-04T03:20:53.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.673+0000 7f36817fa700 1 -- 172.21.15.1:0/3158512516 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f369400f970 con 0x56036591e5d0 2024-05-04T03:20:53.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.673+0000 7f36817fa700 1 -- 172.21.15.1:0/3158512516 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3694039030 con 0x56036591e5d0 2024-05-04T03:20:53.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.673+0000 7f36ae7ebb80 10 client.15462 fetch_fsmap learned FSMap version 26 2024-05-04T03:20:53.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.673+0000 7f36ae7ebb80 1 -- 172.21.15.1:0/3158512516 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5603659a5830 con 0x56036591e5d0 2024-05-04T03:20:53.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.673+0000 7f36817fa700 1 -- 172.21.15.1:0/3158512516 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3694003d30 con 0x56036591e5d0 2024-05-04T03:20:53.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.673+0000 7f36817fa700 1 -- 172.21.15.1:0/3158512516 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f3694022070 con 0x56036591e5d0 2024-05-04T03:20:53.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.673+0000 7f36ae7ebb80 10 client.15462 fetch_fsmap finished waiting for FSMap version 26 2024-05-04T03:20:53.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.673+0000 7f36ae7ebb80 10 client.15462 resolve_mds: validated gid 24494 aka daemon mds.b 2024-05-04T03:20:53.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.673+0000 7f36ae7ebb80 20 client.15462 populate_metadata read hostname 'smithi001' 2024-05-04T03:20:53.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.673+0000 7f36ae7ebb80 1 --2- 172.21.15.1:0/3158512516 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5603659a5e00 0x5603659a8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:53.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.673+0000 7f36ae7ebb80 4 client.15462 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "661992bc", "release": true, "await": true}] 2024-05-04T03:20:53.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.673+0000 7f36ae7ebb80 1 -- 172.21.15.1:0/3158512516 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "quiesce db", "set_id": "661992bc", "release": true, "await": true}) -- 0x5603658493e0 con 0x5603659a5e00 2024-05-04T03:20:53.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.674+0000 7f369d372700 1 --2- 172.21.15.1:0/3158512516 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5603659a5e00 0x5603659a8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:53.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.674+0000 7f369d372700 1 --2- 172.21.15.1:0/3158512516 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5603659a5e00 0x5603659a8220 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3c56aff7cd471f2 server_cookie=f811f3f265096c31 in_seq=0 out_seq=0 2024-05-04T03:20:53.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.674+0000 7f36817fa700 10 client.15462 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:53.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.678+0000 7f36817fa700 1 -- 172.21.15.1:0/3158512516 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x5603658493e0 con 0x5603659a5e00 2024-05-04T03:20:53.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.678+0000 7f36817fa700 10 client.15462 handle_command_reply: tid=0 2024-05-04T03:20:53.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.678+0000 7f36ae7ebb80 1 client.15462 shutdown 2024-05-04T03:20:53.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.678+0000 7f36ae7ebb80 1 -- 172.21.15.1:0/3158512516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56036591e5d0 msgr2=0x560365930cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:53.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.678+0000 7f36ae7ebb80 1 --2- 172.21.15.1:0/3158512516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56036591e5d0 0x560365930cf0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3694009ce0 tx=0x7f3694005780 comp rx=0 tx=0).stop 2024-05-04T03:20:53.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.678+0000 7f36ae7ebb80 1 -- 172.21.15.1:0/3158512516 shutdown_connections 2024-05-04T03:20:53.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.678+0000 7f36ae7ebb80 1 --2- 172.21.15.1:0/3158512516 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5603659a5e00 0x5603659a8220 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.678+0000 7f36ae7ebb80 1 --2- 172.21.15.1:0/3158512516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603659298e0 0x5603659a4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.678+0000 7f36ae7ebb80 1 --2- 172.21.15.1:0/3158512516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56036591e5d0 0x560365930cf0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.678+0000 7f36817fa700 0 client.15462 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:53.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.678+0000 7f36ae7ebb80 1 --2- 172.21.15.1:0/3158512516 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56036591ef20 0x560365931230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.678+0000 7f36ae7ebb80 1 -- 172.21.15.1:0/3158512516 >> 172.21.15.1:0/3158512516 conn(0x5603657a1f40 msgr2=0x560365926ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:53.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.678+0000 7f36ae7ebb80 1 -- 172.21.15.1:0/3158512516 shutdown_connections 2024-05-04T03:20:53.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.678+0000 7f36ae7ebb80 1 -- 172.21.15.1:0/3158512516 wait complete. 2024-05-04T03:20:53.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.678+0000 7f36ae7ebb80 20 client.15462 trim_cache size 0 max 16384 2024-05-04T03:20:53.682 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-04T03:20:53.683 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 26, 2024-05-04T03:20:53.683 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24494, 2024-05-04T03:20:53.683 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 8, 2024-05-04T03:20:53.683 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-05-04T03:20:53.683 INFO:teuthology.orchestra.run.smithi001.stdout: "661992bc": { 2024-05-04T03:20:53.683 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 8, 2024-05-04T03:20:53.683 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-05-04T03:20:53.683 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-05-04T03:20:53.683 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-05-04T03:20:53.683 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-05-04T03:20:53.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:20:53.683 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 60.0, 2024-05-04T03:20:53.683 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 89.2, 2024-05-04T03:20:53.683 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-05-04T03:20:53.683 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-05-04T03:20:53.684 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-05-04T03:20:53.684 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-05-04T03:20:53.684 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-05-04T03:20:53.684 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-05-04T03:20:53.684 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:20:53.684 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:20:53.684 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:20:53.684 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:20:53.684 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:20:53.684 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-04T03:20:53.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.679+0000 7f369fdd7700 1 -- 172.21.15.1:0/649781952 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f36840b2880 msgr2=0x7f36840b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:53.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.679+0000 7f369fdd7700 1 --2- 172.21.15.1:0/649781952 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f36840b2880 0x7f36840b4d90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3694006010 tx=0x7f3694005960 comp rx=0 tx=0).stop 2024-05-04T03:20:53.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.679+0000 7f369fdd7700 1 -- 172.21.15.1:0/649781952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36981098e0 msgr2=0x7f36981a1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:53.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.679+0000 7f369fdd7700 1 --2- 172.21.15.1:0/649781952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36981098e0 0x7f36981a1890 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3690007ee0 tx=0x7f369000d3b0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.679+0000 7f369fdd7700 1 -- 172.21.15.1:0/649781952 shutdown_connections 2024-05-04T03:20:53.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.679+0000 7f369fdd7700 1 --2- 172.21.15.1:0/649781952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3698102b70 0x7f369819d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.679+0000 7f369fdd7700 1 --2- 172.21.15.1:0/649781952 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f36840b2880 0x7f36840b4d90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.679+0000 7f369fdd7700 1 --2- 172.21.15.1:0/649781952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f36981098e0 0x7f36981a1890 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.679+0000 7f369fdd7700 1 --2- 172.21.15.1:0/649781952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3698108f90 0x7f369819d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:53.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.679+0000 7f369fdd7700 1 -- 172.21.15.1:0/649781952 >> 172.21.15.1:0/649781952 conn(0x7f36980fe510 msgr2=0x7f3698110530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:53.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.679+0000 7f369fdd7700 1 -- 172.21.15.1:0/649781952 shutdown_connections 2024-05-04T03:20:53.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:20:53.679+0000 7f369fdd7700 1 -- 172.21.15.1:0/649781952 wait complete. 2024-05-04T03:20:53.686 INFO:tasks.workunit.client.0.smithi001.stderr:+ set -e 2024-05-04T03:20:53.686 INFO:tasks.workunit.client.0.smithi001.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-05-04T03:20:53.687 INFO:tasks.workunit.client.0.smithi001.stderr:Cloning into 'xfstests-dev'... 2024-05-04T03:20:54.454 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '661992bc', seconds elapsed: 42.1 2024-05-04T03:20:54.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:54 smithi001 ceph-mon[29731]: pgmap v443: 97 pgs: 97 active+clean; 682 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-04T03:20:54.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:54 smithi077 ceph-mon[33531]: pgmap v443: 97 pgs: 97 active+clean; 682 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-04T03:20:54.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:54 smithi072 ceph-mon[33332]: pgmap v443: 97 pgs: 97 active+clean; 682 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-05-04T03:20:56.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:56 smithi001 ceph-mon[29731]: pgmap v444: 97 pgs: 97 active+clean; 682 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-04T03:20:56.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:56 smithi001 ceph-mon[29731]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:20:56.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:56 smithi077 ceph-mon[33531]: pgmap v444: 97 pgs: 97 active+clean; 682 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-04T03:20:56.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:56 smithi077 ceph-mon[33531]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:20:56.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:56 smithi072 ceph-mon[33332]: pgmap v444: 97 pgs: 97 active+clean; 682 KiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-04T03:20:56.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:56 smithi072 ceph-mon[33332]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:20:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.544+0000 7fc073c1e640 1 -- 172.21.15.1:0/527559367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc06c104e90 msgr2=0x7fc06c1052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.544+0000 7fc073c1e640 1 --2- 172.21.15.1:0/527559367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc06c104e90 0x7fc06c1052a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc05c009860 tx=0x7fc05c02a340 comp rx=0 tx=0).stop 2024-05-04T03:20:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.544+0000 7fc073c1e640 1 -- 172.21.15.1:0/527559367 shutdown_connections 2024-05-04T03:20:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.544+0000 7fc073c1e640 1 --2- 172.21.15.1:0/527559367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc06c104e90 0x7fc06c1052a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.544+0000 7fc073c1e640 1 --2- 172.21.15.1:0/527559367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc06c109b90 0x7fc06c10bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.544+0000 7fc073c1e640 1 --2- 172.21.15.1:0/527559367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc06c103c50 0x7fc06c104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.544+0000 7fc073c1e640 1 -- 172.21.15.1:0/527559367 >> 172.21.15.1:0/527559367 conn(0x7fc06c0fee60 msgr2=0x7fc06c1012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.545+0000 7fc073c1e640 1 -- 172.21.15.1:0/527559367 shutdown_connections 2024-05-04T03:20:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.545+0000 7fc073c1e640 1 -- 172.21.15.1:0/527559367 wait complete. 2024-05-04T03:20:57.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.545+0000 7fc073c1e640 1 Processor -- start 2024-05-04T03:20:57.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.545+0000 7fc073c1e640 1 -- start start 2024-05-04T03:20:57.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.545+0000 7fc073c1e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc06c103c50 0x7fc06c19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:57.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.545+0000 7fc073c1e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc06c104e90 0x7fc06c19a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:57.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.545+0000 7fc073c1e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc06c109b90 0x7fc06c1a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:57.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.545+0000 7fc073c1e640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c10f780 con 0x7fc06c103c50 2024-05-04T03:20:57.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.545+0000 7fc073c1e640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c10f600 con 0x7fc06c109b90 2024-05-04T03:20:57.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.545+0000 7fc073c1e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c10f900 con 0x7fc06c104e90 2024-05-04T03:20:57.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.546+0000 7fc0719ba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc06c103c50 0x7fc06c19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.546+0000 7fc0719ba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc06c103c50 0x7fc06c19a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54160/0 (socket says 172.21.15.1:54160) 2024-05-04T03:20:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.546+0000 7fc0719ba640 1 -- 172.21.15.1:0/4177737987 learned_addr learned my addr 172.21.15.1:0/4177737987 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.546+0000 7fc0721bb640 1 --2- 172.21.15.1:0/4177737987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc06c109b90 0x7fc06c1a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.546+0000 7fc0711b9640 1 --2- 172.21.15.1:0/4177737987 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc06c104e90 0x7fc06c19a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.546+0000 7fc0719ba640 1 -- 172.21.15.1:0/4177737987 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc06c104e90 msgr2=0x7fc06c19a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.546+0000 7fc0719ba640 1 --2- 172.21.15.1:0/4177737987 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc06c104e90 0x7fc06c19a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.546+0000 7fc0719ba640 1 -- 172.21.15.1:0/4177737987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc06c109b90 msgr2=0x7fc06c1a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.546+0000 7fc0719ba640 1 --2- 172.21.15.1:0/4177737987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc06c109b90 0x7fc06c1a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.546+0000 7fc0719ba640 1 -- 172.21.15.1:0/4177737987 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06c1a1e70 con 0x7fc06c103c50 2024-05-04T03:20:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.546+0000 7fc0719ba640 1 --2- 172.21.15.1:0/4177737987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc06c103c50 0x7fc06c19a330 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc06800efc0 tx=0x7fc06800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.546+0000 7fc062ffd640 1 -- 172.21.15.1:0/4177737987 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068003970 con 0x7fc06c103c50 2024-05-04T03:20:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.546+0000 7fc073c1e640 1 -- 172.21.15.1:0/4177737987 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc06c1a2160 con 0x7fc06c103c50 2024-05-04T03:20:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.546+0000 7fc062ffd640 1 -- 172.21.15.1:0/4177737987 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc068004020 con 0x7fc06c103c50 2024-05-04T03:20:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.546+0000 7fc062ffd640 1 -- 172.21.15.1:0/4177737987 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06800b900 con 0x7fc06c103c50 2024-05-04T03:20:57.552 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.547+0000 7fc073c1e640 1 -- 172.21.15.1:0/4177737987 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc06c1a26b0 con 0x7fc06c103c50 2024-05-04T03:20:57.559 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.549+0000 7fc07aba8740 1 Processor -- start 2024-05-04T03:20:57.559 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.550+0000 7fc062ffd640 1 -- 172.21.15.1:0/4177737987 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7fc06801d020 con 0x7fc06c103c50 2024-05-04T03:20:57.559 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.550+0000 7fc07aba8740 1 -- start start 2024-05-04T03:20:57.559 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.550+0000 7fc07aba8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1b03c1b0 0x558d1b3e1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:57.559 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.550+0000 7fc07aba8740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d1b0132f0 0x558d1b3e23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.550+0000 7fc07aba8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d1b01b4d0 0x558d1b3e92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.550+0000 7fc07aba8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558d1b3425d0 con 0x558d1b01b4d0 2024-05-04T03:20:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.550+0000 7fc07aba8740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x558d1b2fb890 con 0x558d1b0132f0 2024-05-04T03:20:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.550+0000 7fc07aba8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558d1b33e590 con 0x558d1b03c1b0 2024-05-04T03:20:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.550+0000 7fc0711b9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d1b01b4d0 0x558d1b3e92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.550+0000 7fc0719ba640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d1b0132f0 0x558d1b3e23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.550+0000 7fc0711b9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d1b01b4d0 0x558d1b3e92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54162/0 (socket says 172.21.15.1:54162) 2024-05-04T03:20:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.550+0000 7fc0711b9640 1 -- 172.21.15.1:0/2487790224 learned_addr learned my addr 172.21.15.1:0/2487790224 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.550+0000 7fc0721bb640 1 --2- 172.21.15.1:0/2487790224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1b03c1b0 0x558d1b3e1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.550+0000 7fc0711b9640 1 -- 172.21.15.1:0/2487790224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1b03c1b0 msgr2=0x558d1b3e1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.550+0000 7fc0711b9640 1 --2- 172.21.15.1:0/2487790224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1b03c1b0 0x558d1b3e1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.550+0000 7fc0711b9640 1 -- 172.21.15.1:0/2487790224 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d1b0132f0 msgr2=0x558d1b3e23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:57.560 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.551+0000 7fc0711b9640 1 --2- 172.21.15.1:0/2487790224 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d1b0132f0 0x558d1b3e23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.551+0000 7fc0711b9640 1 -- 172.21.15.1:0/2487790224 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d1b054210 con 0x558d1b01b4d0 2024-05-04T03:20:57.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.551+0000 7fc0721bb640 1 --2- 172.21.15.1:0/2487790224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1b03c1b0 0x558d1b3e1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:20:57.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.551+0000 7fc0719ba640 1 --2- 172.21.15.1:0/2487790224 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d1b0132f0 0x558d1b3e23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:20:57.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.551+0000 7fc062ffd640 1 --2- 172.21.15.1:0/4177737987 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc0580b28a0 0x7fc0580b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:57.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.551+0000 7fc0711b9640 1 --2- 172.21.15.1:0/2487790224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d1b01b4d0 0x558d1b3e92f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc05c000c00 tx=0x7fc05c005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1c75b6ee7803a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:57.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.551+0000 7fc0721bb640 1 --2- 172.21.15.1:0/4177737987 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc0580b28a0 0x7fc0580b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:57.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.551+0000 7fc062ffd640 1 -- 172.21.15.1:0/4177737987 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fc0680e2120 con 0x7fc06c103c50 2024-05-04T03:20:57.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.551+0000 7fc057fff640 1 -- 172.21.15.1:0/2487790224 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c037040 con 0x558d1b01b4d0 2024-05-04T03:20:57.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.551+0000 7fc057fff640 1 -- 172.21.15.1:0/2487790224 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc05c030e50 con 0x558d1b01b4d0 2024-05-04T03:20:57.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.551+0000 7fc057fff640 1 -- 172.21.15.1:0/2487790224 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c0054a0 con 0x558d1b01b4d0 2024-05-04T03:20:57.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.552+0000 7fc07aba8740 1 -- 172.21.15.1:0/2487790224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d1b01b4d0 msgr2=0x558d1b3e92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:57.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.552+0000 7fc0721bb640 1 --2- 172.21.15.1:0/4177737987 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc0580b28a0 0x7fc0580b4db0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x558d1b3e3040 tx=0x7fc064005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:57.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.552+0000 7fc07aba8740 1 --2- 172.21.15.1:0/2487790224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d1b01b4d0 0x558d1b3e92f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc05c000c00 tx=0x7fc05c005da0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.561 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.552+0000 7fc07aba8740 1 -- 172.21.15.1:0/2487790224 shutdown_connections 2024-05-04T03:20:57.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.552+0000 7fc07aba8740 1 --2- 172.21.15.1:0/2487790224 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d1b0132f0 0x558d1b3e23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.552+0000 7fc07aba8740 1 --2- 172.21.15.1:0/2487790224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d1b01b4d0 0x558d1b3e92f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.552+0000 7fc07aba8740 1 --2- 172.21.15.1:0/2487790224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1b03c1b0 0x558d1b3e1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.552+0000 7fc07aba8740 1 -- 172.21.15.1:0/2487790224 >> 172.21.15.1:0/2487790224 conn(0x558d1b030300 msgr2=0x558d1b3d93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:57.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.552+0000 7fc07aba8740 1 -- 172.21.15.1:0/2487790224 shutdown_connections 2024-05-04T03:20:57.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.552+0000 7fc07aba8740 1 -- 172.21.15.1:0/2487790224 wait complete. 2024-05-04T03:20:57.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.553+0000 7fc07aba8740 1 -- start start 2024-05-04T03:20:57.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.553+0000 7fc07aba8740 1 Processor -- start 2024-05-04T03:20:57.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.553+0000 7fc07aba8740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d1b0132f0 0x558d1b3eba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:57.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.553+0000 7fc07aba8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1b01b4d0 0x558d1b3ebf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:57.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.553+0000 7fc07aba8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d1b03c1b0 0x558d1b3f2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:57.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.553+0000 7fc07aba8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558d1b3e9ce0 con 0x558d1b03c1b0 2024-05-04T03:20:57.562 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.553+0000 7fc07aba8740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x558d1b2ff370 con 0x558d1b0132f0 2024-05-04T03:20:57.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.553+0000 7fc07aba8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558d1b3e9e60 con 0x558d1b01b4d0 2024-05-04T03:20:57.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.553+0000 7fc07aba8740 10 client.0 _ll_register_callbacks cb 0x558d1b053a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:20:57.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.553+0000 7fc0721bb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d1b03c1b0 0x558d1b3f2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:57.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.553+0000 7fc0721bb640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d1b03c1b0 0x558d1b3f2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54178/0 (socket says 172.21.15.1:54178) 2024-05-04T03:20:57.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.553+0000 7fc0721bb640 1 -- 172.21.15.1:0/912882525 learned_addr learned my addr 172.21.15.1:0/912882525 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:57.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.554+0000 7fc0711b9640 1 --2- 172.21.15.1:0/912882525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1b01b4d0 0x558d1b3ebf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:57.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.554+0000 7fc0719ba640 1 --2- 172.21.15.1:0/912882525 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d1b0132f0 0x558d1b3eba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:57.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.554+0000 7fc0721bb640 1 -- 172.21.15.1:0/912882525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1b01b4d0 msgr2=0x558d1b3ebf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:57.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.554+0000 7fc0721bb640 1 --2- 172.21.15.1:0/912882525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1b01b4d0 0x558d1b3ebf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.563 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.554+0000 7fc0721bb640 1 -- 172.21.15.1:0/912882525 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d1b0132f0 msgr2=0x558d1b3eba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.554+0000 7fc0721bb640 1 --2- 172.21.15.1:0/912882525 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d1b0132f0 0x558d1b3eba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.554+0000 7fc0721bb640 1 -- 172.21.15.1:0/912882525 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d1b120620 con 0x558d1b03c1b0 2024-05-04T03:20:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.554+0000 7fc0721bb640 1 --2- 172.21.15.1:0/912882525 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d1b03c1b0 0x558d1b3f2ef0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc064005880 tx=0x7fc064004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ec9bc76c1d27c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.554+0000 7fc0557fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:20:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.554+0000 7fc0557fa640 1 -- 172.21.15.1:0/912882525 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06401ee40 con 0x558d1b03c1b0 2024-05-04T03:20:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.554+0000 7fc0557fa640 1 -- 172.21.15.1:0/912882525 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fc064018030 con 0x558d1b03c1b0 2024-05-04T03:20:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.554+0000 7fc07aba8740 1 -- 172.21.15.1:0/912882525 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc05c030e50 con 0x558d1b03c1b0 2024-05-04T03:20:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.554+0000 7fc0557fa640 1 -- 172.21.15.1:0/912882525 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc064004020 con 0x558d1b03c1b0 2024-05-04T03:20:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.554+0000 7fc07aba8740 10 client.15474 fetch_fsmap learned FSMap version 27 2024-05-04T03:20:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.554+0000 7fc07aba8740 1 -- 172.21.15.1:0/912882525 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558d1b3f5ef0 con 0x558d1b03c1b0 2024-05-04T03:20:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.555+0000 7fc0557fa640 1 -- 172.21.15.1:0/912882525 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc05c030e50 con 0x558d1b03c1b0 2024-05-04T03:20:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.555+0000 7fc0557fa640 1 -- 172.21.15.1:0/912882525 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 8342+0+0 (secure 0 0 0) 0x7fc064029420 con 0x558d1b03c1b0 2024-05-04T03:20:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.555+0000 7fc07aba8740 10 client.15474 fetch_fsmap finished waiting for FSMap version 27 2024-05-04T03:20:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.555+0000 7fc07aba8740 10 client.15474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:20:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.555+0000 7fc07aba8740 20 client.15474 populate_metadata read hostname 'smithi001' 2024-05-04T03:20:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.555+0000 7fc07aba8740 1 --2- 172.21.15.1:0/912882525 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x558d1affad80 0x558d1b3f96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.555+0000 7fc07aba8740 4 client.15474 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:20:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.555+0000 7fc07aba8740 1 -- 172.21.15.1:0/912882525 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d1b3501e0 con 0x558d1affad80 2024-05-04T03:20:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.555+0000 7fc0719ba640 1 --2- 172.21.15.1:0/912882525 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x558d1affad80 0x558d1b3f96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.556+0000 7fc0557fa640 10 client.15474 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:57.565 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.556+0000 7fc0719ba640 1 --2- 172.21.15.1:0/912882525 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x558d1affad80 0x558d1b3f96a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5ea6d1f8b9e77fd server_cookie=f8dbdcd37191e10a in_seq=0 out_seq=0 2024-05-04T03:20:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.556+0000 7fc0557fa640 1 -- 172.21.15.1:0/912882525 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558d1b3501e0 con 0x558d1affad80 2024-05-04T03:20:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.556+0000 7fc0557fa640 10 client.15474 handle_command_reply: tid=0 2024-05-04T03:20:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.557+0000 7fc07aba8740 1 client.15474 shutdown 2024-05-04T03:20:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.557+0000 7fc07aba8740 1 -- 172.21.15.1:0/912882525 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d1b03c1b0 msgr2=0x558d1b3f2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.557+0000 7fc07aba8740 1 --2- 172.21.15.1:0/912882525 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d1b03c1b0 0x558d1b3f2ef0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc064005880 tx=0x7fc064004730 comp rx=0 tx=0).stop 2024-05-04T03:20:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.557+0000 7fc07aba8740 1 -- 172.21.15.1:0/912882525 shutdown_connections 2024-05-04T03:20:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.557+0000 7fc07aba8740 1 --2- 172.21.15.1:0/912882525 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x558d1affad80 0x558d1b3f96a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.557+0000 7fc07aba8740 1 --2- 172.21.15.1:0/912882525 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d1b03c1b0 0x558d1b3f2ef0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.557+0000 7fc07aba8740 1 --2- 172.21.15.1:0/912882525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1b01b4d0 0x558d1b3ebf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.557+0000 7fc0557fa640 0 client.15474 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.557+0000 7fc07aba8740 1 --2- 172.21.15.1:0/912882525 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d1b0132f0 0x558d1b3eba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.566 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.557+0000 7fc07aba8740 1 -- 172.21.15.1:0/912882525 >> 172.21.15.1:0/912882525 conn(0x558d1b030300 msgr2=0x558d1b3d83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.557+0000 7fc07aba8740 1 -- 172.21.15.1:0/912882525 shutdown_connections 2024-05-04T03:20:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.557+0000 7fc07aba8740 1 -- 172.21.15.1:0/912882525 wait complete. 2024-05-04T03:20:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.557+0000 7fc07aba8740 20 client.15474 trim_cache size 0 max 16384 2024-05-04T03:20:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.573+0000 7fc07aba8740 1 Processor -- start 2024-05-04T03:20:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.573+0000 7fc07aba8740 1 -- start start 2024-05-04T03:20:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.573+0000 7fc07aba8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d1b0132f0 0x558d1b3d9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.573+0000 7fc07aba8740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d1b01b4d0 0x558d1b3f2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.573+0000 7fc07aba8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1b03c1b0 0x558d1b3f3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:57.577 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.573+0000 7fc07aba8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1ab850 con 0x558d1b0132f0 2024-05-04T03:20:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.573+0000 7fc07aba8740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1a9120 con 0x558d1b01b4d0 2024-05-04T03:20:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.573+0000 7fc07aba8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1a6fb0 con 0x558d1b03c1b0 2024-05-04T03:20:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.573+0000 7fc0719ba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d1b0132f0 0x558d1b3d9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.573+0000 7fc0719ba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d1b0132f0 0x558d1b3d9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54192/0 (socket says 172.21.15.1:54192) 2024-05-04T03:20:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.573+0000 7fc0719ba640 1 -- 172.21.15.1:0/1970595065 learned_addr learned my addr 172.21.15.1:0/1970595065 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.573+0000 7fc0711b9640 1 --2- 172.21.15.1:0/1970595065 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d1b01b4d0 0x558d1b3f2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.573+0000 7fc0719ba640 1 -- 172.21.15.1:0/1970595065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1b03c1b0 msgr2=0x558d1b3f3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.573+0000 7fc0719ba640 1 --2- 172.21.15.1:0/1970595065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1b03c1b0 0x558d1b3f3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.578 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.573+0000 7fc0719ba640 1 -- 172.21.15.1:0/1970595065 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d1b01b4d0 msgr2=0x558d1b3f2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.573+0000 7fc0719ba640 1 --2- 172.21.15.1:0/1970595065 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d1b01b4d0 0x558d1b3f2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.573+0000 7fc0719ba640 1 -- 172.21.15.1:0/1970595065 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d1b3d9940 con 0x558d1b0132f0 2024-05-04T03:20:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.573+0000 7fc0711b9640 1 --2- 172.21.15.1:0/1970595065 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d1b01b4d0 0x558d1b3f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:20:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.573+0000 7fc0719ba640 1 --2- 172.21.15.1:0/1970595065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d1b0132f0 0x558d1b3d9400 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc0680a6a70 tx=0x7fc0680e4650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0cd3a12600da784 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.574+0000 7fc0577fe640 1 -- 172.21.15.1:0/1970595065 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0680098d0 con 0x558d1b0132f0 2024-05-04T03:20:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.574+0000 7fc0577fe640 1 -- 172.21.15.1:0/1970595065 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0680a9310 con 0x558d1b0132f0 2024-05-04T03:20:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.574+0000 7fc0577fe640 1 -- 172.21.15.1:0/1970595065 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0680e5e10 con 0x558d1b0132f0 2024-05-04T03:20:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.574+0000 7fc07aba8740 1 -- 172.21.15.1:0/1970595065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d1b0132f0 msgr2=0x558d1b3d9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.574+0000 7fc07aba8740 1 --2- 172.21.15.1:0/1970595065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d1b0132f0 0x558d1b3d9400 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc0680a6a70 tx=0x7fc0680e4650 comp rx=0 tx=0).stop 2024-05-04T03:20:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.574+0000 7fc07aba8740 1 -- 172.21.15.1:0/1970595065 shutdown_connections 2024-05-04T03:20:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.574+0000 7fc07aba8740 1 --2- 172.21.15.1:0/1970595065 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d1b01b4d0 0x558d1b3f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.574+0000 7fc07aba8740 1 --2- 172.21.15.1:0/1970595065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1b03c1b0 0x558d1b3f3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.579 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.575+0000 7fc07aba8740 1 --2- 172.21.15.1:0/1970595065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d1b0132f0 0x558d1b3d9400 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.575+0000 7fc07aba8740 1 -- 172.21.15.1:0/1970595065 >> 172.21.15.1:0/1970595065 conn(0x558d1b030300 msgr2=0x558d1b3d8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:57.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.575+0000 7fc07aba8740 1 -- 172.21.15.1:0/1970595065 shutdown_connections 2024-05-04T03:20:57.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.575+0000 7fc07aba8740 1 -- 172.21.15.1:0/1970595065 wait complete. 2024-05-04T03:20:57.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.575+0000 7fc07aba8740 1 -- start start 2024-05-04T03:20:57.580 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.575+0000 7fc07aba8740 1 Processor -- start 2024-05-04T03:20:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.575+0000 7fc07aba8740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d1b0132f0 0x558d1b3f0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.575+0000 7fc07aba8740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d1b01b4d0 0x558d1b3f0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.575+0000 7fc07aba8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1b03c1b0 0x558d1b45fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:57.583 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.575+0000 7fc07aba8740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1a7600 con 0x558d1b01b4d0 2024-05-04T03:20:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.575+0000 7fc07aba8740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1a8a50 con 0x558d1b0132f0 2024-05-04T03:20:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.575+0000 7fc07aba8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1a7cb0 con 0x558d1b03c1b0 2024-05-04T03:20:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.575+0000 7fc07aba8740 10 client.0 _ll_register_callbacks cb 0x7fc064012f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:20:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.576+0000 7fc0711b9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d1b01b4d0 0x558d1b3f0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.576+0000 7fc0711b9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d1b01b4d0 0x558d1b3f0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54196/0 (socket says 172.21.15.1:54196) 2024-05-04T03:20:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.576+0000 7fc0711b9640 1 -- 172.21.15.1:0/3112163860 learned_addr learned my addr 172.21.15.1:0/3112163860 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:20:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.576+0000 7fc0719ba640 1 --2- 172.21.15.1:0/3112163860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d1b0132f0 0x558d1b3f0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.576+0000 7fc0721bb640 1 --2- 172.21.15.1:0/3112163860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1b03c1b0 0x558d1b45fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.576+0000 7fc0711b9640 1 -- 172.21.15.1:0/3112163860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1b03c1b0 msgr2=0x558d1b45fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.576+0000 7fc0711b9640 1 --2- 172.21.15.1:0/3112163860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1b03c1b0 0x558d1b45fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.576+0000 7fc0711b9640 1 -- 172.21.15.1:0/3112163860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d1b0132f0 msgr2=0x558d1b3f0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:20:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.576+0000 7fc0711b9640 1 --2- 172.21.15.1:0/3112163860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d1b0132f0 0x558d1b3f0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.584 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.576+0000 7fc0711b9640 1 -- 172.21.15.1:0/3112163860 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d1b45ffb0 con 0x558d1b01b4d0 2024-05-04T03:20:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.576+0000 7fc0719ba640 1 --2- 172.21.15.1:0/3112163860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d1b0132f0 0x558d1b3f0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:20:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.576+0000 7fc0711b9640 1 --2- 172.21.15.1:0/3112163860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d1b01b4d0 0x558d1b3f0c40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc05c02a860 tx=0x7fc05c005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c86832bac0177777 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:20:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.576+0000 7fc0557fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:20:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.576+0000 7fc0557fa640 1 -- 172.21.15.1:0/3112163860 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c037040 con 0x558d1b01b4d0 2024-05-04T03:20:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.576+0000 7fc0557fa640 1 -- 172.21.15.1:0/3112163860 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fc05c0307d0 con 0x558d1b01b4d0 2024-05-04T03:20:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.576+0000 7fc07aba8740 1 -- 172.21.15.1:0/3112163860 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0680a9310 con 0x558d1b01b4d0 2024-05-04T03:20:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.576+0000 7fc0557fa640 1 -- 172.21.15.1:0/3112163860 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc05c004030 con 0x558d1b01b4d0 2024-05-04T03:20:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.576+0000 7fc07aba8740 10 client.15480 fetch_fsmap learned FSMap version 27 2024-05-04T03:20:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.576+0000 7fc07aba8740 1 -- 172.21.15.1:0/3112163860 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558d1b461230 con 0x558d1b01b4d0 2024-05-04T03:20:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.577+0000 7fc0557fa640 1 -- 172.21.15.1:0/3112163860 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0680a9310 con 0x558d1b01b4d0 2024-05-04T03:20:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.577+0000 7fc0557fa640 1 -- 172.21.15.1:0/3112163860 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) ==== 8342+0+0 (secure 0 0 0) 0x7fc05c059070 con 0x558d1b01b4d0 2024-05-04T03:20:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.577+0000 7fc07aba8740 10 client.15480 fetch_fsmap finished waiting for FSMap version 27 2024-05-04T03:20:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.577+0000 7fc07aba8740 10 client.15480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:20:57.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.577+0000 7fc07aba8740 20 client.15480 populate_metadata read hostname 'smithi001' 2024-05-04T03:20:57.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.577+0000 7fc07aba8740 1 --2- 172.21.15.1:0/3112163860 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x558d1afd01c0 0x558d1b4637d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:20:57.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.577+0000 7fc07aba8740 4 client.15480 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:20:57.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.577+0000 7fc07aba8740 1 -- 172.21.15.1:0/3112163860 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558d1b34a050 con 0x558d1afd01c0 2024-05-04T03:20:57.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.577+0000 7fc0719ba640 1 --2- 172.21.15.1:0/3112163860 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x558d1afd01c0 0x558d1b4637d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:20:57.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.578+0000 7fc0557fa640 10 client.15480 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:57.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.578+0000 7fc0719ba640 1 --2- 172.21.15.1:0/3112163860 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x558d1afd01c0 0x558d1b4637d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=887bed315e353c5a server_cookie=a5e8fff330560c9d in_seq=0 out_seq=0 2024-05-04T03:20:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.588+0000 7fc0557fa640 1 -- 172.21.15.1:0/3112163860 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558d1b34a050 con 0x558d1afd01c0 2024-05-04T03:20:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.588+0000 7fc0557fa640 10 client.15480 handle_command_reply: tid=0 2024-05-04T03:20:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.588+0000 7fc07aba8740 1 client.15480 shutdown 2024-05-04T03:20:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.588+0000 7fc07aba8740 1 -- 172.21.15.1:0/3112163860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d1b01b4d0 msgr2=0x558d1b3f0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.588+0000 7fc07aba8740 1 --2- 172.21.15.1:0/3112163860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d1b01b4d0 0x558d1b3f0c40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc05c02a860 tx=0x7fc05c005f10 comp rx=0 tx=0).stop 2024-05-04T03:20:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.589+0000 7fc07aba8740 1 -- 172.21.15.1:0/3112163860 shutdown_connections 2024-05-04T03:20:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.589+0000 7fc07aba8740 1 --2- 172.21.15.1:0/3112163860 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x558d1afd01c0 0x558d1b4637d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.589+0000 7fc07aba8740 1 --2- 172.21.15.1:0/3112163860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d1b03c1b0 0x558d1b45fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.589+0000 7fc07aba8740 1 --2- 172.21.15.1:0/3112163860 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d1b01b4d0 0x558d1b3f0c40 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.589+0000 7fc0557fa640 0 client.15480 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:20:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.589+0000 7fc07aba8740 1 --2- 172.21.15.1:0/3112163860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d1b0132f0 0x558d1b3f0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.589+0000 7fc07aba8740 1 -- 172.21.15.1:0/3112163860 >> 172.21.15.1:0/3112163860 conn(0x558d1b030300 msgr2=0x558d1b3dc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.589+0000 7fc07aba8740 1 -- 172.21.15.1:0/3112163860 shutdown_connections 2024-05-04T03:20:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.589+0000 7fc07aba8740 1 -- 172.21.15.1:0/3112163860 wait complete. 2024-05-04T03:20:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.589+0000 7fc07aba8740 20 client.15480 trim_cache size 0 max 16384 2024-05-04T03:20:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:20:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:20:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:20:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:20:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.589+0000 7fc073c1e640 1 -- 172.21.15.1:0/4177737987 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc0580b28a0 msgr2=0x7fc0580b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.589+0000 7fc073c1e640 1 --2- 172.21.15.1:0/4177737987 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc0580b28a0 0x7fc0580b4db0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x558d1b3e3040 tx=0x7fc064005f90 comp rx=0 tx=0).stop 2024-05-04T03:20:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.589+0000 7fc073c1e640 1 -- 172.21.15.1:0/4177737987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc06c103c50 msgr2=0x7fc06c19a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:20:57.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.589+0000 7fc073c1e640 1 --2- 172.21.15.1:0/4177737987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc06c103c50 0x7fc06c19a330 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc06800efc0 tx=0x7fc06800c800 comp rx=0 tx=0).stop 2024-05-04T03:20:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.589+0000 7fc073c1e640 1 -- 172.21.15.1:0/4177737987 shutdown_connections 2024-05-04T03:20:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.589+0000 7fc073c1e640 1 --2- 172.21.15.1:0/4177737987 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc0580b28a0 0x7fc0580b4db0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.589+0000 7fc073c1e640 1 --2- 172.21.15.1:0/4177737987 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc06c109b90 0x7fc06c1a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.589+0000 7fc073c1e640 1 --2- 172.21.15.1:0/4177737987 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc06c104e90 0x7fc06c19a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.589+0000 7fc073c1e640 1 --2- 172.21.15.1:0/4177737987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc06c103c50 0x7fc06c19a330 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:20:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.589+0000 7fc073c1e640 1 -- 172.21.15.1:0/4177737987 >> 172.21.15.1:0/4177737987 conn(0x7fc06c0fee60 msgr2=0x7fc06c105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:20:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.590+0000 7fc073c1e640 1 -- 172.21.15.1:0/4177737987 shutdown_connections 2024-05-04T03:20:57.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:20:57.590+0000 7fc073c1e640 1 -- 172.21.15.1:0/4177737987 wait complete. 2024-05-04T03:20:58.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:20:58 smithi001 ceph-mon[29731]: pgmap v445: 97 pgs: 97 active+clean; 682 KiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 3.7 KiB/s wr, 5 op/s 2024-05-04T03:20:58.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:20:58 smithi077 ceph-mon[33531]: pgmap v445: 97 pgs: 97 active+clean; 682 KiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 3.7 KiB/s wr, 5 op/s 2024-05-04T03:20:58.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:20:58 smithi072 ceph-mon[33332]: pgmap v445: 97 pgs: 97 active+clean; 682 KiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 3.7 KiB/s wr, 5 op/s 2024-05-04T03:21:00.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:00 smithi001 ceph-mon[29731]: pgmap v446: 97 pgs: 97 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.1 MiB/s wr, 15 op/s 2024-05-04T03:21:00.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:00 smithi077 ceph-mon[33531]: pgmap v446: 97 pgs: 97 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.1 MiB/s wr, 15 op/s 2024-05-04T03:21:00.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:00 smithi072 ceph-mon[33332]: pgmap v446: 97 pgs: 97 active+clean; 13 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.1 MiB/s wr, 15 op/s 2024-05-04T03:21:02.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:02 smithi001 ceph-mon[29731]: pgmap v447: 97 pgs: 97 active+clean; 15 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.2 MiB/s wr, 14 op/s 2024-05-04T03:21:02.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:02 smithi077 ceph-mon[33531]: pgmap v447: 97 pgs: 97 active+clean; 15 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.2 MiB/s wr, 14 op/s 2024-05-04T03:21:02.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:02 smithi072 ceph-mon[33332]: pgmap v447: 97 pgs: 97 active+clean; 15 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.2 MiB/s wr, 14 op/s 2024-05-04T03:21:03.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:03 smithi001 ceph-mon[29731]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:21:03.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:03 smithi077 ceph-mon[33531]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:21:03.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:03 smithi072 ceph-mon[33332]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:21:04.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:04 smithi001 ceph-mon[29731]: pgmap v448: 97 pgs: 97 active+clean; 15 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.2 MiB/s wr, 17 op/s 2024-05-04T03:21:04.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:04 smithi077 ceph-mon[33531]: pgmap v448: 97 pgs: 97 active+clean; 15 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.2 MiB/s wr, 17 op/s 2024-05-04T03:21:04.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:04 smithi072 ceph-mon[33332]: pgmap v448: 97 pgs: 97 active+clean; 15 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.2 MiB/s wr, 17 op/s 2024-05-04T03:21:06.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:06 smithi001 ceph-mon[29731]: pgmap v449: 97 pgs: 97 active+clean; 19 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 1.6 MiB/s wr, 23 op/s 2024-05-04T03:21:06.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:06 smithi077 ceph-mon[33531]: pgmap v449: 97 pgs: 97 active+clean; 19 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 1.6 MiB/s wr, 23 op/s 2024-05-04T03:21:06.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:06 smithi072 ceph-mon[33332]: pgmap v449: 97 pgs: 97 active+clean; 19 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 1.6 MiB/s wr, 23 op/s 2024-05-04T03:21:08.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:08 smithi001 ceph-mon[29731]: pgmap v450: 97 pgs: 97 active+clean; 19 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 1.6 MiB/s wr, 23 op/s 2024-05-04T03:21:08.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:08 smithi077 ceph-mon[33531]: pgmap v450: 97 pgs: 97 active+clean; 19 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 1.6 MiB/s wr, 23 op/s 2024-05-04T03:21:08.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:08 smithi072 ceph-mon[33332]: pgmap v450: 97 pgs: 97 active+clean; 19 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 1.6 MiB/s wr, 23 op/s 2024-05-04T03:21:09.463 INFO:tasks.workunit.client.0.smithi001.stderr:Updating files: 11% (491/4287) Updating files: 12% (515/4287) Updating files: 13% (558/4287) Updating files: 14% (601/4287) Updating files: 15% (644/4287) Updating files: 16% (686/4287) Updating files: 17% (729/4287) Updating files: 18% (772/4287) Updating files: 19% (815/4287) Updating files: 19% (825/4287) Updating files: 20% (858/4287) Updating files: 21% (901/4287) Updating files: 22% (944/4287) Updating files: 23% (987/4287) Updating files: 24% (1029/4287) Updating files: 25% (1072/4287) Updating files: 26% (1115/4287) Updating files: 27% (1158/4287) Updating files: 28% (1201/4287) Updating files: 29% (1244/4287) Updating files: 30% (1287/4287) Updating files: 31% (1329/4287) Updating files: 32% (1372/4287) Updating files: 33% (1415/4287) Updating files: 34% (1458/4287) Updating files: 34% (1459/4287) Updating files: 35% (1501/4287) Updating files: 36% (1544/4287) Updating files: 37% (1587/4287) Updating files: 38% (1630/4287) Updating files: 39% (1672/4287) Updating files: 40% (1715/4287) Updating files: 41% (1758/4287) Updating files: 42% (1801/4287) Updating files: 43% (1844/4287) Updating files: 44% (1887/4287) Updating files: 45% (1930/4287) Updating files: 46% (1973/4287) Updating files: 47% (2015/4287) Updating files: 47% (2054/4287) Updating files: 48% (2058/4287) Updating files: 49% (2101/4287) Updating files: 50% (2144/4287) Updating files: 51% (2187/4287) Updating files: 52% (2230/4287) Updating files: 53% (2273/4287) Updating files: 54% (2315/4287) Updating files: 55% (2358/4287) Updating files: 56% (2401/4287) Updating files: 57% (2444/4287) Updating files: 58% (2487/4287) Updating files: 59% (2530/4287) Updating files: 60% (2573/4287) Updating files: 60% (2586/4287) Updating files: 61% (2616/4287) Updating files: 62% (2658/4287) Updating files: 63% (2701/4287) Updating files: 64% (2744/4287) Updating files: 64% (2784/4287) Updating files: 65% (2787/4287) Updating files: 66% (2830/4287) Updating files: 67% (2873/4287) Updating files: 68% (2916/4287) Updating files: 69% (2959/4287) Updating files: 70% (3001/4287) Updating files: 70% (3022/4287) Updating files: 71% (3044/4287) Updating files: 72% (3087/4287) Updating files: 73% (3130/4287) Updating files: 74% (3173/4287) Updating files: 75% (3216/4287) Updating files: 76% (3259/4287) Updating files: 77% (3301/4287) Updating files: 78% (3344/4287) Updating files: 79% (3387/4287) Updating files: 80% (3430/4287) Updating files: 81% (3473/4287) Updating files: 81% (3479/4287) Updating files: 82% (3516/4287) Updating files: 83% (3559/4287) Updating files: 84% (3602/4287) Updating files: 85% (3644/4287) Updating files: 86% (3687/4287) Updating files: 87% (3730/4287) Updating files: 88% (3773/4287) Updating files: 89% (3816/4287) Updating files: 90% (3859/4287) Updating files: 91% (3902/4287) Updating files: 92% (3945/4287) Updating files: 93% (3987/4287) Updating files: 94% (4030/4287) Updating files: 94% (4033/4287) Updating files: 95% (4073/4287) Updating files: 96% (4116/4287) Updating files: 97% (4159/4287) Updating files: 98% (4202/4287) Updating files: 99% (4245/4287) Updating files: 100% (4287/4287) Updating files: 100% (4287/4287), done. 2024-05-04T03:21:09.470 INFO:tasks.workunit.client.0.smithi001.stderr:+ cd xfstests-dev 2024-05-04T03:21:09.470 INFO:tasks.workunit.client.0.smithi001.stderr:+ make -j4 2024-05-04T03:21:09.506 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize -cfi 2024-05-04T03:21:09.890 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: putting auxiliary files in '.'. 2024-05-04T03:21:09.890 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: copying file './config.guess' 2024-05-04T03:21:09.894 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: copying file './config.sub' 2024-05-04T03:21:09.898 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: copying file './install-sh' 2024-05-04T03:21:09.903 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: copying file './ltmain.sh' 2024-05-04T03:21:09.926 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-05-04T03:21:09.927 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-05-04T03:21:09.932 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-05-04T03:21:09.938 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-05-04T03:21:09.944 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-05-04T03:21:09.949 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-05-04T03:21:09.956 INFO:tasks.workunit.client.0.smithi001.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-05-04T03:21:09.957 INFO:tasks.workunit.client.0.smithi001.stdout:cp include/install-sh . 2024-05-04T03:21:09.966 INFO:tasks.workunit.client.0.smithi001.stdout:aclocal -I m4 2024-05-04T03:21:10.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:10 smithi001 ceph-mon[29731]: pgmap v451: 97 pgs: 97 active+clean; 19 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.7 MiB/s wr, 38 op/s 2024-05-04T03:21:10.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:10 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:21:10.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:10 smithi077 ceph-mon[33531]: pgmap v451: 97 pgs: 97 active+clean; 19 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.7 MiB/s wr, 38 op/s 2024-05-04T03:21:10.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:10 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:21:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.689+0000 7fbdfb0a1640 1 -- 172.21.15.1:0/3657609347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbdf4108a90 msgr2=0x7fbdf410aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.689+0000 7fbdfb0a1640 1 --2- 172.21.15.1:0/3657609347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbdf4108a90 0x7fbdf410aeb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fbdec009ca0 tx=0x7fbdec02a8c0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.689+0000 7fbdfb0a1640 1 -- 172.21.15.1:0/3657609347 shutdown_connections 2024-05-04T03:21:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.689+0000 7fbdfb0a1640 1 --2- 172.21.15.1:0/3657609347 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbdf4103d90 0x7fbdf41041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.689+0000 7fbdfb0a1640 1 --2- 172.21.15.1:0/3657609347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbdf4108a90 0x7fbdf410aeb0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.689+0000 7fbdfb0a1640 1 --2- 172.21.15.1:0/3657609347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbdf4102b50 0x7fbdf4103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.689+0000 7fbdfb0a1640 1 -- 172.21.15.1:0/3657609347 >> 172.21.15.1:0/3657609347 conn(0x7fbdf40fdd40 msgr2=0x7fbdf41001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.689+0000 7fbdfb0a1640 1 -- 172.21.15.1:0/3657609347 shutdown_connections 2024-05-04T03:21:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.689+0000 7fbdfb0a1640 1 -- 172.21.15.1:0/3657609347 wait complete. 2024-05-04T03:21:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.690+0000 7fbdfb0a1640 1 Processor -- start 2024-05-04T03:21:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.690+0000 7fbdfb0a1640 1 -- start start 2024-05-04T03:21:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.690+0000 7fbdfb0a1640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbdf4102b50 0x7fbdf419a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.690+0000 7fbdfb0a1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbdf4103d90 0x7fbdf419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.690+0000 7fbdfb0a1640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbdf4108a90 0x7fbdf41a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.690+0000 7fbdfb0a1640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf410e680 con 0x7fbdf4108a90 2024-05-04T03:21:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.690+0000 7fbdfb0a1640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf410e500 con 0x7fbdf4102b50 2024-05-04T03:21:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.690+0000 7fbdfb0a1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf410e800 con 0x7fbdf4103d90 2024-05-04T03:21:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.690+0000 7fbdf963e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbdf4108a90 0x7fbdf41a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.690+0000 7fbdf963e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbdf4108a90 0x7fbdf41a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41946/0 (socket says 172.21.15.1:41946) 2024-05-04T03:21:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.690+0000 7fbdf963e640 1 -- 172.21.15.1:0/125966060 learned_addr learned my addr 172.21.15.1:0/125966060 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:21:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.690+0000 7fbdf8e3d640 1 --2- 172.21.15.1:0/125966060 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbdf4102b50 0x7fbdf419a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.690+0000 7fbdf3fff640 1 --2- 172.21.15.1:0/125966060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbdf4103d90 0x7fbdf419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.690+0000 7fbdf963e640 1 -- 172.21.15.1:0/125966060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbdf4103d90 msgr2=0x7fbdf419a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.690+0000 7fbdf963e640 1 --2- 172.21.15.1:0/125966060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbdf4103d90 0x7fbdf419a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.690+0000 7fbdf963e640 1 -- 172.21.15.1:0/125966060 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbdf4102b50 msgr2=0x7fbdf419a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.690+0000 7fbdf963e640 1 --2- 172.21.15.1:0/125966060 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbdf4102b50 0x7fbdf419a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.690+0000 7fbdf963e640 1 -- 172.21.15.1:0/125966060 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf41a1db0 con 0x7fbdf4108a90 2024-05-04T03:21:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.690+0000 7fbdf8e3d640 1 --2- 172.21.15.1:0/125966060 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbdf4102b50 0x7fbdf419a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:21:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.691+0000 7fbdf963e640 1 --2- 172.21.15.1:0/125966060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbdf4108a90 0x7fbdf41a1740 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbdec0097b0 tx=0x7fbdec031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.691+0000 7fbdf1ffb640 1 -- 172.21.15.1:0/125966060 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdec003bf0 con 0x7fbdf4108a90 2024-05-04T03:21:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.691+0000 7fbdf1ffb640 1 -- 172.21.15.1:0/125966060 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdec003d90 con 0x7fbdf4108a90 2024-05-04T03:21:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.691+0000 7fbdfb0a1640 1 -- 172.21.15.1:0/125966060 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdf41a2040 con 0x7fbdf4108a90 2024-05-04T03:21:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.691+0000 7fbdf1ffb640 1 -- 172.21.15.1:0/125966060 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdec02fc30 con 0x7fbdf4108a90 2024-05-04T03:21:10.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.691+0000 7fbdfb0a1640 1 -- 172.21.15.1:0/125966060 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdf41a2590 con 0x7fbdf4108a90 2024-05-04T03:21:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.694+0000 7fbe02032740 1 Processor -- start 2024-05-04T03:21:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.694+0000 7fbdf1ffb640 1 -- 172.21.15.1:0/125966060 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7fbdec0305c0 con 0x7fbdf4108a90 2024-05-04T03:21:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.694+0000 7fbe02032740 1 -- start start 2024-05-04T03:21:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.694+0000 7fbe02032740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e810ac81b0 0x55e810e69a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.694+0000 7fbe02032740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e810a9f2f0 0x55e810e69fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.694+0000 7fbe02032740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e810aa74d0 0x55e810e70ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.694+0000 7fbe02032740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e810dce5d0 con 0x55e810a9f2f0 2024-05-04T03:21:10.707 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.694+0000 7fbe02032740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55e810d87890 con 0x55e810ac81b0 2024-05-04T03:21:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.694+0000 7fbe02032740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e810dca590 con 0x55e810aa74d0 2024-05-04T03:21:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.694+0000 7fbdf8e3d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e810a9f2f0 0x55e810e69fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.695+0000 7fbdf8e3d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e810a9f2f0 0x55e810e69fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41956/0 (socket says 172.21.15.1:41956) 2024-05-04T03:21:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.695+0000 7fbdf8e3d640 1 -- 172.21.15.1:0/3417275321 learned_addr learned my addr 172.21.15.1:0/3417275321 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:21:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.695+0000 7fbdf3fff640 1 --2- 172.21.15.1:0/3417275321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e810aa74d0 0x55e810e70ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.695+0000 7fbdf1ffb640 1 --2- 172.21.15.1:0/125966060 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fbde40b28a0 0x7fbde40b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.695+0000 7fbdf3fff640 1 --2- 172.21.15.1:0/3417275321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e810ac81b0 0x55e810e69a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.695+0000 7fbdf3fff640 1 -- 172.21.15.1:0/3417275321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e810ac81b0 msgr2=0x55e810e69a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.695+0000 7fbdf3fff640 1 --2- 172.21.15.1:0/3417275321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e810ac81b0 0x55e810e69a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.695+0000 7fbdf3fff640 1 -- 172.21.15.1:0/3417275321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e810a9f2f0 msgr2=0x55e810e69fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.695+0000 7fbdf3fff640 1 --2- 172.21.15.1:0/3417275321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e810a9f2f0 0x55e810e69fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.695+0000 7fbdf3fff640 1 -- 172.21.15.1:0/3417275321 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e810ae0210 con 0x55e810aa74d0 2024-05-04T03:21:10.708 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.695+0000 7fbdf8e3d640 1 --2- 172.21.15.1:0/3417275321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e810a9f2f0 0x55e810e69fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:21:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.695+0000 7fbdf1ffb640 1 -- 172.21.15.1:0/125966060 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fbdec103840 con 0x7fbdf4108a90 2024-05-04T03:21:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.695+0000 7fbdf963e640 1 --2- 172.21.15.1:0/125966060 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fbde40b28a0 0x7fbde40b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.695+0000 7fbdf3fff640 1 --2- 172.21.15.1:0/3417275321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e810aa74d0 0x55e810e70ee0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbdf419b310 tx=0x7fbde0009940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84b8047a077e0cf2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.695+0000 7fbddeffd640 1 -- 172.21.15.1:0/3417275321 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0004050 con 0x55e810aa74d0 2024-05-04T03:21:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.695+0000 7fbddeffd640 1 -- 172.21.15.1:0/3417275321 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbde00054d0 con 0x55e810aa74d0 2024-05-04T03:21:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.695+0000 7fbddeffd640 1 -- 172.21.15.1:0/3417275321 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0004cd0 con 0x55e810aa74d0 2024-05-04T03:21:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.696+0000 7fbe02032740 1 -- 172.21.15.1:0/3417275321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e810aa74d0 msgr2=0x55e810e70ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.696+0000 7fbe02032740 1 --2- 172.21.15.1:0/3417275321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e810aa74d0 0x55e810e70ee0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbdf419b310 tx=0x7fbde0009940 comp rx=0 tx=0).stop 2024-05-04T03:21:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.696+0000 7fbdf963e640 1 --2- 172.21.15.1:0/125966060 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fbde40b28a0 0x7fbde40b4db0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbdec0094e0 tx=0x7fbdec005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.696+0000 7fbe02032740 1 -- 172.21.15.1:0/3417275321 shutdown_connections 2024-05-04T03:21:10.709 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.696+0000 7fbe02032740 1 --2- 172.21.15.1:0/3417275321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e810aa74d0 0x55e810e70ee0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.696+0000 7fbe02032740 1 --2- 172.21.15.1:0/3417275321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e810a9f2f0 0x55e810e69fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.696+0000 7fbe02032740 1 --2- 172.21.15.1:0/3417275321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e810ac81b0 0x55e810e69a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.696+0000 7fbe02032740 1 -- 172.21.15.1:0/3417275321 >> 172.21.15.1:0/3417275321 conn(0x55e810abc300 msgr2=0x55e810e653a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.696+0000 7fbe02032740 1 -- 172.21.15.1:0/3417275321 shutdown_connections 2024-05-04T03:21:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.696+0000 7fbe02032740 1 -- 172.21.15.1:0/3417275321 wait complete. 2024-05-04T03:21:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.697+0000 7fbe02032740 1 -- start start 2024-05-04T03:21:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.697+0000 7fbe02032740 1 Processor -- start 2024-05-04T03:21:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.697+0000 7fbe02032740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e810a9f2f0 0x55e810e73670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.697+0000 7fbe02032740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e810aa74d0 0x55e810e73bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.697+0000 7fbe02032740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e810ac81b0 0x55e810e7ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.697+0000 7fbe02032740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55e810e71930 con 0x55e810a9f2f0 2024-05-04T03:21:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.697+0000 7fbe02032740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55e810d8b370 con 0x55e810ac81b0 2024-05-04T03:21:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.697+0000 7fbe02032740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e810e71ab0 con 0x55e810aa74d0 2024-05-04T03:21:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.697+0000 7fbe02032740 10 client.0 _ll_register_callbacks cb 0x55e810adfa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:21:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.697+0000 7fbdf3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e810a9f2f0 0x55e810e73670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.697+0000 7fbdf3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e810a9f2f0 0x55e810e73670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41964/0 (socket says 172.21.15.1:41964) 2024-05-04T03:21:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.697+0000 7fbdf3fff640 1 -- 172.21.15.1:0/1474300429 learned_addr learned my addr 172.21.15.1:0/1474300429 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:21:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.697+0000 7fbdf8e3d640 1 --2- 172.21.15.1:0/1474300429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e810aa74d0 0x55e810e73bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.698+0000 7fbdf3fff640 1 --2- 172.21.15.1:0/1474300429 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e810ac81b0 0x55e810e7ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.698+0000 7fbdf3fff640 1 -- 172.21.15.1:0/1474300429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e810aa74d0 msgr2=0x55e810e73bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.698+0000 7fbdf3fff640 1 --2- 172.21.15.1:0/1474300429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e810aa74d0 0x55e810e73bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.698+0000 7fbdf3fff640 1 -- 172.21.15.1:0/1474300429 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e810ac81b0 msgr2=0x55e810e7ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.698+0000 7fbdf3fff640 1 --2- 172.21.15.1:0/1474300429 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e810ac81b0 0x55e810e7ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.698+0000 7fbdf3fff640 1 -- 172.21.15.1:0/1474300429 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e810bac620 con 0x55e810a9f2f0 2024-05-04T03:21:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.698+0000 7fbdf3fff640 1 --2- 172.21.15.1:0/1474300429 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e810a9f2f0 0x55e810e73670 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbde00047e0 tx=0x7fbde0005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d596bd79e08b9ed8 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.698+0000 7fbdd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:21:10.711 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.698+0000 7fbdd3fff640 1 -- 172.21.15.1:0/1474300429 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde001e660 con 0x55e810a9f2f0 2024-05-04T03:21:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.698+0000 7fbdd3fff640 1 -- 172.21.15.1:0/1474300429 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fbde001e800 con 0x55e810a9f2f0 2024-05-04T03:21:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.698+0000 7fbe02032740 1 -- 172.21.15.1:0/1474300429 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbde00054d0 con 0x55e810a9f2f0 2024-05-04T03:21:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.698+0000 7fbdd3fff640 1 -- 172.21.15.1:0/1474300429 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbde0032050 con 0x55e810a9f2f0 2024-05-04T03:21:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.698+0000 7fbe02032740 10 client.15492 fetch_fsmap learned FSMap version 28 2024-05-04T03:21:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.698+0000 7fbe02032740 1 -- 172.21.15.1:0/1474300429 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e810e7da40 con 0x55e810a9f2f0 2024-05-04T03:21:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.698+0000 7fbdd3fff640 1 -- 172.21.15.1:0/1474300429 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbde00054d0 con 0x55e810a9f2f0 2024-05-04T03:21:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.699+0000 7fbdd3fff640 1 -- 172.21.15.1:0/1474300429 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7fbde000f9d0 con 0x55e810a9f2f0 2024-05-04T03:21:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.699+0000 7fbe02032740 10 client.15492 fetch_fsmap finished waiting for FSMap version 28 2024-05-04T03:21:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.699+0000 7fbe02032740 10 client.15492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:21:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.699+0000 7fbe02032740 20 client.15492 populate_metadata read hostname 'smithi001' 2024-05-04T03:21:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.699+0000 7fbe02032740 1 --2- 172.21.15.1:0/1474300429 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55e810a86d80 0x55e810e80e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.699+0000 7fbe02032740 4 client.15492 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:21:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.699+0000 7fbe02032740 1 -- 172.21.15.1:0/1474300429 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e810ddc1e0 con 0x55e810a86d80 2024-05-04T03:21:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.699+0000 7fbdf8e3d640 1 --2- 172.21.15.1:0/1474300429 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55e810a86d80 0x55e810e80e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.699+0000 7fbdd3fff640 10 client.15492 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:21:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.699+0000 7fbdf8e3d640 1 --2- 172.21.15.1:0/1474300429 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55e810a86d80 0x55e810e80e60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=623fbbb2837a461f server_cookie=81f6c57358613d18 in_seq=0 out_seq=0 2024-05-04T03:21:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.700+0000 7fbdd3fff640 1 -- 172.21.15.1:0/1474300429 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e810ddc1e0 con 0x55e810a86d80 2024-05-04T03:21:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.700+0000 7fbdd3fff640 10 client.15492 handle_command_reply: tid=0 2024-05-04T03:21:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.700+0000 7fbe02032740 1 client.15492 shutdown 2024-05-04T03:21:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.700+0000 7fbe02032740 1 -- 172.21.15.1:0/1474300429 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e810a9f2f0 msgr2=0x55e810e73670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.700+0000 7fbe02032740 1 --2- 172.21.15.1:0/1474300429 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e810a9f2f0 0x55e810e73670 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbde00047e0 tx=0x7fbde0005d30 comp rx=0 tx=0).stop 2024-05-04T03:21:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.700+0000 7fbe02032740 1 -- 172.21.15.1:0/1474300429 shutdown_connections 2024-05-04T03:21:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.700+0000 7fbe02032740 1 --2- 172.21.15.1:0/1474300429 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55e810a86d80 0x55e810e80e60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.700+0000 7fbe02032740 1 --2- 172.21.15.1:0/1474300429 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e810ac81b0 0x55e810e7ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.700+0000 7fbe02032740 1 --2- 172.21.15.1:0/1474300429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e810aa74d0 0x55e810e73bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.700+0000 7fbe02032740 1 --2- 172.21.15.1:0/1474300429 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e810a9f2f0 0x55e810e73670 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.700+0000 7fbe02032740 1 -- 172.21.15.1:0/1474300429 >> 172.21.15.1:0/1474300429 conn(0x55e810abc300 msgr2=0x55e810e68300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.700+0000 7fbdd3fff640 0 client.15492 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:21:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.700+0000 7fbe02032740 1 -- 172.21.15.1:0/1474300429 shutdown_connections 2024-05-04T03:21:10.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.700+0000 7fbe02032740 1 -- 172.21.15.1:0/1474300429 wait complete. 2024-05-04T03:21:10.714 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.701+0000 7fbe02032740 20 client.15492 trim_cache size 0 max 16384 2024-05-04T03:21:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.715+0000 7fbe02032740 1 Processor -- start 2024-05-04T03:21:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.716+0000 7fbe02032740 1 -- start start 2024-05-04T03:21:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.716+0000 7fbe02032740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e810a9f2f0 0x55e810e64dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.716+0000 7fbe02032740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e810aa74d0 0x55e810e65310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.716+0000 7fbe02032740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e810ac81b0 0x55e810e7aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.716+0000 7fbe02032740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf41ada20 con 0x55e810ac81b0 2024-05-04T03:21:10.719 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.716+0000 7fbe02032740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf41ab2c0 con 0x55e810a9f2f0 2024-05-04T03:21:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.716+0000 7fbe02032740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf41a9130 con 0x55e810aa74d0 2024-05-04T03:21:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.716+0000 7fbdf3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e810ac81b0 0x55e810e7aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.716+0000 7fbdf3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e810ac81b0 0x55e810e7aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41978/0 (socket says 172.21.15.1:41978) 2024-05-04T03:21:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.716+0000 7fbdf3fff640 1 -- 172.21.15.1:0/2337205676 learned_addr learned my addr 172.21.15.1:0/2337205676 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:21:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.716+0000 7fbdf3fff640 1 --2- 172.21.15.1:0/2337205676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e810a9f2f0 0x55e810e64dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.716+0000 7fbdf8e3d640 1 --2- 172.21.15.1:0/2337205676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e810aa74d0 0x55e810e65310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.716+0000 7fbdf3fff640 1 -- 172.21.15.1:0/2337205676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e810aa74d0 msgr2=0x55e810e65310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.716+0000 7fbdf3fff640 1 --2- 172.21.15.1:0/2337205676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e810aa74d0 0x55e810e65310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.716+0000 7fbdf3fff640 1 -- 172.21.15.1:0/2337205676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e810a9f2f0 msgr2=0x55e810e64dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.716+0000 7fbdf3fff640 1 --2- 172.21.15.1:0/2337205676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e810a9f2f0 0x55e810e64dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.716+0000 7fbdf3fff640 1 -- 172.21.15.1:0/2337205676 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e810e6d4f0 con 0x55e810ac81b0 2024-05-04T03:21:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.717+0000 7fbdf3fff640 1 --2- 172.21.15.1:0/2337205676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e810ac81b0 0x55e810e7aaa0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x55e810e69350 tx=0x7fbde0005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd75478fc7a1d40 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.717+0000 7fbdde7fc640 1 -- 172.21.15.1:0/2337205676 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0004810 con 0x55e810ac81b0 2024-05-04T03:21:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.717+0000 7fbdde7fc640 1 -- 172.21.15.1:0/2337205676 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbde0005420 con 0x55e810ac81b0 2024-05-04T03:21:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.717+0000 7fbdde7fc640 1 -- 172.21.15.1:0/2337205676 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde000f990 con 0x55e810ac81b0 2024-05-04T03:21:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.718+0000 7fbe02032740 1 -- 172.21.15.1:0/2337205676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e810ac81b0 msgr2=0x55e810e7aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.718+0000 7fbe02032740 1 --2- 172.21.15.1:0/2337205676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e810ac81b0 0x55e810e7aaa0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x55e810e69350 tx=0x7fbde0005970 comp rx=0 tx=0).stop 2024-05-04T03:21:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.718+0000 7fbe02032740 1 -- 172.21.15.1:0/2337205676 shutdown_connections 2024-05-04T03:21:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.718+0000 7fbe02032740 1 --2- 172.21.15.1:0/2337205676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e810ac81b0 0x55e810e7aaa0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.718+0000 7fbe02032740 1 --2- 172.21.15.1:0/2337205676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e810aa74d0 0x55e810e65310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.718+0000 7fbe02032740 1 --2- 172.21.15.1:0/2337205676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e810a9f2f0 0x55e810e64dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.718+0000 7fbe02032740 1 -- 172.21.15.1:0/2337205676 >> 172.21.15.1:0/2337205676 conn(0x55e810abc300 msgr2=0x55e810e71eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.718+0000 7fbe02032740 1 -- 172.21.15.1:0/2337205676 shutdown_connections 2024-05-04T03:21:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.718+0000 7fbe02032740 1 -- 172.21.15.1:0/2337205676 wait complete. 2024-05-04T03:21:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.718+0000 7fbe02032740 1 -- start start 2024-05-04T03:21:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.718+0000 7fbe02032740 1 Processor -- start 2024-05-04T03:21:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.719+0000 7fbe02032740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e810a9f2f0 0x55e810e783a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.719+0000 7fbe02032740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e810aa74d0 0x55e810e788e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.719+0000 7fbe02032740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e810ac81b0 0x55e810ee70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.719+0000 7fbe02032740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf41a9780 con 0x55e810ac81b0 2024-05-04T03:21:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.719+0000 7fbe02032740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf41aabf0 con 0x55e810aa74d0 2024-05-04T03:21:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.719+0000 7fbe02032740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf41a9e50 con 0x55e810a9f2f0 2024-05-04T03:21:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.719+0000 7fbe02032740 10 client.0 _ll_register_callbacks cb 0x7fbde800b4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:21:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.719+0000 7fbdf3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e810ac81b0 0x55e810ee70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.719+0000 7fbdf8e3d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e810aa74d0 0x55e810e788e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.719+0000 7fbdf3fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e810ac81b0 0x55e810ee70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41986/0 (socket says 172.21.15.1:41986) 2024-05-04T03:21:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.719+0000 7fbdf3fff640 1 -- 172.21.15.1:0/583183401 learned_addr learned my addr 172.21.15.1:0/583183401 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:21:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.719+0000 7fbdf3fff640 1 --2- 172.21.15.1:0/583183401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e810a9f2f0 0x55e810e783a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.719+0000 7fbdf8e3d640 1 -- 172.21.15.1:0/583183401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e810a9f2f0 msgr2=0x55e810e783a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.719+0000 7fbdf8e3d640 1 --2- 172.21.15.1:0/583183401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e810a9f2f0 0x55e810e783a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.719+0000 7fbdf8e3d640 1 -- 172.21.15.1:0/583183401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e810ac81b0 msgr2=0x55e810ee70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.719+0000 7fbdf8e3d640 1 --2- 172.21.15.1:0/583183401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e810ac81b0 0x55e810ee70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.719+0000 7fbdf8e3d640 1 -- 172.21.15.1:0/583183401 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e810ee76a0 con 0x55e810aa74d0 2024-05-04T03:21:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.719+0000 7fbdf8e3d640 1 --2- 172.21.15.1:0/583183401 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e810aa74d0 0x55e810e788e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55e810e746e0 tx=0x7fbde80039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1986a967499d4a41 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.719+0000 7fbdd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-04T03:21:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.720+0000 7fbdd3fff640 1 -- 172.21.15.1:0/583183401 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde801f830 con 0x55e810aa74d0 2024-05-04T03:21:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.720+0000 7fbe02032740 1 -- 172.21.15.1:0/583183401 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbde0005420 con 0x55e810aa74d0 2024-05-04T03:21:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.720+0000 7fbdd3fff640 1 -- 172.21.15.1:0/583183401 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fbde801f9d0 con 0x55e810aa74d0 2024-05-04T03:21:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.720+0000 7fbdd3fff640 1 -- 172.21.15.1:0/583183401 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbde802f050 con 0x55e810aa74d0 2024-05-04T03:21:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.720+0000 7fbe02032740 10 client.25238 fetch_fsmap learned FSMap version 28 2024-05-04T03:21:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.720+0000 7fbe02032740 1 -- 172.21.15.1:0/583183401 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e810ee8a50 con 0x55e810aa74d0 2024-05-04T03:21:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.720+0000 7fbdd3fff640 1 -- 172.21.15.1:0/583183401 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbde0005420 con 0x55e810aa74d0 2024-05-04T03:21:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.720+0000 7fbdd3fff640 1 -- 172.21.15.1:0/583183401 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7fbde80273c0 con 0x55e810aa74d0 2024-05-04T03:21:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.720+0000 7fbe02032740 10 client.25238 fetch_fsmap finished waiting for FSMap version 28 2024-05-04T03:21:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.720+0000 7fbe02032740 10 client.25238 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:21:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.720+0000 7fbe02032740 20 client.25238 populate_metadata read hostname 'smithi001' 2024-05-04T03:21:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.720+0000 7fbe02032740 1 --2- 172.21.15.1:0/583183401 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55e810a5c1c0 0x55e810eeb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.720+0000 7fbe02032740 4 client.25238 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:21:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.720+0000 7fbe02032740 1 -- 172.21.15.1:0/583183401 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e810dd6050 con 0x55e810a5c1c0 2024-05-04T03:21:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.720+0000 7fbdf3fff640 1 --2- 172.21.15.1:0/583183401 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55e810a5c1c0 0x55e810eeb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.721+0000 7fbdd3fff640 10 client.25238 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:21:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.721+0000 7fbdf3fff640 1 --2- 172.21.15.1:0/583183401 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55e810a5c1c0 0x55e810eeb420 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74383e16e009771b server_cookie=30c211d8e73be90a in_seq=0 out_seq=0 2024-05-04T03:21:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.741+0000 7fbdd3fff640 1 -- 172.21.15.1:0/583183401 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e810dd6050 con 0x55e810a5c1c0 2024-05-04T03:21:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.741+0000 7fbdd3fff640 10 client.25238 handle_command_reply: tid=0 2024-05-04T03:21:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.741+0000 7fbe02032740 1 client.25238 shutdown 2024-05-04T03:21:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.741+0000 7fbe02032740 1 -- 172.21.15.1:0/583183401 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e810aa74d0 msgr2=0x55e810e788e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.741+0000 7fbe02032740 1 --2- 172.21.15.1:0/583183401 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e810aa74d0 0x55e810e788e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55e810e746e0 tx=0x7fbde80039c0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.741+0000 7fbe02032740 1 -- 172.21.15.1:0/583183401 shutdown_connections 2024-05-04T03:21:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.741+0000 7fbe02032740 1 --2- 172.21.15.1:0/583183401 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55e810a5c1c0 0x55e810eeb420 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.741+0000 7fbe02032740 1 --2- 172.21.15.1:0/583183401 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e810aa74d0 0x55e810e788e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.741+0000 7fbdd3fff640 0 client.25238 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:21:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.741+0000 7fbe02032740 1 --2- 172.21.15.1:0/583183401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e810ac81b0 0x55e810ee70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.741+0000 7fbe02032740 1 --2- 172.21.15.1:0/583183401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e810a9f2f0 0x55e810e783a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.741+0000 7fbe02032740 1 -- 172.21.15.1:0/583183401 >> 172.21.15.1:0/583183401 conn(0x55e810abc300 msgr2=0x55e810e64d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.741+0000 7fbe02032740 1 -- 172.21.15.1:0/583183401 shutdown_connections 2024-05-04T03:21:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.741+0000 7fbe02032740 1 -- 172.21.15.1:0/583183401 wait complete. 2024-05-04T03:21:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.741+0000 7fbe02032740 20 client.25238 trim_cache size 0 max 16384 2024-05-04T03:21:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:21:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:21:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:21:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:21:10.746 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.742+0000 7fbdfb0a1640 1 -- 172.21.15.1:0/125966060 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fbde40b28a0 msgr2=0x7fbde40b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.742+0000 7fbdfb0a1640 1 --2- 172.21.15.1:0/125966060 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fbde40b28a0 0x7fbde40b4db0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbdec0094e0 tx=0x7fbdec005fd0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.742+0000 7fbdfb0a1640 1 -- 172.21.15.1:0/125966060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbdf4108a90 msgr2=0x7fbdf41a1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.742+0000 7fbdfb0a1640 1 --2- 172.21.15.1:0/125966060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbdf4108a90 0x7fbdf41a1740 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbdec0097b0 tx=0x7fbdec031840 comp rx=0 tx=0).stop 2024-05-04T03:21:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.742+0000 7fbdfb0a1640 1 -- 172.21.15.1:0/125966060 shutdown_connections 2024-05-04T03:21:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.742+0000 7fbdfb0a1640 1 --2- 172.21.15.1:0/125966060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbdf4108a90 0x7fbdf41a1740 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.742+0000 7fbdfb0a1640 1 --2- 172.21.15.1:0/125966060 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbdf4103d90 0x7fbdf419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.742+0000 7fbdfb0a1640 1 --2- 172.21.15.1:0/125966060 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fbde40b28a0 0x7fbde40b4db0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.742+0000 7fbdfb0a1640 1 --2- 172.21.15.1:0/125966060 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbdf4102b50 0x7fbdf419a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:10.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.742+0000 7fbdfb0a1640 1 -- 172.21.15.1:0/125966060 >> 172.21.15.1:0/125966060 conn(0x7fbdf40fdd40 msgr2=0x7fbdf4109ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:10.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.742+0000 7fbdfb0a1640 1 -- 172.21.15.1:0/125966060 shutdown_connections 2024-05-04T03:21:10.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:10.742+0000 7fbdfb0a1640 1 -- 172.21.15.1:0/125966060 wait complete. 2024-05-04T03:21:10.749 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:10 smithi072 ceph-mon[33332]: pgmap v451: 97 pgs: 97 active+clean; 19 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.7 MiB/s wr, 38 op/s 2024-05-04T03:21:10.749 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:10 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:21:11.025 INFO:tasks.workunit.client.0.smithi001.stdout:autoheader 2024-05-04T03:21:11.348 INFO:tasks.workunit.client.0.smithi001.stdout:autoconf 2024-05-04T03:21:11.536 INFO:tasks.workunit.client.0.smithi001.stdout:./configure \ 2024-05-04T03:21:11.536 INFO:tasks.workunit.client.0.smithi001.stdout: --libexecdir=/usr/lib \ 2024-05-04T03:21:11.536 INFO:tasks.workunit.client.0.smithi001.stdout: --exec_prefix=/var/lib 2024-05-04T03:21:11.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:11 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:21:11.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:11 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:21:11.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:11 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:21:11.670 INFO:tasks.workunit.client.0.smithi001.stdout:checking build system type... x86_64-pc-linux-gnu 2024-05-04T03:21:11.670 INFO:tasks.workunit.client.0.smithi001.stdout:checking host system type... x86_64-pc-linux-gnu 2024-05-04T03:21:11.671 INFO:tasks.workunit.client.0.smithi001.stdout:checking how to print strings... printf 2024-05-04T03:21:11.671 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gcc... gcc 2024-05-04T03:21:11.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:11 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:21:11.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:11 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:21:11.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:11 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:21:11.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:11 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:21:11.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:11 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:21:11.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:11 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:21:11.753 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether the C compiler works... yes 2024-05-04T03:21:11.753 INFO:tasks.workunit.client.0.smithi001.stdout:checking for C compiler default output file name... a.out 2024-05-04T03:21:11.791 INFO:tasks.workunit.client.0.smithi001.stdout:checking for suffix of executables... 2024-05-04T03:21:11.829 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether we are cross compiling... no 2024-05-04T03:21:11.866 INFO:tasks.workunit.client.0.smithi001.stdout:checking for suffix of object files... o 2024-05-04T03:21:11.890 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether we are using the GNU C compiler... yes 2024-05-04T03:21:11.914 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether gcc accepts -g... yes 2024-05-04T03:21:12.004 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gcc option to accept ISO C89... none needed 2024-05-04T03:21:12.007 INFO:tasks.workunit.client.0.smithi001.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-05-04T03:21:12.011 INFO:tasks.workunit.client.0.smithi001.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-05-04T03:21:12.013 INFO:tasks.workunit.client.0.smithi001.stdout:checking for egrep... /usr/bin/grep -E 2024-05-04T03:21:12.014 INFO:tasks.workunit.client.0.smithi001.stdout:checking for fgrep... /usr/bin/grep -F 2024-05-04T03:21:12.017 INFO:tasks.workunit.client.0.smithi001.stdout:checking for ld used by gcc... /usr/bin/ld 2024-05-04T03:21:12.018 INFO:tasks.workunit.client.0.smithi001.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-05-04T03:21:12.030 INFO:tasks.workunit.client.0.smithi001.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-05-04T03:21:12.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:12 smithi072 ceph-mon[33332]: pgmap v452: 97 pgs: 97 active+clean; 22 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 903 KiB/s wr, 38 op/s 2024-05-04T03:21:12.878 INFO:tasks.workunit.client.0.smithi001.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-05-04T03:21:12.879 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether ln -s works... yes 2024-05-04T03:21:12.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:12 smithi001 ceph-mon[29731]: pgmap v452: 97 pgs: 97 active+clean; 22 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 903 KiB/s wr, 38 op/s 2024-05-04T03:21:12.899 INFO:tasks.workunit.client.0.smithi001.stdout:checking the maximum length of command line arguments... 1572864 2024-05-04T03:21:12.899 INFO:tasks.workunit.client.0.smithi001.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-05-04T03:21:12.899 INFO:tasks.workunit.client.0.smithi001.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-05-04T03:21:12.899 INFO:tasks.workunit.client.0.smithi001.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-05-04T03:21:12.899 INFO:tasks.workunit.client.0.smithi001.stdout:checking for objdump... objdump 2024-05-04T03:21:12.901 INFO:tasks.workunit.client.0.smithi001.stdout:checking how to recognize dependent libraries... pass_all 2024-05-04T03:21:12.901 INFO:tasks.workunit.client.0.smithi001.stdout:checking for dlltool... no 2024-05-04T03:21:12.901 INFO:tasks.workunit.client.0.smithi001.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-05-04T03:21:12.901 INFO:tasks.workunit.client.0.smithi001.stdout:checking for ar... ar 2024-05-04T03:21:12.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:12 smithi077 ceph-mon[33531]: pgmap v452: 97 pgs: 97 active+clean; 22 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 903 KiB/s wr, 38 op/s 2024-05-04T03:21:12.997 INFO:tasks.workunit.client.0.smithi001.stdout:checking for archiver @FILE support... @ 2024-05-04T03:21:12.997 INFO:tasks.workunit.client.0.smithi001.stdout:checking for strip... strip 2024-05-04T03:21:12.997 INFO:tasks.workunit.client.0.smithi001.stdout:checking for ranlib... ranlib 2024-05-04T03:21:12.997 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gawk... gawk 2024-05-04T03:21:13.105 INFO:tasks.workunit.client.0.smithi001.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-05-04T03:21:13.106 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sysroot... no 2024-05-04T03:21:13.119 INFO:tasks.workunit.client.0.smithi001.stdout:checking for a working dd... /usr/bin/dd 2024-05-04T03:21:13.131 INFO:tasks.workunit.client.0.smithi001.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-05-04T03:21:13.166 INFO:tasks.workunit.client.0.smithi001.stdout:checking for mt... no 2024-05-04T03:21:13.178 INFO:tasks.workunit.client.0.smithi001.stdout:checking if : is a manifest tool... no 2024-05-04T03:21:13.278 INFO:tasks.workunit.client.0.smithi001.stdout:checking how to run the C preprocessor... gcc -E 2024-05-04T03:21:13.478 INFO:tasks.workunit.client.0.smithi001.stdout:checking for ANSI C header files... yes 2024-05-04T03:21:13.514 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/types.h... yes 2024-05-04T03:21:13.556 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/stat.h... yes 2024-05-04T03:21:13.597 INFO:tasks.workunit.client.0.smithi001.stdout:checking for stdlib.h... yes 2024-05-04T03:21:13.641 INFO:tasks.workunit.client.0.smithi001.stdout:checking for string.h... yes 2024-05-04T03:21:13.687 INFO:tasks.workunit.client.0.smithi001.stdout:checking for memory.h... yes 2024-05-04T03:21:13.731 INFO:tasks.workunit.client.0.smithi001.stdout:checking for strings.h... yes 2024-05-04T03:21:13.781 INFO:tasks.workunit.client.0.smithi001.stdout:checking for inttypes.h... yes 2024-05-04T03:21:13.824 INFO:tasks.workunit.client.0.smithi001.stdout:checking for stdint.h... yes 2024-05-04T03:21:13.869 INFO:tasks.workunit.client.0.smithi001.stdout:checking for unistd.h... yes 2024-05-04T03:21:13.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:13 smithi001 ceph-mon[29731]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:21:13.916 INFO:tasks.workunit.client.0.smithi001.stdout:checking for dlfcn.h... yes 2024-05-04T03:21:13.922 INFO:tasks.workunit.client.0.smithi001.stdout:checking for objdir... .libs 2024-05-04T03:21:13.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:13 smithi077 ceph-mon[33531]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:21:13.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:13 smithi072 ceph-mon[33332]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:21:14.051 INFO:tasks.workunit.client.0.smithi001.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-05-04T03:21:14.051 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-05-04T03:21:14.089 INFO:tasks.workunit.client.0.smithi001.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-05-04T03:21:14.136 INFO:tasks.workunit.client.0.smithi001.stdout:checking if gcc static flag -static works... no 2024-05-04T03:21:14.194 INFO:tasks.workunit.client.0.smithi001.stdout:checking if gcc supports -c -o file.o... yes 2024-05-04T03:21:14.194 INFO:tasks.workunit.client.0.smithi001.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-05-04T03:21:14.200 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-05-04T03:21:14.244 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether -lc should be explicitly linked in... no 2024-05-04T03:21:14.306 INFO:tasks.workunit.client.0.smithi001.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-05-04T03:21:14.306 INFO:tasks.workunit.client.0.smithi001.stdout:checking how to hardcode library paths into programs... immediate 2024-05-04T03:21:14.307 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether stripping libraries is possible... yes 2024-05-04T03:21:14.307 INFO:tasks.workunit.client.0.smithi001.stdout:checking if libtool supports shared libraries... yes 2024-05-04T03:21:14.307 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether to build shared libraries... yes 2024-05-04T03:21:14.308 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether to build static libraries... yes 2024-05-04T03:21:14.313 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gcc... (cached) gcc 2024-05-04T03:21:14.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:14 smithi001 ceph-mon[29731]: pgmap v453: 97 pgs: 97 active+clean; 22 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 822 KiB/s wr, 48 op/s 2024-05-04T03:21:14.897 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:14 smithi072 ceph-mon[33332]: pgmap v453: 97 pgs: 97 active+clean; 22 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 822 KiB/s wr, 48 op/s 2024-05-04T03:21:14.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:14 smithi077 ceph-mon[33531]: pgmap v453: 97 pgs: 97 active+clean; 22 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 822 KiB/s wr, 48 op/s 2024-05-04T03:21:16.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:16 smithi001 ceph-mon[29731]: pgmap v454: 97 pgs: 97 active+clean; 24 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 1.0 MiB/s wr, 94 op/s 2024-05-04T03:21:16.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:16 smithi077 ceph-mon[33531]: pgmap v454: 97 pgs: 97 active+clean; 24 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 1.0 MiB/s wr, 94 op/s 2024-05-04T03:21:16.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:16 smithi072 ceph-mon[33332]: pgmap v454: 97 pgs: 97 active+clean; 24 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 748 KiB/s rd, 1.0 MiB/s wr, 94 op/s 2024-05-04T03:21:17.228 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-05-04T03:21:17.228 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether gcc accepts -g... (cached) yes 2024-05-04T03:21:17.228 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-05-04T03:21:17.228 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gmake... /usr/bin/gmake 2024-05-04T03:21:17.229 INFO:tasks.workunit.client.0.smithi001.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-05-04T03:21:17.229 INFO:tasks.workunit.client.0.smithi001.stdout:checking for tar... /bin/tar 2024-05-04T03:21:17.230 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gzip... /bin/gzip 2024-05-04T03:21:17.252 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether gcc -MM is supported... yes 2024-05-04T03:21:17.252 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sort... /bin/sort 2024-05-04T03:21:17.253 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether ln -s works... yes 2024-05-04T03:21:17.253 INFO:tasks.workunit.client.0.smithi001.stdout:checking for rpm... /bin/rpm 2024-05-04T03:21:17.257 INFO:tasks.workunit.client.0.smithi001.stdout:checking for rpmbuild... no 2024-05-04T03:21:17.695 INFO:tasks.workunit.client.0.smithi001.stdout:checking assert.h usability... yes 2024-05-04T03:21:17.713 INFO:tasks.workunit.client.0.smithi001.stdout:checking assert.h presence... yes 2024-05-04T03:21:17.713 INFO:tasks.workunit.client.0.smithi001.stdout:checking for assert.h... yes 2024-05-04T03:21:17.765 INFO:tasks.workunit.client.0.smithi001.stdout:checking bstring.h usability... no 2024-05-04T03:21:17.795 INFO:tasks.workunit.client.0.smithi001.stdout:checking bstring.h presence... no 2024-05-04T03:21:17.795 INFO:tasks.workunit.client.0.smithi001.stdout:checking for bstring.h... no 2024-05-04T03:21:17.835 INFO:tasks.workunit.client.0.smithi001.stdout:checking libgen.h usability... yes 2024-05-04T03:21:17.852 INFO:tasks.workunit.client.0.smithi001.stdout:checking libgen.h presence... yes 2024-05-04T03:21:17.852 INFO:tasks.workunit.client.0.smithi001.stdout:checking for libgen.h... yes 2024-05-04T03:21:17.908 INFO:tasks.workunit.client.0.smithi001.stdout:checking dirent.h usability... yes 2024-05-04T03:21:17.926 INFO:tasks.workunit.client.0.smithi001.stdout:checking dirent.h presence... yes 2024-05-04T03:21:17.926 INFO:tasks.workunit.client.0.smithi001.stdout:checking for dirent.h... yes 2024-05-04T03:21:17.973 INFO:tasks.workunit.client.0.smithi001.stdout:checking errno.h usability... yes 2024-05-04T03:21:17.990 INFO:tasks.workunit.client.0.smithi001.stdout:checking errno.h presence... yes 2024-05-04T03:21:17.990 INFO:tasks.workunit.client.0.smithi001.stdout:checking for errno.h... yes 2024-05-04T03:21:18.038 INFO:tasks.workunit.client.0.smithi001.stdout:checking malloc.h usability... yes 2024-05-04T03:21:18.057 INFO:tasks.workunit.client.0.smithi001.stdout:checking malloc.h presence... yes 2024-05-04T03:21:18.057 INFO:tasks.workunit.client.0.smithi001.stdout:checking for malloc.h... yes 2024-05-04T03:21:18.111 INFO:tasks.workunit.client.0.smithi001.stdout:checking uuid.h usability... no 2024-05-04T03:21:18.141 INFO:tasks.workunit.client.0.smithi001.stdout:checking uuid.h presence... no 2024-05-04T03:21:18.141 INFO:tasks.workunit.client.0.smithi001.stdout:checking for uuid.h... no 2024-05-04T03:21:18.187 INFO:tasks.workunit.client.0.smithi001.stdout:checking uuid/uuid.h usability... yes 2024-05-04T03:21:18.206 INFO:tasks.workunit.client.0.smithi001.stdout:checking uuid/uuid.h presence... yes 2024-05-04T03:21:18.206 INFO:tasks.workunit.client.0.smithi001.stdout:checking for uuid/uuid.h... yes 2024-05-04T03:21:18.259 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/uuid.h usability... no 2024-05-04T03:21:18.288 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/uuid.h presence... no 2024-05-04T03:21:18.288 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/uuid.h... no 2024-05-04T03:21:18.336 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/file.h usability... yes 2024-05-04T03:21:18.353 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/file.h presence... yes 2024-05-04T03:21:18.353 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/file.h... yes 2024-05-04T03:21:18.405 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/fcntl.h usability... yes 2024-05-04T03:21:18.428 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/fcntl.h presence... yes 2024-05-04T03:21:18.428 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/fcntl.h... yes 2024-05-04T03:21:18.506 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/syssgi.h usability... no 2024-05-04T03:21:18.535 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/syssgi.h presence... no 2024-05-04T03:21:18.535 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/syssgi.h... no 2024-05-04T03:21:18.597 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/param.h usability... yes 2024-05-04T03:21:18.619 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/param.h presence... yes 2024-05-04T03:21:18.620 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/param.h... yes 2024-05-04T03:21:18.634 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/stat.h... (cached) yes 2024-05-04T03:21:18.676 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/statvfs.h usability... yes 2024-05-04T03:21:18.697 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/statvfs.h presence... yes 2024-05-04T03:21:18.698 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/statvfs.h... yes 2024-05-04T03:21:18.745 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/time.h usability... yes 2024-05-04T03:21:18.763 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/time.h presence... yes 2024-05-04T03:21:18.763 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/time.h... yes 2024-05-04T03:21:18.811 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/ioctl.h usability... yes 2024-05-04T03:21:18.829 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/ioctl.h presence... yes 2024-05-04T03:21:18.829 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/ioctl.h... yes 2024-05-04T03:21:18.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:18 smithi001 ceph-mon[29731]: pgmap v455: 97 pgs: 97 active+clean; 27 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 977 KiB/s wr, 194 op/s 2024-05-04T03:21:18.883 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/wait.h usability... yes 2024-05-04T03:21:18.903 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/wait.h presence... yes 2024-05-04T03:21:18.903 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/wait.h... yes 2024-05-04T03:21:18.917 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/types.h... (cached) yes 2024-05-04T03:21:18.922 INFO:tasks.workunit.client.0.smithi001.stdout:checking for strings.h... (cached) yes 2024-05-04T03:21:18.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:18 smithi077 ceph-mon[33531]: pgmap v455: 97 pgs: 97 active+clean; 27 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 977 KiB/s wr, 194 op/s 2024-05-04T03:21:18.963 INFO:tasks.workunit.client.0.smithi001.stdout:checking err.h usability... yes 2024-05-04T03:21:18.986 INFO:tasks.workunit.client.0.smithi001.stdout:checking err.h presence... yes 2024-05-04T03:21:18.986 INFO:tasks.workunit.client.0.smithi001.stdout:checking for err.h... yes 2024-05-04T03:21:18.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:18 smithi072 ceph-mon[33332]: pgmap v455: 97 pgs: 97 active+clean; 27 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 977 KiB/s wr, 194 op/s 2024-05-04T03:21:19.032 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/falloc.h usability... yes 2024-05-04T03:21:19.048 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/falloc.h presence... yes 2024-05-04T03:21:19.048 INFO:tasks.workunit.client.0.smithi001.stdout:checking for linux/falloc.h... yes 2024-05-04T03:21:19.102 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-05-04T03:21:19.133 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-05-04T03:21:19.133 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/fs/xfs_fsops.h... no 2024-05-04T03:21:19.178 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/fs/xfs_itable.h usability... no 2024-05-04T03:21:19.208 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/fs/xfs_itable.h presence... no 2024-05-04T03:21:19.208 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/fs/xfs_itable.h... no 2024-05-04T03:21:19.253 INFO:tasks.workunit.client.0.smithi001.stdout:checking xfs/platform_defs.h usability... no 2024-05-04T03:21:19.282 INFO:tasks.workunit.client.0.smithi001.stdout:checking xfs/platform_defs.h presence... no 2024-05-04T03:21:19.282 INFO:tasks.workunit.client.0.smithi001.stdout:checking for xfs/platform_defs.h... no 2024-05-04T03:21:19.325 INFO:tasks.workunit.client.0.smithi001.stdout:checking btrfs/ioctl.h usability... no 2024-05-04T03:21:19.356 INFO:tasks.workunit.client.0.smithi001.stdout:checking btrfs/ioctl.h presence... no 2024-05-04T03:21:19.356 INFO:tasks.workunit.client.0.smithi001.stdout:checking for btrfs/ioctl.h... no 2024-05-04T03:21:19.403 INFO:tasks.workunit.client.0.smithi001.stdout:checking cifs/ioctl.h usability... no 2024-05-04T03:21:19.434 INFO:tasks.workunit.client.0.smithi001.stdout:checking cifs/ioctl.h presence... no 2024-05-04T03:21:19.434 INFO:tasks.workunit.client.0.smithi001.stdout:checking for cifs/ioctl.h... no 2024-05-04T03:21:19.499 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/mman.h usability... yes 2024-05-04T03:21:19.517 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/mman.h presence... yes 2024-05-04T03:21:19.517 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/mman.h... yes 2024-05-04T03:21:19.569 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/ext4.h usability... no 2024-05-04T03:21:19.598 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/ext4.h presence... no 2024-05-04T03:21:19.598 INFO:tasks.workunit.client.0.smithi001.stdout:checking for linux/ext4.h... no 2024-05-04T03:21:19.626 INFO:tasks.workunit.client.0.smithi001.stdout:checking for xfs/xfs_log_format.h... no 2024-05-04T03:21:19.681 INFO:tasks.workunit.client.0.smithi001.stdout:checking for uuid_compare... no 2024-05-04T03:21:19.722 INFO:tasks.workunit.client.0.smithi001.stdout:checking for uuid_compare in -luuid... yes 2024-05-04T03:21:19.787 INFO:tasks.workunit.client.0.smithi001.stdout:checking for xfs/xfs.h... yes 2024-05-04T03:21:19.823 INFO:tasks.workunit.client.0.smithi001.stdout:checking for xfs/libxfs.h... no 2024-05-04T03:21:19.854 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether xlog_assign_lsn is declared... no 2024-05-04T03:21:19.894 INFO:tasks.workunit.client.0.smithi001.stdout:checking for xfs/xqm.h... yes 2024-05-04T03:21:19.947 INFO:tasks.workunit.client.0.smithi001.stdout:checking xfsctl from xfs/xfs.h... ok 2024-05-04T03:21:20.001 INFO:tasks.workunit.client.0.smithi001.stdout:checking xfs/handle.h usability... yes 2024-05-04T03:21:20.016 INFO:tasks.workunit.client.0.smithi001.stdout:checking xfs/handle.h presence... yes 2024-05-04T03:21:20.016 INFO:tasks.workunit.client.0.smithi001.stdout:checking for xfs/handle.h... yes 2024-05-04T03:21:20.058 INFO:tasks.workunit.client.0.smithi001.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-05-04T03:21:20.102 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/acl.h usability... yes 2024-05-04T03:21:20.120 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/acl.h presence... yes 2024-05-04T03:21:20.120 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/acl.h... yes 2024-05-04T03:21:20.153 INFO:tasks.workunit.client.0.smithi001.stdout:checking acl/libacl.h usability... yes 2024-05-04T03:21:20.171 INFO:tasks.workunit.client.0.smithi001.stdout:checking acl/libacl.h presence... yes 2024-05-04T03:21:20.172 INFO:tasks.workunit.client.0.smithi001.stdout:checking for acl/libacl.h... yes 2024-05-04T03:21:20.213 INFO:tasks.workunit.client.0.smithi001.stdout:checking for acl_init in -lacl... yes 2024-05-04T03:21:20.273 INFO:tasks.workunit.client.0.smithi001.stdout:checking gdbm-ndbm.h usability... no 2024-05-04T03:21:20.307 INFO:tasks.workunit.client.0.smithi001.stdout:checking gdbm-ndbm.h presence... no 2024-05-04T03:21:20.307 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gdbm-ndbm.h... no 2024-05-04T03:21:20.346 INFO:tasks.workunit.client.0.smithi001.stdout:checking gdbm/ndbm.h usability... no 2024-05-04T03:21:20.376 INFO:tasks.workunit.client.0.smithi001.stdout:checking gdbm/ndbm.h presence... no 2024-05-04T03:21:20.376 INFO:tasks.workunit.client.0.smithi001.stdout:checking for gdbm/ndbm.h... no 2024-05-04T03:21:20.414 INFO:tasks.workunit.client.0.smithi001.stdout:checking ndbm.h usability... no 2024-05-04T03:21:20.444 INFO:tasks.workunit.client.0.smithi001.stdout:checking ndbm.h presence... no 2024-05-04T03:21:20.444 INFO:tasks.workunit.client.0.smithi001.stdout:checking for ndbm.h... no 2024-05-04T03:21:20.477 INFO:tasks.workunit.client.0.smithi001.stdout:checking libaio.h usability... yes 2024-05-04T03:21:20.495 INFO:tasks.workunit.client.0.smithi001.stdout:checking libaio.h presence... yes 2024-05-04T03:21:20.496 INFO:tasks.workunit.client.0.smithi001.stdout:checking for libaio.h... yes 2024-05-04T03:21:20.543 INFO:tasks.workunit.client.0.smithi001.stdout:checking liburing.h usability... no 2024-05-04T03:21:20.573 INFO:tasks.workunit.client.0.smithi001.stdout:checking liburing.h presence... no 2024-05-04T03:21:20.573 INFO:tasks.workunit.client.0.smithi001.stdout:checking for liburing.h... no 2024-05-04T03:21:20.613 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/capability.h usability... no 2024-05-04T03:21:20.642 INFO:tasks.workunit.client.0.smithi001.stdout:checking sys/capability.h presence... no 2024-05-04T03:21:20.643 INFO:tasks.workunit.client.0.smithi001.stdout:checking for sys/capability.h... no 2024-05-04T03:21:20.682 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/fiemap.h usability... yes 2024-05-04T03:21:20.698 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/fiemap.h presence... yes 2024-05-04T03:21:20.698 INFO:tasks.workunit.client.0.smithi001.stdout:checking for linux/fiemap.h... yes 2024-05-04T03:21:20.736 INFO:tasks.workunit.client.0.smithi001.stdout:checking for fallocate... yes 2024-05-04T03:21:20.785 INFO:tasks.workunit.client.0.smithi001.stdout:checking for open_by_handle_at... yes 2024-05-04T03:21:20.822 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/fs.h usability... yes 2024-05-04T03:21:20.839 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/fs.h presence... yes 2024-05-04T03:21:20.839 INFO:tasks.workunit.client.0.smithi001.stdout:checking for linux/fs.h... yes 2024-05-04T03:21:20.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:20 smithi001 ceph-mon[29731]: pgmap v456: 97 pgs: 97 active+clean; 42 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 392 op/s 2024-05-04T03:21:20.898 INFO:tasks.workunit.client.0.smithi001.stdout:checking btrfsutil.h usability... no 2024-05-04T03:21:20.927 INFO:tasks.workunit.client.0.smithi001.stdout:checking btrfsutil.h presence... no 2024-05-04T03:21:20.927 INFO:tasks.workunit.client.0.smithi001.stdout:checking for btrfsutil.h... no 2024-05-04T03:21:20.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:20 smithi077 ceph-mon[33531]: pgmap v456: 97 pgs: 97 active+clean; 42 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 392 op/s 2024-05-04T03:21:20.968 INFO:tasks.workunit.client.0.smithi001.stdout:checking for copy_file_range... yes 2024-05-04T03:21:20.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:20 smithi072 ceph-mon[33332]: pgmap v456: 97 pgs: 97 active+clean; 42 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 392 op/s 2024-05-04T03:21:21.014 INFO:tasks.workunit.client.0.smithi001.stdout:checking for SEEK_DATA... yes 2024-05-04T03:21:21.074 INFO:tasks.workunit.client.0.smithi001.stdout:checking for BMV_OF_SHARED... yes 2024-05-04T03:21:21.119 INFO:tasks.workunit.client.0.smithi001.stdout:checking for nftw... yes 2024-05-04T03:21:21.173 INFO:tasks.workunit.client.0.smithi001.stdout:checking for RLIMIT_NOFILE... yes 2024-05-04T03:21:21.218 INFO:tasks.workunit.client.0.smithi001.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-05-04T03:21:21.285 INFO:tasks.workunit.client.0.smithi001.stdout:checking for FICLONE... yes 2024-05-04T03:21:21.332 INFO:tasks.workunit.client.0.smithi001.stdout:checking for renameat2... yes 2024-05-04T03:21:21.375 INFO:tasks.workunit.client.0.smithi001.stdout:checking for reallocarray... yes 2024-05-04T03:21:21.441 INFO:tasks.workunit.client.0.smithi001.stdout:checking for struct mount_attr... yes 2024-05-04T03:21:21.509 INFO:tasks.workunit.client.0.smithi001.stdout:checking for struct btrfs_qgroup_limit... yes 2024-05-04T03:21:21.567 INFO:tasks.workunit.client.0.smithi001.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-05-04T03:21:21.635 INFO:tasks.workunit.client.0.smithi001.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-05-04T03:21:21.696 INFO:tasks.workunit.client.0.smithi001.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-05-04T03:21:21.760 INFO:tasks.workunit.client.0.smithi001.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-05-04T03:21:21.819 INFO:tasks.workunit.client.0.smithi001.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-05-04T03:21:21.877 INFO:tasks.workunit.client.0.smithi001.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-05-04T03:21:21.917 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/btrfs.h usability... yes 2024-05-04T03:21:21.933 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/btrfs.h presence... yes 2024-05-04T03:21:21.933 INFO:tasks.workunit.client.0.smithi001.stdout:checking for linux/btrfs.h... yes 2024-05-04T03:21:21.982 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/btrfs_tree.h usability... yes 2024-05-04T03:21:22.000 INFO:tasks.workunit.client.0.smithi001.stdout:checking linux/btrfs_tree.h presence... yes 2024-05-04T03:21:22.000 INFO:tasks.workunit.client.0.smithi001.stdout:checking for linux/btrfs_tree.h... yes 2024-05-04T03:21:22.040 INFO:tasks.workunit.client.0.smithi001.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-05-04T03:21:22.071 INFO:tasks.workunit.client.0.smithi001.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-05-04T03:21:22.091 INFO:tasks.workunit.client.0.smithi001.stdout:configure: creating ./config.status 2024-05-04T03:21:22.425 INFO:tasks.workunit.client.0.smithi001.stdout:config.status: creating include/builddefs 2024-05-04T03:21:22.439 INFO:tasks.workunit.client.0.smithi001.stdout:config.status: creating include/config.h 2024-05-04T03:21:22.449 INFO:tasks.workunit.client.0.smithi001.stdout:config.status: executing libtool commands 2024-05-04T03:21:22.528 INFO:tasks.workunit.client.0.smithi001.stdout:Building include 2024-05-04T03:21:22.528 INFO:tasks.workunit.client.0.smithi001.stdout:Building lib 2024-05-04T03:21:22.528 INFO:tasks.workunit.client.0.smithi001.stdout:Building tests 2024-05-04T03:21:22.536 INFO:tasks.workunit.client.0.smithi001.stdout: [LTDEP] 2024-05-04T03:21:22.538 INFO:tasks.workunit.client.0.smithi001.stdout:Building btrfs 2024-05-04T03:21:22.538 INFO:tasks.workunit.client.0.smithi001.stdout:Building ceph 2024-05-04T03:21:22.538 INFO:tasks.workunit.client.0.smithi001.stdout:Building cifs 2024-05-04T03:21:22.546 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ceph/group.list 2024-05-04T03:21:22.547 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/cifs/group.list 2024-05-04T03:21:22.565 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/btrfs/group.list 2024-05-04T03:21:22.565 INFO:tasks.workunit.client.0.smithi001.stdout:Building ext4 2024-05-04T03:21:22.566 INFO:tasks.workunit.client.0.smithi001.stdout:Building f2fs 2024-05-04T03:21:22.573 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/f2fs/group.list 2024-05-04T03:21:22.586 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ext4/group.list 2024-05-04T03:21:22.593 INFO:tasks.workunit.client.0.smithi001.stdout:Building generic 2024-05-04T03:21:22.624 INFO:tasks.workunit.client.0.smithi001.stdout:Building nfs 2024-05-04T03:21:22.631 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/nfs/group.list 2024-05-04T03:21:22.640 INFO:tasks.workunit.client.0.smithi001.stdout:Building ocfs2 2024-05-04T03:21:22.647 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ocfs2/group.list 2024-05-04T03:21:22.663 INFO:tasks.workunit.client.0.smithi001.stdout:Building overlay 2024-05-04T03:21:22.664 INFO:tasks.workunit.client.0.smithi001.stdout:Building perf 2024-05-04T03:21:22.672 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/perf/group.list 2024-05-04T03:21:22.686 INFO:tasks.workunit.client.0.smithi001.stdout:Building selftest 2024-05-04T03:21:22.689 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/overlay/group.list 2024-05-04T03:21:22.693 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/selftest/group.list 2024-05-04T03:21:22.702 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/generic/group.list 2024-05-04T03:21:22.709 INFO:tasks.workunit.client.0.smithi001.stdout:Building shared 2024-05-04T03:21:22.718 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/shared/group.list 2024-05-04T03:21:22.739 INFO:tasks.workunit.client.0.smithi001.stdout:Building tmpfs 2024-05-04T03:21:22.741 INFO:tasks.workunit.client.0.smithi001.stdout:Building udf 2024-05-04T03:21:22.745 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/tmpfs/group.list 2024-05-04T03:21:22.747 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/udf/group.list 2024-05-04T03:21:22.754 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] dataascii.lo 2024-05-04T03:21:22.762 INFO:tasks.workunit.client.0.smithi001.stdout:Building xfs 2024-05-04T03:21:22.763 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] databin.lo 2024-05-04T03:21:22.826 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] datapid.lo 2024-05-04T03:21:22.858 INFO:tasks.workunit.client.0.smithi001.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/xfs/group.list 2024-05-04T03:21:22.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:22 smithi001 ceph-mon[29731]: pgmap v457: 97 pgs: 97 active+clean; 44 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 427 op/s 2024-05-04T03:21:22.920 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] file_lock.lo 2024-05-04T03:21:22.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:22 smithi077 ceph-mon[33531]: pgmap v457: 97 pgs: 97 active+clean; 44 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 427 op/s 2024-05-04T03:21:22.949 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] forker.lo 2024-05-04T03:21:22.967 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] pattern.lo 2024-05-04T03:21:22.974 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] open_flags.lo 2024-05-04T03:21:22.998 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:22 smithi072 ceph-mon[33332]: pgmap v457: 97 pgs: 97 active+clean; 44 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 427 op/s 2024-05-04T03:21:23.132 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] string_to_tokens.lo 2024-05-04T03:21:23.132 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] random_range.lo 2024-05-04T03:21:23.140 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] str_to_bytes.lo 2024-05-04T03:21:23.156 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] tlibio.lo 2024-05-04T03:21:23.274 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] write_log.lo 2024-05-04T03:21:23.329 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] random.lo 2024-05-04T03:21:23.551 INFO:tasks.workunit.client.0.smithi001.stdout: [LD] 2024-05-04T03:21:23.706 INFO:tasks.workunit.client.0.smithi001.stdout:Building ltp 2024-05-04T03:21:23.706 INFO:tasks.workunit.client.0.smithi001.stdout:Building src 2024-05-04T03:21:23.707 INFO:tasks.workunit.client.0.smithi001.stdout:Building m4 2024-05-04T03:21:23.707 INFO:tasks.workunit.client.0.smithi001.stdout:Building common 2024-05-04T03:21:23.711 INFO:tasks.workunit.client.0.smithi001.stdout: [DEP] 2024-05-04T03:21:23.725 INFO:tasks.workunit.client.0.smithi001.stdout: [DEP] 2024-05-04T03:21:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.848+0000 7f298690d640 1 -- 172.21.15.1:0/1305586729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2980102b50 msgr2=0x7f2980103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.848+0000 7f298690d640 1 --2- 172.21.15.1:0/1305586729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2980102b50 0x7f2980103850 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2970009900 tx=0x7f297002a410 comp rx=0 tx=0).stop 2024-05-04T03:21:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.848+0000 7f298690d640 1 -- 172.21.15.1:0/1305586729 shutdown_connections 2024-05-04T03:21:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.848+0000 7f298690d640 1 --2- 172.21.15.1:0/1305586729 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2980103d90 0x7f29801041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.850 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.848+0000 7f298690d640 1 --2- 172.21.15.1:0/1305586729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2980108a90 0x7f298010aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.848+0000 7f298690d640 1 --2- 172.21.15.1:0/1305586729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2980102b50 0x7f2980103850 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.848+0000 7f298690d640 1 -- 172.21.15.1:0/1305586729 >> 172.21.15.1:0/1305586729 conn(0x7f29800fdd40 msgr2=0x7f29801001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.848+0000 7f298690d640 1 -- 172.21.15.1:0/1305586729 shutdown_connections 2024-05-04T03:21:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.848+0000 7f298690d640 1 -- 172.21.15.1:0/1305586729 wait complete. 2024-05-04T03:21:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.848+0000 7f298690d640 1 Processor -- start 2024-05-04T03:21:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.848+0000 7f298690d640 1 -- start start 2024-05-04T03:21:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.849+0000 7f298690d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2980102b50 0x7f298019a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.849+0000 7f298690d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2980103d90 0x7f298019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.849+0000 7f298690d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2980108a90 0x7f29801a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.849+0000 7f298690d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f298010e680 con 0x7f2980102b50 2024-05-04T03:21:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.849+0000 7f298690d640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f298010e500 con 0x7f2980103d90 2024-05-04T03:21:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.849+0000 7f298690d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f298010e800 con 0x7f2980108a90 2024-05-04T03:21:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.849+0000 7f297ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2980102b50 0x7f298019a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.849+0000 7f297ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2980102b50 0x7f298019a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39198/0 (socket says 172.21.15.1:39198) 2024-05-04T03:21:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.849+0000 7f297ffff640 1 -- 172.21.15.1:0/1038626321 learned_addr learned my addr 172.21.15.1:0/1038626321 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:21:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.849+0000 7f297f7fe640 1 --2- 172.21.15.1:0/1038626321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2980103d90 0x7f298019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.849+0000 7f2984eaa640 1 --2- 172.21.15.1:0/1038626321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2980108a90 0x7f29801a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.849+0000 7f297ffff640 1 -- 172.21.15.1:0/1038626321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2980108a90 msgr2=0x7f29801a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.849+0000 7f297ffff640 1 --2- 172.21.15.1:0/1038626321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2980108a90 0x7f29801a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.849+0000 7f297ffff640 1 -- 172.21.15.1:0/1038626321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2980103d90 msgr2=0x7f298019a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.849+0000 7f297ffff640 1 --2- 172.21.15.1:0/1038626321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2980103d90 0x7f298019a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.850+0000 7f297ffff640 1 -- 172.21.15.1:0/1038626321 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29801a1e00 con 0x7f2980102b50 2024-05-04T03:21:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.850+0000 7f297ffff640 1 --2- 172.21.15.1:0/1038626321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2980102b50 0x7f298019a2c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2970031440 tx=0x7f2970003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.850+0000 7f297d7fa640 1 -- 172.21.15.1:0/1038626321 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297003f070 con 0x7f2980102b50 2024-05-04T03:21:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.850+0000 7f298690d640 1 -- 172.21.15.1:0/1038626321 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29801a2090 con 0x7f2980102b50 2024-05-04T03:21:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.850+0000 7f297d7fa640 1 -- 172.21.15.1:0/1038626321 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2970031c50 con 0x7f2980102b50 2024-05-04T03:21:23.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.850+0000 7f297d7fa640 1 -- 172.21.15.1:0/1038626321 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2970035030 con 0x7f2980102b50 2024-05-04T03:21:23.855 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.850+0000 7f298690d640 1 -- 172.21.15.1:0/1038626321 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29801a25d0 con 0x7f2980102b50 2024-05-04T03:21:23.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.853+0000 7f298d7c1740 1 Processor -- start 2024-05-04T03:21:23.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.853+0000 7f298d7c1740 1 -- start start 2024-05-04T03:21:23.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.853+0000 7f297d7fa640 1 -- 172.21.15.1:0/1038626321 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f2970042070 con 0x7f2980102b50 2024-05-04T03:21:23.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.853+0000 7f298d7c1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f8ba51b0 0x55a3f8f46a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:23.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.853+0000 7f298d7c1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3f8b7c2f0 0x55a3f8f46fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:23.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.853+0000 7f298d7c1740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a3f8b844d0 0x55a3f8f4dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:23.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.853+0000 7f298d7c1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a3f8eab5d0 con 0x55a3f8b7c2f0 2024-05-04T03:21:23.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.853+0000 7f298d7c1740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55a3f8e64890 con 0x55a3f8b844d0 2024-05-04T03:21:23.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.853+0000 7f298d7c1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a3f8ea7590 con 0x55a3f8ba51b0 2024-05-04T03:21:23.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.853+0000 7f297ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3f8b7c2f0 0x55a3f8f46fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:23.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.853+0000 7f297f7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a3f8b844d0 0x55a3f8f4dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:23.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.853+0000 7f297ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3f8b7c2f0 0x55a3f8f46fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39212/0 (socket says 172.21.15.1:39212) 2024-05-04T03:21:23.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.853+0000 7f297ffff640 1 -- 172.21.15.1:0/1454114510 learned_addr learned my addr 172.21.15.1:0/1454114510 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:21:23.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.854+0000 7f2984eaa640 1 --2- 172.21.15.1:0/1454114510 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f8ba51b0 0x55a3f8f46a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:23.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.854+0000 7f297f7fe640 1 -- 172.21.15.1:0/1454114510 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f8ba51b0 msgr2=0x55a3f8f46a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:23.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.854+0000 7f297f7fe640 1 --2- 172.21.15.1:0/1454114510 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f8ba51b0 0x55a3f8f46a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.854+0000 7f297f7fe640 1 -- 172.21.15.1:0/1454114510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3f8b7c2f0 msgr2=0x55a3f8f46fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:23.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.854+0000 7f297f7fe640 1 --2- 172.21.15.1:0/1454114510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3f8b7c2f0 0x55a3f8f46fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.854+0000 7f297f7fe640 1 -- 172.21.15.1:0/1454114510 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3f8bbd210 con 0x55a3f8b844d0 2024-05-04T03:21:23.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.854+0000 7f297d7fa640 1 --2- 172.21.15.1:0/1038626321 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f29680b28a0 0x7f29680b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:23.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.854+0000 7f297f7fe640 1 --2- 172.21.15.1:0/1454114510 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a3f8b844d0 0x55a3f8f4dee0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2974009bf0 tx=0x7f2974011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcf58aae84f4d514 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:23.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.854+0000 7f2984eaa640 1 --2- 172.21.15.1:0/1038626321 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f29680b28a0 0x7f29680b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:23.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.854+0000 7f297d7fa640 1 -- 172.21.15.1:0/1038626321 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f29701028e0 con 0x7f2980102b50 2024-05-04T03:21:23.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.854+0000 7f29667fc640 1 -- 172.21.15.1:0/1454114510 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2974004050 con 0x55a3f8b844d0 2024-05-04T03:21:23.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.854+0000 7f29667fc640 1 -- 172.21.15.1:0/1454114510 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29740054d0 con 0x55a3f8b844d0 2024-05-04T03:21:23.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.854+0000 7f29667fc640 1 -- 172.21.15.1:0/1454114510 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2974004d20 con 0x55a3f8b844d0 2024-05-04T03:21:23.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.855+0000 7f298d7c1740 1 -- 172.21.15.1:0/1454114510 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a3f8b844d0 msgr2=0x55a3f8f4dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:23.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.855+0000 7f2984eaa640 1 --2- 172.21.15.1:0/1038626321 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f29680b28a0 0x7f29680b4db0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55a3f8f47c30 tx=0x7f2978009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:23.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.855+0000 7f298d7c1740 1 --2- 172.21.15.1:0/1454114510 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a3f8b844d0 0x55a3f8f4dee0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2974009bf0 tx=0x7f2974011810 comp rx=0 tx=0).stop 2024-05-04T03:21:23.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.856+0000 7f298d7c1740 1 -- 172.21.15.1:0/1454114510 shutdown_connections 2024-05-04T03:21:23.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.856+0000 7f298d7c1740 1 --2- 172.21.15.1:0/1454114510 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a3f8b844d0 0x55a3f8f4dee0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.856+0000 7f298d7c1740 1 --2- 172.21.15.1:0/1454114510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3f8b7c2f0 0x55a3f8f46fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.856+0000 7f298d7c1740 1 --2- 172.21.15.1:0/1454114510 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f8ba51b0 0x55a3f8f46a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.856+0000 7f298d7c1740 1 -- 172.21.15.1:0/1454114510 >> 172.21.15.1:0/1454114510 conn(0x55a3f8b99300 msgr2=0x55a3f8f423a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.856+0000 7f298d7c1740 1 -- 172.21.15.1:0/1454114510 shutdown_connections 2024-05-04T03:21:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.856+0000 7f298d7c1740 1 -- 172.21.15.1:0/1454114510 wait complete. 2024-05-04T03:21:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.856+0000 7f298d7c1740 1 -- start start 2024-05-04T03:21:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.856+0000 7f298d7c1740 1 Processor -- start 2024-05-04T03:21:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.857+0000 7f298d7c1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3f8b7c2f0 0x55a3f8f50610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.857+0000 7f298d7c1740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a3f8b844d0 0x55a3f8f50b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.857+0000 7f298d7c1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f8ba51b0 0x55a3f8f57ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.857+0000 7f298d7c1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a3f8f4e930 con 0x55a3f8b7c2f0 2024-05-04T03:21:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.857+0000 7f298d7c1740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55a3f8e68370 con 0x55a3f8b844d0 2024-05-04T03:21:23.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.857+0000 7f298d7c1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a3f8f4eab0 con 0x55a3f8ba51b0 2024-05-04T03:21:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.857+0000 7f298d7c1740 10 client.0 _ll_register_callbacks cb 0x55a3f8bbca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:21:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.857+0000 7f297ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3f8b7c2f0 0x55a3f8f50610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.857+0000 7f297f7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a3f8b844d0 0x55a3f8f50b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.857+0000 7f297ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3f8b7c2f0 0x55a3f8f50610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39214/0 (socket says 172.21.15.1:39214) 2024-05-04T03:21:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.857+0000 7f297ffff640 1 -- 172.21.15.1:0/555228373 learned_addr learned my addr 172.21.15.1:0/555228373 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:21:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.857+0000 7f2984eaa640 1 --2- 172.21.15.1:0/555228373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f8ba51b0 0x55a3f8f57ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.857+0000 7f297ffff640 1 -- 172.21.15.1:0/555228373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f8ba51b0 msgr2=0x55a3f8f57ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.857+0000 7f297ffff640 1 --2- 172.21.15.1:0/555228373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f8ba51b0 0x55a3f8f57ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.857+0000 7f297ffff640 1 -- 172.21.15.1:0/555228373 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a3f8b844d0 msgr2=0x55a3f8f50b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.857+0000 7f297ffff640 1 --2- 172.21.15.1:0/555228373 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a3f8b844d0 0x55a3f8f50b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.857+0000 7f297ffff640 1 -- 172.21.15.1:0/555228373 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3f8c89620 con 0x55a3f8b7c2f0 2024-05-04T03:21:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.857+0000 7f297f7fe640 1 --2- 172.21.15.1:0/555228373 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a3f8b844d0 0x55a3f8f50b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:21:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.857+0000 7f297ffff640 1 --2- 172.21.15.1:0/555228373 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3f8b7c2f0 0x55a3f8f50610 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f29700c60f0 tx=0x7f2970005ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694198c0fe229f00 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:23.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.857+0000 7f295f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:21:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.858+0000 7f295f7fe640 1 -- 172.21.15.1:0/555228373 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2970049050 con 0x55a3f8b7c2f0 2024-05-04T03:21:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.858+0000 7f295f7fe640 1 -- 172.21.15.1:0/555228373 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f297003b460 con 0x55a3f8b7c2f0 2024-05-04T03:21:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.858+0000 7f298d7c1740 1 -- 172.21.15.1:0/555228373 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29740054d0 con 0x55a3f8b7c2f0 2024-05-04T03:21:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.858+0000 7f295f7fe640 1 -- 172.21.15.1:0/555228373 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f29700c99c0 con 0x55a3f8b7c2f0 2024-05-04T03:21:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.858+0000 7f298d7c1740 10 client.15510 fetch_fsmap learned FSMap version 29 2024-05-04T03:21:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.858+0000 7f298d7c1740 1 -- 172.21.15.1:0/555228373 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a3f8f5a9e0 con 0x55a3f8b7c2f0 2024-05-04T03:21:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.858+0000 7f295f7fe640 1 -- 172.21.15.1:0/555228373 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29740054d0 con 0x55a3f8b7c2f0 2024-05-04T03:21:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.858+0000 7f295f7fe640 1 -- 172.21.15.1:0/555228373 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f29700cc070 con 0x55a3f8b7c2f0 2024-05-04T03:21:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.858+0000 7f298d7c1740 10 client.15510 fetch_fsmap finished waiting for FSMap version 29 2024-05-04T03:21:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.858+0000 7f298d7c1740 10 client.15510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:21:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.858+0000 7f298d7c1740 20 client.15510 populate_metadata read hostname 'smithi001' 2024-05-04T03:21:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.858+0000 7f298d7c1740 1 --2- 172.21.15.1:0/555228373 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a3f8b63d80 0x55a3f8f5e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:23.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.858+0000 7f298d7c1740 4 client.15510 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:21:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.858+0000 7f298d7c1740 1 -- 172.21.15.1:0/555228373 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3f8eb91e0 con 0x55a3f8b63d80 2024-05-04T03:21:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.859+0000 7f297f7fe640 1 --2- 172.21.15.1:0/555228373 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a3f8b63d80 0x55a3f8f5e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.859+0000 7f295f7fe640 10 client.15510 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:21:23.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.859+0000 7f297f7fe640 1 --2- 172.21.15.1:0/555228373 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a3f8b63d80 0x55a3f8f5e280 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca87c2bb03e75b4a server_cookie=e02d1e73e3a39ff8 in_seq=0 out_seq=0 2024-05-04T03:21:23.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.860+0000 7f295f7fe640 1 -- 172.21.15.1:0/555228373 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a3f8eb91e0 con 0x55a3f8b63d80 2024-05-04T03:21:23.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.860+0000 7f295f7fe640 10 client.15510 handle_command_reply: tid=0 2024-05-04T03:21:23.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.860+0000 7f298d7c1740 1 client.15510 shutdown 2024-05-04T03:21:23.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.860+0000 7f298d7c1740 1 -- 172.21.15.1:0/555228373 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3f8b7c2f0 msgr2=0x55a3f8f50610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:23.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.860+0000 7f298d7c1740 1 --2- 172.21.15.1:0/555228373 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3f8b7c2f0 0x55a3f8f50610 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f29700c60f0 tx=0x7f2970005ed0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.860+0000 7f298d7c1740 1 -- 172.21.15.1:0/555228373 shutdown_connections 2024-05-04T03:21:23.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.860+0000 7f298d7c1740 1 --2- 172.21.15.1:0/555228373 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a3f8b63d80 0x55a3f8f5e280 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.860+0000 7f298d7c1740 1 --2- 172.21.15.1:0/555228373 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a3f8b844d0 0x55a3f8f50b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.860+0000 7f295f7fe640 0 client.15510 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:21:23.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.860+0000 7f298d7c1740 1 --2- 172.21.15.1:0/555228373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f8ba51b0 0x55a3f8f57ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.860+0000 7f298d7c1740 1 --2- 172.21.15.1:0/555228373 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3f8b7c2f0 0x55a3f8f50610 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.860+0000 7f298d7c1740 1 -- 172.21.15.1:0/555228373 >> 172.21.15.1:0/555228373 conn(0x55a3f8b99300 msgr2=0x55a3f8f413a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:23.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.860+0000 7f298d7c1740 1 -- 172.21.15.1:0/555228373 shutdown_connections 2024-05-04T03:21:23.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.860+0000 7f298d7c1740 1 -- 172.21.15.1:0/555228373 wait complete. 2024-05-04T03:21:23.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.860+0000 7f298d7c1740 20 client.15510 trim_cache size 0 max 16384 2024-05-04T03:21:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.876+0000 7f298d7c1740 1 Processor -- start 2024-05-04T03:21:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.876+0000 7f298d7c1740 1 -- start start 2024-05-04T03:21:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.876+0000 7f298d7c1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f8b63d80 0x55a3f8f42400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.876+0000 7f298d7c1740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a3f8b7c2f0 0x55a3f8f501e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.876+0000 7f298d7c1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3f8b844d0 0x55a3f8f50720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.876+0000 7f298d7c1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f29801afc10 con 0x55a3f8b844d0 2024-05-04T03:21:23.880 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.876+0000 7f298d7c1740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f29801ad480 con 0x55a3f8b7c2f0 2024-05-04T03:21:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.876+0000 7f298d7c1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f29801ab2f0 con 0x55a3f8b63d80 2024-05-04T03:21:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.876+0000 7f2984eaa640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3f8b844d0 0x55a3f8f50720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.876+0000 7f2984eaa640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3f8b844d0 0x55a3f8f50720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39220/0 (socket says 172.21.15.1:39220) 2024-05-04T03:21:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.876+0000 7f2984eaa640 1 -- 172.21.15.1:0/4020177844 learned_addr learned my addr 172.21.15.1:0/4020177844 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:21:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.876+0000 7f297ffff640 1 --2- 172.21.15.1:0/4020177844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f8b63d80 0x55a3f8f42400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.876+0000 7f297f7fe640 1 --2- 172.21.15.1:0/4020177844 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a3f8b7c2f0 0x55a3f8f501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.877+0000 7f297ffff640 1 -- 172.21.15.1:0/4020177844 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a3f8b7c2f0 msgr2=0x55a3f8f501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.877+0000 7f297ffff640 1 --2- 172.21.15.1:0/4020177844 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a3f8b7c2f0 0x55a3f8f501e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.877+0000 7f297ffff640 1 -- 172.21.15.1:0/4020177844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3f8b844d0 msgr2=0x55a3f8f50720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.877+0000 7f297ffff640 1 --2- 172.21.15.1:0/4020177844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3f8b844d0 0x55a3f8f50720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.877+0000 7f297ffff640 1 -- 172.21.15.1:0/4020177844 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3f8f42940 con 0x55a3f8b63d80 2024-05-04T03:21:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.877+0000 7f2984eaa640 1 --2- 172.21.15.1:0/4020177844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3f8b844d0 0x55a3f8f50720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:21:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.877+0000 7f297ffff640 1 --2- 172.21.15.1:0/4020177844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f8b63d80 0x55a3f8f42400 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2970000c00 tx=0x7f29700ce4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18ca5baeb76968ab server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:23.881 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.877+0000 7f2965ffb640 1 -- 172.21.15.1:0/4020177844 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2970049070 con 0x55a3f8b63d80 2024-05-04T03:21:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.877+0000 7f2965ffb640 1 -- 172.21.15.1:0/4020177844 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29700c9b80 con 0x55a3f8b63d80 2024-05-04T03:21:23.882 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.877+0000 7f2965ffb640 1 -- 172.21.15.1:0/4020177844 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29700c97e0 con 0x55a3f8b63d80 2024-05-04T03:21:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.878+0000 7f298d7c1740 1 -- 172.21.15.1:0/4020177844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f8b63d80 msgr2=0x55a3f8f42400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.878+0000 7f298d7c1740 1 --2- 172.21.15.1:0/4020177844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f8b63d80 0x55a3f8f42400 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2970000c00 tx=0x7f29700ce4f0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.878+0000 7f298d7c1740 1 -- 172.21.15.1:0/4020177844 shutdown_connections 2024-05-04T03:21:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.878+0000 7f298d7c1740 1 --2- 172.21.15.1:0/4020177844 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a3f8b7c2f0 0x55a3f8f501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.878+0000 7f298d7c1740 1 --2- 172.21.15.1:0/4020177844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3f8b844d0 0x55a3f8f50720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.878+0000 7f298d7c1740 1 --2- 172.21.15.1:0/4020177844 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f8b63d80 0x55a3f8f42400 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.878+0000 7f298d7c1740 1 -- 172.21.15.1:0/4020177844 >> 172.21.15.1:0/4020177844 conn(0x55a3f8b99300 msgr2=0x55a3f8f41cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.878+0000 7f298d7c1740 1 -- 172.21.15.1:0/4020177844 shutdown_connections 2024-05-04T03:21:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.878+0000 7f298d7c1740 1 -- 172.21.15.1:0/4020177844 wait complete. 2024-05-04T03:21:23.886 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.878+0000 7f298d7c1740 1 -- start start 2024-05-04T03:21:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.879+0000 7f298d7c1740 1 Processor -- start 2024-05-04T03:21:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.879+0000 7f298d7c1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f8b63d80 0x55a3f8f54320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.879+0000 7f298d7c1740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a3f8b7c2f0 0x55a3f8f54860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.879+0000 7f298d7c1740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3f8b844d0 0x55a3f8f556b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.879+0000 7f298d7c1740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f29801ab940 con 0x55a3f8b844d0 2024-05-04T03:21:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.879+0000 7f298d7c1740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f29801acdb0 con 0x55a3f8b7c2f0 2024-05-04T03:21:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.879+0000 7f298d7c1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f29801ac010 con 0x55a3f8b63d80 2024-05-04T03:21:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.879+0000 7f298d7c1740 10 client.0 _ll_register_callbacks cb 0x7f2970009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:21:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.879+0000 7f2984eaa640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3f8b844d0 0x55a3f8f556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.879+0000 7f2984eaa640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3f8b844d0 0x55a3f8f556b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39230/0 (socket says 172.21.15.1:39230) 2024-05-04T03:21:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.879+0000 7f2984eaa640 1 -- 172.21.15.1:0/3882042221 learned_addr learned my addr 172.21.15.1:0/3882042221 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:21:23.887 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.879+0000 7f297ffff640 1 --2- 172.21.15.1:0/3882042221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f8b63d80 0x55a3f8f54320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.879+0000 7f297f7fe640 1 --2- 172.21.15.1:0/3882042221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a3f8b7c2f0 0x55a3f8f54860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.879+0000 7f2984eaa640 1 -- 172.21.15.1:0/3882042221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f8b63d80 msgr2=0x55a3f8f54320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.879+0000 7f2984eaa640 1 --2- 172.21.15.1:0/3882042221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f8b63d80 0x55a3f8f54320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.879+0000 7f2984eaa640 1 -- 172.21.15.1:0/3882042221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a3f8b7c2f0 msgr2=0x55a3f8f54860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.879+0000 7f2984eaa640 1 --2- 172.21.15.1:0/3882042221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a3f8b7c2f0 0x55a3f8f54860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.879+0000 7f2984eaa640 1 -- 172.21.15.1:0/3882042221 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3f8f55c70 con 0x55a3f8b844d0 2024-05-04T03:21:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.880+0000 7f2984eaa640 1 --2- 172.21.15.1:0/3882042221 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3f8b844d0 0x55a3f8f556b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2978009c00 tx=0x7f2978004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88240a45cbd5571 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.880+0000 7f295f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:21:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.880+0000 7f295f7fe640 1 -- 172.21.15.1:0/3882042221 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2978004840 con 0x55a3f8b844d0 2024-05-04T03:21:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.880+0000 7f295f7fe640 1 -- 172.21.15.1:0/3882042221 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f2978003de0 con 0x55a3f8b844d0 2024-05-04T03:21:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.880+0000 7f298d7c1740 1 -- 172.21.15.1:0/3882042221 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29700c9b80 con 0x55a3f8b844d0 2024-05-04T03:21:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.880+0000 7f295f7fe640 1 -- 172.21.15.1:0/3882042221 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f297802c420 con 0x55a3f8b844d0 2024-05-04T03:21:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.880+0000 7f298d7c1740 10 client.15516 fetch_fsmap learned FSMap version 29 2024-05-04T03:21:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.880+0000 7f298d7c1740 1 -- 172.21.15.1:0/3882042221 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a3f8f56fd0 con 0x55a3f8b844d0 2024-05-04T03:21:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.880+0000 7f295f7fe640 1 -- 172.21.15.1:0/3882042221 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29700c9b80 con 0x55a3f8b844d0 2024-05-04T03:21:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.880+0000 7f295f7fe640 1 -- 172.21.15.1:0/3882042221 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f2978036020 con 0x55a3f8b844d0 2024-05-04T03:21:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.880+0000 7f298d7c1740 10 client.15516 fetch_fsmap finished waiting for FSMap version 29 2024-05-04T03:21:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.880+0000 7f298d7c1740 10 client.15516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:21:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.880+0000 7f298d7c1740 20 client.15516 populate_metadata read hostname 'smithi001' 2024-05-04T03:21:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.881+0000 7f298d7c1740 1 --2- 172.21.15.1:0/3882042221 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a3f8ba51b0 0x55a3f8fc89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.881+0000 7f298d7c1740 4 client.15516 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:21:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.881+0000 7f298d7c1740 1 -- 172.21.15.1:0/3882042221 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a3f8eb3050 con 0x55a3f8ba51b0 2024-05-04T03:21:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.881+0000 7f297ffff640 1 --2- 172.21.15.1:0/3882042221 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a3f8ba51b0 0x55a3f8fc89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.881+0000 7f295f7fe640 10 client.15516 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:21:23.889 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.881+0000 7f297ffff640 1 --2- 172.21.15.1:0/3882042221 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a3f8ba51b0 0x55a3f8fc89e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fd1f52ee329b7f1 server_cookie=e89aa69fdffbcafb in_seq=0 out_seq=0 2024-05-04T03:21:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.905+0000 7f295f7fe640 1 -- 172.21.15.1:0/3882042221 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a3f8eb3050 con 0x55a3f8ba51b0 2024-05-04T03:21:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.905+0000 7f295f7fe640 10 client.15516 handle_command_reply: tid=0 2024-05-04T03:21:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.905+0000 7f298d7c1740 1 client.15516 shutdown 2024-05-04T03:21:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.905+0000 7f298d7c1740 1 -- 172.21.15.1:0/3882042221 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3f8b844d0 msgr2=0x55a3f8f556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.905+0000 7f298d7c1740 1 --2- 172.21.15.1:0/3882042221 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3f8b844d0 0x55a3f8f556b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2978009c00 tx=0x7f2978004a40 comp rx=0 tx=0).stop 2024-05-04T03:21:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.905+0000 7f298d7c1740 1 -- 172.21.15.1:0/3882042221 shutdown_connections 2024-05-04T03:21:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.905+0000 7f298d7c1740 1 --2- 172.21.15.1:0/3882042221 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a3f8ba51b0 0x55a3f8fc89e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.905+0000 7f298d7c1740 1 --2- 172.21.15.1:0/3882042221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a3f8b7c2f0 0x55a3f8f54860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.905+0000 7f298d7c1740 1 --2- 172.21.15.1:0/3882042221 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3f8b844d0 0x55a3f8f556b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.905+0000 7f298d7c1740 1 --2- 172.21.15.1:0/3882042221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3f8b63d80 0x55a3f8f54320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.905+0000 7f295f7fe640 0 client.15516 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:21:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.905+0000 7f298d7c1740 1 -- 172.21.15.1:0/3882042221 >> 172.21.15.1:0/3882042221 conn(0x55a3f8b99300 msgr2=0x55a3f8f501e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.905+0000 7f298d7c1740 1 -- 172.21.15.1:0/3882042221 shutdown_connections 2024-05-04T03:21:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.905+0000 7f298d7c1740 1 -- 172.21.15.1:0/3882042221 wait complete. 2024-05-04T03:21:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.905+0000 7f298d7c1740 20 client.15516 trim_cache size 0 max 16384 2024-05-04T03:21:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:21:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:21:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:21:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:21:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.906+0000 7f298690d640 1 -- 172.21.15.1:0/1038626321 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f29680b28a0 msgr2=0x7f29680b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.906+0000 7f298690d640 1 --2- 172.21.15.1:0/1038626321 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f29680b28a0 0x7f29680b4db0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55a3f8f47c30 tx=0x7f2978009600 comp rx=0 tx=0).stop 2024-05-04T03:21:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.906+0000 7f298690d640 1 -- 172.21.15.1:0/1038626321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2980102b50 msgr2=0x7f298019a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.906+0000 7f298690d640 1 --2- 172.21.15.1:0/1038626321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2980102b50 0x7f298019a2c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2970031440 tx=0x7f2970003c10 comp rx=0 tx=0).stop 2024-05-04T03:21:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.906+0000 7f298690d640 1 -- 172.21.15.1:0/1038626321 shutdown_connections 2024-05-04T03:21:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.906+0000 7f298690d640 1 --2- 172.21.15.1:0/1038626321 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f29680b28a0 0x7f29680b4db0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.906+0000 7f298690d640 1 --2- 172.21.15.1:0/1038626321 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2980103d90 0x7f298019a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.906+0000 7f298690d640 1 --2- 172.21.15.1:0/1038626321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2980108a90 0x7f29801a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.906+0000 7f298690d640 1 --2- 172.21.15.1:0/1038626321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2980102b50 0x7f298019a2c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.906+0000 7f298690d640 1 -- 172.21.15.1:0/1038626321 >> 172.21.15.1:0/1038626321 conn(0x7f29800fdd40 msgr2=0x7f2980101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.906+0000 7f298690d640 1 -- 172.21.15.1:0/1038626321 shutdown_connections 2024-05-04T03:21:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:23.906+0000 7f298690d640 1 -- 172.21.15.1:0/1038626321 wait complete. 2024-05-04T03:21:23.918 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] doio 2024-05-04T03:21:23.918 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fsstress 2024-05-04T03:21:23.918 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fsx 2024-05-04T03:21:24.633 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] growfiles 2024-05-04T03:21:24.699 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] iogen 2024-05-04T03:21:24.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:24 smithi001 ceph-mon[29731]: pgmap v458: 97 pgs: 97 active+clean; 45 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 427 op/s 2024-05-04T03:21:24.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:24 smithi077 ceph-mon[33531]: pgmap v458: 97 pgs: 97 active+clean; 45 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 427 op/s 2024-05-04T03:21:24.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:24 smithi072 ceph-mon[33332]: pgmap v458: 97 pgs: 97 active+clean; 45 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 427 op/s 2024-05-04T03:21:25.071 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-stress 2024-05-04T03:21:25.652 INFO:tasks.workunit.client.0.smithi001.stdout: [DEP] 2024-05-04T03:21:25.652 INFO:tasks.workunit.client.0.smithi001.stdout: [DEP] 2024-05-04T03:21:25.652 INFO:tasks.workunit.client.0.smithi001.stdout: [DEP] 2024-05-04T03:21:25.653 INFO:tasks.workunit.client.0.smithi001.stdout: [DEP] 2024-05-04T03:21:25.883 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] dirstress 2024-05-04T03:21:25.883 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fill 2024-05-04T03:21:25.883 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fill2 2024-05-04T03:21:25.884 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] getpagesize 2024-05-04T03:21:26.014 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] holes 2024-05-04T03:21:26.050 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] lstat64 2024-05-04T03:21:26.126 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] nametest 2024-05-04T03:21:26.130 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] permname 2024-05-04T03:21:26.233 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] randholes 2024-05-04T03:21:26.239 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] runas 2024-05-04T03:21:26.353 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] truncfile 2024-05-04T03:21:26.365 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] usemem 2024-05-04T03:21:26.412 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] mmapcat 2024-05-04T03:21:26.480 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] append_reader 2024-05-04T03:21:26.515 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] append_writer 2024-05-04T03:21:26.533 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] dirperf 2024-05-04T03:21:26.557 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] metaperf 2024-05-04T03:21:26.631 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] devzero 2024-05-04T03:21:26.668 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] feature 2024-05-04T03:21:26.763 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] alloc 2024-05-04T03:21:26.802 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fault 2024-05-04T03:21:26.864 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fstest 2024-05-04T03:21:26.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:26 smithi001 ceph-mon[29731]: pgmap v459: 97 pgs: 97 active+clean; 47 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 431 op/s 2024-05-04T03:21:26.898 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_access_root 2024-05-04T03:21:26.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:26 smithi077 ceph-mon[33531]: pgmap v459: 97 pgs: 97 active+clean; 47 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 431 op/s 2024-05-04T03:21:26.961 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] godown 2024-05-04T03:21:26.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:26 smithi072 ceph-mon[33332]: pgmap v459: 97 pgs: 97 active+clean; 47 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 431 op/s 2024-05-04T03:21:27.004 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] resvtest 2024-05-04T03:21:27.050 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] writemod 2024-05-04T03:21:27.139 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] writev_on_pagefault 2024-05-04T03:21:27.147 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] makeextents 2024-05-04T03:21:27.180 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] itrash 2024-05-04T03:21:27.209 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] rename 2024-05-04T03:21:27.314 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] multi_open_unlink 2024-05-04T03:21:27.327 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] unwritten_sync 2024-05-04T03:21:27.338 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] genhashnames 2024-05-04T03:21:27.356 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_holes 2024-05-04T03:21:27.481 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mmap_writev 2024-05-04T03:21:27.502 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_truncate_cmtime 2024-05-04T03:21:27.509 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] dirhash_collide 2024-05-04T03:21:27.534 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_rename_overwrite 2024-05-04T03:21:27.624 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] holetest 2024-05-04T03:21:27.662 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_truncate_self 2024-05-04T03:21:27.677 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] af_unix 2024-05-04T03:21:27.714 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mmap_stale_pmd 2024-05-04T03:21:27.798 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mmap_cow_race 2024-05-04T03:21:27.834 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mmap_fallocate 2024-05-04T03:21:27.861 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fsync-err 2024-05-04T03:21:27.879 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mmap_write_ro 2024-05-04T03:21:27.973 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_ext4_dax_journal_corruption 2024-05-04T03:21:27.984 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_ext4_dax_inline_corruption 2024-05-04T03:21:28.034 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_ofd_locks 2024-05-04T03:21:28.074 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mmap_collision 2024-05-04T03:21:28.136 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] mmap-write-concurrent 2024-05-04T03:21:28.153 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_get_file_time 2024-05-04T03:21:28.245 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_create_short_dirs 2024-05-04T03:21:28.303 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_create_long_dirs 2024-05-04T03:21:28.304 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_enospc 2024-05-04T03:21:28.306 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mmap_writev_overlap 2024-05-04T03:21:28.426 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] checkpoint_journal 2024-05-04T03:21:28.468 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] mmap-rw-fault 2024-05-04T03:21:28.469 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] allocstale 2024-05-04T03:21:28.507 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mmap_cow_memory_failure 2024-05-04T03:21:28.593 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fake-dump-rootino 2024-05-04T03:21:28.641 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] dio-buf-fault 2024-05-04T03:21:28.650 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] rewinddir-test 2024-05-04T03:21:28.685 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] readdir-while-renames 2024-05-04T03:21:28.753 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] xfsctl 2024-05-04T03:21:28.803 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] bstat 2024-05-04T03:21:28.815 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mtab 2024-05-04T03:21:28.870 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] getdevicesize 2024-05-04T03:21:28.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:28 smithi001 ceph-mon[29731]: pgmap v460: 97 pgs: 97 active+clean; 47 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 385 op/s 2024-05-04T03:21:28.931 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:28 smithi077 ceph-mon[33531]: pgmap v460: 97 pgs: 97 active+clean; 47 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 385 op/s 2024-05-04T03:21:28.965 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] preallo_rw_pattern_reader 2024-05-04T03:21:28.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:28 smithi072 ceph-mon[33332]: pgmap v460: 97 pgs: 97 active+clean; 47 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 385 op/s 2024-05-04T03:21:29.005 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] preallo_rw_pattern_writer 2024-05-04T03:21:29.014 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] ftrunc 2024-05-04T03:21:29.028 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] trunc 2024-05-04T03:21:29.145 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fs_perms 2024-05-04T03:21:29.159 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] testx 2024-05-04T03:21:29.160 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] looptest 2024-05-04T03:21:29.210 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] locktest 2024-05-04T03:21:29.289 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] unwritten_mmap 2024-05-04T03:21:29.315 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] bulkstat_unlink_test 2024-05-04T03:21:29.352 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] deduperace 2024-05-04T03:21:29.464 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] bulkstat_unlink_test_modified 2024-05-04T03:21:29.505 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_dir_offset 2024-05-04T03:21:29.542 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_futimens 2024-05-04T03:21:29.559 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_immutable 2024-05-04T03:21:29.651 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] stale_handle 2024-05-04T03:21:29.678 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] pwrite_mmap_blocked 2024-05-04T03:21:29.693 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_dir_offset2 2024-05-04T03:21:29.818 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] seek_sanity_test 2024-05-04T03:21:29.832 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] seek_copy_test 2024-05-04T03:21:29.885 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_readdir_1 2024-05-04T03:21:30.014 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_readdir_2 2024-05-04T03:21:30.026 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fsync-tester 2024-05-04T03:21:30.157 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] nsexec 2024-05-04T03:21:30.273 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] cloner 2024-05-04T03:21:30.279 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] renameat2 2024-05-04T03:21:30.282 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_getcwd 2024-05-04T03:21:30.348 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] e4compact 2024-05-04T03:21:30.440 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] test-nextquota 2024-05-04T03:21:30.440 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] punch-alternating 2024-05-04T03:21:30.473 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] attr-list-by-handle-cursor-test 2024-05-04T03:21:30.588 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] listxattr 2024-05-04T03:21:30.611 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] dio-interleaved 2024-05-04T03:21:30.614 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_dir_type 2024-05-04T03:21:30.642 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] dio-invalidate-cache 2024-05-04T03:21:30.739 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] stat_test 2024-05-04T03:21:30.774 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_encrypted_d_revalidate 2024-05-04T03:21:30.789 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] attr_replace_test 2024-05-04T03:21:30.866 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] swapon 2024-05-04T03:21:30.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:30 smithi001 ceph-mon[29731]: pgmap v461: 97 pgs: 97 active+clean; 47 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 291 op/s 2024-05-04T03:21:30.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:30 smithi077 ceph-mon[33531]: pgmap v461: 97 pgs: 97 active+clean; 47 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 291 op/s 2024-05-04T03:21:30.941 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] mkswap 2024-05-04T03:21:30.952 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_attr_corruption 2024-05-04T03:21:30.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:30 smithi072 ceph-mon[33332]: pgmap v461: 97 pgs: 97 active+clean; 47 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 291 op/s 2024-05-04T03:21:31.040 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_open_tmpfiles 2024-05-04T03:21:31.055 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fscrypt-crypt-util 2024-05-04T03:21:31.099 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] bulkstat_null_ocount 2024-05-04T03:21:31.118 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] splice-test 2024-05-04T03:21:31.228 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] chprojid_fail 2024-05-04T03:21:31.254 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] detached_mounts_propagation 2024-05-04T03:21:31.299 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] ext4_resize 2024-05-04T03:21:31.396 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_readdir_3 2024-05-04T03:21:31.449 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] splice2pipe 2024-05-04T03:21:31.466 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] uuid_ioctl 2024-05-04T03:21:31.646 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_snapshot_deleted_subvolume 2024-05-04T03:21:31.650 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fiemap-fault 2024-05-04T03:21:31.652 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fiemap-tester 2024-05-04T03:21:31.815 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_stripealign 2024-05-04T03:21:31.818 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] open_by_handle 2024-05-04T03:21:31.819 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_mmap_dio 2024-05-04T03:21:31.950 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] xfsfind 2024-05-04T03:21:31.962 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] t_reflink_read_race 2024-05-04T03:21:31.971 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] fssum 2024-05-04T03:21:32.108 INFO:tasks.workunit.client.0.smithi001.stdout:Building log-writes 2024-05-04T03:21:32.114 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] replay-log 2024-05-04T03:21:32.165 INFO:tasks.workunit.client.0.smithi001.stdout:Building perf 2024-05-04T03:21:32.168 INFO:tasks.workunit.client.0.smithi001.stdout:Building aio-dio-regress 2024-05-04T03:21:32.169 INFO:tasks.workunit.client.0.smithi001.stdout:Building vfs 2024-05-04T03:21:32.175 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] vfstest 2024-05-04T03:21:32.175 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aiocp 2024-05-04T03:21:32.413 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-append-write-fallocate-race 2024-05-04T03:21:32.453 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] mount-idmapped 2024-05-04T03:21:32.454 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-append-write-read-race 2024-05-04T03:21:32.590 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-cow-race 2024-05-04T03:21:32.632 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-cycle-write 2024-05-04T03:21:32.756 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-eof-race 2024-05-04T03:21:32.817 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-extend-stat 2024-05-04T03:21:32.874 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-fcntl-race 2024-05-04T03:21:32.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:32 smithi001 ceph-mon[29731]: pgmap v462: 97 pgs: 97 active+clean; 47 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 485 KiB/s wr, 92 op/s 2024-05-04T03:21:32.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:32 smithi077 ceph-mon[33531]: pgmap v462: 97 pgs: 97 active+clean; 47 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 485 KiB/s wr, 92 op/s 2024-05-04T03:21:32.964 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-hole-filling-race 2024-05-04T03:21:32.979 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-invalidate-failure 2024-05-04T03:21:32.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:32 smithi072 ceph-mon[33332]: pgmap v462: 97 pgs: 97 active+clean; 47 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 485 KiB/s wr, 92 op/s 2024-05-04T03:21:33.047 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-invalidate-readahead 2024-05-04T03:21:33.117 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aiodio_sparse2 2024-05-04T03:21:33.165 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-subblock-eof-read 2024-05-04T03:21:33.251 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-dio-write-verify 2024-05-04T03:21:33.307 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-05-04T03:21:33.365 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-05-04T03:21:33.455 INFO:tasks.workunit.client.0.smithi001.stdout: [CC] aio-last-ref-held-by-io 2024-05-04T03:21:34.288 INFO:tasks.workunit.client.0.smithi001.stderr:+ cd .. 2024-05-04T03:21:34.288 INFO:tasks.workunit.client.0.smithi001.stderr:+ cp xfstests-dev/ltp/fsx . 2024-05-04T03:21:34.291 INFO:tasks.workunit.client.0.smithi001.stderr:+ OPTIONS=-z 2024-05-04T03:21:34.292 INFO:tasks.workunit.client.0.smithi001.stderr:+ ./fsx -z 1MB -N 50000 -p 10000 -l 1048576 2024-05-04T03:21:34.292 INFO:tasks.workunit.client.0.smithi001.stdout:Seed set to 1 2024-05-04T03:21:34.336 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-05-04T03:21:34.336 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-05-04T03:21:34.338 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-05-04T03:21:34.338 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-05-04T03:21:34.338 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support clone range, disabling! 2024-05-04T03:21:34.342 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support dedupe range, disabling! 2024-05-04T03:21:34.344 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support exchange range, disabling! 2024-05-04T03:21:34.344 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero size read 2024-05-04T03:21:34.345 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero size read 2024-05-04T03:21:34.345 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0x81d54 2024-05-04T03:21:34.350 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xb86d7 2024-05-04T03:21:34.358 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xe627e 2024-05-04T03:21:34.398 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0xf3194 2024-05-04T03:21:34.419 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xf3ceb 2024-05-04T03:21:34.426 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0xf47e4 2024-05-04T03:21:34.519 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xfaccc 2024-05-04T03:21:34.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:34 smithi001 ceph-mon[29731]: pgmap v463: 97 pgs: 97 active+clean; 47 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 288 KiB/s wr, 43 op/s 2024-05-04T03:21:34.905 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xfe423 2024-05-04T03:21:34.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:34 smithi077 ceph-mon[33531]: pgmap v463: 97 pgs: 97 active+clean; 47 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 288 KiB/s wr, 43 op/s 2024-05-04T03:21:34.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:34 smithi072 ceph-mon[33332]: pgmap v463: 97 pgs: 97 active+clean; 47 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 288 KiB/s wr, 43 op/s 2024-05-04T03:21:36.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:36 smithi001 ceph-mon[29731]: pgmap v464: 97 pgs: 97 active+clean; 48 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 323 KiB/s wr, 53 op/s 2024-05-04T03:21:36.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:36 smithi077 ceph-mon[33531]: pgmap v464: 97 pgs: 97 active+clean; 48 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 323 KiB/s wr, 53 op/s 2024-05-04T03:21:36.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:36 smithi072 ceph-mon[33332]: pgmap v464: 97 pgs: 97 active+clean; 48 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 323 KiB/s wr, 53 op/s 2024-05-04T03:21:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.014+0000 7f1d55934640 1 -- 172.21.15.1:0/1715139485 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d501043e0 msgr2=0x7f1d50106800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.014+0000 7f1d55934640 1 --2- 172.21.15.1:0/1715139485 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d501043e0 0x7f1d50106800 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1d48009900 tx=0x7f1d4802a410 comp rx=0 tx=0).stop 2024-05-04T03:21:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.014+0000 7f1d55934640 1 -- 172.21.15.1:0/1715139485 shutdown_connections 2024-05-04T03:21:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.014+0000 7f1d55934640 1 --2- 172.21.15.1:0/1715139485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d501098e0 0x7f1d5010bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.014+0000 7f1d55934640 1 --2- 172.21.15.1:0/1715139485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d50106dd0 0x7f1d50109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.014+0000 7f1d55934640 1 --2- 172.21.15.1:0/1715139485 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d501043e0 0x7f1d50106800 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.014+0000 7f1d55934640 1 -- 172.21.15.1:0/1715139485 >> 172.21.15.1:0/1715139485 conn(0x7f1d500fdd40 msgr2=0x7f1d501001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.014+0000 7f1d55934640 1 -- 172.21.15.1:0/1715139485 shutdown_connections 2024-05-04T03:21:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.014+0000 7f1d55934640 1 -- 172.21.15.1:0/1715139485 wait complete. 2024-05-04T03:21:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.015+0000 7f1d55934640 1 Processor -- start 2024-05-04T03:21:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.015+0000 7f1d55934640 1 -- start start 2024-05-04T03:21:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.015+0000 7f1d55934640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d501043e0 0x7f1d5019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.015+0000 7f1d55934640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d50106dd0 0x7f1d5019a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.015+0000 7f1d55934640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d501098e0 0x7f1d501a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.015+0000 7f1d55934640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5010f7a0 con 0x7f1d50106dd0 2024-05-04T03:21:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.015+0000 7f1d55934640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5010f620 con 0x7f1d501043e0 2024-05-04T03:21:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.015+0000 7f1d55934640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5010f920 con 0x7f1d501098e0 2024-05-04T03:21:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.015+0000 7f1d4e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d50106dd0 0x7f1d5019a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:37.019 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.015+0000 7f1d4effd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d501043e0 0x7f1d5019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.015+0000 7f1d4effd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d501043e0 0x7f1d5019a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:53534/0 (socket says 172.21.15.1:53534) 2024-05-04T03:21:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.015+0000 7f1d4e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d50106dd0 0x7f1d5019a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40874/0 (socket says 172.21.15.1:40874) 2024-05-04T03:21:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.015+0000 7f1d4effd640 1 -- 172.21.15.1:0/1324557479 learned_addr learned my addr 172.21.15.1:0/1324557479 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:21:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.015+0000 7f1d4f7fe640 1 --2- 172.21.15.1:0/1324557479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d501098e0 0x7f1d501a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.016+0000 7f1d4effd640 1 -- 172.21.15.1:0/1324557479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d501098e0 msgr2=0x7f1d501a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.016+0000 7f1d4effd640 1 --2- 172.21.15.1:0/1324557479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d501098e0 0x7f1d501a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.016+0000 7f1d4effd640 1 -- 172.21.15.1:0/1324557479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d50106dd0 msgr2=0x7f1d5019a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.016+0000 7f1d4effd640 1 --2- 172.21.15.1:0/1324557479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d50106dd0 0x7f1d5019a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.016+0000 7f1d4effd640 1 -- 172.21.15.1:0/1324557479 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d501a1dd0 con 0x7f1d501043e0 2024-05-04T03:21:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.016+0000 7f1d4effd640 1 --2- 172.21.15.1:0/1324557479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d501043e0 0x7f1d5019a290 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1d48031440 tx=0x7f1d48003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.016+0000 7f1d54932640 1 -- 172.21.15.1:0/1324557479 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4803f070 con 0x7f1d501043e0 2024-05-04T03:21:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.016+0000 7f1d54932640 1 -- 172.21.15.1:0/1324557479 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d48031c50 con 0x7f1d501043e0 2024-05-04T03:21:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.016+0000 7f1d55934640 1 -- 172.21.15.1:0/1324557479 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d501a2060 con 0x7f1d501043e0 2024-05-04T03:21:37.020 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.016+0000 7f1d54932640 1 -- 172.21.15.1:0/1324557479 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48035030 con 0x7f1d501043e0 2024-05-04T03:21:37.021 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.016+0000 7f1d55934640 1 -- 172.21.15.1:0/1324557479 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d501a25a0 con 0x7f1d501043e0 2024-05-04T03:21:37.022 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.020+0000 7f1d54932640 1 -- 172.21.15.1:0/1324557479 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f1d48042070 con 0x7f1d501043e0 2024-05-04T03:21:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.020+0000 7f1d5c7e7740 1 Processor -- start 2024-05-04T03:21:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.020+0000 7f1d5c7e7740 1 -- start start 2024-05-04T03:21:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.021+0000 7f1d5c7e7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a10a4bd1b0 0x55a10a85ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.021+0000 7f1d5c7e7740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a10a4942f0 0x55a10a85efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.021+0000 7f1d5c7e7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a10a49c4d0 0x55a10a865ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.021+0000 7f1d5c7e7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a10a7c35d0 con 0x55a10a49c4d0 2024-05-04T03:21:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.021+0000 7f1d5c7e7740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55a10a77c890 con 0x55a10a4942f0 2024-05-04T03:21:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.021+0000 7f1d5c7e7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a10a7bf590 con 0x55a10a4bd1b0 2024-05-04T03:21:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.021+0000 7f1d54932640 1 --2- 172.21.15.1:0/1324557479 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1d3c0b2920 0x7f1d3c0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.021+0000 7f1d4f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a10a4bd1b0 0x55a10a85ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.021+0000 7f1d4e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a10a49c4d0 0x55a10a865ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:37.024 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.021+0000 7f1d4f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a10a4bd1b0 0x55a10a85ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.1:50170/0 (socket says 172.21.15.1:50170) 2024-05-04T03:21:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.021+0000 7f1d4f7fe640 1 -- 172.21.15.1:0/1979017552 learned_addr learned my addr 172.21.15.1:0/1979017552 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:21:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.021+0000 7f1d4e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a10a49c4d0 0x55a10a865ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40882/0 (socket says 172.21.15.1:40882) 2024-05-04T03:21:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.021+0000 7f1d4effd640 1 --2- 172.21.15.1:0/1979017552 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a10a4942f0 0x55a10a85efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.021+0000 7f1d4f7fe640 1 --2- 172.21.15.1:0/1324557479 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1d3c0b2920 0x7f1d3c0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.021+0000 7f1d4f7fe640 1 -- 172.21.15.1:0/1979017552 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a10a4942f0 msgr2=0x55a10a85efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.021+0000 7f1d4f7fe640 1 --2- 172.21.15.1:0/1979017552 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a10a4942f0 0x55a10a85efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.021+0000 7f1d4f7fe640 1 -- 172.21.15.1:0/1979017552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a10a49c4d0 msgr2=0x55a10a865ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.021+0000 7f1d4f7fe640 1 --2- 172.21.15.1:0/1979017552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a10a49c4d0 0x55a10a865ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.021+0000 7f1d4f7fe640 1 -- 172.21.15.1:0/1979017552 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a10a4d5210 con 0x55a10a4bd1b0 2024-05-04T03:21:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.021+0000 7f1d54932640 1 -- 172.21.15.1:0/1324557479 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f1d481027f0 con 0x7f1d501043e0 2024-05-04T03:21:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.022+0000 7f1d4f7fe640 1 --2- 172.21.15.1:0/1979017552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a10a4bd1b0 0x55a10a85ea60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1d40005c40 tx=0x7f1d4000f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4389b0bd7b3f5b27 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.022+0000 7f1d45ffb640 1 -- 172.21.15.1:0/1979017552 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d40022020 con 0x55a10a4bd1b0 2024-05-04T03:21:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.022+0000 7f1d4f7fe640 1 --2- 172.21.15.1:0/1324557479 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1d3c0b2920 0x7f1d3c0b4e30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1d40023000 tx=0x7f1d40021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.022+0000 7f1d45ffb640 1 -- 172.21.15.1:0/1979017552 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d40013170 con 0x55a10a4bd1b0 2024-05-04T03:21:37.025 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.022+0000 7f1d45ffb640 1 -- 172.21.15.1:0/1979017552 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d400107d0 con 0x55a10a4bd1b0 2024-05-04T03:21:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.023+0000 7f1d5c7e7740 1 -- 172.21.15.1:0/1979017552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a10a4bd1b0 msgr2=0x55a10a85ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.023+0000 7f1d5c7e7740 1 --2- 172.21.15.1:0/1979017552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a10a4bd1b0 0x55a10a85ea60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1d40005c40 tx=0x7f1d4000f3c0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.023+0000 7f1d5c7e7740 1 -- 172.21.15.1:0/1979017552 shutdown_connections 2024-05-04T03:21:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.023+0000 7f1d5c7e7740 1 --2- 172.21.15.1:0/1979017552 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a10a4942f0 0x55a10a85efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.023+0000 7f1d5c7e7740 1 --2- 172.21.15.1:0/1979017552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a10a49c4d0 0x55a10a865ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.023+0000 7f1d5c7e7740 1 --2- 172.21.15.1:0/1979017552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a10a4bd1b0 0x55a10a85ea60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.023+0000 7f1d5c7e7740 1 -- 172.21.15.1:0/1979017552 >> 172.21.15.1:0/1979017552 conn(0x55a10a4b1300 msgr2=0x55a10a85a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.023+0000 7f1d5c7e7740 1 -- 172.21.15.1:0/1979017552 shutdown_connections 2024-05-04T03:21:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.023+0000 7f1d5c7e7740 1 -- 172.21.15.1:0/1979017552 wait complete. 2024-05-04T03:21:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.023+0000 7f1d5c7e7740 1 -- start start 2024-05-04T03:21:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.024+0000 7f1d5c7e7740 1 Processor -- start 2024-05-04T03:21:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.024+0000 7f1d5c7e7740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a10a4942f0 0x55a10a8686e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.024+0000 7f1d5c7e7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a10a49c4d0 0x55a10a868c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.024+0000 7f1d5c7e7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a10a4bd1b0 0x55a10a86fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.024+0000 7f1d5c7e7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a10a866930 con 0x55a10a4bd1b0 2024-05-04T03:21:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.024+0000 7f1d5c7e7740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55a10a780370 con 0x55a10a4942f0 2024-05-04T03:21:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.024+0000 7f1d5c7e7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a10a866ab0 con 0x55a10a49c4d0 2024-05-04T03:21:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.024+0000 7f1d5c7e7740 10 client.0 _ll_register_callbacks cb 0x55a10a4d4a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:21:37.032 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.024+0000 7f1d4effd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a10a4942f0 0x55a10a8686e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.024+0000 7f1d4e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a10a49c4d0 0x55a10a868c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.024+0000 7f1d4f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a10a4bd1b0 0x55a10a86fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.024+0000 7f1d4effd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a10a4942f0 0x55a10a8686e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:53552/0 (socket says 172.21.15.1:53552) 2024-05-04T03:21:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.024+0000 7f1d4e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a10a49c4d0 0x55a10a868c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.1:50186/0 (socket says 172.21.15.1:50186) 2024-05-04T03:21:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.024+0000 7f1d4effd640 1 -- 172.21.15.1:0/3119290239 learned_addr learned my addr 172.21.15.1:0/3119290239 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:21:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.024+0000 7f1d4f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a10a4bd1b0 0x55a10a86fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40896/0 (socket says 172.21.15.1:40896) 2024-05-04T03:21:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.024+0000 7f1d4effd640 1 -- 172.21.15.1:0/3119290239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a10a49c4d0 msgr2=0x55a10a868c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.024+0000 7f1d4effd640 1 --2- 172.21.15.1:0/3119290239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a10a49c4d0 0x55a10a868c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.024+0000 7f1d4effd640 1 -- 172.21.15.1:0/3119290239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a10a4bd1b0 msgr2=0x55a10a86fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.024+0000 7f1d4effd640 1 --2- 172.21.15.1:0/3119290239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a10a4bd1b0 0x55a10a86fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.024+0000 7f1d4effd640 1 -- 172.21.15.1:0/3119290239 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a10a5a1620 con 0x55a10a4942f0 2024-05-04T03:21:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.025+0000 7f1d4effd640 1 --2- 172.21.15.1:0/3119290239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a10a4942f0 0x55a10a8686e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1d480c9f10 tx=0x7f1d48005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4754983c685c46c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.025+0000 7f1d2effd640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-04T03:21:37.033 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.025+0000 7f1d2effd640 1 -- 172.21.15.1:0/3119290239 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48049040 con 0x55a10a4942f0 2024-05-04T03:21:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.025+0000 7f1d2effd640 1 -- 172.21.15.1:0/3119290239 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f1d480c6210 con 0x55a10a4942f0 2024-05-04T03:21:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.025+0000 7f1d5c7e7740 1 -- 172.21.15.1:0/3119290239 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d40013170 con 0x55a10a4942f0 2024-05-04T03:21:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.025+0000 7f1d2effd640 1 -- 172.21.15.1:0/3119290239 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1d480b1e60 con 0x55a10a4942f0 2024-05-04T03:21:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.025+0000 7f1d5c7e7740 10 client.25268 fetch_fsmap learned FSMap version 29 2024-05-04T03:21:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.025+0000 7f1d5c7e7740 1 -- 172.21.15.1:0/3119290239 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a10a872ab0 con 0x55a10a4942f0 2024-05-04T03:21:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.025+0000 7f1d2effd640 1 -- 172.21.15.1:0/3119290239 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d40013170 con 0x55a10a4942f0 2024-05-04T03:21:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.025+0000 7f1d2effd640 1 -- 172.21.15.1:0/3119290239 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f1d480ca1e0 con 0x55a10a4942f0 2024-05-04T03:21:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.025+0000 7f1d5c7e7740 10 client.25268 fetch_fsmap finished waiting for FSMap version 29 2024-05-04T03:21:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.025+0000 7f1d5c7e7740 10 client.25268 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:21:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.025+0000 7f1d5c7e7740 20 client.25268 populate_metadata read hostname 'smithi001' 2024-05-04T03:21:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.025+0000 7f1d5c7e7740 1 --2- 172.21.15.1:0/3119290239 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a10a47bd80 0x55a10a876200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.025+0000 7f1d5c7e7740 4 client.25268 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:21:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.025+0000 7f1d5c7e7740 1 -- 172.21.15.1:0/3119290239 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a10a7d11e0 con 0x55a10a47bd80 2024-05-04T03:21:37.034 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.026+0000 7f1d4e7fc640 1 --2- 172.21.15.1:0/3119290239 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a10a47bd80 0x55a10a876200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.026+0000 7f1d2effd640 10 client.25268 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:21:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.026+0000 7f1d4e7fc640 1 --2- 172.21.15.1:0/3119290239 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a10a47bd80 0x55a10a876200 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85e4dccad126cd9 server_cookie=331eb879ff487900 in_seq=0 out_seq=0 2024-05-04T03:21:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.027+0000 7f1d2effd640 1 -- 172.21.15.1:0/3119290239 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a10a7d11e0 con 0x55a10a47bd80 2024-05-04T03:21:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.027+0000 7f1d2effd640 10 client.25268 handle_command_reply: tid=0 2024-05-04T03:21:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.027+0000 7f1d5c7e7740 1 client.25268 shutdown 2024-05-04T03:21:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.027+0000 7f1d5c7e7740 1 -- 172.21.15.1:0/3119290239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a10a4942f0 msgr2=0x55a10a8686e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.027+0000 7f1d5c7e7740 1 --2- 172.21.15.1:0/3119290239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a10a4942f0 0x55a10a8686e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1d480c9f10 tx=0x7f1d48005c90 comp rx=0 tx=0).stop 2024-05-04T03:21:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.027+0000 7f1d5c7e7740 1 -- 172.21.15.1:0/3119290239 shutdown_connections 2024-05-04T03:21:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.027+0000 7f1d5c7e7740 1 --2- 172.21.15.1:0/3119290239 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a10a47bd80 0x55a10a876200 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.027+0000 7f1d5c7e7740 1 --2- 172.21.15.1:0/3119290239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a10a4bd1b0 0x55a10a86fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.027+0000 7f1d2effd640 0 client.25268 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:21:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.027+0000 7f1d5c7e7740 1 --2- 172.21.15.1:0/3119290239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a10a49c4d0 0x55a10a868c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.027+0000 7f1d5c7e7740 1 --2- 172.21.15.1:0/3119290239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a10a4942f0 0x55a10a8686e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.027+0000 7f1d5c7e7740 1 -- 172.21.15.1:0/3119290239 >> 172.21.15.1:0/3119290239 conn(0x55a10a4b1300 msgr2=0x55a10a85d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:37.035 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.027+0000 7f1d5c7e7740 1 -- 172.21.15.1:0/3119290239 shutdown_connections 2024-05-04T03:21:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.027+0000 7f1d5c7e7740 1 -- 172.21.15.1:0/3119290239 wait complete. 2024-05-04T03:21:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.027+0000 7f1d5c7e7740 20 client.25268 trim_cache size 0 max 16384 2024-05-04T03:21:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.043+0000 7f1d5c7e7740 1 Processor -- start 2024-05-04T03:21:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.043+0000 7f1d5c7e7740 1 -- start start 2024-05-04T03:21:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.043+0000 7f1d5c7e7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a10a47bd80 0x55a10a859cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.043+0000 7f1d5c7e7740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a10a4942f0 0x55a10a85a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.043+0000 7f1d5c7e7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a10a49c4d0 0x55a10a85d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.043+0000 7f1d5c7e7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1d501ada40 con 0x55a10a47bd80 2024-05-04T03:21:37.046 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.043+0000 7f1d5c7e7740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1d501ab2b0 con 0x55a10a4942f0 2024-05-04T03:21:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.043+0000 7f1d5c7e7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1d501a9100 con 0x55a10a49c4d0 2024-05-04T03:21:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.043+0000 7f1d4effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a10a47bd80 0x55a10a859cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.043+0000 7f1d4effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a10a47bd80 0x55a10a859cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40910/0 (socket says 172.21.15.1:40910) 2024-05-04T03:21:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.043+0000 7f1d4effd640 1 -- 172.21.15.1:0/1557650149 learned_addr learned my addr 172.21.15.1:0/1557650149 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:21:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.043+0000 7f1d4f7fe640 1 --2- 172.21.15.1:0/1557650149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a10a49c4d0 0x55a10a85d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.043+0000 7f1d4e7fc640 1 --2- 172.21.15.1:0/1557650149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a10a4942f0 0x55a10a85a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.044+0000 7f1d4effd640 1 -- 172.21.15.1:0/1557650149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a10a49c4d0 msgr2=0x55a10a85d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.044+0000 7f1d4effd640 1 --2- 172.21.15.1:0/1557650149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a10a49c4d0 0x55a10a85d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.044+0000 7f1d4effd640 1 -- 172.21.15.1:0/1557650149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a10a4942f0 msgr2=0x55a10a85a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.044+0000 7f1d4effd640 1 --2- 172.21.15.1:0/1557650149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a10a4942f0 0x55a10a85a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.044+0000 7f1d4effd640 1 -- 172.21.15.1:0/1557650149 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a10a85f860 con 0x55a10a47bd80 2024-05-04T03:21:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.044+0000 7f1d4effd640 1 --2- 172.21.15.1:0/1557650149 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a10a47bd80 0x55a10a859cc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1d480c6000 tx=0x7f1d4803b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad17d00b3876b84 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.044+0000 7f1d457fa640 1 -- 172.21.15.1:0/1557650149 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48049070 con 0x55a10a47bd80 2024-05-04T03:21:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.044+0000 7f1d457fa640 1 -- 172.21.15.1:0/1557650149 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d480ec040 con 0x55a10a47bd80 2024-05-04T03:21:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.044+0000 7f1d457fa640 1 -- 172.21.15.1:0/1557650149 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d480c6210 con 0x55a10a47bd80 2024-05-04T03:21:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.045+0000 7f1d5c7e7740 1 -- 172.21.15.1:0/1557650149 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a10a47bd80 msgr2=0x55a10a859cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.045+0000 7f1d5c7e7740 1 --2- 172.21.15.1:0/1557650149 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a10a47bd80 0x55a10a859cc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1d480c6000 tx=0x7f1d4803b790 comp rx=0 tx=0).stop 2024-05-04T03:21:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.045+0000 7f1d5c7e7740 1 -- 172.21.15.1:0/1557650149 shutdown_connections 2024-05-04T03:21:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.045+0000 7f1d5c7e7740 1 --2- 172.21.15.1:0/1557650149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a10a4942f0 0x55a10a85a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.045+0000 7f1d5c7e7740 1 --2- 172.21.15.1:0/1557650149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a10a49c4d0 0x55a10a85d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.045+0000 7f1d5c7e7740 1 --2- 172.21.15.1:0/1557650149 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a10a47bd80 0x55a10a859cc0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.051 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.045+0000 7f1d5c7e7740 1 -- 172.21.15.1:0/1557650149 >> 172.21.15.1:0/1557650149 conn(0x55a10a4b1300 msgr2=0x55a10a862b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.045+0000 7f1d5c7e7740 1 -- 172.21.15.1:0/1557650149 shutdown_connections 2024-05-04T03:21:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.045+0000 7f1d5c7e7740 1 -- 172.21.15.1:0/1557650149 wait complete. 2024-05-04T03:21:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.045+0000 7f1d5c7e7740 1 -- start start 2024-05-04T03:21:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.046+0000 7f1d5c7e7740 1 Processor -- start 2024-05-04T03:21:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.046+0000 7f1d5c7e7740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a10a47bd80 0x55a10a86c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.046+0000 7f1d5c7e7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a10a4942f0 0x55a10a86ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.046+0000 7f1d5c7e7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a10a49c4d0 0x55a10a86dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.046+0000 7f1d5c7e7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1d501a9770 con 0x55a10a4942f0 2024-05-04T03:21:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.046+0000 7f1d5c7e7740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1d501aabe0 con 0x55a10a47bd80 2024-05-04T03:21:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.046+0000 7f1d5c7e7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1d501a9e40 con 0x55a10a49c4d0 2024-05-04T03:21:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.046+0000 7f1d5c7e7740 10 client.0 _ll_register_callbacks cb 0x7f1d480c66a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:21:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.046+0000 7f1d4e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a10a4942f0 0x55a10a86ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.046+0000 7f1d4e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a10a4942f0 0x55a10a86ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40918/0 (socket says 172.21.15.1:40918) 2024-05-04T03:21:37.052 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.046+0000 7f1d4e7fc640 1 -- 172.21.15.1:0/1601754804 learned_addr learned my addr 172.21.15.1:0/1601754804 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:21:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.046+0000 7f1d4f7fe640 1 --2- 172.21.15.1:0/1601754804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a10a49c4d0 0x55a10a86dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.046+0000 7f1d4effd640 1 --2- 172.21.15.1:0/1601754804 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a10a47bd80 0x55a10a86c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.046+0000 7f1d4f7fe640 1 -- 172.21.15.1:0/1601754804 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a10a47bd80 msgr2=0x55a10a86c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.046+0000 7f1d4f7fe640 1 --2- 172.21.15.1:0/1601754804 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a10a47bd80 0x55a10a86c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.046+0000 7f1d4f7fe640 1 -- 172.21.15.1:0/1601754804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a10a4942f0 msgr2=0x55a10a86ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.046+0000 7f1d4f7fe640 1 --2- 172.21.15.1:0/1601754804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a10a4942f0 0x55a10a86ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.046+0000 7f1d4f7fe640 1 -- 172.21.15.1:0/1601754804 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a10a8662a0 con 0x55a10a49c4d0 2024-05-04T03:21:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.046+0000 7f1d4e7fc640 1 --2- 172.21.15.1:0/1601754804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a10a4942f0 0x55a10a86ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:21:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.047+0000 7f1d4f7fe640 1 --2- 172.21.15.1:0/1601754804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a10a49c4d0 0x55a10a86dcc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000a010 tx=0x7f1d4000ffc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbc903c9e50ee853 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.047+0000 7f1d2effd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-04T03:21:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.047+0000 7f1d2effd640 1 -- 172.21.15.1:0/1601754804 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d40022030 con 0x55a10a49c4d0 2024-05-04T03:21:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.047+0000 7f1d5c7e7740 1 -- 172.21.15.1:0/1601754804 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d480ec040 con 0x55a10a49c4d0 2024-05-04T03:21:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.047+0000 7f1d2effd640 1 -- 172.21.15.1:0/1601754804 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f1d40005250 con 0x55a10a49c4d0 2024-05-04T03:21:37.053 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.047+0000 7f1d2effd640 1 -- 172.21.15.1:0/1601754804 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1d40009c00 con 0x55a10a49c4d0 2024-05-04T03:21:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.047+0000 7f1d5c7e7740 10 client.25234 fetch_fsmap learned FSMap version 29 2024-05-04T03:21:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.047+0000 7f1d5c7e7740 1 -- 172.21.15.1:0/1601754804 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a10a86f3b0 con 0x55a10a49c4d0 2024-05-04T03:21:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.047+0000 7f1d2effd640 1 -- 172.21.15.1:0/1601754804 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d480ec040 con 0x55a10a49c4d0 2024-05-04T03:21:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.047+0000 7f1d2effd640 1 -- 172.21.15.1:0/1601754804 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f1d40010630 con 0x55a10a49c4d0 2024-05-04T03:21:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.047+0000 7f1d5c7e7740 10 client.25234 fetch_fsmap finished waiting for FSMap version 29 2024-05-04T03:21:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.047+0000 7f1d5c7e7740 10 client.25234 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:21:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.047+0000 7f1d5c7e7740 20 client.25234 populate_metadata read hostname 'smithi001' 2024-05-04T03:21:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.047+0000 7f1d5c7e7740 1 --2- 172.21.15.1:0/1601754804 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a10a4bd1b0 0x55a10a8e0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.048+0000 7f1d5c7e7740 4 client.25234 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:21:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.048+0000 7f1d5c7e7740 1 -- 172.21.15.1:0/1601754804 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a10a7cb050 con 0x55a10a4bd1b0 2024-05-04T03:21:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.048+0000 7f1d4effd640 1 --2- 172.21.15.1:0/1601754804 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a10a4bd1b0 0x55a10a8e0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.048+0000 7f1d2effd640 10 client.25234 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:21:37.054 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.048+0000 7f1d4effd640 1 --2- 172.21.15.1:0/1601754804 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a10a4bd1b0 0x55a10a8e0ba0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5df2d102694edf5 server_cookie=1f8a69927208ddda in_seq=0 out_seq=0 2024-05-04T03:21:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.055+0000 7f1d2effd640 1 -- 172.21.15.1:0/1601754804 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a10a7cb050 con 0x55a10a4bd1b0 2024-05-04T03:21:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.055+0000 7f1d2effd640 10 client.25234 handle_command_reply: tid=0 2024-05-04T03:21:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.056+0000 7f1d5c7e7740 1 client.25234 shutdown 2024-05-04T03:21:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.056+0000 7f1d5c7e7740 1 -- 172.21.15.1:0/1601754804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a10a49c4d0 msgr2=0x55a10a86dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.056+0000 7f1d5c7e7740 1 --2- 172.21.15.1:0/1601754804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a10a49c4d0 0x55a10a86dcc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000a010 tx=0x7f1d4000ffc0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.056+0000 7f1d5c7e7740 1 -- 172.21.15.1:0/1601754804 shutdown_connections 2024-05-04T03:21:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.056+0000 7f1d5c7e7740 1 --2- 172.21.15.1:0/1601754804 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a10a4bd1b0 0x55a10a8e0ba0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.056+0000 7f1d5c7e7740 1 --2- 172.21.15.1:0/1601754804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a10a49c4d0 0x55a10a86dcc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.056+0000 7f1d2effd640 0 client.25234 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:21:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.056+0000 7f1d5c7e7740 1 --2- 172.21.15.1:0/1601754804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a10a4942f0 0x55a10a86ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.056+0000 7f1d5c7e7740 1 --2- 172.21.15.1:0/1601754804 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a10a47bd80 0x55a10a86c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.056+0000 7f1d5c7e7740 1 -- 172.21.15.1:0/1601754804 >> 172.21.15.1:0/1601754804 conn(0x55a10a4b1300 msgr2=0x55a10a85d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.056+0000 7f1d5c7e7740 1 -- 172.21.15.1:0/1601754804 shutdown_connections 2024-05-04T03:21:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.056+0000 7f1d5c7e7740 1 -- 172.21.15.1:0/1601754804 wait complete. 2024-05-04T03:21:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.056+0000 7f1d5c7e7740 20 client.25234 trim_cache size 0 max 16384 2024-05-04T03:21:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:21:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:21:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:21:37.059 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:21:37.060 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.057+0000 7f1d55934640 1 -- 172.21.15.1:0/1324557479 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1d3c0b2920 msgr2=0x7f1d3c0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:37.060 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.057+0000 7f1d55934640 1 --2- 172.21.15.1:0/1324557479 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1d3c0b2920 0x7f1d3c0b4e30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1d40023000 tx=0x7f1d40021000 comp rx=0 tx=0).stop 2024-05-04T03:21:37.060 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.057+0000 7f1d55934640 1 -- 172.21.15.1:0/1324557479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d501043e0 msgr2=0x7f1d5019a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:37.060 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.057+0000 7f1d55934640 1 --2- 172.21.15.1:0/1324557479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d501043e0 0x7f1d5019a290 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1d48031440 tx=0x7f1d48003c10 comp rx=0 tx=0).stop 2024-05-04T03:21:37.060 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.057+0000 7f1d55934640 1 -- 172.21.15.1:0/1324557479 shutdown_connections 2024-05-04T03:21:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.057+0000 7f1d55934640 1 --2- 172.21.15.1:0/1324557479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d501098e0 0x7f1d501a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.057+0000 7f1d55934640 1 --2- 172.21.15.1:0/1324557479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d50106dd0 0x7f1d5019a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.057+0000 7f1d55934640 1 --2- 172.21.15.1:0/1324557479 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1d3c0b2920 0x7f1d3c0b4e30 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.057+0000 7f1d55934640 1 --2- 172.21.15.1:0/1324557479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1d501043e0 0x7f1d5019a290 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.057+0000 7f1d55934640 1 -- 172.21.15.1:0/1324557479 >> 172.21.15.1:0/1324557479 conn(0x7f1d500fdd40 msgr2=0x7f1d50104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.057+0000 7f1d55934640 1 -- 172.21.15.1:0/1324557479 shutdown_connections 2024-05-04T03:21:37.061 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:37.057+0000 7f1d55934640 1 -- 172.21.15.1:0/1324557479 wait complete. 2024-05-04T03:21:37.243 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xfe492 2024-05-04T03:21:38.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:38 smithi001 ceph-mon[29731]: pgmap v465: 97 pgs: 97 active+clean; 48 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 193 KiB/s wr, 38 op/s 2024-05-04T03:21:38.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:38 smithi077 ceph-mon[33531]: pgmap v465: 97 pgs: 97 active+clean; 48 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 193 KiB/s wr, 38 op/s 2024-05-04T03:21:38.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:38 smithi072 ceph-mon[33332]: pgmap v465: 97 pgs: 97 active+clean; 48 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 193 KiB/s wr, 38 op/s 2024-05-04T03:21:39.169 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xff038 2024-05-04T03:21:40.029 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0xffcac 2024-05-04T03:21:40.147 INFO:tasks.workunit.client.0.smithi001.stdout:10000 copy from 0x2d237 to 0x30552, (0x331b bytes) at 0x82aed 2024-05-04T03:21:40.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:40 smithi001 ceph-mon[29731]: pgmap v466: 97 pgs: 97 active+clean; 50 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 547 KiB/s wr, 154 op/s 2024-05-04T03:21:40.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:40 smithi077 ceph-mon[33531]: pgmap v466: 97 pgs: 97 active+clean; 50 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 547 KiB/s wr, 154 op/s 2024-05-04T03:21:40.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:40 smithi072 ceph-mon[33332]: pgmap v466: 97 pgs: 97 active+clean; 50 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 547 KiB/s wr, 154 op/s 2024-05-04T03:21:42.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:42 smithi001 ceph-mon[29731]: pgmap v467: 97 pgs: 97 active+clean; 51 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 356 op/s 2024-05-04T03:21:42.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:42 smithi077 ceph-mon[33531]: pgmap v467: 97 pgs: 97 active+clean; 51 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 356 op/s 2024-05-04T03:21:42.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:42 smithi072 ceph-mon[33332]: pgmap v467: 97 pgs: 97 active+clean; 51 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 356 op/s 2024-05-04T03:21:44.798 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:44 smithi072 ceph-mon[33332]: pgmap v468: 97 pgs: 97 active+clean; 51 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 357 op/s 2024-05-04T03:21:44.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:44 smithi001 ceph-mon[29731]: pgmap v468: 97 pgs: 97 active+clean; 51 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 357 op/s 2024-05-04T03:21:44.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:44 smithi077 ceph-mon[33531]: pgmap v468: 97 pgs: 97 active+clean; 51 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 357 op/s 2024-05-04T03:21:46.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:46 smithi001 ceph-mon[29731]: pgmap v469: 97 pgs: 97 active+clean; 54 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 662 op/s 2024-05-04T03:21:46.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:46 smithi077 ceph-mon[33531]: pgmap v469: 97 pgs: 97 active+clean; 54 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 662 op/s 2024-05-04T03:21:46.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:46 smithi072 ceph-mon[33332]: pgmap v469: 97 pgs: 97 active+clean; 54 MiB data, 640 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 662 op/s 2024-05-04T03:21:48.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:48 smithi001 ceph-mon[29731]: pgmap v470: 97 pgs: 97 active+clean; 54 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.1 MiB/s wr, 642 op/s 2024-05-04T03:21:48.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:48 smithi077 ceph-mon[33531]: pgmap v470: 97 pgs: 97 active+clean; 54 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.1 MiB/s wr, 642 op/s 2024-05-04T03:21:48.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:48 smithi072 ceph-mon[33332]: pgmap v470: 97 pgs: 97 active+clean; 54 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.1 MiB/s wr, 642 op/s 2024-05-04T03:21:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.187+0000 7f2646b72640 1 -- 172.21.15.1:0/262237542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26401121f0 msgr2=0x7f2640114690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.187+0000 7f2646b72640 1 --2- 172.21.15.1:0/262237542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26401121f0 0x7f2640114690 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2638009ca0 tx=0x7f263802a8f0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.188+0000 7f2646b72640 1 -- 172.21.15.1:0/262237542 shutdown_connections 2024-05-04T03:21:50.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.188+0000 7f2646b72640 1 --2- 172.21.15.1:0/262237542 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f264010f890 0x7f2640111cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.188+0000 7f2646b72640 1 --2- 172.21.15.1:0/262237542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26401121f0 0x7f2640114690 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.190 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.188+0000 7f2646b72640 1 --2- 172.21.15.1:0/262237542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f264010a0f0 0x7f264010a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.188+0000 7f2646b72640 1 -- 172.21.15.1:0/262237542 >> 172.21.15.1:0/262237542 conn(0x7f2640070640 msgr2=0x7f2640072af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.189+0000 7f2646b72640 1 -- 172.21.15.1:0/262237542 shutdown_connections 2024-05-04T03:21:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.191+0000 7f2646b72640 1 -- 172.21.15.1:0/262237542 wait complete. 2024-05-04T03:21:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.192+0000 7f2646b72640 1 Processor -- start 2024-05-04T03:21:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.192+0000 7f2646b72640 1 -- start start 2024-05-04T03:21:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.192+0000 7f2646b72640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f264010a0f0 0x7f26401a9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.192+0000 7f2646b72640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f264010f890 0x7f26401a9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.192+0000 7f2646b72640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f26401121f0 0x7f26401b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.192+0000 7f2646b72640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f264011e640 con 0x7f26401121f0 2024-05-04T03:21:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.192+0000 7f2646b72640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f264011e4c0 con 0x7f264010f890 2024-05-04T03:21:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.192+0000 7f2646b72640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f264011e7c0 con 0x7f264010a0f0 2024-05-04T03:21:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.193+0000 7f264490e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f264010a0f0 0x7f26401a9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.193+0000 7f264490e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f264010a0f0 0x7f26401a9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.1:50660/0 (socket says 172.21.15.1:50660) 2024-05-04T03:21:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.193+0000 7f264490e640 1 -- 172.21.15.1:0/3514393386 learned_addr learned my addr 172.21.15.1:0/3514393386 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:21:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.193+0000 7f264510f640 1 --2- 172.21.15.1:0/3514393386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f26401121f0 0x7f26401b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.193+0000 7f263ffff640 1 --2- 172.21.15.1:0/3514393386 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f264010f890 0x7f26401a9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.193+0000 7f264490e640 1 -- 172.21.15.1:0/3514393386 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f264010f890 msgr2=0x7f26401a9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.193+0000 7f264490e640 1 --2- 172.21.15.1:0/3514393386 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f264010f890 0x7f26401a9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.193+0000 7f264490e640 1 -- 172.21.15.1:0/3514393386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f26401121f0 msgr2=0x7f26401b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.193+0000 7f264490e640 1 --2- 172.21.15.1:0/3514393386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f26401121f0 0x7f26401b05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.193+0000 7f264490e640 1 -- 172.21.15.1:0/3514393386 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26401b0c50 con 0x7f264010a0f0 2024-05-04T03:21:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.193+0000 7f264490e640 1 --2- 172.21.15.1:0/3514393386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f264010a0f0 0x7f26401a9110 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f263000d9b0 tx=0x7f263000de70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.194+0000 7f263dffb640 1 -- 172.21.15.1:0/3514393386 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263001a070 con 0x7f264010a0f0 2024-05-04T03:21:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.194+0000 7f263dffb640 1 -- 172.21.15.1:0/3514393386 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2630015410 con 0x7f264010a0f0 2024-05-04T03:21:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.194+0000 7f2646b72640 1 -- 172.21.15.1:0/3514393386 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26401b0f40 con 0x7f264010a0f0 2024-05-04T03:21:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.194+0000 7f263dffb640 1 -- 172.21.15.1:0/3514393386 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2630014610 con 0x7f264010a0f0 2024-05-04T03:21:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.194+0000 7f2646b72640 1 -- 172.21.15.1:0/3514393386 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26401b1350 con 0x7f264010a0f0 2024-05-04T03:21:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.197+0000 7f264da29740 1 Processor -- start 2024-05-04T03:21:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.198+0000 7f264da29740 1 -- start start 2024-05-04T03:21:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.198+0000 7f263dffb640 1 -- 172.21.15.1:0/3514393386 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f263001d070 con 0x7f264010a0f0 2024-05-04T03:21:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.198+0000 7f264da29740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55846deee1b0 0x55846e28fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.198+0000 7f264da29740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55846dec52f0 0x55846e28ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.198+0000 7f264da29740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55846decd4d0 0x55846e296ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.198+0000 7f264da29740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55846e1f45d0 con 0x55846dec52f0 2024-05-04T03:21:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.198+0000 7f264da29740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55846e1ad890 con 0x55846deee1b0 2024-05-04T03:21:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.198+0000 7f264da29740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55846e1f0590 con 0x55846decd4d0 2024-05-04T03:21:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.198+0000 7f263ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55846decd4d0 0x55846e296ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.198+0000 7f263ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55846decd4d0 0x55846e296ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.1:50676/0 (socket says 172.21.15.1:50676) 2024-05-04T03:21:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.198+0000 7f263ffff640 1 -- 172.21.15.1:0/3482081865 learned_addr learned my addr 172.21.15.1:0/3482081865 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:21:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.198+0000 7f264510f640 1 --2- 172.21.15.1:0/3482081865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55846deee1b0 0x55846e28fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.198+0000 7f264490e640 1 --2- 172.21.15.1:0/3482081865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55846dec52f0 0x55846e28ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.199+0000 7f263ffff640 1 -- 172.21.15.1:0/3482081865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55846deee1b0 msgr2=0x55846e28fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.199+0000 7f263ffff640 1 --2- 172.21.15.1:0/3482081865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55846deee1b0 0x55846e28fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.199+0000 7f263ffff640 1 -- 172.21.15.1:0/3482081865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55846dec52f0 msgr2=0x55846e28ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.199+0000 7f263ffff640 1 --2- 172.21.15.1:0/3482081865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55846dec52f0 0x55846e28ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.199+0000 7f263ffff640 1 -- 172.21.15.1:0/3482081865 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55846df06210 con 0x55846decd4d0 2024-05-04T03:21:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.199+0000 7f263ffff640 1 --2- 172.21.15.1:0/3482081865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55846decd4d0 0x55846e296ee0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f263400bc30 tx=0x7f26340107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad85945583876065 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.199+0000 7f263dffb640 1 --2- 172.21.15.1:0/3514393386 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f26280b2970 0x7f26280b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.199+0000 7f2626ffd640 1 -- 172.21.15.1:0/3482081865 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263401c480 con 0x55846decd4d0 2024-05-04T03:21:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.199+0000 7f264510f640 1 --2- 172.21.15.1:0/3514393386 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f26280b2970 0x7f26280b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.199+0000 7f2626ffd640 1 -- 172.21.15.1:0/3482081865 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f263401cad0 con 0x55846decd4d0 2024-05-04T03:21:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.199+0000 7f2626ffd640 1 -- 172.21.15.1:0/3482081865 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2634011bc0 con 0x55846decd4d0 2024-05-04T03:21:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.199+0000 7f263dffb640 1 -- 172.21.15.1:0/3514393386 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f26300e1690 con 0x7f264010a0f0 2024-05-04T03:21:50.208 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.200+0000 7f264da29740 1 -- 172.21.15.1:0/3482081865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55846decd4d0 msgr2=0x55846e296ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.200+0000 7f264da29740 1 --2- 172.21.15.1:0/3482081865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55846decd4d0 0x55846e296ee0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f263400bc30 tx=0x7f26340107a0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.200+0000 7f264510f640 1 --2- 172.21.15.1:0/3514393386 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f26280b2970 0x7f26280b4e80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55846e290c30 tx=0x7f2638004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.200+0000 7f264da29740 1 -- 172.21.15.1:0/3482081865 shutdown_connections 2024-05-04T03:21:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.200+0000 7f264da29740 1 --2- 172.21.15.1:0/3482081865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55846decd4d0 0x55846e296ee0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.200+0000 7f264da29740 1 --2- 172.21.15.1:0/3482081865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55846dec52f0 0x55846e28ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.200+0000 7f264da29740 1 --2- 172.21.15.1:0/3482081865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55846deee1b0 0x55846e28fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.201+0000 7f264da29740 1 -- 172.21.15.1:0/3482081865 >> 172.21.15.1:0/3482081865 conn(0x55846dee2300 msgr2=0x55846e28b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.201+0000 7f264da29740 1 -- 172.21.15.1:0/3482081865 shutdown_connections 2024-05-04T03:21:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.201+0000 7f264da29740 1 -- 172.21.15.1:0/3482081865 wait complete. 2024-05-04T03:21:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.201+0000 7f264da29740 1 -- start start 2024-05-04T03:21:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.201+0000 7f264da29740 1 Processor -- start 2024-05-04T03:21:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.202+0000 7f264da29740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55846dec52f0 0x55846e299760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.202+0000 7f264da29740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55846decd4d0 0x55846e299ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.202+0000 7f264da29740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55846deee1b0 0x55846e2a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.202+0000 7f264da29740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55846e297930 con 0x55846decd4d0 2024-05-04T03:21:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.202+0000 7f264da29740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55846e1b1370 con 0x55846deee1b0 2024-05-04T03:21:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.202+0000 7f264da29740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55846e297ab0 con 0x55846dec52f0 2024-05-04T03:21:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.202+0000 7f264da29740 10 client.0 _ll_register_callbacks cb 0x55846df05a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:21:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.202+0000 7f264510f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55846deee1b0 0x55846e2a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.202+0000 7f263ffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55846decd4d0 0x55846e299ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:50.210 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.202+0000 7f264490e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55846dec52f0 0x55846e299760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.202+0000 7f264510f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55846deee1b0 0x55846e2a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:58088/0 (socket says 172.21.15.1:58088) 2024-05-04T03:21:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.202+0000 7f264490e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55846dec52f0 0x55846e299760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.1:50684/0 (socket says 172.21.15.1:50684) 2024-05-04T03:21:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.202+0000 7f264510f640 1 -- 172.21.15.1:0/528262143 learned_addr learned my addr 172.21.15.1:0/528262143 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:21:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.202+0000 7f264510f640 1 -- 172.21.15.1:0/528262143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55846dec52f0 msgr2=0x55846e299760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.202+0000 7f264510f640 1 --2- 172.21.15.1:0/528262143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55846dec52f0 0x55846e299760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.202+0000 7f264510f640 1 -- 172.21.15.1:0/528262143 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55846decd4d0 msgr2=0x55846e299ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.202+0000 7f264510f640 1 --2- 172.21.15.1:0/528262143 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55846decd4d0 0x55846e299ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.202+0000 7f264510f640 1 -- 172.21.15.1:0/528262143 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55846dfd2620 con 0x55846deee1b0 2024-05-04T03:21:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.202+0000 7f264490e640 1 --2- 172.21.15.1:0/528262143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55846dec52f0 0x55846e299760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:21:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.202+0000 7f264510f640 1 --2- 172.21.15.1:0/528262143 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55846deee1b0 0x55846e2a0b90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2638035000 tx=0x7f263802a8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22b17f27db609ecd server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.202+0000 7f261ffff640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-04T03:21:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.203+0000 7f261ffff640 1 -- 172.21.15.1:0/528262143 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638056060 con 0x55846deee1b0 2024-05-04T03:21:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.203+0000 7f261ffff640 1 -- 172.21.15.1:0/528262143 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f2638041030 con 0x55846deee1b0 2024-05-04T03:21:50.211 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.203+0000 7f264da29740 1 -- 172.21.15.1:0/528262143 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f263401cad0 con 0x55846deee1b0 2024-05-04T03:21:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.203+0000 7f261ffff640 1 -- 172.21.15.1:0/528262143 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2638030d00 con 0x55846deee1b0 2024-05-04T03:21:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.203+0000 7f264da29740 10 client.25286 fetch_fsmap learned FSMap version 29 2024-05-04T03:21:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.203+0000 7f264da29740 1 -- 172.21.15.1:0/528262143 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55846e2a3b30 con 0x55846deee1b0 2024-05-04T03:21:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.203+0000 7f261ffff640 1 -- 172.21.15.1:0/528262143 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f263401cad0 con 0x55846deee1b0 2024-05-04T03:21:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.203+0000 7f261ffff640 1 -- 172.21.15.1:0/528262143 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f2638046410 con 0x55846deee1b0 2024-05-04T03:21:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.203+0000 7f264da29740 10 client.25286 fetch_fsmap finished waiting for FSMap version 29 2024-05-04T03:21:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.203+0000 7f264da29740 10 client.25286 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:21:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.203+0000 7f264da29740 20 client.25286 populate_metadata read hostname 'smithi001' 2024-05-04T03:21:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.203+0000 7f264da29740 1 --2- 172.21.15.1:0/528262143 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55846deacd80 0x55846e2a7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.203+0000 7f264da29740 4 client.25286 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:21:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.203+0000 7f264da29740 1 -- 172.21.15.1:0/528262143 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55846e2021e0 con 0x55846deacd80 2024-05-04T03:21:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.204+0000 7f264490e640 1 --2- 172.21.15.1:0/528262143 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55846deacd80 0x55846e2a7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.204+0000 7f261ffff640 10 client.25286 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:21:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.204+0000 7f264490e640 1 --2- 172.21.15.1:0/528262143 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55846deacd80 0x55846e2a7310 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3381bcde47272bf3 server_cookie=2b2c5342a96bc84e in_seq=0 out_seq=0 2024-05-04T03:21:50.212 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.205+0000 7f261ffff640 1 -- 172.21.15.1:0/528262143 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55846e2021e0 con 0x55846deacd80 2024-05-04T03:21:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.205+0000 7f261ffff640 10 client.25286 handle_command_reply: tid=0 2024-05-04T03:21:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.205+0000 7f264da29740 1 client.25286 shutdown 2024-05-04T03:21:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.205+0000 7f264da29740 1 -- 172.21.15.1:0/528262143 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55846deee1b0 msgr2=0x55846e2a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.205+0000 7f264da29740 1 --2- 172.21.15.1:0/528262143 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55846deee1b0 0x55846e2a0b90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2638035000 tx=0x7f263802a8f0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.205+0000 7f264da29740 1 -- 172.21.15.1:0/528262143 shutdown_connections 2024-05-04T03:21:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.205+0000 7f264da29740 1 --2- 172.21.15.1:0/528262143 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55846deacd80 0x55846e2a7310 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.205+0000 7f264da29740 1 --2- 172.21.15.1:0/528262143 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55846deee1b0 0x55846e2a0b90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.205+0000 7f264da29740 1 --2- 172.21.15.1:0/528262143 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55846decd4d0 0x55846e299ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.205+0000 7f264da29740 1 --2- 172.21.15.1:0/528262143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55846dec52f0 0x55846e299760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.205+0000 7f261ffff640 0 client.25286 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:21:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.205+0000 7f264da29740 1 -- 172.21.15.1:0/528262143 >> 172.21.15.1:0/528262143 conn(0x55846dee2300 msgr2=0x55846e28a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.206+0000 7f264da29740 1 -- 172.21.15.1:0/528262143 shutdown_connections 2024-05-04T03:21:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.206+0000 7f264da29740 1 -- 172.21.15.1:0/528262143 wait complete. 2024-05-04T03:21:50.213 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.206+0000 7f264da29740 20 client.25286 trim_cache size 0 max 16384 2024-05-04T03:21:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.221+0000 7f264da29740 1 Processor -- start 2024-05-04T03:21:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.221+0000 7f264da29740 1 -- start start 2024-05-04T03:21:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.221+0000 7f264da29740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55846dec52f0 0x55846e28b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.221+0000 7f264da29740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55846decd4d0 0x55846e2a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.221+0000 7f264da29740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55846deee1b0 0x55846e2a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.221+0000 7f264da29740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f26401bc7d0 con 0x55846deee1b0 2024-05-04T03:21:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.221+0000 7f264da29740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f26401ba040 con 0x55846decd4d0 2024-05-04T03:21:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.221+0000 7f264da29740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f26401b7eb0 con 0x55846dec52f0 2024-05-04T03:21:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.221+0000 7f263ffff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55846decd4d0 0x55846e2a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.221+0000 7f264510f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55846deee1b0 0x55846e2a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.221+0000 7f263ffff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55846decd4d0 0x55846e2a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:58100/0 (socket says 172.21.15.1:58100) 2024-05-04T03:21:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.221+0000 7f264510f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55846deee1b0 0x55846e2a0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41086/0 (socket says 172.21.15.1:41086) 2024-05-04T03:21:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.221+0000 7f264490e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55846dec52f0 0x55846e28b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.221+0000 7f264510f640 1 -- 172.21.15.1:0/2644243396 learned_addr learned my addr 172.21.15.1:0/2644243396 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:21:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.222+0000 7f264510f640 1 -- 172.21.15.1:0/2644243396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55846dec52f0 msgr2=0x55846e28b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.222+0000 7f264510f640 1 --2- 172.21.15.1:0/2644243396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55846dec52f0 0x55846e28b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.222+0000 7f264510f640 1 -- 172.21.15.1:0/2644243396 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55846decd4d0 msgr2=0x55846e2a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.222+0000 7f264510f640 1 --2- 172.21.15.1:0/2644243396 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55846decd4d0 0x55846e2a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.222+0000 7f264510f640 1 -- 172.21.15.1:0/2644243396 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55846e28b940 con 0x55846deee1b0 2024-05-04T03:21:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.222+0000 7f264490e640 1 --2- 172.21.15.1:0/2644243396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55846dec52f0 0x55846e28b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:21:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.222+0000 7f264510f640 1 --2- 172.21.15.1:0/2644243396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55846deee1b0 0x55846e2a0ff0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2638046800 tx=0x7f2638009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5c6c5a342d1329 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.222+0000 7f26267fc640 1 -- 172.21.15.1:0/2644243396 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638061070 con 0x55846deee1b0 2024-05-04T03:21:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.222+0000 7f26267fc640 1 -- 172.21.15.1:0/2644243396 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2638036040 con 0x55846deee1b0 2024-05-04T03:21:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.222+0000 7f26267fc640 1 -- 172.21.15.1:0/2644243396 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638046a70 con 0x55846deee1b0 2024-05-04T03:21:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.223+0000 7f264da29740 1 -- 172.21.15.1:0/2644243396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55846deee1b0 msgr2=0x55846e2a0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.223+0000 7f264da29740 1 --2- 172.21.15.1:0/2644243396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55846deee1b0 0x55846e2a0ff0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2638046800 tx=0x7f2638009680 comp rx=0 tx=0).stop 2024-05-04T03:21:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.223+0000 7f264da29740 1 -- 172.21.15.1:0/2644243396 shutdown_connections 2024-05-04T03:21:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.223+0000 7f264da29740 1 --2- 172.21.15.1:0/2644243396 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55846decd4d0 0x55846e2a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.223+0000 7f264da29740 1 --2- 172.21.15.1:0/2644243396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55846deee1b0 0x55846e2a0ff0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.223+0000 7f264da29740 1 --2- 172.21.15.1:0/2644243396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55846dec52f0 0x55846e28b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.223+0000 7f264da29740 1 -- 172.21.15.1:0/2644243396 >> 172.21.15.1:0/2644243396 conn(0x55846dee2300 msgr2=0x55846e28acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.223+0000 7f264da29740 1 -- 172.21.15.1:0/2644243396 shutdown_connections 2024-05-04T03:21:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.223+0000 7f264da29740 1 -- 172.21.15.1:0/2644243396 wait complete. 2024-05-04T03:21:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.223+0000 7f264da29740 1 -- start start 2024-05-04T03:21:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.223+0000 7f264da29740 1 Processor -- start 2024-05-04T03:21:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.224+0000 7f264da29740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55846dec52f0 0x55846e29e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.224+0000 7f264da29740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55846decd4d0 0x55846e29ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.224+0000 7f264da29740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55846deee1b0 0x55846e29f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.224+0000 7f264da29740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f26401b8500 con 0x55846decd4d0 2024-05-04T03:21:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.224+0000 7f264da29740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f26401b9970 con 0x55846dec52f0 2024-05-04T03:21:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.224+0000 7f264da29740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f26401b8bd0 con 0x55846deee1b0 2024-05-04T03:21:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.224+0000 7f264da29740 10 client.0 _ll_register_callbacks cb 0x7f26380099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:21:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.224+0000 7f264490e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55846dec52f0 0x55846e29e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.224+0000 7f264490e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55846dec52f0 0x55846e29e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:58112/0 (socket says 172.21.15.1:58112) 2024-05-04T03:21:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.224+0000 7f264490e640 1 -- 172.21.15.1:0/3619493918 learned_addr learned my addr 172.21.15.1:0/3619493918 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:21:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.224+0000 7f264510f640 1 --2- 172.21.15.1:0/3619493918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55846deee1b0 0x55846e29f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.224+0000 7f263ffff640 1 --2- 172.21.15.1:0/3619493918 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55846decd4d0 0x55846e29ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.224+0000 7f264490e640 1 -- 172.21.15.1:0/3619493918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55846deee1b0 msgr2=0x55846e29f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.224+0000 7f264490e640 1 --2- 172.21.15.1:0/3619493918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55846deee1b0 0x55846e29f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.224+0000 7f264490e640 1 -- 172.21.15.1:0/3619493918 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55846decd4d0 msgr2=0x55846e29ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:21:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.224+0000 7f264490e640 1 --2- 172.21.15.1:0/3619493918 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55846decd4d0 0x55846e29ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.224+0000 7f264490e640 1 -- 172.21.15.1:0/3619493918 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55846e29fe90 con 0x55846dec52f0 2024-05-04T03:21:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.224+0000 7f264510f640 1 --2- 172.21.15.1:0/3619493918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55846deee1b0 0x55846e29f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:21:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.225+0000 7f264490e640 1 --2- 172.21.15.1:0/3619493918 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55846dec52f0 0x55846e29e4d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f263000c6a0 tx=0x7f26300e7060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cc4fd337e83e0bb server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:21:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.225+0000 7f261ffff640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-04T03:21:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.225+0000 7f261ffff640 1 -- 172.21.15.1:0/3619493918 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26300c33a0 con 0x55846dec52f0 2024-05-04T03:21:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.225+0000 7f261ffff640 1 -- 172.21.15.1:0/3619493918 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f26300155c0 con 0x55846dec52f0 2024-05-04T03:21:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.225+0000 7f264da29740 1 -- 172.21.15.1:0/3619493918 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2638036040 con 0x55846dec52f0 2024-05-04T03:21:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.225+0000 7f261ffff640 1 -- 172.21.15.1:0/3619493918 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f26300ad080 con 0x55846dec52f0 2024-05-04T03:21:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.225+0000 7f264da29740 10 client.25292 fetch_fsmap learned FSMap version 29 2024-05-04T03:21:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.225+0000 7f264da29740 1 -- 172.21.15.1:0/3619493918 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55846e2a1190 con 0x55846dec52f0 2024-05-04T03:21:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.225+0000 7f261ffff640 1 -- 172.21.15.1:0/3619493918 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2638036040 con 0x55846dec52f0 2024-05-04T03:21:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.225+0000 7f261ffff640 1 -- 172.21.15.1:0/3619493918 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f2630090040 con 0x55846dec52f0 2024-05-04T03:21:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.225+0000 7f264da29740 10 client.25292 fetch_fsmap finished waiting for FSMap version 29 2024-05-04T03:21:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.225+0000 7f264da29740 10 client.25292 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:21:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.225+0000 7f264da29740 20 client.25292 populate_metadata read hostname 'smithi001' 2024-05-04T03:21:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.225+0000 7f264da29740 1 --2- 172.21.15.1:0/3619493918 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55846de821c0 0x55846e311940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:21:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.225+0000 7f264da29740 4 client.25292 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:21:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.225+0000 7f264da29740 1 -- 172.21.15.1:0/3619493918 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55846e1fc050 con 0x55846de821c0 2024-05-04T03:21:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.226+0000 7f263ffff640 1 --2- 172.21.15.1:0/3619493918 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55846de821c0 0x55846e311940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:21:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.226+0000 7f263ffff640 1 --2- 172.21.15.1:0/3619493918 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55846de821c0 0x55846e311940 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=779746ec7b0748c7 server_cookie=1663a05573d871d5 in_seq=0 out_seq=0 2024-05-04T03:21:50.233 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.226+0000 7f261ffff640 10 client.25292 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:21:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.233+0000 7f261ffff640 1 -- 172.21.15.1:0/3619493918 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55846e1fc050 con 0x55846de821c0 2024-05-04T03:21:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.233+0000 7f261ffff640 10 client.25292 handle_command_reply: tid=0 2024-05-04T03:21:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.233+0000 7f264da29740 1 client.25292 shutdown 2024-05-04T03:21:50.235 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.233+0000 7f264da29740 1 -- 172.21.15.1:0/3619493918 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55846dec52f0 msgr2=0x55846e29e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.233+0000 7f264da29740 1 --2- 172.21.15.1:0/3619493918 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55846dec52f0 0x55846e29e4d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f263000c6a0 tx=0x7f26300e7060 comp rx=0 tx=0).stop 2024-05-04T03:21:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.233+0000 7f264da29740 1 -- 172.21.15.1:0/3619493918 shutdown_connections 2024-05-04T03:21:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.233+0000 7f264da29740 1 --2- 172.21.15.1:0/3619493918 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55846de821c0 0x55846e311940 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.233+0000 7f264da29740 1 --2- 172.21.15.1:0/3619493918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55846deee1b0 0x55846e29f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.233+0000 7f264da29740 1 --2- 172.21.15.1:0/3619493918 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55846decd4d0 0x55846e29ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.233+0000 7f261ffff640 0 client.25292 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:21:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.233+0000 7f264da29740 1 --2- 172.21.15.1:0/3619493918 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55846dec52f0 0x55846e29e4d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.233+0000 7f264da29740 1 -- 172.21.15.1:0/3619493918 >> 172.21.15.1:0/3619493918 conn(0x55846dee2300 msgr2=0x55846e289970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.233+0000 7f264da29740 1 -- 172.21.15.1:0/3619493918 shutdown_connections 2024-05-04T03:21:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.233+0000 7f264da29740 1 -- 172.21.15.1:0/3619493918 wait complete. 2024-05-04T03:21:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.233+0000 7f264da29740 20 client.25292 trim_cache size 0 max 16384 2024-05-04T03:21:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:21:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:21:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:21:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:21:50.237 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.234+0000 7f2646b72640 1 -- 172.21.15.1:0/3514393386 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f26280b2970 msgr2=0x7f26280b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.234+0000 7f2646b72640 1 --2- 172.21.15.1:0/3514393386 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f26280b2970 0x7f26280b4e80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55846e290c30 tx=0x7f2638004a40 comp rx=0 tx=0).stop 2024-05-04T03:21:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.234+0000 7f2646b72640 1 -- 172.21.15.1:0/3514393386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f264010a0f0 msgr2=0x7f26401a9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:21:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.234+0000 7f2646b72640 1 --2- 172.21.15.1:0/3514393386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f264010a0f0 0x7f26401a9110 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f263000d9b0 tx=0x7f263000de70 comp rx=0 tx=0).stop 2024-05-04T03:21:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.234+0000 7f2646b72640 1 -- 172.21.15.1:0/3514393386 shutdown_connections 2024-05-04T03:21:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.234+0000 7f2646b72640 1 --2- 172.21.15.1:0/3514393386 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f26280b2970 0x7f26280b4e80 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.234+0000 7f2646b72640 1 --2- 172.21.15.1:0/3514393386 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f264010f890 0x7f26401a9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.234+0000 7f2646b72640 1 --2- 172.21.15.1:0/3514393386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f26401121f0 0x7f26401b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.234+0000 7f2646b72640 1 --2- 172.21.15.1:0/3514393386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f264010a0f0 0x7f26401a9110 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:21:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.234+0000 7f2646b72640 1 -- 172.21.15.1:0/3514393386 >> 172.21.15.1:0/3514393386 conn(0x7f2640070640 msgr2=0x7f26401106d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:21:50.238 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.234+0000 7f2646b72640 1 -- 172.21.15.1:0/3514393386 shutdown_connections 2024-05-04T03:21:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:21:50.234+0000 7f2646b72640 1 -- 172.21.15.1:0/3514393386 wait complete. 2024-05-04T03:21:50.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:50 smithi001 ceph-mon[29731]: pgmap v471: 97 pgs: 97 active+clean; 56 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.3 MiB/s wr, 727 op/s 2024-05-04T03:21:50.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:50 smithi077 ceph-mon[33531]: pgmap v471: 97 pgs: 97 active+clean; 56 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.3 MiB/s wr, 727 op/s 2024-05-04T03:21:50.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:50 smithi072 ceph-mon[33332]: pgmap v471: 97 pgs: 97 active+clean; 56 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.3 MiB/s wr, 727 op/s 2024-05-04T03:21:52.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:52 smithi001 ceph-mon[29731]: pgmap v472: 97 pgs: 97 active+clean; 55 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.3 MiB/s wr, 784 op/s 2024-05-04T03:21:52.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:52 smithi077 ceph-mon[33531]: pgmap v472: 97 pgs: 97 active+clean; 55 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.3 MiB/s wr, 784 op/s 2024-05-04T03:21:52.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:52 smithi072 ceph-mon[33332]: pgmap v472: 97 pgs: 97 active+clean; 55 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.3 MiB/s wr, 784 op/s 2024-05-04T03:21:54.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:54 smithi001 ceph-mon[29731]: pgmap v473: 97 pgs: 97 active+clean; 55 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 582 op/s 2024-05-04T03:21:54.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:54 smithi077 ceph-mon[33531]: pgmap v473: 97 pgs: 97 active+clean; 55 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 582 op/s 2024-05-04T03:21:54.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:54 smithi072 ceph-mon[33332]: pgmap v473: 97 pgs: 97 active+clean; 55 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 582 op/s 2024-05-04T03:21:55.095 INFO:tasks.workunit.client.0.smithi001.stdout:30000 trunc from 0x7bb2f to 0xcdf83 2024-05-04T03:21:55.598 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xffcdd 2024-05-04T03:21:56.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:56 smithi001 ceph-mon[29731]: pgmap v474: 97 pgs: 97 active+clean; 58 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.1 MiB/s wr, 831 op/s 2024-05-04T03:21:56.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:56 smithi077 ceph-mon[33531]: pgmap v474: 97 pgs: 97 active+clean; 58 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.1 MiB/s wr, 831 op/s 2024-05-04T03:21:56.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:56 smithi072 ceph-mon[33332]: pgmap v474: 97 pgs: 97 active+clean; 58 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.1 MiB/s wr, 831 op/s 2024-05-04T03:21:57.377 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xffd97 2024-05-04T03:21:58.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:21:58 smithi001 ceph-mon[29731]: pgmap v475: 97 pgs: 97 active+clean; 58 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 533 op/s 2024-05-04T03:21:58.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:21:58 smithi077 ceph-mon[33531]: pgmap v475: 97 pgs: 97 active+clean; 58 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 533 op/s 2024-05-04T03:21:58.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:21:58 smithi072 ceph-mon[33332]: pgmap v475: 97 pgs: 97 active+clean; 58 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 533 op/s 2024-05-04T03:22:00.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:00 smithi001 ceph-mon[29731]: pgmap v476: 97 pgs: 97 active+clean; 62 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 637 op/s 2024-05-04T03:22:00.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:00 smithi077 ceph-mon[33531]: pgmap v476: 97 pgs: 97 active+clean; 62 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 637 op/s 2024-05-04T03:22:00.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:00 smithi072 ceph-mon[33332]: pgmap v476: 97 pgs: 97 active+clean; 62 MiB data, 665 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 637 op/s 2024-05-04T03:22:02.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:02 smithi001 ceph-mon[29731]: pgmap v477: 97 pgs: 97 active+clean; 62 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.6 MiB/s wr, 703 op/s 2024-05-04T03:22:02.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:02 smithi077 ceph-mon[33531]: pgmap v477: 97 pgs: 97 active+clean; 62 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.6 MiB/s wr, 703 op/s 2024-05-04T03:22:02.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:02 smithi072 ceph-mon[33332]: pgmap v477: 97 pgs: 97 active+clean; 62 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.6 MiB/s wr, 703 op/s 2024-05-04T03:22:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.341+0000 7ff625815640 1 -- 172.21.15.1:0/809437306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff620108aa0 msgr2=0x7ff62010aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.341+0000 7ff625815640 1 --2- 172.21.15.1:0/809437306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff620108aa0 0x7ff62010aec0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff618009ca0 tx=0x7ff61802a8c0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.341+0000 7ff625815640 1 -- 172.21.15.1:0/809437306 shutdown_connections 2024-05-04T03:22:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.341+0000 7ff625815640 1 --2- 172.21.15.1:0/809437306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff620108aa0 0x7ff62010aec0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.341+0000 7ff625815640 1 --2- 172.21.15.1:0/809437306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff620103da0 0x7ff6201041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.341+0000 7ff625815640 1 --2- 172.21.15.1:0/809437306 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff620102b60 0x7ff620103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.341+0000 7ff625815640 1 -- 172.21.15.1:0/809437306 >> 172.21.15.1:0/809437306 conn(0x7ff6200fdd50 msgr2=0x7ff620100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.342+0000 7ff625815640 1 -- 172.21.15.1:0/809437306 shutdown_connections 2024-05-04T03:22:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.342+0000 7ff625815640 1 -- 172.21.15.1:0/809437306 wait complete. 2024-05-04T03:22:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.342+0000 7ff625815640 1 Processor -- start 2024-05-04T03:22:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.342+0000 7ff625815640 1 -- start start 2024-05-04T03:22:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.343+0000 7ff625815640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff620102b60 0x7ff620199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.343+0000 7ff625815640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff620103da0 0x7ff6201996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.343+0000 7ff625815640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff620108aa0 0x7ff6201a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.343+0000 7ff625815640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff62010e690 con 0x7ff620103da0 2024-05-04T03:22:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.343+0000 7ff625815640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff62010e510 con 0x7ff620102b60 2024-05-04T03:22:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.343+0000 7ff625815640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff62010e810 con 0x7ff620108aa0 2024-05-04T03:22:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.343+0000 7ff61e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff620103da0 0x7ff6201996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.343+0000 7ff61f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff620108aa0 0x7ff6201a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.343+0000 7ff61e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff620103da0 0x7ff6201996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57460/0 (socket says 172.21.15.1:57460) 2024-05-04T03:22:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.343+0000 7ff61e7fc640 1 -- 172.21.15.1:0/1853660145 learned_addr learned my addr 172.21.15.1:0/1853660145 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.343+0000 7ff61effd640 1 --2- 172.21.15.1:0/1853660145 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff620102b60 0x7ff620199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.343+0000 7ff61e7fc640 1 -- 172.21.15.1:0/1853660145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff620108aa0 msgr2=0x7ff6201a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.343+0000 7ff61e7fc640 1 --2- 172.21.15.1:0/1853660145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff620108aa0 0x7ff6201a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.343+0000 7ff61e7fc640 1 -- 172.21.15.1:0/1853660145 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff620102b60 msgr2=0x7ff620199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.343+0000 7ff61e7fc640 1 --2- 172.21.15.1:0/1853660145 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff620102b60 0x7ff620199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.343+0000 7ff61e7fc640 1 -- 172.21.15.1:0/1853660145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff620074f00 con 0x7ff620103da0 2024-05-04T03:22:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.343+0000 7ff61f7fe640 1 --2- 172.21.15.1:0/1853660145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff620108aa0 0x7ff6201a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:22:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.343+0000 7ff61e7fc640 1 --2- 172.21.15.1:0/1853660145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff620103da0 0x7ff6201996b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff61400e9f0 tx=0x7ff61400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.344+0000 7ff624813640 1 -- 172.21.15.1:0/1853660145 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61400cd80 con 0x7ff620103da0 2024-05-04T03:22:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.344+0000 7ff624813640 1 -- 172.21.15.1:0/1853660145 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6140054d0 con 0x7ff620103da0 2024-05-04T03:22:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.344+0000 7ff625815640 1 -- 172.21.15.1:0/1853660145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6200751f0 con 0x7ff620103da0 2024-05-04T03:22:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.344+0000 7ff624813640 1 -- 172.21.15.1:0/1853660145 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff614010690 con 0x7ff620103da0 2024-05-04T03:22:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.344+0000 7ff625815640 1 -- 172.21.15.1:0/1853660145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6200756f0 con 0x7ff620103da0 2024-05-04T03:22:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.347+0000 7ff62c7a4740 1 Processor -- start 2024-05-04T03:22:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.347+0000 7ff62c7a4740 1 -- start start 2024-05-04T03:22:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.347+0000 7ff62c7a4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f69022f1b0 0x55f6905d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.347+0000 7ff62c7a4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6902062f0 0x55f6905d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.347+0000 7ff62c7a4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f69020e4d0 0x55f6905d7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.347+0000 7ff62c7a4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f6905355d0 con 0x55f6902062f0 2024-05-04T03:22:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.347+0000 7ff62c7a4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55f6904ee890 con 0x55f69020e4d0 2024-05-04T03:22:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.347+0000 7ff62c7a4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f690531590 con 0x55f69022f1b0 2024-05-04T03:22:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.347+0000 7ff61effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6902062f0 0x55f6905d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.347+0000 7ff61effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6902062f0 0x55f6905d0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57474/0 (socket says 172.21.15.1:57474) 2024-05-04T03:22:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.347+0000 7ff61effd640 1 -- 172.21.15.1:0/2555974677 learned_addr learned my addr 172.21.15.1:0/2555974677 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.347+0000 7ff61f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f69022f1b0 0x55f6905d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.347+0000 7ff624813640 1 -- 172.21.15.1:0/1853660145 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7ff614004e90 con 0x7ff620103da0 2024-05-04T03:22:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.347+0000 7ff61effd640 1 -- 172.21.15.1:0/2555974677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f69022f1b0 msgr2=0x55f6905d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.347+0000 7ff61effd640 1 --2- 172.21.15.1:0/2555974677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f69022f1b0 0x55f6905d0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.347+0000 7ff61effd640 1 -- 172.21.15.1:0/2555974677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f69020e4d0 msgr2=0x55f6905d7ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-04T03:22:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.347+0000 7ff61effd640 1 --2- 172.21.15.1:0/2555974677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f69020e4d0 0x55f6905d7ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.347+0000 7ff61effd640 1 -- 172.21.15.1:0/2555974677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f690247210 con 0x55f6902062f0 2024-05-04T03:22:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.347+0000 7ff61f7fe640 1 --2- 172.21.15.1:0/2555974677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f69022f1b0 0x55f6905d0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:22:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.348+0000 7ff61effd640 1 --2- 172.21.15.1:0/2555974677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6902062f0 0x55f6905d0fe0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff610009c60 tx=0x7ff610011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce06632ed29d1c68 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.348+0000 7ff605ffb640 1 -- 172.21.15.1:0/2555974677 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff610004220 con 0x55f6902062f0 2024-05-04T03:22:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.348+0000 7ff605ffb640 1 -- 172.21.15.1:0/2555974677 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6100043c0 con 0x55f6902062f0 2024-05-04T03:22:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.348+0000 7ff605ffb640 1 -- 172.21.15.1:0/2555974677 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff610004ce0 con 0x55f6902062f0 2024-05-04T03:22:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.348+0000 7ff624813640 1 --2- 172.21.15.1:0/1853660145 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ff6080b28a0 0x7ff6080b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.348+0000 7ff61f7fe640 1 --2- 172.21.15.1:0/1853660145 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ff6080b28a0 0x7ff6080b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.348+0000 7ff624813640 1 -- 172.21.15.1:0/1853660145 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7ff6140e2570 con 0x7ff620103da0 2024-05-04T03:22:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.349+0000 7ff62c7a4740 1 -- 172.21.15.1:0/2555974677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6902062f0 msgr2=0x55f6905d0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.349+0000 7ff62c7a4740 1 --2- 172.21.15.1:0/2555974677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6902062f0 0x55f6905d0fe0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff610009c60 tx=0x7ff610011810 comp rx=0 tx=0).stop 2024-05-04T03:22:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.349+0000 7ff61f7fe640 1 --2- 172.21.15.1:0/1853660145 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ff6080b28a0 0x7ff6080b4db0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55f6905d1c30 tx=0x7ff61802f490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.349+0000 7ff62c7a4740 1 -- 172.21.15.1:0/2555974677 shutdown_connections 2024-05-04T03:22:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.349+0000 7ff62c7a4740 1 --2- 172.21.15.1:0/2555974677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f69020e4d0 0x55f6905d7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.349+0000 7ff62c7a4740 1 --2- 172.21.15.1:0/2555974677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6902062f0 0x55f6905d0fe0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.349+0000 7ff62c7a4740 1 --2- 172.21.15.1:0/2555974677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f69022f1b0 0x55f6905d0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.349+0000 7ff62c7a4740 1 -- 172.21.15.1:0/2555974677 >> 172.21.15.1:0/2555974677 conn(0x55f690223300 msgr2=0x55f6905cc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.349+0000 7ff62c7a4740 1 -- 172.21.15.1:0/2555974677 shutdown_connections 2024-05-04T03:22:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.349+0000 7ff62c7a4740 1 -- 172.21.15.1:0/2555974677 wait complete. 2024-05-04T03:22:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.349+0000 7ff62c7a4740 1 -- start start 2024-05-04T03:22:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.350+0000 7ff62c7a4740 1 Processor -- start 2024-05-04T03:22:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.350+0000 7ff62c7a4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6902062f0 0x55f6905da6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.350+0000 7ff62c7a4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f69020e4d0 0x55f6905dac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.350+0000 7ff62c7a4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f69022f1b0 0x55f6905e1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.350+0000 7ff62c7a4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f6905d8930 con 0x55f6902062f0 2024-05-04T03:22:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.350+0000 7ff62c7a4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55f6904f2370 con 0x55f69020e4d0 2024-05-04T03:22:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.350+0000 7ff62c7a4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f6905d8ab0 con 0x55f69022f1b0 2024-05-04T03:22:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.350+0000 7ff62c7a4740 10 client.0 _ll_register_callbacks cb 0x55f690246a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:22:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.350+0000 7ff61e7fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f69020e4d0 0x55f6905dac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.350+0000 7ff61effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6902062f0 0x55f6905da6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.350+0000 7ff61f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f69022f1b0 0x55f6905e1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.350+0000 7ff61e7fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f69020e4d0 0x55f6905dac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:44064/0 (socket says 172.21.15.1:44064) 2024-05-04T03:22:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.350+0000 7ff61e7fc640 1 -- 172.21.15.1:0/102421507 learned_addr learned my addr 172.21.15.1:0/102421507 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.350+0000 7ff61e7fc640 1 -- 172.21.15.1:0/102421507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f69022f1b0 msgr2=0x55f6905e1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.350+0000 7ff61e7fc640 1 --2- 172.21.15.1:0/102421507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f69022f1b0 0x55f6905e1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.350+0000 7ff61e7fc640 1 -- 172.21.15.1:0/102421507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6902062f0 msgr2=0x55f6905da6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.350+0000 7ff61e7fc640 1 --2- 172.21.15.1:0/102421507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6902062f0 0x55f6905da6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.358 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.350+0000 7ff61e7fc640 1 -- 172.21.15.1:0/102421507 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f690313620 con 0x55f69020e4d0 2024-05-04T03:22:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.351+0000 7ff61e7fc640 1 --2- 172.21.15.1:0/102421507 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f69020e4d0 0x55f6905dac10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff61400c6a0 tx=0x7ff6140a9660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb8bcb793f8bd1c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.351+0000 7ff5feffd640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-04T03:22:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.351+0000 7ff5feffd640 1 -- 172.21.15.1:0/102421507 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6140a9760 con 0x55f69020e4d0 2024-05-04T03:22:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.351+0000 7ff5feffd640 1 -- 172.21.15.1:0/102421507 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7ff6140c42e0 con 0x55f69020e4d0 2024-05-04T03:22:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.351+0000 7ff62c7a4740 1 -- 172.21.15.1:0/102421507 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6100043c0 con 0x55f69020e4d0 2024-05-04T03:22:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.351+0000 7ff5feffd640 1 -- 172.21.15.1:0/102421507 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff6140aa350 con 0x55f69020e4d0 2024-05-04T03:22:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.351+0000 7ff62c7a4740 10 client.25301 fetch_fsmap learned FSMap version 29 2024-05-04T03:22:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.351+0000 7ff62c7a4740 1 -- 172.21.15.1:0/102421507 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f6905e4b70 con 0x55f69020e4d0 2024-05-04T03:22:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.351+0000 7ff5feffd640 1 -- 172.21.15.1:0/102421507 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6100043c0 con 0x55f69020e4d0 2024-05-04T03:22:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.351+0000 7ff5feffd640 1 -- 172.21.15.1:0/102421507 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7ff6140e20f0 con 0x55f69020e4d0 2024-05-04T03:22:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.351+0000 7ff62c7a4740 10 client.25301 fetch_fsmap finished waiting for FSMap version 29 2024-05-04T03:22:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.351+0000 7ff62c7a4740 10 client.25301 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:22:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.351+0000 7ff62c7a4740 20 client.25301 populate_metadata read hostname 'smithi001' 2024-05-04T03:22:03.359 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.352+0000 7ff62c7a4740 1 --2- 172.21.15.1:0/102421507 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55f6901edd80 0x55f6905e8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.352+0000 7ff62c7a4740 4 client.25301 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:22:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.352+0000 7ff62c7a4740 1 -- 172.21.15.1:0/102421507 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6905431e0 con 0x55f6901edd80 2024-05-04T03:22:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.352+0000 7ff61effd640 1 --2- 172.21.15.1:0/102421507 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55f6901edd80 0x55f6905e8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.352+0000 7ff5feffd640 10 client.25301 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:22:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.352+0000 7ff61effd640 1 --2- 172.21.15.1:0/102421507 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55f6901edd80 0x55f6905e8220 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5cc36be4d0eba5c server_cookie=589189cf13982448 in_seq=0 out_seq=0 2024-05-04T03:22:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.353+0000 7ff5feffd640 1 -- 172.21.15.1:0/102421507 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f6905431e0 con 0x55f6901edd80 2024-05-04T03:22:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.353+0000 7ff5feffd640 10 client.25301 handle_command_reply: tid=0 2024-05-04T03:22:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.353+0000 7ff62c7a4740 1 client.25301 shutdown 2024-05-04T03:22:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.353+0000 7ff62c7a4740 1 -- 172.21.15.1:0/102421507 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f69020e4d0 msgr2=0x55f6905dac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.353+0000 7ff62c7a4740 1 --2- 172.21.15.1:0/102421507 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f69020e4d0 0x55f6905dac10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff61400c6a0 tx=0x7ff6140a9660 comp rx=0 tx=0).stop 2024-05-04T03:22:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.353+0000 7ff62c7a4740 1 -- 172.21.15.1:0/102421507 shutdown_connections 2024-05-04T03:22:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.353+0000 7ff62c7a4740 1 --2- 172.21.15.1:0/102421507 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55f6901edd80 0x55f6905e8220 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.353+0000 7ff62c7a4740 1 --2- 172.21.15.1:0/102421507 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f69020e4d0 0x55f6905dac10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.353+0000 7ff62c7a4740 1 --2- 172.21.15.1:0/102421507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f69022f1b0 0x55f6905e1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.353+0000 7ff5feffd640 0 client.25301 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:22:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.353+0000 7ff62c7a4740 1 --2- 172.21.15.1:0/102421507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6902062f0 0x55f6905da6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.353+0000 7ff62c7a4740 1 -- 172.21.15.1:0/102421507 >> 172.21.15.1:0/102421507 conn(0x55f690223300 msgr2=0x55f6905cf300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.353+0000 7ff62c7a4740 1 -- 172.21.15.1:0/102421507 shutdown_connections 2024-05-04T03:22:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.353+0000 7ff62c7a4740 1 -- 172.21.15.1:0/102421507 wait complete. 2024-05-04T03:22:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.353+0000 7ff62c7a4740 20 client.25301 trim_cache size 0 max 16384 2024-05-04T03:22:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.370+0000 7ff62c7a4740 1 Processor -- start 2024-05-04T03:22:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.370+0000 7ff62c7a4740 1 -- start start 2024-05-04T03:22:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.370+0000 7ff62c7a4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f6901edd80 0x55f6905d95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.370+0000 7ff62c7a4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6902062f0 0x55f6905e1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.370+0000 7ff62c7a4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f69020e4d0 0x55f6905e1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.370+0000 7ff62c7a4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff6201ab800 con 0x55f69020e4d0 2024-05-04T03:22:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.370+0000 7ff62c7a4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff6201a9070 con 0x55f6901edd80 2024-05-04T03:22:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.370+0000 7ff62c7a4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff6201a6ee0 con 0x55f6902062f0 2024-05-04T03:22:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.370+0000 7ff61f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f69020e4d0 0x55f6905e1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.370+0000 7ff61f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f69020e4d0 0x55f6905e1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57492/0 (socket says 172.21.15.1:57492) 2024-05-04T03:22:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.370+0000 7ff61f7fe640 1 -- 172.21.15.1:0/3590903983 learned_addr learned my addr 172.21.15.1:0/3590903983 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.370+0000 7ff61e7fc640 1 --2- 172.21.15.1:0/3590903983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6902062f0 0x55f6905e1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.370+0000 7ff61effd640 1 --2- 172.21.15.1:0/3590903983 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f6901edd80 0x55f6905d95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.371+0000 7ff61f7fe640 1 -- 172.21.15.1:0/3590903983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6902062f0 msgr2=0x55f6905e1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.371+0000 7ff61f7fe640 1 --2- 172.21.15.1:0/3590903983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6902062f0 0x55f6905e1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.371+0000 7ff61f7fe640 1 -- 172.21.15.1:0/3590903983 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f6901edd80 msgr2=0x55f6905d95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.371+0000 7ff61f7fe640 1 --2- 172.21.15.1:0/3590903983 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f6901edd80 0x55f6905d95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.371+0000 7ff61f7fe640 1 -- 172.21.15.1:0/3590903983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6905e2510 con 0x55f69020e4d0 2024-05-04T03:22:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.371+0000 7ff61effd640 1 --2- 172.21.15.1:0/3590903983 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f6901edd80 0x55f6905d95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:22:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.371+0000 7ff61f7fe640 1 --2- 172.21.15.1:0/3590903983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f69020e4d0 0x55f6905e1fd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff618009ca0 tx=0x7ff618045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec06d9573b361c63 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.371+0000 7ff6057fa640 1 -- 172.21.15.1:0/3590903983 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff618004040 con 0x55f69020e4d0 2024-05-04T03:22:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.371+0000 7ff6057fa640 1 -- 172.21.15.1:0/3590903983 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff618036040 con 0x55f69020e4d0 2024-05-04T03:22:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.371+0000 7ff6057fa640 1 -- 172.21.15.1:0/3590903983 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61802fc70 con 0x55f69020e4d0 2024-05-04T03:22:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.372+0000 7ff62c7a4740 1 -- 172.21.15.1:0/3590903983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f69020e4d0 msgr2=0x55f6905e1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.372+0000 7ff62c7a4740 1 --2- 172.21.15.1:0/3590903983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f69020e4d0 0x55f6905e1fd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff618009ca0 tx=0x7ff618045000 comp rx=0 tx=0).stop 2024-05-04T03:22:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.372+0000 7ff62c7a4740 1 -- 172.21.15.1:0/3590903983 shutdown_connections 2024-05-04T03:22:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.372+0000 7ff62c7a4740 1 --2- 172.21.15.1:0/3590903983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f69020e4d0 0x55f6905e1fd0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.372+0000 7ff62c7a4740 1 --2- 172.21.15.1:0/3590903983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f6902062f0 0x55f6905e1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.372+0000 7ff62c7a4740 1 --2- 172.21.15.1:0/3590903983 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f6901edd80 0x55f6905d95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.372+0000 7ff62c7a4740 1 -- 172.21.15.1:0/3590903983 >> 172.21.15.1:0/3590903983 conn(0x55f690223300 msgr2=0x55f6905d8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.372+0000 7ff62c7a4740 1 -- 172.21.15.1:0/3590903983 shutdown_connections 2024-05-04T03:22:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.372+0000 7ff62c7a4740 1 -- 172.21.15.1:0/3590903983 wait complete. 2024-05-04T03:22:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.372+0000 7ff62c7a4740 1 -- start start 2024-05-04T03:22:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.373+0000 7ff62c7a4740 1 Processor -- start 2024-05-04T03:22:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.373+0000 7ff62c7a4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6901edd80 0x55f6905df430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.373+0000 7ff62c7a4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f6902062f0 0x55f6905df970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.373+0000 7ff62c7a4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f69020e4d0 0x55f6905e08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.373+0000 7ff62c7a4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ff6201a7530 con 0x55f6901edd80 2024-05-04T03:22:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.373+0000 7ff62c7a4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff6201a89a0 con 0x55f6902062f0 2024-05-04T03:22:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.373+0000 7ff62c7a4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff6201a7c00 con 0x55f69020e4d0 2024-05-04T03:22:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.373+0000 7ff62c7a4740 10 client.0 _ll_register_callbacks cb 0x7ff61400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:22:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.373+0000 7ff61effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6901edd80 0x55f6905df430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.373+0000 7ff61effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6901edd80 0x55f6905df430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57496/0 (socket says 172.21.15.1:57496) 2024-05-04T03:22:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.373+0000 7ff61effd640 1 -- 172.21.15.1:0/2053977850 learned_addr learned my addr 172.21.15.1:0/2053977850 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.373+0000 7ff61f7fe640 1 --2- 172.21.15.1:0/2053977850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f69020e4d0 0x55f6905e08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.373+0000 7ff61e7fc640 1 --2- 172.21.15.1:0/2053977850 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f6902062f0 0x55f6905df970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.373+0000 7ff61effd640 1 -- 172.21.15.1:0/2053977850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f69020e4d0 msgr2=0x55f6905e08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.373+0000 7ff61effd640 1 --2- 172.21.15.1:0/2053977850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f69020e4d0 0x55f6905e08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.373+0000 7ff61effd640 1 -- 172.21.15.1:0/2053977850 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f6902062f0 msgr2=0x55f6905df970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.373+0000 7ff61effd640 1 --2- 172.21.15.1:0/2053977850 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f6902062f0 0x55f6905df970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.373+0000 7ff61effd640 1 -- 172.21.15.1:0/2053977850 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6905e0e60 con 0x55f6901edd80 2024-05-04T03:22:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.373+0000 7ff61f7fe640 1 --2- 172.21.15.1:0/2053977850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f69020e4d0 0x55f6905e08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:22:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.374+0000 7ff61effd640 1 --2- 172.21.15.1:0/2053977850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6901edd80 0x55f6905df430 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff610005710 tx=0x7ff6100099a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4e1f16c87e30bf server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.374+0000 7ff5feffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:22:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.374+0000 7ff5feffd640 1 -- 172.21.15.1:0/2053977850 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff610039060 con 0x55f6901edd80 2024-05-04T03:22:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.374+0000 7ff5feffd640 1 -- 172.21.15.1:0/2053977850 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7ff610004040 con 0x55f6901edd80 2024-05-04T03:22:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.374+0000 7ff62c7a4740 1 -- 172.21.15.1:0/2053977850 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff618036040 con 0x55f6901edd80 2024-05-04T03:22:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.374+0000 7ff5feffd640 1 -- 172.21.15.1:0/2053977850 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff61001d040 con 0x55f6901edd80 2024-05-04T03:22:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.374+0000 7ff62c7a4740 10 client.15570 fetch_fsmap learned FSMap version 29 2024-05-04T03:22:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.374+0000 7ff62c7a4740 1 -- 172.21.15.1:0/2053977850 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f6905e2160 con 0x55f6901edd80 2024-05-04T03:22:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.374+0000 7ff5feffd640 1 -- 172.21.15.1:0/2053977850 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff618036040 con 0x55f6901edd80 2024-05-04T03:22:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.374+0000 7ff5feffd640 1 -- 172.21.15.1:0/2053977850 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7ff610031470 con 0x55f6901edd80 2024-05-04T03:22:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.374+0000 7ff62c7a4740 10 client.15570 fetch_fsmap finished waiting for FSMap version 29 2024-05-04T03:22:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.374+0000 7ff62c7a4740 10 client.15570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:22:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.374+0000 7ff62c7a4740 20 client.15570 populate_metadata read hostname 'smithi001' 2024-05-04T03:22:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.374+0000 7ff62c7a4740 1 --2- 172.21.15.1:0/2053977850 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55f69022f1b0 0x55f690652a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.375+0000 7ff62c7a4740 4 client.15570 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:22:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.375+0000 7ff61e7fc640 1 --2- 172.21.15.1:0/2053977850 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55f69022f1b0 0x55f690652a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.375+0000 7ff62c7a4740 1 -- 172.21.15.1:0/2053977850 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f69053d050 con 0x55f69022f1b0 2024-05-04T03:22:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.375+0000 7ff5feffd640 10 client.15570 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:22:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.375+0000 7ff61e7fc640 1 --2- 172.21.15.1:0/2053977850 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55f69022f1b0 0x55f690652a90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be5f683759eb5702 server_cookie=c3497d2c8e4f4d2b in_seq=0 out_seq=0 2024-05-04T03:22:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.385+0000 7ff5feffd640 1 -- 172.21.15.1:0/2053977850 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f69053d050 con 0x55f69022f1b0 2024-05-04T03:22:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.385+0000 7ff5feffd640 10 client.15570 handle_command_reply: tid=0 2024-05-04T03:22:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.386+0000 7ff62c7a4740 1 client.15570 shutdown 2024-05-04T03:22:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.386+0000 7ff62c7a4740 1 -- 172.21.15.1:0/2053977850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6901edd80 msgr2=0x55f6905df430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.386+0000 7ff62c7a4740 1 --2- 172.21.15.1:0/2053977850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6901edd80 0x55f6905df430 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff610005710 tx=0x7ff6100099a0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.386+0000 7ff62c7a4740 1 -- 172.21.15.1:0/2053977850 shutdown_connections 2024-05-04T03:22:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.386+0000 7ff62c7a4740 1 --2- 172.21.15.1:0/2053977850 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55f69022f1b0 0x55f690652a90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.386+0000 7ff62c7a4740 1 --2- 172.21.15.1:0/2053977850 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f6902062f0 0x55f6905df970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.386+0000 7ff5feffd640 0 client.15570 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:22:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.386+0000 7ff62c7a4740 1 --2- 172.21.15.1:0/2053977850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f69020e4d0 0x55f6905e08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.386+0000 7ff62c7a4740 1 --2- 172.21.15.1:0/2053977850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6901edd80 0x55f6905df430 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.386+0000 7ff62c7a4740 1 -- 172.21.15.1:0/2053977850 >> 172.21.15.1:0/2053977850 conn(0x55f690223300 msgr2=0x55f6905cbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.386+0000 7ff62c7a4740 1 -- 172.21.15.1:0/2053977850 shutdown_connections 2024-05-04T03:22:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.386+0000 7ff62c7a4740 1 -- 172.21.15.1:0/2053977850 wait complete. 2024-05-04T03:22:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.386+0000 7ff62c7a4740 20 client.15570 trim_cache size 0 max 16384 2024-05-04T03:22:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:22:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:22:03.389 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:22:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:22:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.387+0000 7ff625815640 1 -- 172.21.15.1:0/1853660145 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ff6080b28a0 msgr2=0x7ff6080b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.387+0000 7ff625815640 1 --2- 172.21.15.1:0/1853660145 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ff6080b28a0 0x7ff6080b4db0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55f6905d1c30 tx=0x7ff61802f490 comp rx=0 tx=0).stop 2024-05-04T03:22:03.390 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.387+0000 7ff625815640 1 -- 172.21.15.1:0/1853660145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff620103da0 msgr2=0x7ff6201996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.387+0000 7ff625815640 1 --2- 172.21.15.1:0/1853660145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff620103da0 0x7ff6201996b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff61400e9f0 tx=0x7ff61400edf0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.387+0000 7ff625815640 1 -- 172.21.15.1:0/1853660145 shutdown_connections 2024-05-04T03:22:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.387+0000 7ff625815640 1 --2- 172.21.15.1:0/1853660145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff620108aa0 0x7ff6201a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.387+0000 7ff625815640 1 --2- 172.21.15.1:0/1853660145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff620103da0 0x7ff6201996b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.387+0000 7ff625815640 1 --2- 172.21.15.1:0/1853660145 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ff6080b28a0 0x7ff6080b4db0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.387+0000 7ff625815640 1 --2- 172.21.15.1:0/1853660145 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff620102b60 0x7ff620199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.387+0000 7ff625815640 1 -- 172.21.15.1:0/1853660145 >> 172.21.15.1:0/1853660145 conn(0x7ff6200fdd50 msgr2=0x7ff6201001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.387+0000 7ff625815640 1 -- 172.21.15.1:0/1853660145 shutdown_connections 2024-05-04T03:22:03.391 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:03.387+0000 7ff625815640 1 -- 172.21.15.1:0/1853660145 wait complete. 2024-05-04T03:22:04.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:04 smithi001 ceph-mon[29731]: pgmap v478: 97 pgs: 97 active+clean; 63 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 531 op/s 2024-05-04T03:22:04.932 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:04 smithi077 ceph-mon[33531]: pgmap v478: 97 pgs: 97 active+clean; 63 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 531 op/s 2024-05-04T03:22:04.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:04 smithi072 ceph-mon[33332]: pgmap v478: 97 pgs: 97 active+clean; 63 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.4 MiB/s wr, 531 op/s 2024-05-04T03:22:06.581 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0xffdb8 2024-05-04T03:22:06.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:06 smithi001 ceph-mon[29731]: pgmap v479: 97 pgs: 97 active+clean; 66 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.6 MiB/s wr, 712 op/s 2024-05-04T03:22:06.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:06 smithi077 ceph-mon[33531]: pgmap v479: 97 pgs: 97 active+clean; 66 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.6 MiB/s wr, 712 op/s 2024-05-04T03:22:06.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:06 smithi072 ceph-mon[33332]: pgmap v479: 97 pgs: 97 active+clean; 66 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.6 MiB/s wr, 712 op/s 2024-05-04T03:22:08.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:08 smithi001 ceph-mon[29731]: pgmap v480: 97 pgs: 97 active+clean; 67 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 464 op/s 2024-05-04T03:22:08.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:08 smithi077 ceph-mon[33531]: pgmap v480: 97 pgs: 97 active+clean; 67 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 464 op/s 2024-05-04T03:22:08.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:08 smithi072 ceph-mon[33332]: pgmap v480: 97 pgs: 97 active+clean; 67 MiB data, 707 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 464 op/s 2024-05-04T03:22:10.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:10 smithi001 ceph-mon[29731]: pgmap v481: 97 pgs: 97 active+clean; 68 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 478 op/s 2024-05-04T03:22:10.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:10 smithi077 ceph-mon[33531]: pgmap v481: 97 pgs: 97 active+clean; 68 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 478 op/s 2024-05-04T03:22:10.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:10 smithi072 ceph-mon[33332]: pgmap v481: 97 pgs: 97 active+clean; 68 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 478 op/s 2024-05-04T03:22:11.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:11 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:22:11.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:11 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:22:11.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:11 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:22:11.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:11 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:22:11.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:11 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:22:11.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:11 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:22:11.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:11 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:22:11.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:11 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:22:11.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:11 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:22:11.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:11 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:22:11.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:11 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:22:11.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:11 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:22:12.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:12 smithi001 ceph-mon[29731]: pgmap v482: 97 pgs: 97 active+clean; 70 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 558 op/s 2024-05-04T03:22:12.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:12 smithi077 ceph-mon[33531]: pgmap v482: 97 pgs: 97 active+clean; 70 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 558 op/s 2024-05-04T03:22:12.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:12 smithi072 ceph-mon[33332]: pgmap v482: 97 pgs: 97 active+clean; 70 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.0 MiB/s wr, 558 op/s 2024-05-04T03:22:14.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:14 smithi001 ceph-mon[29731]: pgmap v483: 97 pgs: 97 active+clean; 70 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 410 op/s 2024-05-04T03:22:14.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:14 smithi077 ceph-mon[33531]: pgmap v483: 97 pgs: 97 active+clean; 70 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 410 op/s 2024-05-04T03:22:14.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:14 smithi072 ceph-mon[33332]: pgmap v483: 97 pgs: 97 active+clean; 70 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 410 op/s 2024-05-04T03:22:15.952 INFO:tasks.workunit.client.0.smithi001.stdout:50000 copy from 0x898b8 to 0x92015, (0x875d bytes) at 0xb142c 2024-05-04T03:22:15.952 INFO:tasks.workunit.client.0.smithi001.stdout:All 50000 operations completed A-OK! 2024-05-04T03:22:15.953 INFO:tasks.workunit.client.0.smithi001.stderr:+ ./fsx -z 10MB -N 50000 -p 10000 -l 10485760 2024-05-04T03:22:15.953 INFO:tasks.workunit.client.0.smithi001.stdout:Seed set to 1 2024-05-04T03:22:16.081 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-05-04T03:22:16.082 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-05-04T03:22:16.084 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-05-04T03:22:16.084 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-05-04T03:22:16.084 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support clone range, disabling! 2024-05-04T03:22:16.087 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support dedupe range, disabling! 2024-05-04T03:22:16.089 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero length punch hole 2024-05-04T03:22:16.090 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero length punch hole 2024-05-04T03:22:16.090 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero length punch hole 2024-05-04T03:22:16.090 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero size read 2024-05-04T03:22:16.090 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero length punch hole 2024-05-04T03:22:16.090 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support exchange range, disabling! 2024-05-04T03:22:16.090 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero size read 2024-05-04T03:22:16.090 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero length punch hole 2024-05-04T03:22:16.091 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0x4310e9 2024-05-04T03:22:16.096 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0x954a45 2024-05-04T03:22:16.126 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0x9bc582 2024-05-04T03:22:16.147 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0x9eecbc 2024-05-04T03:22:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.498+0000 7fd04914d640 1 -- 172.21.15.1:0/3270888762 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd044103c40 msgr2=0x7fd044104940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:16.500 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.498+0000 7fd04914d640 1 --2- 172.21.15.1:0/3270888762 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd044103c40 0x7fd044104940 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd034009860 tx=0x7fd03402a340 comp rx=0 tx=0).stop 2024-05-04T03:22:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.498+0000 7fd04914d640 1 -- 172.21.15.1:0/3270888762 shutdown_connections 2024-05-04T03:22:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.498+0000 7fd04914d640 1 --2- 172.21.15.1:0/3270888762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd044109b80 0x7fd04410bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.498+0000 7fd04914d640 1 --2- 172.21.15.1:0/3270888762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd044104e80 0x7fd044105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.498+0000 7fd04914d640 1 --2- 172.21.15.1:0/3270888762 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd044103c40 0x7fd044104940 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.498+0000 7fd04914d640 1 -- 172.21.15.1:0/3270888762 >> 172.21.15.1:0/3270888762 conn(0x7fd0440fee70 msgr2=0x7fd0441012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.498+0000 7fd04914d640 1 -- 172.21.15.1:0/3270888762 shutdown_connections 2024-05-04T03:22:16.501 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.498+0000 7fd04914d640 1 -- 172.21.15.1:0/3270888762 wait complete. 2024-05-04T03:22:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.499+0000 7fd04914d640 1 Processor -- start 2024-05-04T03:22:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.499+0000 7fd04914d640 1 -- start start 2024-05-04T03:22:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.499+0000 7fd04914d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd044103c40 0x7fd04419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.499+0000 7fd04914d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd044104e80 0x7fd04419a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.499+0000 7fd04914d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd044109b80 0x7fd0441a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.499+0000 7fd04914d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd04410f720 con 0x7fd044109b80 2024-05-04T03:22:16.503 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.499+0000 7fd04914d640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd04410f5a0 con 0x7fd044103c40 2024-05-04T03:22:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.499+0000 7fd04914d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd04410f8a0 con 0x7fd044104e80 2024-05-04T03:22:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.500+0000 7fd04359e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd044109b80 0x7fd0441a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.500+0000 7fd042d9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd044103c40 0x7fd04419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.500+0000 7fd04359e640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd044109b80 0x7fd0441a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37280/0 (socket says 172.21.15.1:37280) 2024-05-04T03:22:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.500+0000 7fd04359e640 1 -- 172.21.15.1:0/1898117002 learned_addr learned my addr 172.21.15.1:0/1898117002 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.500+0000 7fd03a59c640 1 --2- 172.21.15.1:0/1898117002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd044104e80 0x7fd04419a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.500+0000 7fd04359e640 1 -- 172.21.15.1:0/1898117002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd044104e80 msgr2=0x7fd04419a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.500+0000 7fd04359e640 1 --2- 172.21.15.1:0/1898117002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd044104e80 0x7fd04419a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.500+0000 7fd04359e640 1 -- 172.21.15.1:0/1898117002 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd044103c40 msgr2=0x7fd04419a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.500+0000 7fd04359e640 1 --2- 172.21.15.1:0/1898117002 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd044103c40 0x7fd04419a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.500+0000 7fd04359e640 1 -- 172.21.15.1:0/1898117002 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0441a1dd0 con 0x7fd044109b80 2024-05-04T03:22:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.500+0000 7fd042d9d640 1 --2- 172.21.15.1:0/1898117002 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd044103c40 0x7fd04419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:22:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.500+0000 7fd04359e640 1 --2- 172.21.15.1:0/1898117002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd044109b80 0x7fd0441a1760 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00eb10 tx=0x7fd03c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.500+0000 7fd040d99640 1 -- 172.21.15.1:0/1898117002 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c004b20 con 0x7fd044109b80 2024-05-04T03:22:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.500+0000 7fd04914d640 1 -- 172.21.15.1:0/1898117002 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0441a20c0 con 0x7fd044109b80 2024-05-04T03:22:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.500+0000 7fd040d99640 1 -- 172.21.15.1:0/1898117002 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd03c004cc0 con 0x7fd044109b80 2024-05-04T03:22:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.500+0000 7fd04914d640 1 -- 172.21.15.1:0/1898117002 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0441a2610 con 0x7fd044109b80 2024-05-04T03:22:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.500+0000 7fd040d99640 1 -- 172.21.15.1:0/1898117002 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c018630 con 0x7fd044109b80 2024-05-04T03:22:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.503+0000 7fd040d99640 1 -- 172.21.15.1:0/1898117002 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7fd03c0187d0 con 0x7fd044109b80 2024-05-04T03:22:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.503+0000 7fd050003740 1 Processor -- start 2024-05-04T03:22:16.505 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.504+0000 7fd050003740 1 -- start start 2024-05-04T03:22:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.504+0000 7fd050003740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582028211b0 0x558202bc6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.504+0000 7fd050003740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582027f82f0 0x558202bc73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.504+0000 7fd050003740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5582028004d0 0x558202bce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.504+0000 7fd050003740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558202b275d0 con 0x5582027f82f0 2024-05-04T03:22:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.504+0000 7fd050003740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x558202ae0890 con 0x5582028004d0 2024-05-04T03:22:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.504+0000 7fd050003740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558202b23590 con 0x5582028211b0 2024-05-04T03:22:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.504+0000 7fd03a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582028211b0 0x558202bc6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.504+0000 7fd042d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582027f82f0 0x558202bc73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.504+0000 7fd042d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582027f82f0 0x558202bc73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37284/0 (socket says 172.21.15.1:37284) 2024-05-04T03:22:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.504+0000 7fd042d9d640 1 -- 172.21.15.1:0/2498549816 learned_addr learned my addr 172.21.15.1:0/2498549816 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.505+0000 7fd03a59c640 1 --2- 172.21.15.1:0/2498549816 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5582028004d0 0x558202bce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.505+0000 7fd040d99640 1 --2- 172.21.15.1:0/1898117002 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd0300b28a0 0x7fd0300b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.505+0000 7fd042d9d640 1 -- 172.21.15.1:0/2498549816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582028211b0 msgr2=0x558202bc6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.505+0000 7fd042d9d640 1 --2- 172.21.15.1:0/2498549816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582028211b0 0x558202bc6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.505+0000 7fd042d9d640 1 -- 172.21.15.1:0/2498549816 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5582028004d0 msgr2=0x558202bce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.505+0000 7fd042d9d640 1 --2- 172.21.15.1:0/2498549816 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5582028004d0 0x558202bce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.505+0000 7fd042d9d640 1 -- 172.21.15.1:0/2498549816 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558202839210 con 0x5582027f82f0 2024-05-04T03:22:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.505+0000 7fd04359e640 1 --2- 172.21.15.1:0/1898117002 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd0300b28a0 0x7fd0300b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.505+0000 7fd040d99640 1 -- 172.21.15.1:0/1898117002 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fd03c0e2890 con 0x7fd044109b80 2024-05-04T03:22:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.505+0000 7fd042d9d640 1 --2- 172.21.15.1:0/2498549816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582027f82f0 0x558202bc73f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd03402a930 tx=0x7fd034005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b3c599bc1550bd server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.505+0000 7fd03959a640 1 -- 172.21.15.1:0/2498549816 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd034037040 con 0x5582027f82f0 2024-05-04T03:22:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.505+0000 7fd03959a640 1 -- 172.21.15.1:0/2498549816 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd034030e10 con 0x5582027f82f0 2024-05-04T03:22:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.505+0000 7fd03959a640 1 -- 172.21.15.1:0/2498549816 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd034032ac0 con 0x5582027f82f0 2024-05-04T03:22:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.506+0000 7fd050003740 1 -- 172.21.15.1:0/2498549816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582027f82f0 msgr2=0x558202bc73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.506+0000 7fd04359e640 1 --2- 172.21.15.1:0/1898117002 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd0300b28a0 0x7fd0300b4db0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0a5fd0 tx=0x7fd03c0704c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.506+0000 7fd050003740 1 --2- 172.21.15.1:0/2498549816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582027f82f0 0x558202bc73f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd03402a930 tx=0x7fd034005cd0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.506+0000 7fd050003740 1 -- 172.21.15.1:0/2498549816 shutdown_connections 2024-05-04T03:22:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.506+0000 7fd050003740 1 --2- 172.21.15.1:0/2498549816 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5582028004d0 0x558202bce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.506+0000 7fd050003740 1 --2- 172.21.15.1:0/2498549816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582027f82f0 0x558202bc73f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.506+0000 7fd050003740 1 --2- 172.21.15.1:0/2498549816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582028211b0 0x558202bc6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.506+0000 7fd050003740 1 -- 172.21.15.1:0/2498549816 >> 172.21.15.1:0/2498549816 conn(0x558202815300 msgr2=0x558202bbe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.507+0000 7fd050003740 1 -- 172.21.15.1:0/2498549816 shutdown_connections 2024-05-04T03:22:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.507+0000 7fd050003740 1 -- 172.21.15.1:0/2498549816 wait complete. 2024-05-04T03:22:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.507+0000 7fd050003740 1 -- start start 2024-05-04T03:22:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.507+0000 7fd050003740 1 Processor -- start 2024-05-04T03:22:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.507+0000 7fd050003740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582027f82f0 0x558202bd09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.507+0000 7fd050003740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5582028004d0 0x558202bd0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.507+0000 7fd050003740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582028211b0 0x558202bd7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.507+0000 7fd050003740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558202bcece0 con 0x5582028211b0 2024-05-04T03:22:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.507+0000 7fd050003740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x558202ae4370 con 0x5582028004d0 2024-05-04T03:22:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.507+0000 7fd050003740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558202bcee60 con 0x5582027f82f0 2024-05-04T03:22:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.507+0000 7fd050003740 10 client.0 _ll_register_callbacks cb 0x558202838a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:22:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.508+0000 7fd03a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582028211b0 0x558202bd7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.508+0000 7fd042d9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5582028004d0 0x558202bd0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.508+0000 7fd03a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582028211b0 0x558202bd7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37286/0 (socket says 172.21.15.1:37286) 2024-05-04T03:22:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.508+0000 7fd03a59c640 1 -- 172.21.15.1:0/2492254588 learned_addr learned my addr 172.21.15.1:0/2492254588 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.508+0000 7fd03a59c640 1 --2- 172.21.15.1:0/2492254588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582027f82f0 0x558202bd09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.508+0000 7fd042d9d640 1 -- 172.21.15.1:0/2492254588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582027f82f0 msgr2=0x558202bd09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.508+0000 7fd042d9d640 1 --2- 172.21.15.1:0/2492254588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582027f82f0 0x558202bd09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.508+0000 7fd042d9d640 1 -- 172.21.15.1:0/2492254588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582028211b0 msgr2=0x558202bd7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.508+0000 7fd042d9d640 1 --2- 172.21.15.1:0/2492254588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582028211b0 0x558202bd7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.508+0000 7fd042d9d640 1 -- 172.21.15.1:0/2492254588 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558202905620 con 0x5582028004d0 2024-05-04T03:22:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.508+0000 7fd042d9d640 1 --2- 172.21.15.1:0/2492254588 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5582028004d0 0x558202bd0f00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd0340303f0 tx=0x7fd034005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79b812f260e956b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.508+0000 7fd02a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-04T03:22:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.508+0000 7fd02a7fc640 1 -- 172.21.15.1:0/2492254588 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd034037030 con 0x5582028004d0 2024-05-04T03:22:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.508+0000 7fd02a7fc640 1 -- 172.21.15.1:0/2492254588 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fd034030760 con 0x5582028004d0 2024-05-04T03:22:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.508+0000 7fd050003740 1 -- 172.21.15.1:0/2492254588 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd034030e10 con 0x5582028004d0 2024-05-04T03:22:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.509+0000 7fd02a7fc640 1 -- 172.21.15.1:0/2492254588 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd034058050 con 0x5582028004d0 2024-05-04T03:22:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.509+0000 7fd050003740 10 client.25319 fetch_fsmap learned FSMap version 29 2024-05-04T03:22:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.509+0000 7fd050003740 1 -- 172.21.15.1:0/2492254588 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558202bdad90 con 0x5582028004d0 2024-05-04T03:22:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.509+0000 7fd02a7fc640 1 -- 172.21.15.1:0/2492254588 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd034030e10 con 0x5582028004d0 2024-05-04T03:22:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.509+0000 7fd02a7fc640 1 -- 172.21.15.1:0/2492254588 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7fd034003b90 con 0x5582028004d0 2024-05-04T03:22:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.509+0000 7fd050003740 10 client.25319 fetch_fsmap finished waiting for FSMap version 29 2024-05-04T03:22:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.509+0000 7fd050003740 10 client.25319 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:22:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.509+0000 7fd050003740 20 client.25319 populate_metadata read hostname 'smithi001' 2024-05-04T03:22:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.509+0000 7fd050003740 1 --2- 172.21.15.1:0/2492254588 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5582027dfd80 0x558202bde0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.509+0000 7fd050003740 4 client.25319 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:22:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.509+0000 7fd050003740 1 -- 172.21.15.1:0/2492254588 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558202b351e0 con 0x5582027dfd80 2024-05-04T03:22:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.509+0000 7fd03a59c640 1 --2- 172.21.15.1:0/2492254588 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5582027dfd80 0x558202bde0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.510+0000 7fd02a7fc640 10 client.25319 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:22:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.510+0000 7fd03a59c640 1 --2- 172.21.15.1:0/2492254588 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5582027dfd80 0x558202bde0f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fc2092af8c809a8 server_cookie=4b5a399ff392dc24 in_seq=0 out_seq=0 2024-05-04T03:22:16.517 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.510+0000 7fd02a7fc640 1 -- 172.21.15.1:0/2492254588 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558202b351e0 con 0x5582027dfd80 2024-05-04T03:22:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.510+0000 7fd02a7fc640 10 client.25319 handle_command_reply: tid=0 2024-05-04T03:22:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.511+0000 7fd050003740 1 client.25319 shutdown 2024-05-04T03:22:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.511+0000 7fd050003740 1 -- 172.21.15.1:0/2492254588 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5582028004d0 msgr2=0x558202bd0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.511+0000 7fd050003740 1 --2- 172.21.15.1:0/2492254588 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5582028004d0 0x558202bd0f00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd0340303f0 tx=0x7fd034005fd0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.511+0000 7fd050003740 1 -- 172.21.15.1:0/2492254588 shutdown_connections 2024-05-04T03:22:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.511+0000 7fd050003740 1 --2- 172.21.15.1:0/2492254588 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5582027dfd80 0x558202bde0f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.511+0000 7fd050003740 1 --2- 172.21.15.1:0/2492254588 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5582028004d0 0x558202bd0f00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.511+0000 7fd050003740 1 --2- 172.21.15.1:0/2492254588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582028211b0 0x558202bd7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.511+0000 7fd050003740 1 --2- 172.21.15.1:0/2492254588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582027f82f0 0x558202bd09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.511+0000 7fd02a7fc640 0 client.25319 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:22:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.511+0000 7fd050003740 1 -- 172.21.15.1:0/2492254588 >> 172.21.15.1:0/2492254588 conn(0x558202815300 msgr2=0x558202bc1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.511+0000 7fd050003740 1 -- 172.21.15.1:0/2492254588 shutdown_connections 2024-05-04T03:22:16.518 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.511+0000 7fd050003740 1 -- 172.21.15.1:0/2492254588 wait complete. 2024-05-04T03:22:16.519 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.511+0000 7fd050003740 20 client.25319 trim_cache size 0 max 16384 2024-05-04T03:22:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.529+0000 7fd050003740 1 Processor -- start 2024-05-04T03:22:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.529+0000 7fd050003740 1 -- start start 2024-05-04T03:22:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.529+0000 7fd050003740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582027f82f0 0x558202bca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.529+0000 7fd050003740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5582028004d0 0x558202bcf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.529+0000 7fd050003740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582028211b0 0x558202bcf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.530+0000 7fd050003740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd0441ab830 con 0x5582027f82f0 2024-05-04T03:22:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.530+0000 7fd050003740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd0441a90a0 con 0x5582028004d0 2024-05-04T03:22:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.530+0000 7fd050003740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd0441a6f40 con 0x5582028211b0 2024-05-04T03:22:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.530+0000 7fd03a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582027f82f0 0x558202bca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.530+0000 7fd042d9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5582028004d0 0x558202bcf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.530+0000 7fd03a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582027f82f0 0x558202bca5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37294/0 (socket says 172.21.15.1:37294) 2024-05-04T03:22:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.530+0000 7fd03a59c640 1 -- 172.21.15.1:0/4114845769 learned_addr learned my addr 172.21.15.1:0/4114845769 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.530+0000 7fd03a59c640 1 --2- 172.21.15.1:0/4114845769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582028211b0 0x558202bcf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.530+0000 7fd042d9d640 1 -- 172.21.15.1:0/4114845769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582028211b0 msgr2=0x558202bcf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.530+0000 7fd042d9d640 1 --2- 172.21.15.1:0/4114845769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582028211b0 0x558202bcf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.530+0000 7fd042d9d640 1 -- 172.21.15.1:0/4114845769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582027f82f0 msgr2=0x558202bca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.530+0000 7fd042d9d640 1 --2- 172.21.15.1:0/4114845769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582027f82f0 0x558202bca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.530+0000 7fd042d9d640 1 -- 172.21.15.1:0/4114845769 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558202bc76b0 con 0x5582028004d0 2024-05-04T03:22:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.530+0000 7fd03a59c640 1 --2- 172.21.15.1:0/4114845769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582027f82f0 0x558202bca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:22:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.530+0000 7fd042d9d640 1 --2- 172.21.15.1:0/4114845769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5582028004d0 0x558202bcf230 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd034003f60 tx=0x7fd0340045a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0583a351f4cb465 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.530+0000 7fd038d99640 1 -- 172.21.15.1:0/4114845769 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd034037070 con 0x5582028004d0 2024-05-04T03:22:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.531+0000 7fd038d99640 1 -- 172.21.15.1:0/4114845769 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd03403de30 con 0x5582028004d0 2024-05-04T03:22:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.531+0000 7fd038d99640 1 -- 172.21.15.1:0/4114845769 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd034032ac0 con 0x5582028004d0 2024-05-04T03:22:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.531+0000 7fd050003740 1 -- 172.21.15.1:0/4114845769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5582028004d0 msgr2=0x558202bcf230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.531+0000 7fd050003740 1 --2- 172.21.15.1:0/4114845769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5582028004d0 0x558202bcf230 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd034003f60 tx=0x7fd0340045a0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.532+0000 7fd050003740 1 -- 172.21.15.1:0/4114845769 shutdown_connections 2024-05-04T03:22:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.532+0000 7fd050003740 1 --2- 172.21.15.1:0/4114845769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5582028004d0 0x558202bcf230 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.532+0000 7fd050003740 1 --2- 172.21.15.1:0/4114845769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582028211b0 0x558202bcf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.532+0000 7fd050003740 1 --2- 172.21.15.1:0/4114845769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582027f82f0 0x558202bca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.532+0000 7fd050003740 1 -- 172.21.15.1:0/4114845769 >> 172.21.15.1:0/4114845769 conn(0x558202815300 msgr2=0x558202bc9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.532+0000 7fd050003740 1 -- 172.21.15.1:0/4114845769 shutdown_connections 2024-05-04T03:22:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.532+0000 7fd050003740 1 -- 172.21.15.1:0/4114845769 wait complete. 2024-05-04T03:22:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.532+0000 7fd050003740 1 -- start start 2024-05-04T03:22:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.532+0000 7fd050003740 1 Processor -- start 2024-05-04T03:22:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.532+0000 7fd050003740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582027f82f0 0x558202bd4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.532+0000 7fd050003740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582028004d0 0x558202bd46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.532+0000 7fd050003740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5582028211b0 0x558202c443b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.532+0000 7fd050003740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fd0441a7560 con 0x5582027f82f0 2024-05-04T03:22:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.532+0000 7fd050003740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd0441a89d0 con 0x5582028211b0 2024-05-04T03:22:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.532+0000 7fd050003740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd0441a7c30 con 0x5582028004d0 2024-05-04T03:22:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.532+0000 7fd050003740 10 client.0 _ll_register_callbacks cb 0x7fd02c009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:22:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.532+0000 7fd03a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582027f82f0 0x558202bd4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.532+0000 7fd03a59c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582027f82f0 0x558202bd4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37296/0 (socket says 172.21.15.1:37296) 2024-05-04T03:22:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.532+0000 7fd03a59c640 1 -- 172.21.15.1:0/3820109309 learned_addr learned my addr 172.21.15.1:0/3820109309 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.532+0000 7fd042d9d640 1 --2- 172.21.15.1:0/3820109309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582028004d0 0x558202bd46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.533+0000 7fd03a59c640 1 --2- 172.21.15.1:0/3820109309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5582028211b0 0x558202c443b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.533+0000 7fd042d9d640 1 -- 172.21.15.1:0/3820109309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5582028211b0 msgr2=0x558202c443b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.533+0000 7fd042d9d640 1 --2- 172.21.15.1:0/3820109309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5582028211b0 0x558202c443b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.533+0000 7fd042d9d640 1 -- 172.21.15.1:0/3820109309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582027f82f0 msgr2=0x558202bd4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.533+0000 7fd042d9d640 1 --2- 172.21.15.1:0/3820109309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582027f82f0 0x558202bd4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.533+0000 7fd042d9d640 1 -- 172.21.15.1:0/3820109309 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558202bd7660 con 0x5582028004d0 2024-05-04T03:22:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.533+0000 7fd03a59c640 1 --2- 172.21.15.1:0/3820109309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582027f82f0 0x558202bd4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:22:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.533+0000 7fd042d9d640 1 --2- 172.21.15.1:0/3820109309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582028004d0 0x558202bd46a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd034003f60 tx=0x7fd03402a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2674c4d05e77fede server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.533+0000 7fd02a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-04T03:22:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.533+0000 7fd02a7fc640 1 -- 172.21.15.1:0/3820109309 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd034037040 con 0x5582028004d0 2024-05-04T03:22:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.533+0000 7fd050003740 1 -- 172.21.15.1:0/3820109309 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd03403de30 con 0x5582028004d0 2024-05-04T03:22:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.533+0000 7fd02a7fc640 1 -- 172.21.15.1:0/3820109309 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fd03403db80 con 0x5582028004d0 2024-05-04T03:22:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.533+0000 7fd02a7fc640 1 -- 172.21.15.1:0/3820109309 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd03406b050 con 0x5582028004d0 2024-05-04T03:22:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.533+0000 7fd050003740 10 client.25288 fetch_fsmap learned FSMap version 29 2024-05-04T03:22:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.533+0000 7fd050003740 1 -- 172.21.15.1:0/3820109309 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558202c457f0 con 0x5582028004d0 2024-05-04T03:22:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.534+0000 7fd02a7fc640 1 -- 172.21.15.1:0/3820109309 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd03403de30 con 0x5582028004d0 2024-05-04T03:22:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.534+0000 7fd02a7fc640 1 -- 172.21.15.1:0/3820109309 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7fd034032880 con 0x5582028004d0 2024-05-04T03:22:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.534+0000 7fd050003740 10 client.25288 fetch_fsmap finished waiting for FSMap version 29 2024-05-04T03:22:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.534+0000 7fd050003740 10 client.25288 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:22:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.534+0000 7fd050003740 20 client.25288 populate_metadata read hostname 'smithi001' 2024-05-04T03:22:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.534+0000 7fd050003740 1 --2- 172.21.15.1:0/3820109309 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5582027b51c0 0x558202c48200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.534+0000 7fd050003740 4 client.25288 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:22:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.534+0000 7fd050003740 1 -- 172.21.15.1:0/3820109309 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558202b2f050 con 0x5582027b51c0 2024-05-04T03:22:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.534+0000 7fd03a59c640 1 --2- 172.21.15.1:0/3820109309 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5582027b51c0 0x558202c48200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.534+0000 7fd03a59c640 1 --2- 172.21.15.1:0/3820109309 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5582027b51c0 0x558202c48200 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f619432410aca2fa server_cookie=fb2b456c3f0c6aa3 in_seq=0 out_seq=0 2024-05-04T03:22:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.534+0000 7fd02a7fc640 10 client.25288 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:22:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.542+0000 7fd02a7fc640 1 -- 172.21.15.1:0/3820109309 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558202b2f050 con 0x5582027b51c0 2024-05-04T03:22:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.542+0000 7fd02a7fc640 10 client.25288 handle_command_reply: tid=0 2024-05-04T03:22:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.543+0000 7fd050003740 1 client.25288 shutdown 2024-05-04T03:22:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.543+0000 7fd050003740 1 -- 172.21.15.1:0/3820109309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582028004d0 msgr2=0x558202bd46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.543+0000 7fd050003740 1 --2- 172.21.15.1:0/3820109309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582028004d0 0x558202bd46a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd034003f60 tx=0x7fd03402a340 comp rx=0 tx=0).stop 2024-05-04T03:22:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.543+0000 7fd050003740 1 -- 172.21.15.1:0/3820109309 shutdown_connections 2024-05-04T03:22:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.543+0000 7fd050003740 1 --2- 172.21.15.1:0/3820109309 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5582027b51c0 0x558202c48200 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.543+0000 7fd050003740 1 --2- 172.21.15.1:0/3820109309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5582028211b0 0x558202c443b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.543+0000 7fd050003740 1 --2- 172.21.15.1:0/3820109309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582028004d0 0x558202bd46a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.543+0000 7fd02a7fc640 0 client.25288 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:22:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.543+0000 7fd050003740 1 --2- 172.21.15.1:0/3820109309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582027f82f0 0x558202bd4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.543+0000 7fd050003740 1 -- 172.21.15.1:0/3820109309 >> 172.21.15.1:0/3820109309 conn(0x558202815300 msgr2=0x558202bcf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.543+0000 7fd050003740 1 -- 172.21.15.1:0/3820109309 shutdown_connections 2024-05-04T03:22:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.543+0000 7fd050003740 1 -- 172.21.15.1:0/3820109309 wait complete. 2024-05-04T03:22:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.543+0000 7fd050003740 20 client.25288 trim_cache size 0 max 16384 2024-05-04T03:22:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:22:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:22:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:22:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:22:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.544+0000 7fd04914d640 1 -- 172.21.15.1:0/1898117002 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd0300b28a0 msgr2=0x7fd0300b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.544+0000 7fd04914d640 1 --2- 172.21.15.1:0/1898117002 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd0300b28a0 0x7fd0300b4db0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0a5fd0 tx=0x7fd03c0704c0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.544+0000 7fd04914d640 1 -- 172.21.15.1:0/1898117002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd044109b80 msgr2=0x7fd0441a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.544+0000 7fd04914d640 1 --2- 172.21.15.1:0/1898117002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd044109b80 0x7fd0441a1760 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00eb10 tx=0x7fd03c00c2d0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.544+0000 7fd04914d640 1 -- 172.21.15.1:0/1898117002 shutdown_connections 2024-05-04T03:22:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.544+0000 7fd04914d640 1 --2- 172.21.15.1:0/1898117002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd044109b80 0x7fd0441a1760 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.544+0000 7fd04914d640 1 --2- 172.21.15.1:0/1898117002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd044104e80 0x7fd04419a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.544+0000 7fd04914d640 1 --2- 172.21.15.1:0/1898117002 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fd0300b28a0 0x7fd0300b4db0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.544+0000 7fd04914d640 1 --2- 172.21.15.1:0/1898117002 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd044103c40 0x7fd04419a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:16.548 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.544+0000 7fd04914d640 1 -- 172.21.15.1:0/1898117002 >> 172.21.15.1:0/1898117002 conn(0x7fd0440fee70 msgr2=0x7fd044105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.544+0000 7fd04914d640 1 -- 172.21.15.1:0/1898117002 shutdown_connections 2024-05-04T03:22:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:16.544+0000 7fd04914d640 1 -- 172.21.15.1:0/1898117002 wait complete. 2024-05-04T03:22:16.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:16 smithi001 ceph-mon[29731]: pgmap v484: 97 pgs: 97 active+clean; 72 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 579 op/s 2024-05-04T03:22:16.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:16 smithi077 ceph-mon[33531]: pgmap v484: 97 pgs: 97 active+clean; 72 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 579 op/s 2024-05-04T03:22:16.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:16 smithi072 ceph-mon[33332]: pgmap v484: 97 pgs: 97 active+clean; 72 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 579 op/s 2024-05-04T03:22:18.006 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0x9fdd84 2024-05-04T03:22:18.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:18 smithi001 ceph-mon[29731]: pgmap v485: 97 pgs: 97 active+clean; 72 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 385 op/s 2024-05-04T03:22:18.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:18 smithi077 ceph-mon[33531]: pgmap v485: 97 pgs: 97 active+clean; 72 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 385 op/s 2024-05-04T03:22:18.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:18 smithi072 ceph-mon[33332]: pgmap v485: 97 pgs: 97 active+clean; 72 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 385 op/s 2024-05-04T03:22:20.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:20 smithi001 ceph-mon[29731]: pgmap v486: 97 pgs: 97 active+clean; 72 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 471 op/s 2024-05-04T03:22:20.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:20 smithi077 ceph-mon[33531]: pgmap v486: 97 pgs: 97 active+clean; 72 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 471 op/s 2024-05-04T03:22:20.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:20 smithi072 ceph-mon[33332]: pgmap v486: 97 pgs: 97 active+clean; 72 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 471 op/s 2024-05-04T03:22:22.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:22 smithi001 ceph-mon[29731]: pgmap v487: 97 pgs: 97 active+clean; 78 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.6 MiB/s wr, 630 op/s 2024-05-04T03:22:22.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:22 smithi077 ceph-mon[33531]: pgmap v487: 97 pgs: 97 active+clean; 78 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.6 MiB/s wr, 630 op/s 2024-05-04T03:22:22.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:22 smithi072 ceph-mon[33332]: pgmap v487: 97 pgs: 97 active+clean; 78 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.6 MiB/s wr, 630 op/s 2024-05-04T03:22:24.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:24 smithi001 ceph-mon[29731]: pgmap v488: 97 pgs: 97 active+clean; 78 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.4 MiB/s wr, 447 op/s 2024-05-04T03:22:24.930 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:24 smithi077 ceph-mon[33531]: pgmap v488: 97 pgs: 97 active+clean; 78 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.4 MiB/s wr, 447 op/s 2024-05-04T03:22:24.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:24 smithi072 ceph-mon[33332]: pgmap v488: 97 pgs: 97 active+clean; 78 MiB data, 800 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.4 MiB/s wr, 447 op/s 2024-05-04T03:22:26.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:26 smithi001 ceph-mon[29731]: pgmap v489: 97 pgs: 97 active+clean; 80 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 4.3 MiB/s wr, 795 op/s 2024-05-04T03:22:26.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:26 smithi077 ceph-mon[33531]: pgmap v489: 97 pgs: 97 active+clean; 80 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 4.3 MiB/s wr, 795 op/s 2024-05-04T03:22:26.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:26 smithi072 ceph-mon[33332]: pgmap v489: 97 pgs: 97 active+clean; 80 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 4.3 MiB/s wr, 795 op/s 2024-05-04T03:22:28.444 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0x9ff773 2024-05-04T03:22:28.824 INFO:tasks.workunit.client.0.smithi001.stdout:20000 punch from 0x34ddcb to 0x35324f, (0x5484 bytes) 2024-05-04T03:22:28.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:28 smithi001 ceph-mon[29731]: pgmap v490: 97 pgs: 97 active+clean; 80 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.3 MiB/s wr, 620 op/s 2024-05-04T03:22:28.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:28 smithi077 ceph-mon[33531]: pgmap v490: 97 pgs: 97 active+clean; 80 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.3 MiB/s wr, 620 op/s 2024-05-04T03:22:28.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:28 smithi072 ceph-mon[33332]: pgmap v490: 97 pgs: 97 active+clean; 80 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.3 MiB/s wr, 620 op/s 2024-05-04T03:22:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.656+0000 7f95f530d640 1 -- 172.21.15.1:0/2127667359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95f0103dc0 msgr2=0x7f95f01041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.656+0000 7f95f530d640 1 --2- 172.21.15.1:0/2127667359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95f0103dc0 0x7f95f01041d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f95e4009900 tx=0x7f95e402a410 comp rx=0 tx=0).stop 2024-05-04T03:22:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.656+0000 7f95f530d640 1 -- 172.21.15.1:0/2127667359 shutdown_connections 2024-05-04T03:22:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.656+0000 7f95f530d640 1 --2- 172.21.15.1:0/2127667359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95f0108ac0 0x7f95f010aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.656+0000 7f95f530d640 1 --2- 172.21.15.1:0/2127667359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95f0103dc0 0x7f95f01041d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.656+0000 7f95f530d640 1 --2- 172.21.15.1:0/2127667359 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95f0102b80 0x7f95f0103880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.656+0000 7f95f530d640 1 -- 172.21.15.1:0/2127667359 >> 172.21.15.1:0/2127667359 conn(0x7f95f00fdd90 msgr2=0x7f95f0100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.656+0000 7f95f530d640 1 -- 172.21.15.1:0/2127667359 shutdown_connections 2024-05-04T03:22:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.656+0000 7f95f530d640 1 -- 172.21.15.1:0/2127667359 wait complete. 2024-05-04T03:22:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.657+0000 7f95f530d640 1 Processor -- start 2024-05-04T03:22:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.657+0000 7f95f530d640 1 -- start start 2024-05-04T03:22:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.657+0000 7f95f530d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95f0102b80 0x7f95f019a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.657+0000 7f95f530d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95f0108ac0 0x7f95f019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.657+0000 7f95f530d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95f019f7c0 0x7f95f01a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.657+0000 7f95f530d640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f95f010e6b0 con 0x7f95f0108ac0 2024-05-04T03:22:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.657+0000 7f95f530d640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f95f010e530 con 0x7f95f0102b80 2024-05-04T03:22:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.657+0000 7f95f530d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f95f010e830 con 0x7f95f019f7c0 2024-05-04T03:22:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.657+0000 7f95ee7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95f0108ac0 0x7f95f019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.658+0000 7f95ee7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95f0108ac0 0x7f95f019a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37788/0 (socket says 172.21.15.1:37788) 2024-05-04T03:22:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.658+0000 7f95ee7fc640 1 -- 172.21.15.1:0/1690968358 learned_addr learned my addr 172.21.15.1:0/1690968358 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.658+0000 7f95ef7fe640 1 --2- 172.21.15.1:0/1690968358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95f019f7c0 0x7f95f01a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.658+0000 7f95eeffd640 1 --2- 172.21.15.1:0/1690968358 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95f0102b80 0x7f95f019a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.658+0000 7f95ee7fc640 1 -- 172.21.15.1:0/1690968358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95f019f7c0 msgr2=0x7f95f01a1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.658+0000 7f95ee7fc640 1 --2- 172.21.15.1:0/1690968358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95f019f7c0 0x7f95f01a1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.658+0000 7f95ee7fc640 1 -- 172.21.15.1:0/1690968358 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95f0102b80 msgr2=0x7f95f019a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.658+0000 7f95ee7fc640 1 --2- 172.21.15.1:0/1690968358 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95f0102b80 0x7f95f019a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.658+0000 7f95ee7fc640 1 -- 172.21.15.1:0/1690968358 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f01a2310 con 0x7f95f0108ac0 2024-05-04T03:22:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.658+0000 7f95ef7fe640 1 --2- 172.21.15.1:0/1690968358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95f019f7c0 0x7f95f01a1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:22:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.658+0000 7f95ee7fc640 1 --2- 172.21.15.1:0/1690968358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95f0108ac0 0x7f95f019a8c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f95e4006010 tx=0x7f95e40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.658+0000 7f95d7fff640 1 -- 172.21.15.1:0/1690968358 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e403f030 con 0x7f95f0108ac0 2024-05-04T03:22:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.658+0000 7f95d7fff640 1 -- 172.21.15.1:0/1690968358 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95e4031910 con 0x7f95f0108ac0 2024-05-04T03:22:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.658+0000 7f95f530d640 1 -- 172.21.15.1:0/1690968358 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f01a25a0 con 0x7f95f0108ac0 2024-05-04T03:22:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.658+0000 7f95d7fff640 1 -- 172.21.15.1:0/1690968358 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e4035070 con 0x7f95f0108ac0 2024-05-04T03:22:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.659+0000 7f95f530d640 1 -- 172.21.15.1:0/1690968358 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f01a2a40 con 0x7f95f0108ac0 2024-05-04T03:22:29.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.661+0000 7f95d7fff640 1 -- 172.21.15.1:0/1690968358 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f95e4042030 con 0x7f95f0108ac0 2024-05-04T03:22:29.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.661+0000 7f95fc1c4740 1 Processor -- start 2024-05-04T03:22:29.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.661+0000 7f95fc1c4740 1 -- start start 2024-05-04T03:22:29.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.661+0000 7f95fc1c4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558baa4181b0 0x558baa7b9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:29.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.661+0000 7f95fc1c4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558baa3ef2f0 0x558baa7b9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:29.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.661+0000 7f95fc1c4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558baa3f74d0 0x558baa7c0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:29.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.661+0000 7f95fc1c4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558baa71e5d0 con 0x558baa4181b0 2024-05-04T03:22:29.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.661+0000 7f95fc1c4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x558baa6d7890 con 0x558baa3ef2f0 2024-05-04T03:22:29.670 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.661+0000 7f95fc1c4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558baa71a590 con 0x558baa3f74d0 2024-05-04T03:22:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.662+0000 7f95ef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558baa4181b0 0x558baa7b9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.662+0000 7f95ee7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558baa3f74d0 0x558baa7c0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.662+0000 7f95ef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558baa4181b0 0x558baa7b9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37790/0 (socket says 172.21.15.1:37790) 2024-05-04T03:22:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.662+0000 7f95ee7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558baa3f74d0 0x558baa7c0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.1:55776/0 (socket says 172.21.15.1:55776) 2024-05-04T03:22:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.662+0000 7f95eeffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558baa3ef2f0 0x558baa7b9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.662+0000 7f95ee7fc640 1 -- 172.21.15.1:0/3452763999 learned_addr learned my addr 172.21.15.1:0/3452763999 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.662+0000 7f95d7fff640 1 --2- 172.21.15.1:0/1690968358 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f95d80b28a0 0x7f95d80b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.662+0000 7f95eeffd640 1 -- 172.21.15.1:0/3452763999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558baa3f74d0 msgr2=0x558baa7c0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.662+0000 7f95eeffd640 1 --2- 172.21.15.1:0/3452763999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558baa3f74d0 0x558baa7c0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.662+0000 7f95eeffd640 1 -- 172.21.15.1:0/3452763999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558baa4181b0 msgr2=0x558baa7b9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.662+0000 7f95eeffd640 1 --2- 172.21.15.1:0/3452763999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558baa4181b0 0x558baa7b9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.662+0000 7f95eeffd640 1 -- 172.21.15.1:0/3452763999 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558baa430210 con 0x558baa3ef2f0 2024-05-04T03:22:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.662+0000 7f95ef7fe640 1 --2- 172.21.15.1:0/1690968358 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f95d80b28a0 0x7f95d80b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.662+0000 7f95d7fff640 1 -- 172.21.15.1:0/1690968358 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f95e41024c0 con 0x7f95f0108ac0 2024-05-04T03:22:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.662+0000 7f95eeffd640 1 --2- 172.21.15.1:0/3452763999 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558baa3ef2f0 0x558baa7b9fe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f95e0005fd0 tx=0x7f95e000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5497f55836a565c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.663+0000 7f95d57fa640 1 -- 172.21.15.1:0/3452763999 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e0015070 con 0x558baa3ef2f0 2024-05-04T03:22:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.663+0000 7f95d57fa640 1 -- 172.21.15.1:0/3452763999 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95e00054d0 con 0x558baa3ef2f0 2024-05-04T03:22:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.663+0000 7f95d57fa640 1 -- 172.21.15.1:0/3452763999 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e0011440 con 0x558baa3ef2f0 2024-05-04T03:22:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.664+0000 7f95fc1c4740 1 -- 172.21.15.1:0/3452763999 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558baa3ef2f0 msgr2=0x558baa7b9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.664+0000 7f95ef7fe640 1 --2- 172.21.15.1:0/1690968358 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f95d80b28a0 0x7f95d80b4db0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f95e8005c40 tx=0x7f95e8004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.664+0000 7f95fc1c4740 1 --2- 172.21.15.1:0/3452763999 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558baa3ef2f0 0x558baa7b9fe0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f95e0005fd0 tx=0x7f95e000f730 comp rx=0 tx=0).stop 2024-05-04T03:22:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.664+0000 7f95fc1c4740 1 -- 172.21.15.1:0/3452763999 shutdown_connections 2024-05-04T03:22:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.664+0000 7f95fc1c4740 1 --2- 172.21.15.1:0/3452763999 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558baa3ef2f0 0x558baa7b9fe0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.664+0000 7f95fc1c4740 1 --2- 172.21.15.1:0/3452763999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558baa3f74d0 0x558baa7c0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.664+0000 7f95fc1c4740 1 --2- 172.21.15.1:0/3452763999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558baa4181b0 0x558baa7b9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.664+0000 7f95fc1c4740 1 -- 172.21.15.1:0/3452763999 >> 172.21.15.1:0/3452763999 conn(0x558baa40c300 msgr2=0x558baa7b53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.664+0000 7f95fc1c4740 1 -- 172.21.15.1:0/3452763999 shutdown_connections 2024-05-04T03:22:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.664+0000 7f95fc1c4740 1 -- 172.21.15.1:0/3452763999 wait complete. 2024-05-04T03:22:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.664+0000 7f95fc1c4740 1 -- start start 2024-05-04T03:22:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.664+0000 7f95fc1c4740 1 Processor -- start 2024-05-04T03:22:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.664+0000 7f95fc1c4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558baa3ef2f0 0x558baa7c36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.665+0000 7f95fc1c4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558baa3f74d0 0x558baa7c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.665+0000 7f95fc1c4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558baa4181b0 0x558baa7cab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.665+0000 7f95fc1c4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558baa7c1930 con 0x558baa4181b0 2024-05-04T03:22:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.665+0000 7f95fc1c4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x558baa6db370 con 0x558baa3ef2f0 2024-05-04T03:22:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.665+0000 7f95fc1c4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558baa7c1ab0 con 0x558baa3f74d0 2024-05-04T03:22:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.665+0000 7f95fc1c4740 10 client.0 _ll_register_callbacks cb 0x558baa42fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:22:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.665+0000 7f95ef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558baa4181b0 0x558baa7cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.665+0000 7f95ee7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558baa3f74d0 0x558baa7c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.665+0000 7f95ee7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558baa3f74d0 0x558baa7c3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.1:55792/0 (socket says 172.21.15.1:55792) 2024-05-04T03:22:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.665+0000 7f95ef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558baa4181b0 0x558baa7cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37806/0 (socket says 172.21.15.1:37806) 2024-05-04T03:22:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.665+0000 7f95ee7fc640 1 -- 172.21.15.1:0/3947335287 learned_addr learned my addr 172.21.15.1:0/3947335287 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.665+0000 7f95eeffd640 1 --2- 172.21.15.1:0/3947335287 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558baa3ef2f0 0x558baa7c36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.665+0000 7f95ee7fc640 1 -- 172.21.15.1:0/3947335287 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558baa3ef2f0 msgr2=0x558baa7c36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.665+0000 7f95ee7fc640 1 --2- 172.21.15.1:0/3947335287 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558baa3ef2f0 0x558baa7c36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.665+0000 7f95ee7fc640 1 -- 172.21.15.1:0/3947335287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558baa4181b0 msgr2=0x558baa7cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.665+0000 7f95ee7fc640 1 --2- 172.21.15.1:0/3947335287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558baa4181b0 0x558baa7cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.665+0000 7f95ee7fc640 1 -- 172.21.15.1:0/3947335287 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558baa4fc620 con 0x558baa3f74d0 2024-05-04T03:22:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.665+0000 7f95ee7fc640 1 --2- 172.21.15.1:0/3947335287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558baa3f74d0 0x558baa7c3be0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f95e4032af0 tx=0x7f95e40c6060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=810f491920d53c27 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.665+0000 7f95d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-04T03:22:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.665+0000 7f95d2ff5640 1 -- 172.21.15.1:0/3947335287 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e4049030 con 0x558baa3f74d0 2024-05-04T03:22:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.665+0000 7f95d2ff5640 1 -- 172.21.15.1:0/3947335287 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f95e40ca010 con 0x558baa3f74d0 2024-05-04T03:22:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.665+0000 7f95fc1c4740 1 -- 172.21.15.1:0/3947335287 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95e00054d0 con 0x558baa3f74d0 2024-05-04T03:22:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.666+0000 7f95d2ff5640 1 -- 172.21.15.1:0/3947335287 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f95e40c9970 con 0x558baa3f74d0 2024-05-04T03:22:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.666+0000 7f95fc1c4740 10 client.25300 fetch_fsmap learned FSMap version 29 2024-05-04T03:22:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.666+0000 7f95fc1c4740 1 -- 172.21.15.1:0/3947335287 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558baa7c1930 con 0x558baa3f74d0 2024-05-04T03:22:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.666+0000 7f95d2ff5640 1 -- 172.21.15.1:0/3947335287 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95e00054d0 con 0x558baa3f74d0 2024-05-04T03:22:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.666+0000 7f95d2ff5640 1 -- 172.21.15.1:0/3947335287 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f95e404c060 con 0x558baa3f74d0 2024-05-04T03:22:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.666+0000 7f95fc1c4740 10 client.25300 fetch_fsmap finished waiting for FSMap version 29 2024-05-04T03:22:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.666+0000 7f95fc1c4740 10 client.25300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:22:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.666+0000 7f95fc1c4740 20 client.25300 populate_metadata read hostname 'smithi001' 2024-05-04T03:22:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.666+0000 7f95fc1c4740 1 --2- 172.21.15.1:0/3947335287 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x558baa3d6d80 0x558baa7d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.666+0000 7f95fc1c4740 4 client.25300 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:22:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.666+0000 7f95fc1c4740 1 -- 172.21.15.1:0/3947335287 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558baa72c1e0 con 0x558baa3d6d80 2024-05-04T03:22:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.666+0000 7f95eeffd640 1 --2- 172.21.15.1:0/3947335287 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x558baa3d6d80 0x558baa7d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.667+0000 7f95d2ff5640 10 client.25300 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:22:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.667+0000 7f95eeffd640 1 --2- 172.21.15.1:0/3947335287 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x558baa3d6d80 0x558baa7d1310 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b87c57f8f510be6 server_cookie=5c9d54bf47d9e374 in_seq=0 out_seq=0 2024-05-04T03:22:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.667+0000 7f95d2ff5640 1 -- 172.21.15.1:0/3947335287 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558baa72c1e0 con 0x558baa3d6d80 2024-05-04T03:22:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.667+0000 7f95d2ff5640 10 client.25300 handle_command_reply: tid=0 2024-05-04T03:22:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.667+0000 7f95fc1c4740 1 client.25300 shutdown 2024-05-04T03:22:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.668+0000 7f95fc1c4740 1 -- 172.21.15.1:0/3947335287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558baa3f74d0 msgr2=0x558baa7c3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.668+0000 7f95fc1c4740 1 --2- 172.21.15.1:0/3947335287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558baa3f74d0 0x558baa7c3be0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f95e4032af0 tx=0x7f95e40c6060 comp rx=0 tx=0).stop 2024-05-04T03:22:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.668+0000 7f95fc1c4740 1 -- 172.21.15.1:0/3947335287 shutdown_connections 2024-05-04T03:22:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.668+0000 7f95fc1c4740 1 --2- 172.21.15.1:0/3947335287 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x558baa3d6d80 0x558baa7d1310 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.668+0000 7f95fc1c4740 1 --2- 172.21.15.1:0/3947335287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558baa4181b0 0x558baa7cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.668+0000 7f95d2ff5640 0 client.25300 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:22:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.668+0000 7f95fc1c4740 1 --2- 172.21.15.1:0/3947335287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558baa3f74d0 0x558baa7c3be0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.668+0000 7f95fc1c4740 1 --2- 172.21.15.1:0/3947335287 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558baa3ef2f0 0x558baa7c36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.668+0000 7f95fc1c4740 1 -- 172.21.15.1:0/3947335287 >> 172.21.15.1:0/3947335287 conn(0x558baa40c300 msgr2=0x558baa7b8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.668+0000 7f95fc1c4740 1 -- 172.21.15.1:0/3947335287 shutdown_connections 2024-05-04T03:22:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.668+0000 7f95fc1c4740 1 -- 172.21.15.1:0/3947335287 wait complete. 2024-05-04T03:22:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.668+0000 7f95fc1c4740 20 client.25300 trim_cache size 0 max 16384 2024-05-04T03:22:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.686+0000 7f95fc1c4740 1 Processor -- start 2024-05-04T03:22:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.686+0000 7f95fc1c4740 1 -- start start 2024-05-04T03:22:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.686+0000 7f95fc1c4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558baa3ef2f0 0x558baa7b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.686+0000 7f95fc1c4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558baa3f74d0 0x558baa7b8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.686+0000 7f95fc1c4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558baa4181b0 0x558baa7b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:29.689 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.686+0000 7f95fc1c4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f95f01ae020 con 0x558baa4181b0 2024-05-04T03:22:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.686+0000 7f95fc1c4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f95f01ab890 con 0x558baa3ef2f0 2024-05-04T03:22:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.686+0000 7f95fc1c4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f95f01a9700 con 0x558baa3f74d0 2024-05-04T03:22:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.686+0000 7f95ef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558baa4181b0 0x558baa7b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.686+0000 7f95ee7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558baa3f74d0 0x558baa7b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.686+0000 7f95ef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558baa4181b0 0x558baa7b4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37822/0 (socket says 172.21.15.1:37822) 2024-05-04T03:22:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.686+0000 7f95ef7fe640 1 -- 172.21.15.1:0/3720352711 learned_addr learned my addr 172.21.15.1:0/3720352711 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.686+0000 7f95ee7fc640 1 --2- 172.21.15.1:0/3720352711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558baa3f74d0 0x558baa7b8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.1:55794/0 (socket says 172.21.15.1:55794) 2024-05-04T03:22:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.686+0000 7f95eeffd640 1 --2- 172.21.15.1:0/3720352711 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558baa3ef2f0 0x558baa7b81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.687+0000 7f95ee7fc640 1 -- 172.21.15.1:0/3720352711 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558baa3ef2f0 msgr2=0x558baa7b81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.687+0000 7f95ee7fc640 1 --2- 172.21.15.1:0/3720352711 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558baa3ef2f0 0x558baa7b81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.687+0000 7f95ee7fc640 1 -- 172.21.15.1:0/3720352711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558baa4181b0 msgr2=0x558baa7b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:29.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.687+0000 7f95ee7fc640 1 --2- 172.21.15.1:0/3720352711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558baa4181b0 0x558baa7b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.687+0000 7f95ee7fc640 1 -- 172.21.15.1:0/3720352711 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558baa7af4d0 con 0x558baa3f74d0 2024-05-04T03:22:29.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.687+0000 7f95eeffd640 1 --2- 172.21.15.1:0/3720352711 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558baa3ef2f0 0x558baa7b81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:22:29.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.687+0000 7f95ee7fc640 1 --2- 172.21.15.1:0/3720352711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558baa3f74d0 0x558baa7b8730 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f95e40c9d60 tx=0x7f95e4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=920a49157a60fed server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:29.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.687+0000 7f95d4ff9640 1 -- 172.21.15.1:0/3720352711 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e4049030 con 0x558baa3f74d0 2024-05-04T03:22:29.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.687+0000 7f95d4ff9640 1 -- 172.21.15.1:0/3720352711 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95e40eae50 con 0x558baa3f74d0 2024-05-04T03:22:29.691 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.687+0000 7f95d4ff9640 1 -- 172.21.15.1:0/3720352711 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e40c9310 con 0x558baa3f74d0 2024-05-04T03:22:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.688+0000 7f95fc1c4740 1 -- 172.21.15.1:0/3720352711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558baa3f74d0 msgr2=0x558baa7b8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.688+0000 7f95fc1c4740 1 --2- 172.21.15.1:0/3720352711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558baa3f74d0 0x558baa7b8730 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f95e40c9d60 tx=0x7f95e4009260 comp rx=0 tx=0).stop 2024-05-04T03:22:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.688+0000 7f95fc1c4740 1 -- 172.21.15.1:0/3720352711 shutdown_connections 2024-05-04T03:22:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.688+0000 7f95fc1c4740 1 --2- 172.21.15.1:0/3720352711 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558baa4181b0 0x558baa7b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.688+0000 7f95fc1c4740 1 --2- 172.21.15.1:0/3720352711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558baa3f74d0 0x558baa7b8730 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.688+0000 7f95fc1c4740 1 --2- 172.21.15.1:0/3720352711 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558baa3ef2f0 0x558baa7b81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.688+0000 7f95fc1c4740 1 -- 172.21.15.1:0/3720352711 >> 172.21.15.1:0/3720352711 conn(0x558baa40c300 msgr2=0x558baa7bdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.688+0000 7f95fc1c4740 1 -- 172.21.15.1:0/3720352711 shutdown_connections 2024-05-04T03:22:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.688+0000 7f95fc1c4740 1 -- 172.21.15.1:0/3720352711 wait complete. 2024-05-04T03:22:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.688+0000 7f95fc1c4740 1 -- start start 2024-05-04T03:22:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.689+0000 7f95fc1c4740 1 Processor -- start 2024-05-04T03:22:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.689+0000 7f95fc1c4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558baa3ef2f0 0x558baa7cb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.689+0000 7f95fc1c4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558baa3f74d0 0x558baa7c8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.689+0000 7f95fc1c4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558baa4181b0 0x558baa7c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.689+0000 7f95fc1c4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f95f01a9d50 con 0x558baa4181b0 2024-05-04T03:22:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.689+0000 7f95fc1c4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f95f01ab1c0 con 0x558baa3f74d0 2024-05-04T03:22:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.689+0000 7f95fc1c4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f95f01aa420 con 0x558baa3ef2f0 2024-05-04T03:22:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.689+0000 7f95fc1c4740 10 client.0 _ll_register_callbacks cb 0x7f95e4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:22:29.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.689+0000 7f95ee7fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558baa3f74d0 0x558baa7c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:29.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.689+0000 7f95ef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558baa4181b0 0x558baa7c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:29.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.689+0000 7f95ee7fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558baa3f74d0 0x558baa7c8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:56810/0 (socket says 172.21.15.1:56810) 2024-05-04T03:22:29.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.689+0000 7f95ef7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558baa4181b0 0x558baa7c8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37828/0 (socket says 172.21.15.1:37828) 2024-05-04T03:22:29.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.689+0000 7f95eeffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558baa3ef2f0 0x558baa7cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:29.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.689+0000 7f95ee7fc640 1 -- 172.21.15.1:0/1208307014 learned_addr learned my addr 172.21.15.1:0/1208307014 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:29.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.689+0000 7f95ef7fe640 1 -- 172.21.15.1:0/1208307014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558baa3ef2f0 msgr2=0x558baa7cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:29.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.689+0000 7f95ef7fe640 1 --2- 172.21.15.1:0/1208307014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558baa3ef2f0 0x558baa7cb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.689+0000 7f95ef7fe640 1 -- 172.21.15.1:0/1208307014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558baa3f74d0 msgr2=0x558baa7c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:29.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.689+0000 7f95ef7fe640 1 --2- 172.21.15.1:0/1208307014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558baa3f74d0 0x558baa7c8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.697 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.689+0000 7f95ef7fe640 1 -- 172.21.15.1:0/1208307014 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558baa837730 con 0x558baa4181b0 2024-05-04T03:22:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.689+0000 7f95ee7fc640 1 --2- 172.21.15.1:0/1208307014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558baa3f74d0 0x558baa7c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:22:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.690+0000 7f95ef7fe640 1 --2- 172.21.15.1:0/1208307014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558baa4181b0 0x558baa7c8ac0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f95e801f6f0 tx=0x7f95e8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30506adb1deda929 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.690+0000 7f95d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:22:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.690+0000 7f95d2ff5640 1 -- 172.21.15.1:0/1208307014 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e802f020 con 0x558baa4181b0 2024-05-04T03:22:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.690+0000 7f95d2ff5640 1 -- 172.21.15.1:0/1208307014 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f95e801c7a0 con 0x558baa4181b0 2024-05-04T03:22:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.690+0000 7f95fc1c4740 1 -- 172.21.15.1:0/1208307014 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95e40eae50 con 0x558baa4181b0 2024-05-04T03:22:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.690+0000 7f95d2ff5640 1 -- 172.21.15.1:0/1208307014 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f95e801c9a0 con 0x558baa4181b0 2024-05-04T03:22:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.690+0000 7f95fc1c4740 10 client.15606 fetch_fsmap learned FSMap version 29 2024-05-04T03:22:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.690+0000 7f95fc1c4740 1 -- 172.21.15.1:0/1208307014 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558baa838a60 con 0x558baa4181b0 2024-05-04T03:22:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.690+0000 7f95d2ff5640 1 -- 172.21.15.1:0/1208307014 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95e40eae50 con 0x558baa4181b0 2024-05-04T03:22:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.690+0000 7f95d2ff5640 1 -- 172.21.15.1:0/1208307014 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f95e802bc50 con 0x558baa4181b0 2024-05-04T03:22:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.690+0000 7f95fc1c4740 10 client.15606 fetch_fsmap finished waiting for FSMap version 29 2024-05-04T03:22:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.690+0000 7f95fc1c4740 10 client.15606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:22:29.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.690+0000 7f95fc1c4740 20 client.15606 populate_metadata read hostname 'smithi001' 2024-05-04T03:22:29.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.690+0000 7f95fc1c4740 1 --2- 172.21.15.1:0/1208307014 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x558baa3ac1c0 0x558baa83b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:29.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.690+0000 7f95fc1c4740 4 client.15606 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:22:29.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.690+0000 7f95fc1c4740 1 -- 172.21.15.1:0/1208307014 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558baa726050 con 0x558baa3ac1c0 2024-05-04T03:22:29.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.691+0000 7f95eeffd640 1 --2- 172.21.15.1:0/1208307014 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x558baa3ac1c0 0x558baa83b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:29.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.691+0000 7f95d2ff5640 10 client.15606 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:22:29.699 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.691+0000 7f95eeffd640 1 --2- 172.21.15.1:0/1208307014 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x558baa3ac1c0 0x558baa83b4b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7b7c6dc45e56cf1 server_cookie=629c3f9375b48888 in_seq=0 out_seq=0 2024-05-04T03:22:29.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.698+0000 7f95d2ff5640 1 -- 172.21.15.1:0/1208307014 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558baa726050 con 0x558baa3ac1c0 2024-05-04T03:22:29.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.698+0000 7f95d2ff5640 10 client.15606 handle_command_reply: tid=0 2024-05-04T03:22:29.700 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.698+0000 7f95fc1c4740 1 client.15606 shutdown 2024-05-04T03:22:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.698+0000 7f95fc1c4740 1 -- 172.21.15.1:0/1208307014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558baa4181b0 msgr2=0x558baa7c8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.698+0000 7f95fc1c4740 1 --2- 172.21.15.1:0/1208307014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558baa4181b0 0x558baa7c8ac0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f95e801f6f0 tx=0x7f95e8004ab0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.698+0000 7f95fc1c4740 1 -- 172.21.15.1:0/1208307014 shutdown_connections 2024-05-04T03:22:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.698+0000 7f95fc1c4740 1 --2- 172.21.15.1:0/1208307014 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x558baa3ac1c0 0x558baa83b4b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.698+0000 7f95fc1c4740 1 --2- 172.21.15.1:0/1208307014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558baa3f74d0 0x558baa7c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.698+0000 7f95d2ff5640 0 client.15606 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:22:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.698+0000 7f95fc1c4740 1 --2- 172.21.15.1:0/1208307014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558baa4181b0 0x558baa7c8ac0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.698+0000 7f95fc1c4740 1 --2- 172.21.15.1:0/1208307014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558baa3ef2f0 0x558baa7cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.698+0000 7f95fc1c4740 1 -- 172.21.15.1:0/1208307014 >> 172.21.15.1:0/1208307014 conn(0x558baa40c300 msgr2=0x558baa7b81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.698+0000 7f95fc1c4740 1 -- 172.21.15.1:0/1208307014 shutdown_connections 2024-05-04T03:22:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.698+0000 7f95fc1c4740 1 -- 172.21.15.1:0/1208307014 wait complete. 2024-05-04T03:22:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.699+0000 7f95fc1c4740 20 client.15606 trim_cache size 0 max 16384 2024-05-04T03:22:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:22:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:22:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:22:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:22:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.699+0000 7f95f530d640 1 -- 172.21.15.1:0/1690968358 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f95d80b28a0 msgr2=0x7f95d80b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.699+0000 7f95f530d640 1 --2- 172.21.15.1:0/1690968358 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f95d80b28a0 0x7f95d80b4db0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f95e8005c40 tx=0x7f95e8004a40 comp rx=0 tx=0).stop 2024-05-04T03:22:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.699+0000 7f95f530d640 1 -- 172.21.15.1:0/1690968358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95f0108ac0 msgr2=0x7f95f019a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.699+0000 7f95f530d640 1 --2- 172.21.15.1:0/1690968358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95f0108ac0 0x7f95f019a8c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f95e4006010 tx=0x7f95e40042f0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.699+0000 7f95f530d640 1 -- 172.21.15.1:0/1690968358 shutdown_connections 2024-05-04T03:22:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.699+0000 7f95f530d640 1 --2- 172.21.15.1:0/1690968358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f95f019f7c0 0x7f95f01a1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.699+0000 7f95f530d640 1 --2- 172.21.15.1:0/1690968358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95f0108ac0 0x7f95f019a8c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.699+0000 7f95f530d640 1 --2- 172.21.15.1:0/1690968358 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f95d80b28a0 0x7f95d80b4db0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.699+0000 7f95f530d640 1 --2- 172.21.15.1:0/1690968358 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95f0102b80 0x7f95f019a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.699+0000 7f95f530d640 1 -- 172.21.15.1:0/1690968358 >> 172.21.15.1:0/1690968358 conn(0x7f95f00fdd90 msgr2=0x7f95f010a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:29.703 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.699+0000 7f95f530d640 1 -- 172.21.15.1:0/1690968358 shutdown_connections 2024-05-04T03:22:29.704 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:29.699+0000 7f95f530d640 1 -- 172.21.15.1:0/1690968358 wait complete. 2024-05-04T03:22:30.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:30 smithi001 ceph-mon[29731]: pgmap v491: 97 pgs: 97 active+clean; 86 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.3 MiB/s wr, 828 op/s 2024-05-04T03:22:30.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:30 smithi077 ceph-mon[33531]: pgmap v491: 97 pgs: 97 active+clean; 86 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.3 MiB/s wr, 828 op/s 2024-05-04T03:22:30.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:30 smithi072 ceph-mon[33332]: pgmap v491: 97 pgs: 97 active+clean; 86 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.3 MiB/s wr, 828 op/s 2024-05-04T03:22:32.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:32 smithi001 ceph-mon[29731]: pgmap v492: 97 pgs: 97 active+clean; 86 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 873 op/s 2024-05-04T03:22:32.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:32 smithi077 ceph-mon[33531]: pgmap v492: 97 pgs: 97 active+clean; 86 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 873 op/s 2024-05-04T03:22:32.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:32 smithi072 ceph-mon[33332]: pgmap v492: 97 pgs: 97 active+clean; 86 MiB data, 946 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 873 op/s 2024-05-04T03:22:34.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:34 smithi001 ceph-mon[29731]: pgmap v493: 97 pgs: 97 active+clean; 86 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 3.8 MiB/s wr, 695 op/s 2024-05-04T03:22:34.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:34 smithi077 ceph-mon[33531]: pgmap v493: 97 pgs: 97 active+clean; 86 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 3.8 MiB/s wr, 695 op/s 2024-05-04T03:22:34.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:34 smithi072 ceph-mon[33332]: pgmap v493: 97 pgs: 97 active+clean; 86 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 3.8 MiB/s wr, 695 op/s 2024-05-04T03:22:36.372 INFO:tasks.workunit.client.0.smithi001.stdout:30000 trunc from 0x7a5444 to 0x143a5 2024-05-04T03:22:36.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:36 smithi001 ceph-mon[29731]: pgmap v494: 97 pgs: 97 active+clean; 85 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.4 MiB/s wr, 998 op/s 2024-05-04T03:22:36.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:36 smithi077 ceph-mon[33531]: pgmap v494: 97 pgs: 97 active+clean; 85 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.4 MiB/s wr, 998 op/s 2024-05-04T03:22:36.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:36 smithi072 ceph-mon[33332]: pgmap v494: 97 pgs: 97 active+clean; 85 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.4 MiB/s wr, 998 op/s 2024-05-04T03:22:38.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:38 smithi001 ceph-mon[29731]: pgmap v495: 97 pgs: 97 active+clean; 85 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.5 MiB/s wr, 649 op/s 2024-05-04T03:22:38.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:38 smithi077 ceph-mon[33531]: pgmap v495: 97 pgs: 97 active+clean; 85 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.5 MiB/s wr, 649 op/s 2024-05-04T03:22:38.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:38 smithi072 ceph-mon[33332]: pgmap v495: 97 pgs: 97 active+clean; 85 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.5 MiB/s wr, 649 op/s 2024-05-04T03:22:40.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:40 smithi001 ceph-mon[29731]: pgmap v496: 97 pgs: 97 active+clean; 91 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.3 MiB/s wr, 810 op/s 2024-05-04T03:22:40.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:40 smithi077 ceph-mon[33531]: pgmap v496: 97 pgs: 97 active+clean; 91 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.3 MiB/s wr, 810 op/s 2024-05-04T03:22:40.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:40 smithi072 ceph-mon[33332]: pgmap v496: 97 pgs: 97 active+clean; 91 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.3 MiB/s wr, 810 op/s 2024-05-04T03:22:41.065 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0x9ffb01 2024-05-04T03:22:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.810+0000 7f853d5ea640 1 -- 172.21.15.1:0/1206742010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8538108a90 msgr2=0x7f853810aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.810+0000 7f853d5ea640 1 --2- 172.21.15.1:0/1206742010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8538108a90 0x7f853810aeb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8530009ca0 tx=0x7f853002a8c0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.810+0000 7f853d5ea640 1 -- 172.21.15.1:0/1206742010 shutdown_connections 2024-05-04T03:22:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.810+0000 7f853d5ea640 1 --2- 172.21.15.1:0/1206742010 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8538103d90 0x7f85381041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.810+0000 7f853d5ea640 1 --2- 172.21.15.1:0/1206742010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8538108a90 0x7f853810aeb0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.810+0000 7f853d5ea640 1 --2- 172.21.15.1:0/1206742010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8538102b50 0x7f8538103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.810+0000 7f853d5ea640 1 -- 172.21.15.1:0/1206742010 >> 172.21.15.1:0/1206742010 conn(0x7f85380fdd40 msgr2=0x7f85381001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.810+0000 7f853d5ea640 1 -- 172.21.15.1:0/1206742010 shutdown_connections 2024-05-04T03:22:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.811+0000 7f853d5ea640 1 -- 172.21.15.1:0/1206742010 wait complete. 2024-05-04T03:22:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.811+0000 7f853d5ea640 1 Processor -- start 2024-05-04T03:22:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.811+0000 7f853d5ea640 1 -- start start 2024-05-04T03:22:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.812+0000 7f853d5ea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8538102b50 0x7f853819a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.812+0000 7f853d5ea640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8538103d90 0x7f853819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.812+0000 7f853d5ea640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8538108a90 0x7f85381a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.812+0000 7f853d5ea640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f853810e660 con 0x7f8538103d90 2024-05-04T03:22:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.812+0000 7f853d5ea640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f853810e4e0 con 0x7f8538108a90 2024-05-04T03:22:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.812+0000 7f853d5ea640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f853810e7e0 con 0x7f8538102b50 2024-05-04T03:22:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.812+0000 7f85377fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8538108a90 0x7f85381a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.812+0000 7f8536ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8538102b50 0x7f853819a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.812+0000 7f85377fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8538108a90 0x7f85381a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:58414/0 (socket says 172.21.15.1:58414) 2024-05-04T03:22:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.812+0000 7f8536ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8538102b50 0x7f853819a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.1:39770/0 (socket says 172.21.15.1:39770) 2024-05-04T03:22:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.812+0000 7f85377fe640 1 -- 172.21.15.1:0/1871261015 learned_addr learned my addr 172.21.15.1:0/1871261015 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.812+0000 7f85367fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8538103d90 0x7f853819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.812+0000 7f8536ffd640 1 -- 172.21.15.1:0/1871261015 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8538108a90 msgr2=0x7f85381a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.812+0000 7f8536ffd640 1 --2- 172.21.15.1:0/1871261015 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8538108a90 0x7f85381a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.812+0000 7f8536ffd640 1 -- 172.21.15.1:0/1871261015 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8538103d90 msgr2=0x7f853819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.812+0000 7f8536ffd640 1 --2- 172.21.15.1:0/1871261015 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8538103d90 0x7f853819a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.812+0000 7f8536ffd640 1 -- 172.21.15.1:0/1871261015 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85381a1d70 con 0x7f8538102b50 2024-05-04T03:22:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.813+0000 7f8536ffd640 1 --2- 172.21.15.1:0/1871261015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8538102b50 0x7f853819a230 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8538102420 tx=0x7f852800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.813+0000 7f852ffff640 1 -- 172.21.15.1:0/1871261015 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852800cc40 con 0x7f8538102b50 2024-05-04T03:22:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.813+0000 7f852ffff640 1 -- 172.21.15.1:0/1871261015 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f852800cde0 con 0x7f8538102b50 2024-05-04T03:22:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.813+0000 7f852ffff640 1 -- 172.21.15.1:0/1871261015 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8528010430 con 0x7f8538102b50 2024-05-04T03:22:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.813+0000 7f853d5ea640 1 -- 172.21.15.1:0/1871261015 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85381a2060 con 0x7f8538102b50 2024-05-04T03:22:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.813+0000 7f853d5ea640 1 -- 172.21.15.1:0/1871261015 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85381a25b0 con 0x7f8538102b50 2024-05-04T03:22:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.816+0000 7f8544571740 1 Processor -- start 2024-05-04T03:22:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.816+0000 7f8544571740 1 -- start start 2024-05-04T03:22:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.817+0000 7f852ffff640 1 -- 172.21.15.1:0/1871261015 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f85280105d0 con 0x7f8538102b50 2024-05-04T03:22:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.817+0000 7f8544571740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a7e41a91b0 0x55a7e454aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.817+0000 7f8544571740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7e41802f0 0x55a7e454afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.817+0000 7f8544571740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7e41884d0 0x55a7e4551ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.817+0000 7f8544571740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a7e44af5d0 con 0x55a7e41802f0 2024-05-04T03:22:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.817+0000 7f8544571740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55a7e4468890 con 0x55a7e41a91b0 2024-05-04T03:22:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.817+0000 7f8544571740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a7e44ab590 con 0x55a7e41884d0 2024-05-04T03:22:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.817+0000 7f85377fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a7e41a91b0 0x55a7e454aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.817+0000 7f85367fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7e41884d0 0x55a7e4551ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.817+0000 7f8536ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7e41802f0 0x55a7e454afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.817+0000 7f85377fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a7e41a91b0 0x55a7e454aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:58424/0 (socket says 172.21.15.1:58424) 2024-05-04T03:22:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.817+0000 7f85377fe640 1 -- 172.21.15.1:0/2257750491 learned_addr learned my addr 172.21.15.1:0/2257750491 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.817+0000 7f85377fe640 1 -- 172.21.15.1:0/2257750491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7e41884d0 msgr2=0x55a7e4551ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.817+0000 7f85377fe640 1 --2- 172.21.15.1:0/2257750491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7e41884d0 0x55a7e4551ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.817+0000 7f85377fe640 1 -- 172.21.15.1:0/2257750491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7e41802f0 msgr2=0x55a7e454afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.817+0000 7f85377fe640 1 --2- 172.21.15.1:0/2257750491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7e41802f0 0x55a7e454afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.817+0000 7f85377fe640 1 -- 172.21.15.1:0/2257750491 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7e41c1210 con 0x55a7e41a91b0 2024-05-04T03:22:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.818+0000 7f85377fe640 1 --2- 172.21.15.1:0/2257750491 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a7e41a91b0 0x55a7e454aa60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8530009800 tx=0x7f8530009830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=982d05f40711fc2b server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.818+0000 7f852ffff640 1 --2- 172.21.15.1:0/1871261015 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f85240b28a0 0x7f85240b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.818+0000 7f852d7fa640 1 -- 172.21.15.1:0/2257750491 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530031650 con 0x55a7e41a91b0 2024-05-04T03:22:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.818+0000 7f852d7fa640 1 -- 172.21.15.1:0/2257750491 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8530036030 con 0x55a7e41a91b0 2024-05-04T03:22:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.818+0000 7f852d7fa640 1 -- 172.21.15.1:0/2257750491 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530004800 con 0x55a7e41a91b0 2024-05-04T03:22:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.818+0000 7f8536ffd640 1 --2- 172.21.15.1:0/1871261015 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f85240b28a0 0x7f85240b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.818+0000 7f852ffff640 1 -- 172.21.15.1:0/1871261015 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f85280e2540 con 0x7f8538102b50 2024-05-04T03:22:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.819+0000 7f8544571740 1 -- 172.21.15.1:0/2257750491 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a7e41a91b0 msgr2=0x55a7e454aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.819+0000 7f8544571740 1 --2- 172.21.15.1:0/2257750491 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a7e41a91b0 0x55a7e454aa60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8530009800 tx=0x7f8530009830 comp rx=0 tx=0).stop 2024-05-04T03:22:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.819+0000 7f8536ffd640 1 --2- 172.21.15.1:0/1871261015 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f85240b28a0 0x7f85240b4db0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f85280a5ca0 tx=0x7f85280e7040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.819+0000 7f8544571740 1 -- 172.21.15.1:0/2257750491 shutdown_connections 2024-05-04T03:22:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.819+0000 7f8544571740 1 --2- 172.21.15.1:0/2257750491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7e41884d0 0x55a7e4551ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.819+0000 7f8544571740 1 --2- 172.21.15.1:0/2257750491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7e41802f0 0x55a7e454afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.819+0000 7f8544571740 1 --2- 172.21.15.1:0/2257750491 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a7e41a91b0 0x55a7e454aa60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.819+0000 7f8544571740 1 -- 172.21.15.1:0/2257750491 >> 172.21.15.1:0/2257750491 conn(0x55a7e419d300 msgr2=0x55a7e45463a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.819+0000 7f8544571740 1 -- 172.21.15.1:0/2257750491 shutdown_connections 2024-05-04T03:22:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.819+0000 7f8544571740 1 -- 172.21.15.1:0/2257750491 wait complete. 2024-05-04T03:22:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.820+0000 7f8544571740 1 -- start start 2024-05-04T03:22:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.820+0000 7f8544571740 1 Processor -- start 2024-05-04T03:22:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.820+0000 7f8544571740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7e41802f0 0x55a7e4554460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.820+0000 7f8544571740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a7e41884d0 0x55a7e45549a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.820+0000 7f8544571740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7e41a91b0 0x55a7e455b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.820+0000 7f8544571740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a7e4552750 con 0x55a7e41802f0 2024-05-04T03:22:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.820+0000 7f8544571740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55a7e446c370 con 0x55a7e41884d0 2024-05-04T03:22:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.820+0000 7f8544571740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a7e45528d0 con 0x55a7e41a91b0 2024-05-04T03:22:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.820+0000 7f8544571740 10 client.0 _ll_register_callbacks cb 0x55a7e41c0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:22:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.820+0000 7f85377fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7e41802f0 0x55a7e4554460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.820+0000 7f85377fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7e41802f0 0x55a7e4554460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39512/0 (socket says 172.21.15.1:39512) 2024-05-04T03:22:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.820+0000 7f85377fe640 1 -- 172.21.15.1:0/1930184532 learned_addr learned my addr 172.21.15.1:0/1930184532 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.821+0000 7f85367fc640 1 --2- 172.21.15.1:0/1930184532 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a7e41884d0 0x55a7e45549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.821+0000 7f85377fe640 1 --2- 172.21.15.1:0/1930184532 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7e41a91b0 0x55a7e455b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.821+0000 7f85377fe640 1 -- 172.21.15.1:0/1930184532 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7e41a91b0 msgr2=0x55a7e455b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.821+0000 7f85377fe640 1 --2- 172.21.15.1:0/1930184532 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7e41a91b0 0x55a7e455b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.821+0000 7f85377fe640 1 -- 172.21.15.1:0/1930184532 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a7e41884d0 msgr2=0x55a7e45549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.821+0000 7f85377fe640 1 --2- 172.21.15.1:0/1930184532 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a7e41884d0 0x55a7e45549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.821+0000 7f85377fe640 1 -- 172.21.15.1:0/1930184532 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7e428d620 con 0x55a7e41802f0 2024-05-04T03:22:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.821+0000 7f85377fe640 1 --2- 172.21.15.1:0/1930184532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7e41802f0 0x55a7e4554460 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8530009620 tx=0x7f853002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f367985cdeed6db1 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.821+0000 7f851e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:22:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.821+0000 7f851e7fc640 1 -- 172.21.15.1:0/1930184532 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530059060 con 0x55a7e41802f0 2024-05-04T03:22:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.821+0000 7f851e7fc640 1 -- 172.21.15.1:0/1930184532 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f8530003da0 con 0x55a7e41802f0 2024-05-04T03:22:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.821+0000 7f8544571740 1 -- 172.21.15.1:0/1930184532 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8530036030 con 0x55a7e41802f0 2024-05-04T03:22:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.822+0000 7f851e7fc640 1 -- 172.21.15.1:0/1930184532 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8530004810 con 0x55a7e41802f0 2024-05-04T03:22:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.822+0000 7f8544571740 10 client.15618 fetch_fsmap learned FSMap version 29 2024-05-04T03:22:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.822+0000 7f8544571740 1 -- 172.21.15.1:0/1930184532 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a7e4552750 con 0x55a7e41802f0 2024-05-04T03:22:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.822+0000 7f851e7fc640 1 -- 172.21.15.1:0/1930184532 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8530036030 con 0x55a7e41802f0 2024-05-04T03:22:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.822+0000 7f851e7fc640 1 -- 172.21.15.1:0/1930184532 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f8530043070 con 0x55a7e41802f0 2024-05-04T03:22:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.822+0000 7f8544571740 10 client.15618 fetch_fsmap finished waiting for FSMap version 29 2024-05-04T03:22:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.822+0000 7f8544571740 10 client.15618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:22:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.822+0000 7f8544571740 20 client.15618 populate_metadata read hostname 'smithi001' 2024-05-04T03:22:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.822+0000 7f8544571740 1 --2- 172.21.15.1:0/1930184532 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a7e4167d80 0x55a7e4561ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.822+0000 7f8544571740 4 client.15618 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:22:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.822+0000 7f8544571740 1 -- 172.21.15.1:0/1930184532 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7e44bd1e0 con 0x55a7e4167d80 2024-05-04T03:22:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.822+0000 7f85367fc640 1 --2- 172.21.15.1:0/1930184532 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a7e4167d80 0x55a7e4561ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.823+0000 7f851e7fc640 10 client.15618 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:22:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.823+0000 7f85367fc640 1 --2- 172.21.15.1:0/1930184532 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a7e4167d80 0x55a7e4561ba0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24932cb5626c3422 server_cookie=4fe75e607e2e01f4 in_seq=0 out_seq=0 2024-05-04T03:22:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.823+0000 7f851e7fc640 1 -- 172.21.15.1:0/1930184532 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a7e44bd1e0 con 0x55a7e4167d80 2024-05-04T03:22:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.824+0000 7f851e7fc640 10 client.15618 handle_command_reply: tid=0 2024-05-04T03:22:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.824+0000 7f8544571740 1 client.15618 shutdown 2024-05-04T03:22:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.824+0000 7f8544571740 1 -- 172.21.15.1:0/1930184532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7e41802f0 msgr2=0x55a7e4554460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:42.831 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.824+0000 7f8544571740 1 --2- 172.21.15.1:0/1930184532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7e41802f0 0x55a7e4554460 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8530009620 tx=0x7f853002a8c0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.824+0000 7f8544571740 1 -- 172.21.15.1:0/1930184532 shutdown_connections 2024-05-04T03:22:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.824+0000 7f8544571740 1 --2- 172.21.15.1:0/1930184532 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a7e4167d80 0x55a7e4561ba0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.824+0000 7f8544571740 1 --2- 172.21.15.1:0/1930184532 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a7e41884d0 0x55a7e45549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.824+0000 7f851e7fc640 0 client.15618 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:22:42.832 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.824+0000 7f8544571740 1 --2- 172.21.15.1:0/1930184532 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7e41a91b0 0x55a7e455b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.824+0000 7f8544571740 1 --2- 172.21.15.1:0/1930184532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7e41802f0 0x55a7e4554460 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.824+0000 7f8544571740 1 -- 172.21.15.1:0/1930184532 >> 172.21.15.1:0/1930184532 conn(0x55a7e419d300 msgr2=0x55a7e45453a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.824+0000 7f8544571740 1 -- 172.21.15.1:0/1930184532 shutdown_connections 2024-05-04T03:22:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.824+0000 7f8544571740 1 -- 172.21.15.1:0/1930184532 wait complete. 2024-05-04T03:22:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.824+0000 7f8544571740 20 client.15618 trim_cache size 0 max 16384 2024-05-04T03:22:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.840+0000 7f8544571740 1 Processor -- start 2024-05-04T03:22:42.842 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.840+0000 7f8544571740 1 -- start start 2024-05-04T03:22:42.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.841+0000 7f8544571740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7e4167d80 0x55a7e4554740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:42.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.841+0000 7f8544571740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7e41802f0 0x55a7e455b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:42.854 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.841+0000 7f8544571740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a7e41884d0 0x55a7e455bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:42.855 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.841+0000 7f8544571740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f85381ad8d0 con 0x55a7e4167d80 2024-05-04T03:22:42.855 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.841+0000 7f8544571740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f85381ab140 con 0x55a7e41884d0 2024-05-04T03:22:42.855 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.841+0000 7f8544571740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f85381a8f90 con 0x55a7e41802f0 2024-05-04T03:22:42.855 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.841+0000 7f85377fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7e4167d80 0x55a7e4554740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:42.855 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.841+0000 7f85367fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7e41802f0 0x55a7e455b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:42.855 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.841+0000 7f85367fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7e41802f0 0x55a7e455b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.1:39802/0 (socket says 172.21.15.1:39802) 2024-05-04T03:22:42.855 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.841+0000 7f85377fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7e4167d80 0x55a7e4554740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39514/0 (socket says 172.21.15.1:39514) 2024-05-04T03:22:42.855 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.841+0000 7f85367fc640 1 -- 172.21.15.1:0/4098455730 learned_addr learned my addr 172.21.15.1:0/4098455730 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.841+0000 7f85377fe640 1 --2- 172.21.15.1:0/4098455730 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a7e41884d0 0x55a7e455bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.841+0000 7f85367fc640 1 -- 172.21.15.1:0/4098455730 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a7e41884d0 msgr2=0x55a7e455bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.841+0000 7f85367fc640 1 --2- 172.21.15.1:0/4098455730 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a7e41884d0 0x55a7e455bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.841+0000 7f85367fc640 1 -- 172.21.15.1:0/4098455730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7e4167d80 msgr2=0x55a7e4554740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.841+0000 7f85367fc640 1 --2- 172.21.15.1:0/4098455730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7e4167d80 0x55a7e4554740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.841+0000 7f85367fc640 1 -- 172.21.15.1:0/4098455730 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7e45554d0 con 0x55a7e41802f0 2024-05-04T03:22:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.841+0000 7f85377fe640 1 --2- 172.21.15.1:0/4098455730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7e4167d80 0x55a7e4554740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:22:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.842+0000 7f85367fc640 1 --2- 172.21.15.1:0/4098455730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7e41802f0 0x55a7e455b890 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55a7e4547180 tx=0x7f8520005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23541160bb320eee server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.842+0000 7f852cff9640 1 -- 172.21.15.1:0/4098455730 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852001f430 con 0x55a7e41802f0 2024-05-04T03:22:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.842+0000 7f852cff9640 1 -- 172.21.15.1:0/4098455730 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f852001fa80 con 0x55a7e41802f0 2024-05-04T03:22:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.842+0000 7f852cff9640 1 -- 172.21.15.1:0/4098455730 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8520010690 con 0x55a7e41802f0 2024-05-04T03:22:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.843+0000 7f8544571740 1 -- 172.21.15.1:0/4098455730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7e41802f0 msgr2=0x55a7e455b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.843+0000 7f8544571740 1 --2- 172.21.15.1:0/4098455730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7e41802f0 0x55a7e455b890 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55a7e4547180 tx=0x7f8520005f90 comp rx=0 tx=0).stop 2024-05-04T03:22:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.843+0000 7f8544571740 1 -- 172.21.15.1:0/4098455730 shutdown_connections 2024-05-04T03:22:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.843+0000 7f8544571740 1 --2- 172.21.15.1:0/4098455730 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a7e41884d0 0x55a7e455bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.843+0000 7f8544571740 1 --2- 172.21.15.1:0/4098455730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7e41802f0 0x55a7e455b890 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.843+0000 7f8544571740 1 --2- 172.21.15.1:0/4098455730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7e4167d80 0x55a7e4554740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.843+0000 7f8544571740 1 -- 172.21.15.1:0/4098455730 >> 172.21.15.1:0/4098455730 conn(0x55a7e419d300 msgr2=0x55a7e4554030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:42.862 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.843+0000 7f8544571740 1 -- 172.21.15.1:0/4098455730 shutdown_connections 2024-05-04T03:22:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.843+0000 7f8544571740 1 -- 172.21.15.1:0/4098455730 wait complete. 2024-05-04T03:22:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.844+0000 7f8544571740 1 -- start start 2024-05-04T03:22:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.844+0000 7f8544571740 1 Processor -- start 2024-05-04T03:22:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.844+0000 7f8544571740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a7e4167d80 0x55a7e455a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.844+0000 7f8544571740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7e41802f0 0x55a7e455ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.844+0000 7f8544571740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7e41884d0 0x55a7e455b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.844+0000 7f8544571740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f85381a9600 con 0x55a7e41884d0 2024-05-04T03:22:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.844+0000 7f8544571740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f85381aaa70 con 0x55a7e4167d80 2024-05-04T03:22:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.844+0000 7f8544571740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f85381a9cd0 con 0x55a7e41802f0 2024-05-04T03:22:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.844+0000 7f8544571740 10 client.0 _ll_register_callbacks cb 0x7f85280e5f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:22:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.844+0000 7f85377fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7e41884d0 0x55a7e455b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:42.863 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.844+0000 7f85377fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7e41884d0 0x55a7e455b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39530/0 (socket says 172.21.15.1:39530) 2024-05-04T03:22:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.844+0000 7f85377fe640 1 -- 172.21.15.1:0/122560250 learned_addr learned my addr 172.21.15.1:0/122560250 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.844+0000 7f85367fc640 1 --2- 172.21.15.1:0/122560250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7e41802f0 0x55a7e455ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.844+0000 7f85377fe640 1 --2- 172.21.15.1:0/122560250 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a7e4167d80 0x55a7e455a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.845+0000 7f85377fe640 1 -- 172.21.15.1:0/122560250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7e41802f0 msgr2=0x55a7e455ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.845+0000 7f85377fe640 1 --2- 172.21.15.1:0/122560250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7e41802f0 0x55a7e455ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.845+0000 7f85377fe640 1 -- 172.21.15.1:0/122560250 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a7e4167d80 msgr2=0x55a7e455a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.845+0000 7f85377fe640 1 --2- 172.21.15.1:0/122560250 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a7e4167d80 0x55a7e455a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.845+0000 7f85377fe640 1 -- 172.21.15.1:0/122560250 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7e45c7f40 con 0x55a7e41884d0 2024-05-04T03:22:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.845+0000 7f85377fe640 1 --2- 172.21.15.1:0/122560250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7e41884d0 0x55a7e455b190 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f85300047e0 tx=0x7f8530005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc908beb197a9fc server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.845+0000 7f851e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:22:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.845+0000 7f851e7fc640 1 -- 172.21.15.1:0/122560250 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530063040 con 0x55a7e41884d0 2024-05-04T03:22:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.845+0000 7f8544571740 1 -- 172.21.15.1:0/122560250 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f852001fa80 con 0x55a7e41884d0 2024-05-04T03:22:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.845+0000 7f851e7fc640 1 -- 172.21.15.1:0/122560250 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f8530003d80 con 0x55a7e41884d0 2024-05-04T03:22:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.845+0000 7f851e7fc640 1 -- 172.21.15.1:0/122560250 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8530030ad0 con 0x55a7e41884d0 2024-05-04T03:22:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.845+0000 7f8544571740 10 client.15624 fetch_fsmap learned FSMap version 29 2024-05-04T03:22:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.845+0000 7f8544571740 1 -- 172.21.15.1:0/122560250 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a7e45c91b0 con 0x55a7e41884d0 2024-05-04T03:22:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.846+0000 7f851e7fc640 1 -- 172.21.15.1:0/122560250 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f852001fa80 con 0x55a7e41884d0 2024-05-04T03:22:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.846+0000 7f851e7fc640 1 -- 172.21.15.1:0/122560250 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f8530043030 con 0x55a7e41884d0 2024-05-04T03:22:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.846+0000 7f8544571740 10 client.15624 fetch_fsmap finished waiting for FSMap version 29 2024-05-04T03:22:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.846+0000 7f8544571740 10 client.15624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:22:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.846+0000 7f8544571740 20 client.15624 populate_metadata read hostname 'smithi001' 2024-05-04T03:22:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.846+0000 7f8544571740 1 --2- 172.21.15.1:0/122560250 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a7e41a91b0 0x55a7e45cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.846+0000 7f8544571740 4 client.15624 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:22:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.846+0000 7f8544571740 1 -- 172.21.15.1:0/122560250 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a7e44b7050 con 0x55a7e41a91b0 2024-05-04T03:22:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.846+0000 7f85367fc640 1 --2- 172.21.15.1:0/122560250 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a7e41a91b0 0x55a7e45cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.846+0000 7f851e7fc640 10 client.15624 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:22:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.847+0000 7f85367fc640 1 --2- 172.21.15.1:0/122560250 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a7e41a91b0 0x55a7e45cbb50 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a7fe349fa47e468 server_cookie=9d62cac48c53d8ee in_seq=0 out_seq=0 2024-05-04T03:22:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.854+0000 7f851e7fc640 1 -- 172.21.15.1:0/122560250 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a7e44b7050 con 0x55a7e41a91b0 2024-05-04T03:22:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.854+0000 7f851e7fc640 10 client.15624 handle_command_reply: tid=0 2024-05-04T03:22:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.854+0000 7f8544571740 1 client.15624 shutdown 2024-05-04T03:22:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.854+0000 7f8544571740 1 -- 172.21.15.1:0/122560250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7e41884d0 msgr2=0x55a7e455b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.854+0000 7f8544571740 1 --2- 172.21.15.1:0/122560250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7e41884d0 0x55a7e455b190 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f85300047e0 tx=0x7f8530005fd0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.854+0000 7f8544571740 1 -- 172.21.15.1:0/122560250 shutdown_connections 2024-05-04T03:22:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.854+0000 7f8544571740 1 --2- 172.21.15.1:0/122560250 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a7e41a91b0 0x55a7e45cbb50 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.854+0000 7f8544571740 1 --2- 172.21.15.1:0/122560250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7e41884d0 0x55a7e455b190 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.854+0000 7f851e7fc640 0 client.15624 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:22:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.854+0000 7f8544571740 1 --2- 172.21.15.1:0/122560250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a7e41802f0 0x55a7e455ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.854+0000 7f8544571740 1 --2- 172.21.15.1:0/122560250 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a7e4167d80 0x55a7e455a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.854+0000 7f8544571740 1 -- 172.21.15.1:0/122560250 >> 172.21.15.1:0/122560250 conn(0x55a7e419d300 msgr2=0x55a7e45547c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.854+0000 7f8544571740 1 -- 172.21.15.1:0/122560250 shutdown_connections 2024-05-04T03:22:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.854+0000 7f8544571740 1 -- 172.21.15.1:0/122560250 wait complete. 2024-05-04T03:22:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.854+0000 7f8544571740 20 client.15624 trim_cache size 0 max 16384 2024-05-04T03:22:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:22:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:22:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:22:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:22:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.855+0000 7f853d5ea640 1 -- 172.21.15.1:0/1871261015 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f85240b28a0 msgr2=0x7f85240b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.855+0000 7f853d5ea640 1 --2- 172.21.15.1:0/1871261015 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f85240b28a0 0x7f85240b4db0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f85280a5ca0 tx=0x7f85280e7040 comp rx=0 tx=0).stop 2024-05-04T03:22:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.855+0000 7f853d5ea640 1 -- 172.21.15.1:0/1871261015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8538102b50 msgr2=0x7f853819a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.855+0000 7f853d5ea640 1 --2- 172.21.15.1:0/1871261015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8538102b50 0x7f853819a230 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8538102420 tx=0x7f852800ee50 comp rx=0 tx=0).stop 2024-05-04T03:22:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.855+0000 7f853d5ea640 1 -- 172.21.15.1:0/1871261015 shutdown_connections 2024-05-04T03:22:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.855+0000 7f853d5ea640 1 --2- 172.21.15.1:0/1871261015 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f85240b28a0 0x7f85240b4db0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.855+0000 7f853d5ea640 1 --2- 172.21.15.1:0/1871261015 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8538108a90 0x7f85381a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.855+0000 7f853d5ea640 1 --2- 172.21.15.1:0/1871261015 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8538103d90 0x7f853819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.855+0000 7f853d5ea640 1 --2- 172.21.15.1:0/1871261015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8538102b50 0x7f853819a230 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.855+0000 7f853d5ea640 1 -- 172.21.15.1:0/1871261015 >> 172.21.15.1:0/1871261015 conn(0x7f85380fdd40 msgr2=0x7f8538109ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.856+0000 7f853d5ea640 1 -- 172.21.15.1:0/1871261015 shutdown_connections 2024-05-04T03:22:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:42.856+0000 7f853d5ea640 1 -- 172.21.15.1:0/1871261015 wait complete. 2024-05-04T03:22:42.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:42 smithi001 ceph-mon[29731]: pgmap v497: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.1 MiB/s wr, 710 op/s 2024-05-04T03:22:42.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:42 smithi077 ceph-mon[33531]: pgmap v497: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.1 MiB/s wr, 710 op/s 2024-05-04T03:22:42.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:42 smithi072 ceph-mon[33332]: pgmap v497: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.1 MiB/s wr, 710 op/s 2024-05-04T03:22:43.098 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-04T03:22:43.098 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-05-04T03:22:43.099 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:22:43.385 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:22:44.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:44 smithi001 ceph-mon[29731]: pgmap v498: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.2 MiB/s wr, 578 op/s 2024-05-04T03:22:44.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:44 smithi077 ceph-mon[33531]: pgmap v498: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.2 MiB/s wr, 578 op/s 2024-05-04T03:22:44.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:44 smithi072 ceph-mon[33332]: pgmap v498: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.2 MiB/s wr, 578 op/s 2024-05-04T03:22:45.311 INFO:tasks.workunit.client.0.smithi001.stdout:40000 mapwrite 0x1becad thru 0x1c6a92 (0x7de6 bytes) 2024-05-04T03:22:45.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.328+0000 7fc9b5e15700 1 -- 172.21.15.1:0/683405788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9b0107df0 msgr2=0x7fc9b0108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:45.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.328+0000 7fc9b5e15700 1 --2- 172.21.15.1:0/683405788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9b0107df0 0x7fc9b0108200 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc9a40099c0 tx=0x7fc9a4009cd0 comp rx=0 tx=0).stop 2024-05-04T03:22:45.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.329+0000 7fc9b5e15700 1 -- 172.21.15.1:0/683405788 shutdown_connections 2024-05-04T03:22:45.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.329+0000 7fc9b5e15700 1 --2- 172.21.15.1:0/683405788 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9b0108740 0x7fc9b01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:45.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.329+0000 7fc9b5e15700 1 --2- 172.21.15.1:0/683405788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9b01034c0 0x7fc9b01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:45.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.329+0000 7fc9b5e15700 1 --2- 172.21.15.1:0/683405788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9b0107df0 0x7fc9b0108200 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:45.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.329+0000 7fc9b5e15700 1 -- 172.21.15.1:0/683405788 >> 172.21.15.1:0/683405788 conn(0x7fc9b00fedf0 msgr2=0x7fc9b0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:45.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.329+0000 7fc9b5e15700 1 -- 172.21.15.1:0/683405788 shutdown_connections 2024-05-04T03:22:45.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.329+0000 7fc9b5e15700 1 -- 172.21.15.1:0/683405788 wait complete. 2024-05-04T03:22:45.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.330+0000 7fc9b5e15700 1 Processor -- start 2024-05-04T03:22:45.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.330+0000 7fc9b5e15700 1 -- start start 2024-05-04T03:22:45.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.330+0000 7fc9b5e15700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9b01034c0 0x7fc9b0110a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:45.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.330+0000 7fc9b5e15700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9b0107df0 0x7fc9b0110fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:45.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.330+0000 7fc9b5e15700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9b0108740 0x7fc9b0111500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:45.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.331+0000 7fc9b5e15700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b0114510 con 0x7fc9b0108740 2024-05-04T03:22:45.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.331+0000 7fc9b5e15700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b0114390 con 0x7fc9b0107df0 2024-05-04T03:22:45.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.331+0000 7fc9b5e15700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b0114690 con 0x7fc9b01034c0 2024-05-04T03:22:45.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.331+0000 7fc9affff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9b0108740 0x7fc9b0111500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:45.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.331+0000 7fc9affff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9b0108740 0x7fc9b0111500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39548/0 (socket says 172.21.15.1:39548) 2024-05-04T03:22:45.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.331+0000 7fc9affff700 1 -- 172.21.15.1:0/2406212328 learned_addr learned my addr 172.21.15.1:0/2406212328 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:45.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.331+0000 7fc9aeffd700 1 --2- 172.21.15.1:0/2406212328 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9b0107df0 0x7fc9b0110fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:45.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.331+0000 7fc9af7fe700 1 --2- 172.21.15.1:0/2406212328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9b01034c0 0x7fc9b0110a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:45.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.331+0000 7fc9affff700 1 -- 172.21.15.1:0/2406212328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9b01034c0 msgr2=0x7fc9b0110a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:45.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.331+0000 7fc9affff700 1 --2- 172.21.15.1:0/2406212328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9b01034c0 0x7fc9b0110a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:45.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.331+0000 7fc9affff700 1 -- 172.21.15.1:0/2406212328 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9b0107df0 msgr2=0x7fc9b0110fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:45.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.331+0000 7fc9affff700 1 --2- 172.21.15.1:0/2406212328 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9b0107df0 0x7fc9b0110fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:45.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.331+0000 7fc9affff700 1 -- 172.21.15.1:0/2406212328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b01a9240 con 0x7fc9b0108740 2024-05-04T03:22:45.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.331+0000 7fc9affff700 1 --2- 172.21.15.1:0/2406212328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9b0108740 0x7fc9b0111500 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800c370 tx=0x7fc9a800c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:45.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.331+0000 7fc9acff9700 1 -- 172.21.15.1:0/2406212328 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a80092e0 con 0x7fc9b0108740 2024-05-04T03:22:45.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.332+0000 7fc9b5e15700 1 -- 172.21.15.1:0/2406212328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b01a94d0 con 0x7fc9b0108740 2024-05-04T03:22:45.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.332+0000 7fc9b5e15700 1 -- 172.21.15.1:0/2406212328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b01a99e0 con 0x7fc9b0108740 2024-05-04T03:22:45.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.332+0000 7fc9acff9700 1 -- 172.21.15.1:0/2406212328 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9a8003a00 con 0x7fc9b0108740 2024-05-04T03:22:45.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.332+0000 7fc9acff9700 1 -- 172.21.15.1:0/2406212328 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a80046f0 con 0x7fc9b0108740 2024-05-04T03:22:45.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.333+0000 7fc9967fc700 1 -- 172.21.15.1:0/2406212328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9b0104970 con 0x7fc9b0108740 2024-05-04T03:22:45.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.335+0000 7fc9acff9700 1 -- 172.21.15.1:0/2406212328 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7fc9a801a020 con 0x7fc9b0108740 2024-05-04T03:22:45.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.335+0000 7fc9acff9700 1 --2- 172.21.15.1:0/2406212328 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc9980b2900 0x7fc9980b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:45.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.336+0000 7fc9af7fe700 1 --2- 172.21.15.1:0/2406212328 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc9980b2900 0x7fc9980b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:45.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.336+0000 7fc9acff9700 1 -- 172.21.15.1:0/2406212328 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fc9a80e18e0 con 0x7fc9b0108740 2024-05-04T03:22:45.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.336+0000 7fc9af7fe700 1 --2- 172.21.15.1:0/2406212328 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc9980b2900 0x7fc9980b4e10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009c60 tx=0x7fc9a0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:45.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.342+0000 7fc9acff9700 1 -- 172.21.15.1:0/2406212328 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc9a8010040 con 0x7fc9b0108740 2024-05-04T03:22:45.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.656+0000 7fc9967fc700 1 -- 172.21.15.1:0/2406212328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9b0111fb0 con 0x7fc9b0108740 2024-05-04T03:22:45.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.657+0000 7fc9acff9700 1 -- 172.21.15.1:0/2406212328 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11848 (secure 0 0 0) 0x7fc9a80a96c0 con 0x7fc9b0108740 2024-05-04T03:22:45.660 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:22:45.660 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2186681910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2186681910},{"type":"v1","addr":"172.21.15.1:6835","nonce":2186681910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2747191047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2747191047},{"type":"v1","addr":"172.21.15.1:6837","nonce":2747191047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4115120540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4115120540},{"type":"v1","addr":"172.21.15.1:6839","nonce":4115120540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3405709780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3405709780},{"type":"v1","addr":"172.21.15.1:6841","nonce":3405709780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/3816098761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":3816098761},{"type":"v1","addr":"172.21.15.72:6833","nonce":3816098761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/379026128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":379026128},{"type":"v1","addr":"172.21.15.77:6833","nonce":379026128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:15:49.823471+0000","modified":"2024-05-04T03:21:11.571220+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24472,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/3930984976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3930984976},{"type":"v1","addr":"172.21.15.72:6835","nonce":3930984976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/4279951305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4279951305},{"type":"v1","addr":"172.21.15.77:6835","nonce":4279951305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2003521417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2003521417},{"type":"v1","addr":"172.21.15.77:6837","nonce":2003521417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/1417614203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1417614203},{"type":"v1","addr":"172.21.15.72:6837","nonce":1417614203}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.77:6839/922723189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":922723189},{"type":"v1","addr":"172.21.15.77:6839","nonce":922723189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2976050650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2976050650},{"type":"v1","addr":"172.21.15.72:6839","nonce":2976050650}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24467,24472,24494]},"id":1}]} 2024-05-04T03:22:45.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.660+0000 7fc9967fc700 1 -- 172.21.15.1:0/2406212328 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc9980b2900 msgr2=0x7fc9980b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:45.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.660+0000 7fc9967fc700 1 --2- 172.21.15.1:0/2406212328 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc9980b2900 0x7fc9980b4e10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009c60 tx=0x7fc9a0009260 comp rx=0 tx=0).stop 2024-05-04T03:22:45.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.660+0000 7fc9967fc700 1 -- 172.21.15.1:0/2406212328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9b0108740 msgr2=0x7fc9b0111500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:45.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.660+0000 7fc9967fc700 1 --2- 172.21.15.1:0/2406212328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9b0108740 0x7fc9b0111500 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800c370 tx=0x7fc9a800c680 comp rx=0 tx=0).stop 2024-05-04T03:22:45.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.660+0000 7fc9967fc700 1 -- 172.21.15.1:0/2406212328 shutdown_connections 2024-05-04T03:22:45.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.660+0000 7fc9967fc700 1 --2- 172.21.15.1:0/2406212328 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc9980b2900 0x7fc9980b4e10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:45.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.660+0000 7fc9967fc700 1 --2- 172.21.15.1:0/2406212328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc9b0108740 0x7fc9b0111500 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:45.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.660+0000 7fc9967fc700 1 --2- 172.21.15.1:0/2406212328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9b01034c0 0x7fc9b0110a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:45.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.660+0000 7fc9967fc700 1 --2- 172.21.15.1:0/2406212328 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc9b0107df0 0x7fc9b0110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:45.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.660+0000 7fc9967fc700 1 -- 172.21.15.1:0/2406212328 >> 172.21.15.1:0/2406212328 conn(0x7fc9b00fedf0 msgr2=0x7fc9b0101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:45.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.660+0000 7fc9967fc700 1 -- 172.21.15.1:0/2406212328 shutdown_connections 2024-05-04T03:22:45.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:45.661+0000 7fc9967fc700 1 -- 172.21.15.1:0/2406212328 wait complete. 2024-05-04T03:22:45.664 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 29 2024-05-04T03:22:46.491 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0' 2024-05-04T03:22:46.492 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24494 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0 2024-05-04T03:22:46.784 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:22:46.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:46 smithi001 ceph-mon[29731]: pgmap v499: 97 pgs: 97 active+clean; 93 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.5 MiB/s wr, 816 op/s 2024-05-04T03:22:46.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:46 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2406212328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:22:46.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:46 smithi077 ceph-mon[33531]: pgmap v499: 97 pgs: 97 active+clean; 93 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.5 MiB/s wr, 816 op/s 2024-05-04T03:22:46.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:46 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/2406212328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:22:46.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:46 smithi072 ceph-mon[33332]: pgmap v499: 97 pgs: 97 active+clean; 93 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.5 MiB/s wr, 816 op/s 2024-05-04T03:22:46.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:46 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/2406212328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:22:48.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:48 smithi001 ceph-mon[29731]: pgmap v500: 97 pgs: 97 active+clean; 93 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.9 MiB/s wr, 512 op/s 2024-05-04T03:22:48.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.637+0000 7fc88cadc700 1 -- 172.21.15.1:0/1582362076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc888107e10 msgr2=0x7fc888108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:48.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.637+0000 7fc88cadc700 1 --2- 172.21.15.1:0/1582362076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc888107e10 0x7fc888108220 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc87c0099c0 tx=0x7fc87c009cd0 comp rx=0 tx=0).stop 2024-05-04T03:22:48.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.637+0000 7fc88cadc700 1 -- 172.21.15.1:0/1582362076 shutdown_connections 2024-05-04T03:22:48.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.637+0000 7fc88cadc700 1 --2- 172.21.15.1:0/1582362076 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc8881034e0 0x7fc8881038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:48.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.637+0000 7fc88cadc700 1 --2- 172.21.15.1:0/1582362076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc888108760 0x7fc8881100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:48.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.637+0000 7fc88cadc700 1 --2- 172.21.15.1:0/1582362076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc888107e10 0x7fc888108220 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:48.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.637+0000 7fc88cadc700 1 -- 172.21.15.1:0/1582362076 >> 172.21.15.1:0/1582362076 conn(0x7fc8880fef30 msgr2=0x7fc888101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:48.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.638+0000 7fc88cadc700 1 -- 172.21.15.1:0/1582362076 shutdown_connections 2024-05-04T03:22:48.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.638+0000 7fc88cadc700 1 -- 172.21.15.1:0/1582362076 wait complete. 2024-05-04T03:22:48.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.638+0000 7fc88cadc700 1 Processor -- start 2024-05-04T03:22:48.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.639+0000 7fc88cadc700 1 -- start start 2024-05-04T03:22:48.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.639+0000 7fc88cadc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc8881034e0 0x7fc88819ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:48.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.639+0000 7fc88cadc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc888107e10 0x7fc88819b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:48.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.639+0000 7fc88cadc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc888108760 0x7fc88819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:48.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.639+0000 7fc88cadc700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc888113390 con 0x7fc888107e10 2024-05-04T03:22:48.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.639+0000 7fc88cadc700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc888113210 con 0x7fc8881034e0 2024-05-04T03:22:48.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.639+0000 7fc88cadc700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc888113510 con 0x7fc888108760 2024-05-04T03:22:48.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.639+0000 7fc88659c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc8881034e0 0x7fc88819ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:48.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.639+0000 7fc886d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc888108760 0x7fc88819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:48.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.639+0000 7fc88659c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc8881034e0 0x7fc88819ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:58498/0 (socket says 172.21.15.1:58498) 2024-05-04T03:22:48.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.639+0000 7fc88659c700 1 -- 172.21.15.1:0/3766624064 learned_addr learned my addr 172.21.15.1:0/3766624064 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:48.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.639+0000 7fc886d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc888108760 0x7fc88819f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.1:39856/0 (socket says 172.21.15.1:39856) 2024-05-04T03:22:48.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.639+0000 7fc885d9b700 1 --2- 172.21.15.1:0/3766624064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc888107e10 0x7fc88819b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:48.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.640+0000 7fc88659c700 1 -- 172.21.15.1:0/3766624064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc888108760 msgr2=0x7fc88819f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:48.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.640+0000 7fc88659c700 1 --2- 172.21.15.1:0/3766624064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc888108760 0x7fc88819f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:48.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.640+0000 7fc88659c700 1 -- 172.21.15.1:0/3766624064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc888107e10 msgr2=0x7fc88819b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:48.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.640+0000 7fc88659c700 1 --2- 172.21.15.1:0/3766624064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc888107e10 0x7fc88819b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:48.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.640+0000 7fc88659c700 1 -- 172.21.15.1:0/3766624064 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88819fd20 con 0x7fc8881034e0 2024-05-04T03:22:48.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.640+0000 7fc886d9d700 1 --2- 172.21.15.1:0/3766624064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc888108760 0x7fc88819f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:22:48.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.640+0000 7fc885d9b700 1 --2- 172.21.15.1:0/3766624064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc888107e10 0x7fc88819b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:22:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.640+0000 7fc88659c700 1 --2- 172.21.15.1:0/3766624064 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc8881034e0 0x7fc88819ae50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc878010e40 tx=0x7fc87800b5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.640+0000 7fc8777fe700 1 -- 172.21.15.1:0/3766624064 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87800bd10 con 0x7fc8881034e0 2024-05-04T03:22:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.640+0000 7fc8777fe700 1 -- 172.21.15.1:0/3766624064 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8780044d0 con 0x7fc8881034e0 2024-05-04T03:22:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.640+0000 7fc8777fe700 1 -- 172.21.15.1:0/3766624064 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc878019410 con 0x7fc8881034e0 2024-05-04T03:22:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.640+0000 7fc88cadc700 1 -- 172.21.15.1:0/3766624064 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8881a0010 con 0x7fc8881034e0 2024-05-04T03:22:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.640+0000 7fc88cadc700 1 -- 172.21.15.1:0/3766624064 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8880ff340 con 0x7fc8881034e0 2024-05-04T03:22:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.643+0000 7fc8777fe700 1 -- 172.21.15.1:0/3766624064 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7fc8780195b0 con 0x7fc8881034e0 2024-05-04T03:22:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.644+0000 7fc8777fe700 1 --2- 172.21.15.1:0/3766624064 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc8700b2880 0x7fc8700b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:48.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.644+0000 7fc885d9b700 1 --2- 172.21.15.1:0/3766624064 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc8700b2880 0x7fc8700b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:48.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.644+0000 7fc8777fe700 1 -- 172.21.15.1:0/3766624064 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fc8780e59d0 con 0x7fc8881034e0 2024-05-04T03:22:48.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.645+0000 7fc885d9b700 1 --2- 172.21.15.1:0/3766624064 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc8700b2880 0x7fc8700b4d90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc87c006010 tx=0x7fc87c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:48.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.645+0000 7fc89b4f0b80 1 Processor -- start 2024-05-04T03:22:48.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.645+0000 7fc89b4f0b80 1 -- start start 2024-05-04T03:22:48.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.646+0000 7fc89b4f0b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55757317fb60 0x55757317ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:48.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.646+0000 7fc89b4f0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575731804b0 0x55757318a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:48.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.646+0000 7fc89b4f0b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757318acd0 0x55757318d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:48.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.646+0000 7fc89b4f0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5575730a2ad0 con 0x5575731804b0 2024-05-04T03:22:48.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.646+0000 7fc89b4f0b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x557573007f30 con 0x55757317fb60 2024-05-04T03:22:48.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.646+0000 7fc89b4f0b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55757309ea90 con 0x55757318acd0 2024-05-04T03:22:48.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.646+0000 7fc885d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575731804b0 0x55757318a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:48.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.646+0000 7fc88659c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55757317fb60 0x55757317ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:48.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.646+0000 7fc88659c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55757317fb60 0x55757317ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:58512/0 (socket says 172.21.15.1:58512) 2024-05-04T03:22:48.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.646+0000 7fc88659c700 1 -- 172.21.15.1:0/3578753017 learned_addr learned my addr 172.21.15.1:0/3578753017 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:48.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.646+0000 7fc885d9b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575731804b0 0x55757318a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39578/0 (socket says 172.21.15.1:39578) 2024-05-04T03:22:48.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.646+0000 7fc886d9d700 1 --2- 172.21.15.1:0/3578753017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757318acd0 0x55757318d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:48.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.646+0000 7fc88659c700 1 -- 172.21.15.1:0/3578753017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757318acd0 msgr2=0x55757318d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:48.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.646+0000 7fc88659c700 1 --2- 172.21.15.1:0/3578753017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757318acd0 0x55757318d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:48.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.646+0000 7fc88659c700 1 -- 172.21.15.1:0/3578753017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575731804b0 msgr2=0x55757318a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:48.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.646+0000 7fc88659c700 1 --2- 172.21.15.1:0/3578753017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575731804b0 0x55757318a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:48.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.646+0000 7fc88659c700 1 -- 172.21.15.1:0/3578753017 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557572fefa60 con 0x55757317fb60 2024-05-04T03:22:48.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.646+0000 7fc88659c700 1 --2- 172.21.15.1:0/3578753017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55757317fb60 0x55757317ff70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc8780e3530 tx=0x7fc878005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b44b911ec91d9617 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:48.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.646+0000 7fc874ff9700 1 -- 172.21.15.1:0/3578753017 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8780ae690 con 0x55757317fb60 2024-05-04T03:22:48.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.647+0000 7fc874ff9700 1 -- 172.21.15.1:0/3578753017 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8780adf90 con 0x55757317fb60 2024-05-04T03:22:48.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.647+0000 7fc874ff9700 1 -- 172.21.15.1:0/3578753017 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc878003fd0 con 0x55757317fb60 2024-05-04T03:22:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.647+0000 7fc89b4f0b80 1 -- 172.21.15.1:0/3578753017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55757317fb60 msgr2=0x55757317ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.647+0000 7fc89b4f0b80 1 --2- 172.21.15.1:0/3578753017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55757317fb60 0x55757317ff70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc8780e3530 tx=0x7fc878005880 comp rx=0 tx=0).stop 2024-05-04T03:22:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.648+0000 7fc89b4f0b80 1 -- 172.21.15.1:0/3578753017 shutdown_connections 2024-05-04T03:22:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.648+0000 7fc89b4f0b80 1 --2- 172.21.15.1:0/3578753017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55757317fb60 0x55757317ff70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.648+0000 7fc89b4f0b80 1 --2- 172.21.15.1:0/3578753017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55757318acd0 0x55757318d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.648+0000 7fc89b4f0b80 1 --2- 172.21.15.1:0/3578753017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575731804b0 0x55757318a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.648+0000 7fc89b4f0b80 1 -- 172.21.15.1:0/3578753017 >> 172.21.15.1:0/3578753017 conn(0x557573002fd0 msgr2=0x55757317ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.648+0000 7fc89b4f0b80 1 -- 172.21.15.1:0/3578753017 shutdown_connections 2024-05-04T03:22:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.648+0000 7fc89b4f0b80 1 -- 172.21.15.1:0/3578753017 wait complete. 2024-05-04T03:22:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.648+0000 7fc89b4f0b80 1 -- start start 2024-05-04T03:22:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.648+0000 7fc89b4f0b80 1 Processor -- start 2024-05-04T03:22:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.649+0000 7fc89b4f0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55757317fb60 0x557573187d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.649+0000 7fc89b4f0b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575731804b0 0x557573188290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.649+0000 7fc89b4f0b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55757318acd0 0x5575731887d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.649+0000 7fc89b4f0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55757318daa0 con 0x55757317fb60 2024-05-04T03:22:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.649+0000 7fc89b4f0b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55757305c060 con 0x55757318acd0 2024-05-04T03:22:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.649+0000 7fc89b4f0b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55757318dc20 con 0x5575731804b0 2024-05-04T03:22:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.649+0000 7fc89b4f0b80 10 client.0 _ll_register_callbacks cb 0x55757303aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:22:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.649+0000 7fc88659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55757317fb60 0x557573187d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.649+0000 7fc885d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575731804b0 0x557573188290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.649+0000 7fc88659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55757317fb60 0x557573187d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39584/0 (socket says 172.21.15.1:39584) 2024-05-04T03:22:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.649+0000 7fc88659c700 1 -- 172.21.15.1:0/3053484779 learned_addr learned my addr 172.21.15.1:0/3053484779 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.649+0000 7fc886d9d700 1 --2- 172.21.15.1:0/3053484779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55757318acd0 0x5575731887d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.649+0000 7fc88659c700 1 -- 172.21.15.1:0/3053484779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575731804b0 msgr2=0x557573188290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:48.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.649+0000 7fc88659c700 1 --2- 172.21.15.1:0/3053484779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575731804b0 0x557573188290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:48.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.649+0000 7fc88659c700 1 -- 172.21.15.1:0/3053484779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55757318acd0 msgr2=0x5575731887d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:48.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.649+0000 7fc88659c700 1 --2- 172.21.15.1:0/3053484779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55757318acd0 0x5575731887d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:48.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.649+0000 7fc88659c700 1 -- 172.21.15.1:0/3053484779 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575730598f0 con 0x55757317fb60 2024-05-04T03:22:48.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.650+0000 7fc88659c700 1 --2- 172.21.15.1:0/3053484779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55757317fb60 0x557573187d50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc8780adc90 tx=0x7fc8780b2340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f07b54f4a17f31 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:48.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.650+0000 7fc86dffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:22:48.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.650+0000 7fc86dffb700 1 -- 172.21.15.1:0/3053484779 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc878072060 con 0x55757317fb60 2024-05-04T03:22:48.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.650+0000 7fc86dffb700 1 -- 172.21.15.1:0/3053484779 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fc8780ae400 con 0x55757317fb60 2024-05-04T03:22:48.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.650+0000 7fc89b4f0b80 1 -- 172.21.15.1:0/3053484779 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8780adf90 con 0x55757317fb60 2024-05-04T03:22:48.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.650+0000 7fc86dffb700 1 -- 172.21.15.1:0/3053484779 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc8780ad1f0 con 0x55757317fb60 2024-05-04T03:22:48.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.650+0000 7fc89b4f0b80 10 client.15642 fetch_fsmap learned FSMap version 29 2024-05-04T03:22:48.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.650+0000 7fc89b4f0b80 1 -- 172.21.15.1:0/3053484779 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557573198740 con 0x55757317fb60 2024-05-04T03:22:48.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.651+0000 7fc86dffb700 1 -- 172.21.15.1:0/3053484779 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8780adf90 con 0x55757317fb60 2024-05-04T03:22:48.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.651+0000 7fc86dffb700 1 -- 172.21.15.1:0/3053484779 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7fc8780acc60 con 0x55757317fb60 2024-05-04T03:22:48.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.651+0000 7fc89b4f0b80 10 client.15642 fetch_fsmap finished waiting for FSMap version 29 2024-05-04T03:22:48.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.651+0000 7fc89b4f0b80 10 client.15642 resolve_mds: validated gid 24494 aka daemon mds.b 2024-05-04T03:22:48.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.651+0000 7fc89b4f0b80 20 client.15642 populate_metadata read hostname 'smithi001' 2024-05-04T03:22:48.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.651+0000 7fc89b4f0b80 1 --2- 172.21.15.1:0/3053484779 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55757319ac10 0x55757319d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:48.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.651+0000 7fc89b4f0b80 4 client.15642 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:22:48.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.651+0000 7fc89b4f0b80 1 -- 172.21.15.1:0/3053484779 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575730b04b0 con 0x55757319ac10 2024-05-04T03:22:48.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.651+0000 7fc885d9b700 1 --2- 172.21.15.1:0/3053484779 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55757319ac10 0x55757319d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:48.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.652+0000 7fc885d9b700 1 --2- 172.21.15.1:0/3053484779 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55757319ac10 0x55757319d030 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd6f9b86dedbd987 server_cookie=e53c09c9fc60efaa in_seq=0 out_seq=0 2024-05-04T03:22:48.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.652+0000 7fc86dffb700 10 client.15642 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:22:48.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.653+0000 7fc86dffb700 1 -- 172.21.15.1:0/3053484779 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5575730b04b0 con 0x55757319ac10 2024-05-04T03:22:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.653+0000 7fc86dffb700 10 client.15642 handle_command_reply: tid=0 2024-05-04T03:22:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.653+0000 7fc89b4f0b80 1 client.15642 shutdown 2024-05-04T03:22:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.653+0000 7fc89b4f0b80 1 -- 172.21.15.1:0/3053484779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55757317fb60 msgr2=0x557573187d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.653+0000 7fc89b4f0b80 1 --2- 172.21.15.1:0/3053484779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55757317fb60 0x557573187d50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc8780adc90 tx=0x7fc8780b2340 comp rx=0 tx=0).stop 2024-05-04T03:22:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.653+0000 7fc89b4f0b80 1 -- 172.21.15.1:0/3053484779 shutdown_connections 2024-05-04T03:22:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.653+0000 7fc89b4f0b80 1 --2- 172.21.15.1:0/3053484779 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55757319ac10 0x55757319d030 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.653+0000 7fc89b4f0b80 1 --2- 172.21.15.1:0/3053484779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55757318acd0 0x5575731887d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.653+0000 7fc89b4f0b80 1 --2- 172.21.15.1:0/3053484779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55757317fb60 0x557573187d50 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.653+0000 7fc89b4f0b80 1 --2- 172.21.15.1:0/3053484779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575731804b0 0x557573188290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.653+0000 7fc86dffb700 0 client.15642 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:22:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.653+0000 7fc89b4f0b80 1 -- 172.21.15.1:0/3053484779 >> 172.21.15.1:0/3053484779 conn(0x557573002fd0 msgr2=0x557573181530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.653+0000 7fc89b4f0b80 1 -- 172.21.15.1:0/3053484779 shutdown_connections 2024-05-04T03:22:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.653+0000 7fc89b4f0b80 1 -- 172.21.15.1:0/3053484779 wait complete. 2024-05-04T03:22:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.653+0000 7fc89b4f0b80 20 client.15642 trim_cache size 0 max 16384 2024-05-04T03:22:48.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.675+0000 7fc89b4f0b80 1 Processor -- start 2024-05-04T03:22:48.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.675+0000 7fc89b4f0b80 1 -- start start 2024-05-04T03:22:48.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.675+0000 7fc89b4f0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55757317fb60 0x55757317b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:48.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.675+0000 7fc89b4f0b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575731804b0 0x5575731836c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:48.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.676+0000 7fc89b4f0b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55757318acd0 0x557573196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:48.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.676+0000 7fc89b4f0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc8881af1e0 con 0x55757317fb60 2024-05-04T03:22:48.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.676+0000 7fc89b4f0b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc8881aca10 con 0x55757318acd0 2024-05-04T03:22:48.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.676+0000 7fc89b4f0b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc8881aa8a0 con 0x5575731804b0 2024-05-04T03:22:48.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.676+0000 7fc886d9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55757318acd0 0x557573196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:48.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.676+0000 7fc88659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55757317fb60 0x55757317b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:48.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.676+0000 7fc88659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55757317fb60 0x55757317b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39594/0 (socket says 172.21.15.1:39594) 2024-05-04T03:22:48.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.676+0000 7fc88659c700 1 -- 172.21.15.1:0/4122835748 learned_addr learned my addr 172.21.15.1:0/4122835748 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:48.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.676+0000 7fc886d9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55757318acd0 0x557573196ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:58534/0 (socket says 172.21.15.1:58534) 2024-05-04T03:22:48.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.676+0000 7fc885d9b700 1 --2- 172.21.15.1:0/4122835748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575731804b0 0x5575731836c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:48.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.676+0000 7fc886d9d700 1 -- 172.21.15.1:0/4122835748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575731804b0 msgr2=0x5575731836c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:48.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.676+0000 7fc886d9d700 1 --2- 172.21.15.1:0/4122835748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575731804b0 0x5575731836c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:48.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.676+0000 7fc886d9d700 1 -- 172.21.15.1:0/4122835748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55757317fb60 msgr2=0x55757317b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:48.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.676+0000 7fc886d9d700 1 --2- 172.21.15.1:0/4122835748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55757317fb60 0x55757317b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:48.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.676+0000 7fc886d9d700 1 -- 172.21.15.1:0/4122835748 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55757318ab00 con 0x55757318acd0 2024-05-04T03:22:48.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.676+0000 7fc88659c700 1 --2- 172.21.15.1:0/4122835748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55757317fb60 0x55757317b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:22:48.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.676+0000 7fc886d9d700 1 --2- 172.21.15.1:0/4122835748 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55757318acd0 0x557573196ec0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc880003d50 tx=0x7fc880009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a8dc201f744ac2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:48.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.677+0000 7fc86ffff700 1 -- 172.21.15.1:0/4122835748 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880003ea0 con 0x55757318acd0 2024-05-04T03:22:48.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.677+0000 7fc86ffff700 1 -- 172.21.15.1:0/4122835748 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8800048c0 con 0x55757318acd0 2024-05-04T03:22:48.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.677+0000 7fc86ffff700 1 -- 172.21.15.1:0/4122835748 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880005280 con 0x55757318acd0 2024-05-04T03:22:48.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.678+0000 7fc89b4f0b80 1 -- 172.21.15.1:0/4122835748 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55757318acd0 msgr2=0x557573196ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:48.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.678+0000 7fc89b4f0b80 1 --2- 172.21.15.1:0/4122835748 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55757318acd0 0x557573196ec0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc880003d50 tx=0x7fc880009f90 comp rx=0 tx=0).stop 2024-05-04T03:22:48.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.678+0000 7fc89b4f0b80 1 -- 172.21.15.1:0/4122835748 shutdown_connections 2024-05-04T03:22:48.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.678+0000 7fc89b4f0b80 1 --2- 172.21.15.1:0/4122835748 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55757318acd0 0x557573196ec0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:48.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.678+0000 7fc89b4f0b80 1 --2- 172.21.15.1:0/4122835748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55757317fb60 0x55757317b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:48.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.678+0000 7fc89b4f0b80 1 --2- 172.21.15.1:0/4122835748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575731804b0 0x5575731836c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:48.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.678+0000 7fc89b4f0b80 1 -- 172.21.15.1:0/4122835748 >> 172.21.15.1:0/4122835748 conn(0x557573002fd0 msgr2=0x557573181490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:48.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.678+0000 7fc89b4f0b80 1 -- 172.21.15.1:0/4122835748 shutdown_connections 2024-05-04T03:22:48.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.678+0000 7fc89b4f0b80 1 -- 172.21.15.1:0/4122835748 wait complete. 2024-05-04T03:22:48.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.679+0000 7fc89b4f0b80 1 -- start start 2024-05-04T03:22:48.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.679+0000 7fc89b4f0b80 1 Processor -- start 2024-05-04T03:22:48.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.679+0000 7fc89b4f0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55757317fb60 0x55757319da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:48.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.679+0000 7fc89b4f0b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575731804b0 0x55757319df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:48.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.679+0000 7fc89b4f0b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55757318acd0 0x55757319e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:48.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.679+0000 7fc89b4f0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fc8881aaef0 con 0x55757317fb60 2024-05-04T03:22:48.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.679+0000 7fc89b4f0b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc8881ac340 con 0x55757318acd0 2024-05-04T03:22:48.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.679+0000 7fc89b4f0b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc8881ab5a0 con 0x5575731804b0 2024-05-04T03:22:48.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.679+0000 7fc89b4f0b80 10 client.0 _ll_register_callbacks cb 0x55757303aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:22:48.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.679+0000 7fc88659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55757317fb60 0x55757319da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:48.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.679+0000 7fc88659c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55757317fb60 0x55757319da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39596/0 (socket says 172.21.15.1:39596) 2024-05-04T03:22:48.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.679+0000 7fc88659c700 1 -- 172.21.15.1:0/2986421733 learned_addr learned my addr 172.21.15.1:0/2986421733 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:48.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.680+0000 7fc885d9b700 1 --2- 172.21.15.1:0/2986421733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575731804b0 0x55757319df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:48.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.680+0000 7fc886d9d700 1 --2- 172.21.15.1:0/2986421733 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55757318acd0 0x55757319e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:48.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.680+0000 7fc88659c700 1 -- 172.21.15.1:0/2986421733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575731804b0 msgr2=0x55757319df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:48.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.680+0000 7fc88659c700 1 --2- 172.21.15.1:0/2986421733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575731804b0 0x55757319df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:48.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.680+0000 7fc88659c700 1 -- 172.21.15.1:0/2986421733 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55757318acd0 msgr2=0x55757319e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:48.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.680+0000 7fc88659c700 1 --2- 172.21.15.1:0/2986421733 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55757318acd0 0x55757319e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:48.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.680+0000 7fc88659c700 1 -- 172.21.15.1:0/2986421733 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575732042d0 con 0x55757317fb60 2024-05-04T03:22:48.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.680+0000 7fc88659c700 1 --2- 172.21.15.1:0/2986421733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55757317fb60 0x55757319da10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc8780ad000 tx=0x7fc878003ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a259a0fccb51519 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:48.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.680+0000 7fc86dffb700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:22:48.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.680+0000 7fc86dffb700 1 -- 172.21.15.1:0/2986421733 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8780b6d70 con 0x55757317fb60 2024-05-04T03:22:48.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.680+0000 7fc86dffb700 1 -- 172.21.15.1:0/2986421733 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fc8780acc60 con 0x55757317fb60 2024-05-04T03:22:48.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.680+0000 7fc89b4f0b80 1 -- 172.21.15.1:0/2986421733 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8800048c0 con 0x55757317fb60 2024-05-04T03:22:48.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.681+0000 7fc86dffb700 1 -- 172.21.15.1:0/2986421733 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc878009500 con 0x55757317fb60 2024-05-04T03:22:48.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.681+0000 7fc89b4f0b80 10 client.15648 fetch_fsmap learned FSMap version 29 2024-05-04T03:22:48.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.681+0000 7fc89b4f0b80 1 -- 172.21.15.1:0/2986421733 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5575732042d0 con 0x55757317fb60 2024-05-04T03:22:48.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.681+0000 7fc86dffb700 1 -- 172.21.15.1:0/2986421733 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8800048c0 con 0x55757317fb60 2024-05-04T03:22:48.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.681+0000 7fc86dffb700 1 -- 172.21.15.1:0/2986421733 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7fc87806da00 con 0x55757317fb60 2024-05-04T03:22:48.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.681+0000 7fc89b4f0b80 10 client.15648 fetch_fsmap finished waiting for FSMap version 29 2024-05-04T03:22:48.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.681+0000 7fc89b4f0b80 10 client.15648 resolve_mds: validated gid 24494 aka daemon mds.b 2024-05-04T03:22:48.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.681+0000 7fc89b4f0b80 20 client.15648 populate_metadata read hostname 'smithi001' 2024-05-04T03:22:48.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.681+0000 7fc89b4f0b80 1 --2- 172.21.15.1:0/2986421733 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5575732058a0 0x557573207cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:48.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.681+0000 7fc89b4f0b80 4 client.15648 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}] 2024-05-04T03:22:48.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.681+0000 7fc89b4f0b80 1 -- 172.21.15.1:0/2986421733 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}) -- 0x5575730aa550 con 0x5575732058a0 2024-05-04T03:22:48.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.681+0000 7fc885d9b700 1 --2- 172.21.15.1:0/2986421733 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5575732058a0 0x557573207cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:48.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.682+0000 7fc86dffb700 10 client.15648 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:22:48.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:48.682+0000 7fc885d9b700 1 --2- 172.21.15.1:0/2986421733 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5575732058a0 0x557573207cc0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0370d11bb9d1136 server_cookie=e706339c7437bfd9 in_seq=0 out_seq=0 2024-05-04T03:22:48.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:48 smithi077 ceph-mon[33531]: pgmap v500: 97 pgs: 97 active+clean; 93 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.9 MiB/s wr, 512 op/s 2024-05-04T03:22:48.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:48 smithi072 ceph-mon[33332]: pgmap v500: 97 pgs: 97 active+clean; 93 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.9 MiB/s wr, 512 op/s 2024-05-04T03:22:50.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:50 smithi001 ceph-mon[29731]: pgmap v501: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.5 MiB/s wr, 639 op/s 2024-05-04T03:22:50.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:50 smithi077 ceph-mon[33531]: pgmap v501: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.5 MiB/s wr, 639 op/s 2024-05-04T03:22:50.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:50.969+0000 7fc86dffb700 1 -- 172.21.15.1:0/2986421733 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5575730aa550 con 0x5575732058a0 2024-05-04T03:22:50.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:50.969+0000 7fc86dffb700 10 client.15648 handle_command_reply: tid=0 2024-05-04T03:22:50.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:50.970+0000 7fc89b4f0b80 1 client.15648 shutdown 2024-05-04T03:22:50.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:50.970+0000 7fc89b4f0b80 1 -- 172.21.15.1:0/2986421733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55757317fb60 msgr2=0x55757319da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:50.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:50.970+0000 7fc89b4f0b80 1 --2- 172.21.15.1:0/2986421733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55757317fb60 0x55757319da10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc8780ad000 tx=0x7fc878003ab0 comp rx=0 tx=0).stop 2024-05-04T03:22:50.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:50.970+0000 7fc89b4f0b80 1 -- 172.21.15.1:0/2986421733 shutdown_connections 2024-05-04T03:22:50.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:50.970+0000 7fc89b4f0b80 1 --2- 172.21.15.1:0/2986421733 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5575732058a0 0x557573207cc0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:50.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:50.970+0000 7fc89b4f0b80 1 --2- 172.21.15.1:0/2986421733 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55757318acd0 0x55757319e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:50.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:50.970+0000 7fc89b4f0b80 1 --2- 172.21.15.1:0/2986421733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55757317fb60 0x55757319da10 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:50.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:50.970+0000 7fc89b4f0b80 1 --2- 172.21.15.1:0/2986421733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5575731804b0 0x55757319df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:50.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:50.970+0000 7fc86dffb700 0 client.15648 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:22:50.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:50.970+0000 7fc89b4f0b80 1 -- 172.21.15.1:0/2986421733 >> 172.21.15.1:0/2986421733 conn(0x557573002fd0 msgr2=0x557573181ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:50.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:50.970+0000 7fc89b4f0b80 1 -- 172.21.15.1:0/2986421733 shutdown_connections 2024-05-04T03:22:50.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:50.970+0000 7fc89b4f0b80 1 -- 172.21.15.1:0/2986421733 wait complete. 2024-05-04T03:22:50.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:50.970+0000 7fc89b4f0b80 20 client.15648 trim_cache size 0 max 16384 2024-05-04T03:22:50.975 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-04T03:22:50.975 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 29, 2024-05-04T03:22:50.975 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24494, 2024-05-04T03:22:50.975 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 10, 2024-05-04T03:22:50.975 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-05-04T03:22:50.975 INFO:teuthology.orchestra.run.smithi001.stdout: "88323539": { 2024-05-04T03:22:50.975 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 10, 2024-05-04T03:22:50.975 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-05-04T03:22:50.975 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-05-04T03:22:50.975 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-05-04T03:22:50.976 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-05-04T03:22:50.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:22:50.976 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 60.0, 2024-05-04T03:22:50.976 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 90.0, 2024-05-04T03:22:50.976 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-05-04T03:22:50.976 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-05-04T03:22:50.976 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-05-04T03:22:50.976 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-05-04T03:22:50.976 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-05-04T03:22:50.976 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-05-04T03:22:50.976 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:22:50.976 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:22:50.976 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:22:50.976 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:22:50.976 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:22:50.976 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-04T03:22:50.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:50.971+0000 7fc88cadc700 1 -- 172.21.15.1:0/3766624064 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc8700b2880 msgr2=0x7fc8700b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:50.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:50.971+0000 7fc88cadc700 1 --2- 172.21.15.1:0/3766624064 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc8700b2880 0x7fc8700b4d90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc87c006010 tx=0x7fc87c01a040 comp rx=0 tx=0).stop 2024-05-04T03:22:50.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:50.971+0000 7fc88cadc700 1 -- 172.21.15.1:0/3766624064 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc8881034e0 msgr2=0x7fc88819ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:50.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:50.971+0000 7fc88cadc700 1 --2- 172.21.15.1:0/3766624064 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc8881034e0 0x7fc88819ae50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc878010e40 tx=0x7fc87800b5a0 comp rx=0 tx=0).stop 2024-05-04T03:22:50.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:50.971+0000 7fc88cadc700 1 -- 172.21.15.1:0/3766624064 shutdown_connections 2024-05-04T03:22:50.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:50.971+0000 7fc88cadc700 1 --2- 172.21.15.1:0/3766624064 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc8881034e0 0x7fc88819ae50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:50.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:50.971+0000 7fc88cadc700 1 --2- 172.21.15.1:0/3766624064 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fc8700b2880 0x7fc8700b4d90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:50.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:50.971+0000 7fc88cadc700 1 --2- 172.21.15.1:0/3766624064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc888108760 0x7fc88819f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:50.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:50.971+0000 7fc88cadc700 1 --2- 172.21.15.1:0/3766624064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc888107e10 0x7fc88819b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:50.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:50.971+0000 7fc88cadc700 1 -- 172.21.15.1:0/3766624064 >> 172.21.15.1:0/3766624064 conn(0x7fc8880fef30 msgr2=0x7fc88806c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:50.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:50.971+0000 7fc88cadc700 1 -- 172.21.15.1:0/3766624064 shutdown_connections 2024-05-04T03:22:50.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:22:50.971+0000 7fc88cadc700 1 -- 172.21.15.1:0/3766624064 wait complete. 2024-05-04T03:22:50.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:50 smithi072 ceph-mon[33332]: pgmap v501: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.5 MiB/s wr, 639 op/s 2024-05-04T03:22:51.747 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '88323539', quiesce took 8.6 seconds. Will release after: 30.0 2024-05-04T03:22:52.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:52 smithi001 ceph-mon[29731]: pgmap v502: 97 pgs: 97 active+clean; 90 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.3 MiB/s wr, 569 op/s 2024-05-04T03:22:52.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:52 smithi077 ceph-mon[33531]: pgmap v502: 97 pgs: 97 active+clean; 90 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.3 MiB/s wr, 569 op/s 2024-05-04T03:22:52.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:52 smithi072 ceph-mon[33332]: pgmap v502: 97 pgs: 97 active+clean; 90 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.3 MiB/s wr, 569 op/s 2024-05-04T03:22:54.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:54 smithi001 ceph-mon[29731]: pgmap v503: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.6 MiB/s wr, 485 op/s 2024-05-04T03:22:54.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:54 smithi077 ceph-mon[33531]: pgmap v503: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.6 MiB/s wr, 485 op/s 2024-05-04T03:22:54.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:54 smithi072 ceph-mon[33332]: pgmap v503: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.6 MiB/s wr, 485 op/s 2024-05-04T03:22:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.969+0000 7fddc4765640 1 -- 172.21.15.1:0/1406058011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fddbc103c60 msgr2=0x7fddbc104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.969+0000 7fddc4765640 1 --2- 172.21.15.1:0/1406058011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fddbc103c60 0x7fddbc104960 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fddb8009900 tx=0x7fddb802a410 comp rx=0 tx=0).stop 2024-05-04T03:22:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.969+0000 7fddc4765640 1 -- 172.21.15.1:0/1406058011 shutdown_connections 2024-05-04T03:22:55.972 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.969+0000 7fddc4765640 1 --2- 172.21.15.1:0/1406058011 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fddbc104ea0 0x7fddbc1052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.969+0000 7fddc4765640 1 --2- 172.21.15.1:0/1406058011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddbc109ba0 0x7fddbc10bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.969+0000 7fddc4765640 1 --2- 172.21.15.1:0/1406058011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fddbc103c60 0x7fddbc104960 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.969+0000 7fddc4765640 1 -- 172.21.15.1:0/1406058011 >> 172.21.15.1:0/1406058011 conn(0x7fddbc0fee70 msgr2=0x7fddbc101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.969+0000 7fddc4765640 1 -- 172.21.15.1:0/1406058011 shutdown_connections 2024-05-04T03:22:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.969+0000 7fddc4765640 1 -- 172.21.15.1:0/1406058011 wait complete. 2024-05-04T03:22:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.969+0000 7fddc4765640 1 Processor -- start 2024-05-04T03:22:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.970+0000 7fddc4765640 1 -- start start 2024-05-04T03:22:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.970+0000 7fddc4765640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fddbc103c60 0x7fddbc19a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.970+0000 7fddc4765640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddbc104ea0 0x7fddbc19a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.970+0000 7fddc4765640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fddbc109ba0 0x7fddbc1a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.970+0000 7fddc4765640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc10f740 con 0x7fddbc109ba0 2024-05-04T03:22:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.970+0000 7fddc4765640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc10f5c0 con 0x7fddbc103c60 2024-05-04T03:22:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.970+0000 7fddc4765640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc10f8c0 con 0x7fddbc104ea0 2024-05-04T03:22:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.970+0000 7fddc2d02640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fddbc109ba0 0x7fddbc1a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.970+0000 7fddc2d02640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fddbc109ba0 0x7fddbc1a1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59374/0 (socket says 172.21.15.1:59374) 2024-05-04T03:22:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.970+0000 7fddc2d02640 1 -- 172.21.15.1:0/2346088834 learned_addr learned my addr 172.21.15.1:0/2346088834 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.970+0000 7fddc2501640 1 --2- 172.21.15.1:0/2346088834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fddbc103c60 0x7fddbc19a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.970+0000 7fddc1d00640 1 --2- 172.21.15.1:0/2346088834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddbc104ea0 0x7fddbc19a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.970+0000 7fddc2d02640 1 -- 172.21.15.1:0/2346088834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddbc104ea0 msgr2=0x7fddbc19a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.970+0000 7fddc2d02640 1 --2- 172.21.15.1:0/2346088834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddbc104ea0 0x7fddbc19a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.970+0000 7fddc2d02640 1 -- 172.21.15.1:0/2346088834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fddbc103c60 msgr2=0x7fddbc19a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.970+0000 7fddc2d02640 1 --2- 172.21.15.1:0/2346088834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fddbc103c60 0x7fddbc19a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.970+0000 7fddc2d02640 1 -- 172.21.15.1:0/2346088834 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddbc1a1e00 con 0x7fddbc109ba0 2024-05-04T03:22:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.971+0000 7fddc2d02640 1 --2- 172.21.15.1:0/2346088834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fddbc109ba0 0x7fddbc1a1790 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fddb400cad0 tx=0x7fddb400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.971+0000 7fddb37fe640 1 -- 172.21.15.1:0/2346088834 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb400ae20 con 0x7fddbc109ba0 2024-05-04T03:22:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.971+0000 7fddb37fe640 1 -- 172.21.15.1:0/2346088834 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddb4005020 con 0x7fddbc109ba0 2024-05-04T03:22:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.971+0000 7fddc4765640 1 -- 172.21.15.1:0/2346088834 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddbc1a20f0 con 0x7fddbc109ba0 2024-05-04T03:22:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.971+0000 7fddb37fe640 1 -- 172.21.15.1:0/2346088834 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb400f6b0 con 0x7fddbc109ba0 2024-05-04T03:22:55.977 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.971+0000 7fddc4765640 1 -- 172.21.15.1:0/2346088834 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddbc1a25f0 con 0x7fddbc109ba0 2024-05-04T03:22:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.974+0000 7fddcb618740 1 Processor -- start 2024-05-04T03:22:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.974+0000 7fddb37fe640 1 -- 172.21.15.1:0/2346088834 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7fddb40051d0 con 0x7fddbc109ba0 2024-05-04T03:22:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.974+0000 7fddcb618740 1 -- start start 2024-05-04T03:22:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.974+0000 7fddcb618740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca84e61b0 0x555ca8887a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.974+0000 7fddcb618740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ca84bd2f0 0x555ca8887fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.974+0000 7fddcb618740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555ca84c54d0 0x555ca888eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.974+0000 7fddcb618740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555ca87ec5d0 con 0x555ca84e61b0 2024-05-04T03:22:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.974+0000 7fddcb618740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x555ca87a5890 con 0x555ca84c54d0 2024-05-04T03:22:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.974+0000 7fddcb618740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555ca87e8590 con 0x555ca84bd2f0 2024-05-04T03:22:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.975+0000 7fddc1d00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca84e61b0 0x555ca8887a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.975+0000 7fddc1d00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca84e61b0 0x555ca8887a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59376/0 (socket says 172.21.15.1:59376) 2024-05-04T03:22:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.975+0000 7fddc1d00640 1 -- 172.21.15.1:0/3146096657 learned_addr learned my addr 172.21.15.1:0/3146096657 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.975+0000 7fddc1d00640 1 --2- 172.21.15.1:0/3146096657 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555ca84c54d0 0x555ca888eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.975+0000 7fddc2501640 1 --2- 172.21.15.1:0/3146096657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ca84bd2f0 0x555ca8887fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.975+0000 7fddb37fe640 1 --2- 172.21.15.1:0/2346088834 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fdda80b28a0 0x7fdda80b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.990 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.975+0000 7fddc1d00640 1 -- 172.21.15.1:0/3146096657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ca84bd2f0 msgr2=0x555ca8887fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:55.990 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.975+0000 7fddc1d00640 1 --2- 172.21.15.1:0/3146096657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ca84bd2f0 0x555ca8887fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.990 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.975+0000 7fddc1d00640 1 -- 172.21.15.1:0/3146096657 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555ca84c54d0 msgr2=0x555ca888eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:55.990 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.975+0000 7fddc1d00640 1 --2- 172.21.15.1:0/3146096657 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555ca84c54d0 0x555ca888eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.990 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.975+0000 7fddc1d00640 1 -- 172.21.15.1:0/3146096657 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ca84fe210 con 0x555ca84e61b0 2024-05-04T03:22:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.975+0000 7fddc2d02640 1 --2- 172.21.15.1:0/2346088834 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fdda80b28a0 0x7fdda80b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.975+0000 7fddc1d00640 1 --2- 172.21.15.1:0/3146096657 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca84e61b0 0x555ca8887a60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fddac0125e0 tx=0x7fddac00b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cbbe8d9f6793c24 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.975+0000 7fddb37fe640 1 -- 172.21.15.1:0/2346088834 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fddb40e2100 con 0x7fddbc109ba0 2024-05-04T03:22:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.975+0000 7fddb0ff9640 1 -- 172.21.15.1:0/3146096657 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac004030 con 0x555ca84e61b0 2024-05-04T03:22:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.975+0000 7fddb0ff9640 1 -- 172.21.15.1:0/3146096657 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddac0041d0 con 0x555ca84e61b0 2024-05-04T03:22:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.975+0000 7fddb0ff9640 1 -- 172.21.15.1:0/3146096657 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac010660 con 0x555ca84e61b0 2024-05-04T03:22:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.976+0000 7fddcb618740 1 -- 172.21.15.1:0/3146096657 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca84e61b0 msgr2=0x555ca8887a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.976+0000 7fddc2d02640 1 --2- 172.21.15.1:0/2346088834 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fdda80b28a0 0x7fdda80b4db0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fddb40a9820 tx=0x7fddb40a5040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.976+0000 7fddcb618740 1 --2- 172.21.15.1:0/3146096657 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca84e61b0 0x555ca8887a60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fddac0125e0 tx=0x7fddac00b820 comp rx=0 tx=0).stop 2024-05-04T03:22:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.976+0000 7fddcb618740 1 -- 172.21.15.1:0/3146096657 shutdown_connections 2024-05-04T03:22:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.976+0000 7fddcb618740 1 --2- 172.21.15.1:0/3146096657 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555ca84c54d0 0x555ca888eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.976+0000 7fddcb618740 1 --2- 172.21.15.1:0/3146096657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ca84bd2f0 0x555ca8887fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.976+0000 7fddcb618740 1 --2- 172.21.15.1:0/3146096657 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca84e61b0 0x555ca8887a60 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.976+0000 7fddcb618740 1 -- 172.21.15.1:0/3146096657 >> 172.21.15.1:0/3146096657 conn(0x555ca84da300 msgr2=0x555ca88833a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.977+0000 7fddcb618740 1 -- 172.21.15.1:0/3146096657 shutdown_connections 2024-05-04T03:22:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.977+0000 7fddcb618740 1 -- 172.21.15.1:0/3146096657 wait complete. 2024-05-04T03:22:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.977+0000 7fddcb618740 1 -- start start 2024-05-04T03:22:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.977+0000 7fddcb618740 1 Processor -- start 2024-05-04T03:22:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.977+0000 7fddcb618740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555ca84bd2f0 0x555ca8891670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.977+0000 7fddcb618740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca84c54d0 0x555ca8891bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.977+0000 7fddcb618740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ca84e61b0 0x555ca8898aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.977+0000 7fddcb618740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555ca888f930 con 0x555ca84c54d0 2024-05-04T03:22:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.977+0000 7fddcb618740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x555ca87a9370 con 0x555ca84bd2f0 2024-05-04T03:22:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.977+0000 7fddcb618740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555ca888fab0 con 0x555ca84e61b0 2024-05-04T03:22:55.993 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.977+0000 7fddcb618740 10 client.0 _ll_register_callbacks cb 0x555ca84fda00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:22:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.977+0000 7fddc2501640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca84c54d0 0x555ca8891bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.977+0000 7fddc2501640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca84c54d0 0x555ca8891bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59382/0 (socket says 172.21.15.1:59382) 2024-05-04T03:22:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.977+0000 7fddc2501640 1 -- 172.21.15.1:0/685934244 learned_addr learned my addr 172.21.15.1:0/685934244 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.978+0000 7fddc1d00640 1 --2- 172.21.15.1:0/685934244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555ca84bd2f0 0x555ca8891670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.978+0000 7fddc1d00640 1 --2- 172.21.15.1:0/685934244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ca84e61b0 0x555ca8898aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.978+0000 7fddc2501640 1 -- 172.21.15.1:0/685934244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ca84e61b0 msgr2=0x555ca8898aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.978+0000 7fddc2501640 1 --2- 172.21.15.1:0/685934244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ca84e61b0 0x555ca8898aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.978+0000 7fddc2501640 1 -- 172.21.15.1:0/685934244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555ca84bd2f0 msgr2=0x555ca8891670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.978+0000 7fddc2501640 1 --2- 172.21.15.1:0/685934244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555ca84bd2f0 0x555ca8891670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.978+0000 7fddc2501640 1 -- 172.21.15.1:0/685934244 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ca85ca620 con 0x555ca84c54d0 2024-05-04T03:22:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.978+0000 7fddc1d00640 1 --2- 172.21.15.1:0/685934244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ca84e61b0 0x555ca8898aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:22:55.994 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.978+0000 7fddc2501640 1 --2- 172.21.15.1:0/685934244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca84c54d0 0x555ca8891bb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fddb8009230 tx=0x7fddb8031e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc51048ae4c24621 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.978+0000 7fdda5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:22:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.978+0000 7fdda5ffb640 1 -- 172.21.15.1:0/685934244 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8035070 con 0x555ca84c54d0 2024-05-04T03:22:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.978+0000 7fdda5ffb640 1 -- 172.21.15.1:0/685934244 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fddb8004240 con 0x555ca84c54d0 2024-05-04T03:22:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.978+0000 7fddcb618740 1 -- 172.21.15.1:0/685934244 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddac0041d0 con 0x555ca84c54d0 2024-05-04T03:22:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.978+0000 7fdda5ffb640 1 -- 172.21.15.1:0/685934244 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fddb80318f0 con 0x555ca84c54d0 2024-05-04T03:22:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.979+0000 7fddcb618740 10 client.15660 fetch_fsmap learned FSMap version 29 2024-05-04T03:22:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.979+0000 7fddcb618740 1 -- 172.21.15.1:0/685934244 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555ca889ba40 con 0x555ca84c54d0 2024-05-04T03:22:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.979+0000 7fdda5ffb640 1 -- 172.21.15.1:0/685934244 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddac0041d0 con 0x555ca84c54d0 2024-05-04T03:22:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.979+0000 7fdda5ffb640 1 -- 172.21.15.1:0/685934244 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7fddb8043020 con 0x555ca84c54d0 2024-05-04T03:22:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.979+0000 7fddcb618740 10 client.15660 fetch_fsmap finished waiting for FSMap version 29 2024-05-04T03:22:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.979+0000 7fddcb618740 10 client.15660 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:22:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.979+0000 7fddcb618740 20 client.15660 populate_metadata read hostname 'smithi001' 2024-05-04T03:22:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.979+0000 7fddcb618740 1 --2- 172.21.15.1:0/685934244 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x555ca84a4d80 0x555ca889f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.979+0000 7fddcb618740 4 client.15660 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:22:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.979+0000 7fddcb618740 1 -- 172.21.15.1:0/685934244 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ca87fa1e0 con 0x555ca84a4d80 2024-05-04T03:22:55.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.979+0000 7fddc1d00640 1 --2- 172.21.15.1:0/685934244 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x555ca84a4d80 0x555ca889f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:55.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.980+0000 7fddc1d00640 1 --2- 172.21.15.1:0/685934244 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x555ca84a4d80 0x555ca889f140 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5a2d94816461270 server_cookie=1650c5e1c89c31df in_seq=0 out_seq=0 2024-05-04T03:22:55.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.980+0000 7fdda5ffb640 10 client.15660 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:22:55.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.980+0000 7fdda5ffb640 1 -- 172.21.15.1:0/685934244 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555ca87fa1e0 con 0x555ca84a4d80 2024-05-04T03:22:55.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.980+0000 7fdda5ffb640 10 client.15660 handle_command_reply: tid=0 2024-05-04T03:22:55.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.980+0000 7fddcb618740 1 client.15660 shutdown 2024-05-04T03:22:55.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.981+0000 7fddcb618740 1 -- 172.21.15.1:0/685934244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca84c54d0 msgr2=0x555ca8891bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:55.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.981+0000 7fddcb618740 1 --2- 172.21.15.1:0/685934244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca84c54d0 0x555ca8891bb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fddb8009230 tx=0x7fddb8031e80 comp rx=0 tx=0).stop 2024-05-04T03:22:55.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.981+0000 7fddcb618740 1 -- 172.21.15.1:0/685934244 shutdown_connections 2024-05-04T03:22:55.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.981+0000 7fddcb618740 1 --2- 172.21.15.1:0/685934244 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x555ca84a4d80 0x555ca889f140 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.981+0000 7fddcb618740 1 --2- 172.21.15.1:0/685934244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ca84e61b0 0x555ca8898aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.981+0000 7fddcb618740 1 --2- 172.21.15.1:0/685934244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca84c54d0 0x555ca8891bb0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.981+0000 7fdda5ffb640 0 client.15660 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:22:55.996 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.981+0000 7fddcb618740 1 --2- 172.21.15.1:0/685934244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555ca84bd2f0 0x555ca8891670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.981+0000 7fddcb618740 1 -- 172.21.15.1:0/685934244 >> 172.21.15.1:0/685934244 conn(0x555ca84da300 msgr2=0x555ca8886300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.981+0000 7fddcb618740 1 -- 172.21.15.1:0/685934244 shutdown_connections 2024-05-04T03:22:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.981+0000 7fddcb618740 1 -- 172.21.15.1:0/685934244 wait complete. 2024-05-04T03:22:55.997 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.981+0000 7fddcb618740 20 client.15660 trim_cache size 0 max 16384 2024-05-04T03:22:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.996+0000 7fddcb618740 1 Processor -- start 2024-05-04T03:22:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.996+0000 7fddcb618740 1 -- start start 2024-05-04T03:22:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.996+0000 7fddcb618740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca84bd2f0 0x555ca8886900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.996+0000 7fddcb618740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555ca84c54d0 0x555ca88a4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.996+0000 7fddcb618740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ca84e61b0 0x555ca88a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:56.000 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.996+0000 7fddcb618740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc1adac0 con 0x555ca84bd2f0 2024-05-04T03:22:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.996+0000 7fddcb618740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc1ab330 con 0x555ca84c54d0 2024-05-04T03:22:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.996+0000 7fddcb618740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc1a9180 con 0x555ca84e61b0 2024-05-04T03:22:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.996+0000 7fddc1d00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca84bd2f0 0x555ca8886900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.996+0000 7fddc1d00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca84bd2f0 0x555ca8886900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59390/0 (socket says 172.21.15.1:59390) 2024-05-04T03:22:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.996+0000 7fddc1d00640 1 -- 172.21.15.1:0/1854893358 learned_addr learned my addr 172.21.15.1:0/1854893358 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.997+0000 7fddc2501640 1 --2- 172.21.15.1:0/1854893358 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555ca84c54d0 0x555ca88a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.997+0000 7fddc1d00640 1 -- 172.21.15.1:0/1854893358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ca84e61b0 msgr2=0x555ca88a4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.997+0000 7fddc1d00640 1 --2- 172.21.15.1:0/1854893358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ca84e61b0 0x555ca88a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.997+0000 7fddc1d00640 1 -- 172.21.15.1:0/1854893358 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555ca84c54d0 msgr2=0x555ca88a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.997+0000 7fddc1d00640 1 --2- 172.21.15.1:0/1854893358 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555ca84c54d0 0x555ca88a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.997+0000 7fddc1d00640 1 -- 172.21.15.1:0/1854893358 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ca8886e40 con 0x555ca84bd2f0 2024-05-04T03:22:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.997+0000 7fddc2501640 1 --2- 172.21.15.1:0/1854893358 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555ca84c54d0 0x555ca88a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:22:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.997+0000 7fddc1d00640 1 --2- 172.21.15.1:0/1854893358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca84bd2f0 0x555ca8886900 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fddac0125e0 tx=0x7fddac0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86da66763d9255fa server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:56.001 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.997+0000 7fdda7fff640 1 -- 172.21.15.1:0/1854893358 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac005050 con 0x555ca84bd2f0 2024-05-04T03:22:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.997+0000 7fdda7fff640 1 -- 172.21.15.1:0/1854893358 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddac0120d0 con 0x555ca84bd2f0 2024-05-04T03:22:56.002 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.997+0000 7fdda7fff640 1 -- 172.21.15.1:0/1854893358 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac02a360 con 0x555ca84bd2f0 2024-05-04T03:22:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.998+0000 7fddcb618740 1 -- 172.21.15.1:0/1854893358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca84bd2f0 msgr2=0x555ca8886900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.998+0000 7fddcb618740 1 --2- 172.21.15.1:0/1854893358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca84bd2f0 0x555ca8886900 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fddac0125e0 tx=0x7fddac0058a0 comp rx=0 tx=0).stop 2024-05-04T03:22:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.998+0000 7fddcb618740 1 -- 172.21.15.1:0/1854893358 shutdown_connections 2024-05-04T03:22:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.998+0000 7fddcb618740 1 --2- 172.21.15.1:0/1854893358 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555ca84c54d0 0x555ca88a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.998+0000 7fddcb618740 1 --2- 172.21.15.1:0/1854893358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ca84e61b0 0x555ca88a4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.998+0000 7fddcb618740 1 --2- 172.21.15.1:0/1854893358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca84bd2f0 0x555ca8886900 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.998+0000 7fddcb618740 1 -- 172.21.15.1:0/1854893358 >> 172.21.15.1:0/1854893358 conn(0x555ca84da300 msgr2=0x555ca88861f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.998+0000 7fddcb618740 1 -- 172.21.15.1:0/1854893358 shutdown_connections 2024-05-04T03:22:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.998+0000 7fddcb618740 1 -- 172.21.15.1:0/1854893358 wait complete. 2024-05-04T03:22:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.998+0000 7fddcb618740 1 -- start start 2024-05-04T03:22:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.998+0000 7fddcb618740 1 Processor -- start 2024-05-04T03:22:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.999+0000 7fddcb618740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca84bd2f0 0x555ca8898730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.999+0000 7fddcb618740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ca84c54d0 0x555ca8898c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.999+0000 7fddcb618740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555ca84e61b0 0x555ca89058d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.999+0000 7fddcb618740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc1a97f0 con 0x555ca84bd2f0 2024-05-04T03:22:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.999+0000 7fddcb618740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc1aac60 con 0x555ca84e61b0 2024-05-04T03:22:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.999+0000 7fddcb618740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc1a9ec0 con 0x555ca84c54d0 2024-05-04T03:22:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.999+0000 7fddcb618740 10 client.0 _ll_register_callbacks cb 0x7fddb8009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:22:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.999+0000 7fddc1d00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca84bd2f0 0x555ca8898730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.999+0000 7fddc1d00640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca84bd2f0 0x555ca8898730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59392/0 (socket says 172.21.15.1:59392) 2024-05-04T03:22:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.999+0000 7fddc1d00640 1 -- 172.21.15.1:0/4027359014 learned_addr learned my addr 172.21.15.1:0/4027359014 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:22:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.999+0000 7fddc1d00640 1 --2- 172.21.15.1:0/4027359014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555ca84e61b0 0x555ca89058d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.999+0000 7fddc2501640 1 --2- 172.21.15.1:0/4027359014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ca84c54d0 0x555ca8898c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.999+0000 7fddc1d00640 1 -- 172.21.15.1:0/4027359014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ca84c54d0 msgr2=0x555ca8898c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.999+0000 7fddc1d00640 1 --2- 172.21.15.1:0/4027359014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ca84c54d0 0x555ca8898c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.999+0000 7fddc1d00640 1 -- 172.21.15.1:0/4027359014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555ca84e61b0 msgr2=0x555ca89058d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:22:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.999+0000 7fddc1d00640 1 --2- 172.21.15.1:0/4027359014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555ca84e61b0 0x555ca89058d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.999+0000 7fddc1d00640 1 -- 172.21.15.1:0/4027359014 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ca8905e90 con 0x555ca84bd2f0 2024-05-04T03:22:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.999+0000 7fddc1d00640 1 --2- 172.21.15.1:0/4027359014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca84bd2f0 0x555ca8898730 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fddac0125b0 tx=0x7fddac012280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24438511089cd269 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:22:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.999+0000 7fdda5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:22:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.999+0000 7fdda5ffb640 1 -- 172.21.15.1:0/4027359014 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac03d060 con 0x555ca84bd2f0 2024-05-04T03:22:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.999+0000 7fdda5ffb640 1 -- 172.21.15.1:0/4027359014 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fddac0051e0 con 0x555ca84bd2f0 2024-05-04T03:22:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:55.999+0000 7fddcb618740 1 -- 172.21.15.1:0/4027359014 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddac0120d0 con 0x555ca84bd2f0 2024-05-04T03:22:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.000+0000 7fdda5ffb640 1 -- 172.21.15.1:0/4027359014 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fddac005040 con 0x555ca84bd2f0 2024-05-04T03:22:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.000+0000 7fddcb618740 10 client.15666 fetch_fsmap learned FSMap version 29 2024-05-04T03:22:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.000+0000 7fddcb618740 1 -- 172.21.15.1:0/4027359014 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555ca8907090 con 0x555ca84bd2f0 2024-05-04T03:22:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.000+0000 7fdda5ffb640 1 -- 172.21.15.1:0/4027359014 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddac0120d0 con 0x555ca84bd2f0 2024-05-04T03:22:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.000+0000 7fdda5ffb640 1 -- 172.21.15.1:0/4027359014 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7fddac010460 con 0x555ca84bd2f0 2024-05-04T03:22:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.000+0000 7fddcb618740 10 client.15666 fetch_fsmap finished waiting for FSMap version 29 2024-05-04T03:22:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.000+0000 7fddcb618740 10 client.15666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:22:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.000+0000 7fddcb618740 20 client.15666 populate_metadata read hostname 'smithi001' 2024-05-04T03:22:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.000+0000 7fddcb618740 1 --2- 172.21.15.1:0/4027359014 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x555ca847a1c0 0x555ca8909810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:22:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.000+0000 7fddcb618740 4 client.15666 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:22:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.000+0000 7fddcb618740 1 -- 172.21.15.1:0/4027359014 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555ca87f4050 con 0x555ca847a1c0 2024-05-04T03:22:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.000+0000 7fddc2501640 1 --2- 172.21.15.1:0/4027359014 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x555ca847a1c0 0x555ca8909810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:22:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.001+0000 7fdda5ffb640 10 client.15666 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:22:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.001+0000 7fddc2501640 1 --2- 172.21.15.1:0/4027359014 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x555ca847a1c0 0x555ca8909810 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81b04873320a1ad0 server_cookie=a5320500d9076a48 in_seq=0 out_seq=0 2024-05-04T03:22:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.009+0000 7fdda5ffb640 1 -- 172.21.15.1:0/4027359014 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555ca87f4050 con 0x555ca847a1c0 2024-05-04T03:22:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.009+0000 7fdda5ffb640 10 client.15666 handle_command_reply: tid=0 2024-05-04T03:22:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.009+0000 7fddcb618740 1 client.15666 shutdown 2024-05-04T03:22:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.010+0000 7fddcb618740 1 -- 172.21.15.1:0/4027359014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca84bd2f0 msgr2=0x555ca8898730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.010+0000 7fddcb618740 1 --2- 172.21.15.1:0/4027359014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca84bd2f0 0x555ca8898730 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fddac0125b0 tx=0x7fddac012280 comp rx=0 tx=0).stop 2024-05-04T03:22:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.010+0000 7fddcb618740 1 -- 172.21.15.1:0/4027359014 shutdown_connections 2024-05-04T03:22:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.010+0000 7fddcb618740 1 --2- 172.21.15.1:0/4027359014 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x555ca847a1c0 0x555ca8909810 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.010+0000 7fddcb618740 1 --2- 172.21.15.1:0/4027359014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555ca84e61b0 0x555ca89058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.010+0000 7fddcb618740 1 --2- 172.21.15.1:0/4027359014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ca84c54d0 0x555ca8898c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.010+0000 7fdda5ffb640 0 client.15666 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:22:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.010+0000 7fddcb618740 1 --2- 172.21.15.1:0/4027359014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555ca84bd2f0 0x555ca8898730 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.010+0000 7fddcb618740 1 -- 172.21.15.1:0/4027359014 >> 172.21.15.1:0/4027359014 conn(0x555ca84da300 msgr2=0x555ca88a3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.010+0000 7fddcb618740 1 -- 172.21.15.1:0/4027359014 shutdown_connections 2024-05-04T03:22:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.010+0000 7fddcb618740 1 -- 172.21.15.1:0/4027359014 wait complete. 2024-05-04T03:22:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.010+0000 7fddcb618740 20 client.15666 trim_cache size 0 max 16384 2024-05-04T03:22:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:22:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:22:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:22:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:22:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.010+0000 7fddc4765640 1 -- 172.21.15.1:0/2346088834 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fdda80b28a0 msgr2=0x7fdda80b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.010+0000 7fddc4765640 1 --2- 172.21.15.1:0/2346088834 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fdda80b28a0 0x7fdda80b4db0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fddb40a9820 tx=0x7fddb40a5040 comp rx=0 tx=0).stop 2024-05-04T03:22:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.010+0000 7fddc4765640 1 -- 172.21.15.1:0/2346088834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fddbc109ba0 msgr2=0x7fddbc1a1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:22:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.010+0000 7fddc4765640 1 --2- 172.21.15.1:0/2346088834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fddbc109ba0 0x7fddbc1a1790 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fddb400cad0 tx=0x7fddb400cf90 comp rx=0 tx=0).stop 2024-05-04T03:22:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.010+0000 7fddc4765640 1 -- 172.21.15.1:0/2346088834 shutdown_connections 2024-05-04T03:22:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.010+0000 7fddc4765640 1 --2- 172.21.15.1:0/2346088834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fddbc109ba0 0x7fddbc1a1790 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.010+0000 7fddc4765640 1 --2- 172.21.15.1:0/2346088834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fddbc104ea0 0x7fddbc19a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.010+0000 7fddc4765640 1 --2- 172.21.15.1:0/2346088834 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fdda80b28a0 0x7fdda80b4db0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.010+0000 7fddc4765640 1 --2- 172.21.15.1:0/2346088834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fddbc103c60 0x7fddbc19a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:22:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.010+0000 7fddc4765640 1 -- 172.21.15.1:0/2346088834 >> 172.21.15.1:0/2346088834 conn(0x7fddbc0fee70 msgr2=0x7fddbc105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:22:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.011+0000 7fddc4765640 1 -- 172.21.15.1:0/2346088834 shutdown_connections 2024-05-04T03:22:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:22:56.011+0000 7fddc4765640 1 -- 172.21.15.1:0/2346088834 wait complete. 2024-05-04T03:22:56.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:56 smithi001 ceph-mon[29731]: pgmap v504: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.6 MiB/s wr, 485 op/s 2024-05-04T03:22:56.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:56 smithi077 ceph-mon[33531]: pgmap v504: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.6 MiB/s wr, 485 op/s 2024-05-04T03:22:56.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:56 smithi072 ceph-mon[33332]: pgmap v504: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.6 MiB/s wr, 485 op/s 2024-05-04T03:22:58.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:22:58 smithi001 ceph-mon[29731]: pgmap v505: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.3 MiB/s wr, 245 op/s 2024-05-04T03:22:58.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:22:58 smithi077 ceph-mon[33531]: pgmap v505: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.3 MiB/s wr, 245 op/s 2024-05-04T03:22:58.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:22:58 smithi072 ceph-mon[33332]: pgmap v505: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.3 MiB/s wr, 245 op/s 2024-05-04T03:23:00.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:00 smithi001 ceph-mon[29731]: pgmap v506: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.3 MiB/s wr, 249 op/s 2024-05-04T03:23:00.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:00 smithi077 ceph-mon[33531]: pgmap v506: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.3 MiB/s wr, 249 op/s 2024-05-04T03:23:00.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:00 smithi072 ceph-mon[33332]: pgmap v506: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.3 MiB/s wr, 249 op/s 2024-05-04T03:23:02.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:02 smithi001 ceph-mon[29731]: pgmap v507: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 732 KiB/s wr, 122 op/s 2024-05-04T03:23:02.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:02 smithi077 ceph-mon[33531]: pgmap v507: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 732 KiB/s wr, 122 op/s 2024-05-04T03:23:02.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:02 smithi072 ceph-mon[33332]: pgmap v507: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 732 KiB/s wr, 122 op/s 2024-05-04T03:23:04.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:04 smithi001 ceph-mon[29731]: pgmap v508: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 66 KiB/s wr, 32 op/s 2024-05-04T03:23:04.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:04 smithi077 ceph-mon[33531]: pgmap v508: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 66 KiB/s wr, 32 op/s 2024-05-04T03:23:04.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:04 smithi072 ceph-mon[33332]: pgmap v508: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 66 KiB/s wr, 32 op/s 2024-05-04T03:23:06.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:06 smithi077 ceph-mon[33531]: pgmap v509: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-05-04T03:23:06.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:06 smithi072 ceph-mon[33332]: pgmap v509: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-05-04T03:23:07.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:06 smithi001 ceph-mon[29731]: pgmap v509: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-05-04T03:23:08.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:08 smithi077 ceph-mon[33531]: pgmap v510: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-05-04T03:23:08.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:08 smithi072 ceph-mon[33332]: pgmap v510: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-05-04T03:23:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.124+0000 7f54cf8f0640 1 -- 172.21.15.1:0/3873865752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c8102b30 msgr2=0x7f54c8103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.124+0000 7f54cf8f0640 1 --2- 172.21.15.1:0/3873865752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c8102b30 0x7f54c8103830 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f54c4009900 tx=0x7f54c402a410 comp rx=0 tx=0).stop 2024-05-04T03:23:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.124+0000 7f54cf8f0640 1 -- 172.21.15.1:0/3873865752 shutdown_connections 2024-05-04T03:23:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.124+0000 7f54cf8f0640 1 --2- 172.21.15.1:0/3873865752 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54c8108a70 0x7f54c810ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.124+0000 7f54cf8f0640 1 --2- 172.21.15.1:0/3873865752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54c8103d70 0x7f54c8104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.124+0000 7f54cf8f0640 1 --2- 172.21.15.1:0/3873865752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c8102b30 0x7f54c8103830 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.124+0000 7f54cf8f0640 1 -- 172.21.15.1:0/3873865752 >> 172.21.15.1:0/3873865752 conn(0x7f54c80fdd60 msgr2=0x7f54c81001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.124+0000 7f54cf8f0640 1 -- 172.21.15.1:0/3873865752 shutdown_connections 2024-05-04T03:23:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.124+0000 7f54cf8f0640 1 -- 172.21.15.1:0/3873865752 wait complete. 2024-05-04T03:23:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.125+0000 7f54cf8f0640 1 Processor -- start 2024-05-04T03:23:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.125+0000 7f54cf8f0640 1 -- start start 2024-05-04T03:23:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.125+0000 7f54cf8f0640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c8102b30 0x7f54c819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.125+0000 7f54cf8f0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54c8103d70 0x7f54c819a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.125+0000 7f54cf8f0640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54c8108a70 0x7f54c81a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.125+0000 7f54cf8f0640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f54c810f760 con 0x7f54c8102b30 2024-05-04T03:23:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.125+0000 7f54cf8f0640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f54c810f5e0 con 0x7f54c8108a70 2024-05-04T03:23:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.125+0000 7f54cf8f0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f54c810f8e0 con 0x7f54c8103d70 2024-05-04T03:23:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.125+0000 7f54cd68c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c8102b30 0x7f54c819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.125+0000 7f54cd68c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c8102b30 0x7f54c819a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44712/0 (socket says 172.21.15.1:44712) 2024-05-04T03:23:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.125+0000 7f54cd68c640 1 -- 172.21.15.1:0/1364558587 learned_addr learned my addr 172.21.15.1:0/1364558587 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.125+0000 7f54cce8b640 1 --2- 172.21.15.1:0/1364558587 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54c8103d70 0x7f54c819a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.125+0000 7f54cde8d640 1 --2- 172.21.15.1:0/1364558587 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54c8108a70 0x7f54c81a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.126+0000 7f54cd68c640 1 -- 172.21.15.1:0/1364558587 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54c8103d70 msgr2=0x7f54c819a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.126+0000 7f54cd68c640 1 --2- 172.21.15.1:0/1364558587 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54c8103d70 0x7f54c819a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.126+0000 7f54cd68c640 1 -- 172.21.15.1:0/1364558587 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54c8108a70 msgr2=0x7f54c81a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.126+0000 7f54cd68c640 1 --2- 172.21.15.1:0/1364558587 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54c8108a70 0x7f54c81a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.126+0000 7f54cd68c640 1 -- 172.21.15.1:0/1364558587 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c81a1e20 con 0x7f54c8102b30 2024-05-04T03:23:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.126+0000 7f54cd68c640 1 --2- 172.21.15.1:0/1364558587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c8102b30 0x7f54c819a2e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f54c4006010 tx=0x7f54c4031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:09.130 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.126+0000 7f54be7fc640 1 -- 172.21.15.1:0/1364558587 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c403f030 con 0x7f54c8102b30 2024-05-04T03:23:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.126+0000 7f54be7fc640 1 -- 172.21.15.1:0/1364558587 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54c4003aa0 con 0x7f54c8102b30 2024-05-04T03:23:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.126+0000 7f54cf8f0640 1 -- 172.21.15.1:0/1364558587 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c81a20b0 con 0x7f54c8102b30 2024-05-04T03:23:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.126+0000 7f54be7fc640 1 -- 172.21.15.1:0/1364558587 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c4035070 con 0x7f54c8102b30 2024-05-04T03:23:09.131 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.126+0000 7f54cf8f0640 1 -- 172.21.15.1:0/1364558587 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c81a25a0 con 0x7f54c8102b30 2024-05-04T03:23:09.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:08 smithi001 ceph-mon[29731]: pgmap v510: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-05-04T03:23:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.129+0000 7f54d679f740 1 Processor -- start 2024-05-04T03:23:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.129+0000 7f54be7fc640 1 -- 172.21.15.1:0/1364558587 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f54c4042030 con 0x7f54c8102b30 2024-05-04T03:23:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.129+0000 7f54d679f740 1 -- start start 2024-05-04T03:23:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.129+0000 7f54d679f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a5bd9891b0 0x55a5bdd2aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.129+0000 7f54d679f740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5bd9602f0 0x55a5bdd2afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.129+0000 7f54d679f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5bd9684d0 0x55a5bdd31ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.129+0000 7f54d679f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a5bdc8f5d0 con 0x55a5bd9891b0 2024-05-04T03:23:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.129+0000 7f54d679f740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55a5bdc48890 con 0x55a5bd9602f0 2024-05-04T03:23:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.129+0000 7f54d679f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a5bdc8b590 con 0x55a5bd9684d0 2024-05-04T03:23:09.137 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.129+0000 7f54cde8d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a5bd9891b0 0x55a5bdd2aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.129+0000 7f54cde8d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a5bd9891b0 0x55a5bdd2aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44718/0 (socket says 172.21.15.1:44718) 2024-05-04T03:23:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.130+0000 7f54cde8d640 1 -- 172.21.15.1:0/515894458 learned_addr learned my addr 172.21.15.1:0/515894458 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.130+0000 7f54cd68c640 1 --2- 172.21.15.1:0/515894458 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5bd9602f0 0x55a5bdd2afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.130+0000 7f54cce8b640 1 --2- 172.21.15.1:0/515894458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5bd9684d0 0x55a5bdd31ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.130+0000 7f54be7fc640 1 --2- 172.21.15.1:0/1364558587 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f54b40b28a0 0x7f54b40b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.130+0000 7f54cde8d640 1 -- 172.21.15.1:0/515894458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5bd9684d0 msgr2=0x55a5bdd31ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.130+0000 7f54cde8d640 1 --2- 172.21.15.1:0/515894458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5bd9684d0 0x55a5bdd31ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.130+0000 7f54cde8d640 1 -- 172.21.15.1:0/515894458 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5bd9602f0 msgr2=0x55a5bdd2afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.130+0000 7f54cde8d640 1 --2- 172.21.15.1:0/515894458 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5bd9602f0 0x55a5bdd2afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.130+0000 7f54cde8d640 1 -- 172.21.15.1:0/515894458 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5bd9a1210 con 0x55a5bd9891b0 2024-05-04T03:23:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.130+0000 7f54cd68c640 1 --2- 172.21.15.1:0/515894458 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5bd9602f0 0x55a5bdd2afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:23:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.130+0000 7f54be7fc640 1 -- 172.21.15.1:0/1364558587 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f54c4102660 con 0x7f54c8102b30 2024-05-04T03:23:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.130+0000 7f54cde8d640 1 --2- 172.21.15.1:0/1364558587 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f54b40b28a0 0x7f54b40b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.130+0000 7f54cde8d640 1 --2- 172.21.15.1:0/515894458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a5bd9891b0 0x55a5bdd2aa60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f54c0010510 tx=0x7f54c0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27df7ea8e46bc40d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.130+0000 7f54b37fe640 1 -- 172.21.15.1:0/515894458 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c0022070 con 0x55a5bd9891b0 2024-05-04T03:23:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.130+0000 7f54b37fe640 1 -- 172.21.15.1:0/515894458 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54c000e370 con 0x55a5bd9891b0 2024-05-04T03:23:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.130+0000 7f54b37fe640 1 -- 172.21.15.1:0/515894458 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c0025600 con 0x55a5bd9891b0 2024-05-04T03:23:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.131+0000 7f54d679f740 1 -- 172.21.15.1:0/515894458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a5bd9891b0 msgr2=0x55a5bdd2aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.131+0000 7f54cde8d640 1 --2- 172.21.15.1:0/1364558587 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f54b40b28a0 0x7f54b40b4db0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f54c001f000 tx=0x7f54c001b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.131+0000 7f54d679f740 1 --2- 172.21.15.1:0/515894458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a5bd9891b0 0x55a5bdd2aa60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f54c0010510 tx=0x7f54c0009f90 comp rx=0 tx=0).stop 2024-05-04T03:23:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.131+0000 7f54d679f740 1 -- 172.21.15.1:0/515894458 shutdown_connections 2024-05-04T03:23:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.131+0000 7f54d679f740 1 --2- 172.21.15.1:0/515894458 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5bd9602f0 0x55a5bdd2afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.131+0000 7f54d679f740 1 --2- 172.21.15.1:0/515894458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5bd9684d0 0x55a5bdd31ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.131+0000 7f54d679f740 1 --2- 172.21.15.1:0/515894458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a5bd9891b0 0x55a5bdd2aa60 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.131+0000 7f54d679f740 1 -- 172.21.15.1:0/515894458 >> 172.21.15.1:0/515894458 conn(0x55a5bd97d300 msgr2=0x55a5bdd263a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.131+0000 7f54d679f740 1 -- 172.21.15.1:0/515894458 shutdown_connections 2024-05-04T03:23:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.131+0000 7f54d679f740 1 -- 172.21.15.1:0/515894458 wait complete. 2024-05-04T03:23:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.132+0000 7f54d679f740 1 -- start start 2024-05-04T03:23:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.132+0000 7f54d679f740 1 Processor -- start 2024-05-04T03:23:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.132+0000 7f54d679f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5bd9602f0 0x55a5bdd34670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.132+0000 7f54d679f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a5bd9684d0 0x55a5bdd34bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.132+0000 7f54d679f740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5bd9891b0 0x55a5bdd3bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.132+0000 7f54d679f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a5bdd32930 con 0x55a5bd9684d0 2024-05-04T03:23:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.132+0000 7f54d679f740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55a5bdc4c370 con 0x55a5bd9891b0 2024-05-04T03:23:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.132+0000 7f54d679f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a5bdd32ab0 con 0x55a5bd9602f0 2024-05-04T03:23:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.132+0000 7f54d679f740 10 client.0 _ll_register_callbacks cb 0x55a5bd9a0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:23:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.133+0000 7f54cce8b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a5bd9684d0 0x55a5bdd34bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.133+0000 7f54cce8b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a5bd9684d0 0x55a5bdd34bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44724/0 (socket says 172.21.15.1:44724) 2024-05-04T03:23:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.133+0000 7f54cce8b640 1 -- 172.21.15.1:0/696791343 learned_addr learned my addr 172.21.15.1:0/696791343 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.133+0000 7f54cd68c640 1 --2- 172.21.15.1:0/696791343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5bd9602f0 0x55a5bdd34670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.133+0000 7f54cde8d640 1 --2- 172.21.15.1:0/696791343 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5bd9891b0 0x55a5bdd3bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.133+0000 7f54cce8b640 1 -- 172.21.15.1:0/696791343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5bd9602f0 msgr2=0x55a5bdd34670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.133+0000 7f54cce8b640 1 --2- 172.21.15.1:0/696791343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5bd9602f0 0x55a5bdd34670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.133+0000 7f54cce8b640 1 -- 172.21.15.1:0/696791343 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5bd9891b0 msgr2=0x55a5bdd3bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.133+0000 7f54cce8b640 1 --2- 172.21.15.1:0/696791343 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5bd9891b0 0x55a5bdd3bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.133+0000 7f54cce8b640 1 -- 172.21.15.1:0/696791343 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5bda6d620 con 0x55a5bd9684d0 2024-05-04T03:23:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.133+0000 7f54cce8b640 1 --2- 172.21.15.1:0/696791343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a5bd9684d0 0x55a5bdd34bb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f54b800b920 tx=0x7f54b800bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcff7db05f2e7593 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.133+0000 7f54b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:23:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.133+0000 7f54b0ff9640 1 -- 172.21.15.1:0/696791343 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b801b440 con 0x55a5bd9684d0 2024-05-04T03:23:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.133+0000 7f54b0ff9640 1 -- 172.21.15.1:0/696791343 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f54b801b5e0 con 0x55a5bd9684d0 2024-05-04T03:23:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.133+0000 7f54d679f740 1 -- 172.21.15.1:0/696791343 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54c000e370 con 0x55a5bd9684d0 2024-05-04T03:23:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.134+0000 7f54b0ff9640 1 -- 172.21.15.1:0/696791343 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f54b801b7e0 con 0x55a5bd9684d0 2024-05-04T03:23:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.134+0000 7f54d679f740 10 client.15678 fetch_fsmap learned FSMap version 29 2024-05-04T03:23:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.134+0000 7f54d679f740 1 -- 172.21.15.1:0/696791343 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a5bdd3eb10 con 0x55a5bd9684d0 2024-05-04T03:23:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.134+0000 7f54b0ff9640 1 -- 172.21.15.1:0/696791343 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54c000e370 con 0x55a5bd9684d0 2024-05-04T03:23:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.134+0000 7f54b0ff9640 1 -- 172.21.15.1:0/696791343 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f54b8025070 con 0x55a5bd9684d0 2024-05-04T03:23:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.134+0000 7f54d679f740 10 client.15678 fetch_fsmap finished waiting for FSMap version 29 2024-05-04T03:23:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.134+0000 7f54d679f740 10 client.15678 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:23:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.134+0000 7f54d679f740 20 client.15678 populate_metadata read hostname 'smithi001' 2024-05-04T03:23:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.134+0000 7f54d679f740 1 --2- 172.21.15.1:0/696791343 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a5bd947d80 0x55a5bdd422d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.134+0000 7f54d679f740 4 client.15678 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:23:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.134+0000 7f54d679f740 1 -- 172.21.15.1:0/696791343 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5bdc9d1e0 con 0x55a5bd947d80 2024-05-04T03:23:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.134+0000 7f54cd68c640 1 --2- 172.21.15.1:0/696791343 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a5bd947d80 0x55a5bdd422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.135+0000 7f54b0ff9640 10 client.15678 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:23:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.135+0000 7f54cd68c640 1 --2- 172.21.15.1:0/696791343 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a5bd947d80 0x55a5bdd422d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=795df8ecab8d003f server_cookie=e5cf5724078a5a45 in_seq=0 out_seq=0 2024-05-04T03:23:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.136+0000 7f54b0ff9640 1 -- 172.21.15.1:0/696791343 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a5bdc9d1e0 con 0x55a5bd947d80 2024-05-04T03:23:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.136+0000 7f54b0ff9640 10 client.15678 handle_command_reply: tid=0 2024-05-04T03:23:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.136+0000 7f54d679f740 1 client.15678 shutdown 2024-05-04T03:23:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.136+0000 7f54d679f740 1 -- 172.21.15.1:0/696791343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a5bd9684d0 msgr2=0x55a5bdd34bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.136+0000 7f54d679f740 1 --2- 172.21.15.1:0/696791343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a5bd9684d0 0x55a5bdd34bb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f54b800b920 tx=0x7f54b800bde0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.136+0000 7f54d679f740 1 -- 172.21.15.1:0/696791343 shutdown_connections 2024-05-04T03:23:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.136+0000 7f54d679f740 1 --2- 172.21.15.1:0/696791343 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a5bd947d80 0x55a5bdd422d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.136+0000 7f54d679f740 1 --2- 172.21.15.1:0/696791343 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5bd9891b0 0x55a5bdd3bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.136+0000 7f54d679f740 1 --2- 172.21.15.1:0/696791343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a5bd9684d0 0x55a5bdd34bb0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.136+0000 7f54d679f740 1 --2- 172.21.15.1:0/696791343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5bd9602f0 0x55a5bdd34670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.136+0000 7f54d679f740 1 -- 172.21.15.1:0/696791343 >> 172.21.15.1:0/696791343 conn(0x55a5bd97d300 msgr2=0x55a5bdd253a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.136+0000 7f54b0ff9640 0 client.15678 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:23:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.136+0000 7f54d679f740 1 -- 172.21.15.1:0/696791343 shutdown_connections 2024-05-04T03:23:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.136+0000 7f54d679f740 1 -- 172.21.15.1:0/696791343 wait complete. 2024-05-04T03:23:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.136+0000 7f54d679f740 20 client.15678 trim_cache size 0 max 16384 2024-05-04T03:23:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.152+0000 7f54d679f740 1 Processor -- start 2024-05-04T03:23:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.152+0000 7f54d679f740 1 -- start start 2024-05-04T03:23:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.152+0000 7f54d679f740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5bd9602f0 0x55a5bdd33590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.152+0000 7f54d679f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a5bd9684d0 0x55a5bdd3ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.152+0000 7f54d679f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5bd9891b0 0x55a5bdd3bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.152+0000 7f54d679f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f54c81ad9f0 con 0x55a5bd9684d0 2024-05-04T03:23:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.152+0000 7f54d679f740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f54c81ab260 con 0x55a5bd9602f0 2024-05-04T03:23:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.152+0000 7f54d679f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f54c81a90d0 con 0x55a5bd9891b0 2024-05-04T03:23:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.152+0000 7f54cce8b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a5bd9684d0 0x55a5bdd3ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.152+0000 7f54cce8b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a5bd9684d0 0x55a5bdd3ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44732/0 (socket says 172.21.15.1:44732) 2024-05-04T03:23:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.152+0000 7f54cce8b640 1 -- 172.21.15.1:0/3578184096 learned_addr learned my addr 172.21.15.1:0/3578184096 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.152+0000 7f54cd68c640 1 --2- 172.21.15.1:0/3578184096 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5bd9602f0 0x55a5bdd33590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.152+0000 7f54cde8d640 1 --2- 172.21.15.1:0/3578184096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5bd9891b0 0x55a5bdd3bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.152+0000 7f54cce8b640 1 -- 172.21.15.1:0/3578184096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5bd9891b0 msgr2=0x55a5bdd3bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.152+0000 7f54cce8b640 1 --2- 172.21.15.1:0/3578184096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5bd9891b0 0x55a5bdd3bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.152+0000 7f54cce8b640 1 -- 172.21.15.1:0/3578184096 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5bd9602f0 msgr2=0x55a5bdd33590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.152+0000 7f54cce8b640 1 --2- 172.21.15.1:0/3578184096 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5bd9602f0 0x55a5bdd33590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.152+0000 7f54cce8b640 1 -- 172.21.15.1:0/3578184096 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5bdd26d00 con 0x55a5bd9684d0 2024-05-04T03:23:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.152+0000 7f54cde8d640 1 --2- 172.21.15.1:0/3578184096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5bd9891b0 0x55a5bdd3bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:23:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.152+0000 7f54cd68c640 1 --2- 172.21.15.1:0/3578184096 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5bd9602f0 0x55a5bdd33590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:23:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.152+0000 7f54cce8b640 1 --2- 172.21.15.1:0/3578184096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a5bd9684d0 0x55a5bdd3ba30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f54b8005c20 tx=0x7f54b8004520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1242ec8786620a0d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.153+0000 7f54b2ffd640 1 -- 172.21.15.1:0/3578184096 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b801b640 con 0x55a5bd9684d0 2024-05-04T03:23:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.153+0000 7f54b2ffd640 1 -- 172.21.15.1:0/3578184096 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54b8022070 con 0x55a5bd9684d0 2024-05-04T03:23:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.153+0000 7f54b2ffd640 1 -- 172.21.15.1:0/3578184096 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b8005040 con 0x55a5bd9684d0 2024-05-04T03:23:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.153+0000 7f54d679f740 1 -- 172.21.15.1:0/3578184096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a5bd9684d0 msgr2=0x55a5bdd3ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.153+0000 7f54d679f740 1 --2- 172.21.15.1:0/3578184096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a5bd9684d0 0x55a5bdd3ba30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f54b8005c20 tx=0x7f54b8004520 comp rx=0 tx=0).stop 2024-05-04T03:23:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.154+0000 7f54d679f740 1 -- 172.21.15.1:0/3578184096 shutdown_connections 2024-05-04T03:23:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.154+0000 7f54d679f740 1 --2- 172.21.15.1:0/3578184096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5bd9891b0 0x55a5bdd3bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.154+0000 7f54d679f740 1 --2- 172.21.15.1:0/3578184096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a5bd9684d0 0x55a5bdd3ba30 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.154+0000 7f54d679f740 1 --2- 172.21.15.1:0/3578184096 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5bd9602f0 0x55a5bdd33590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.154+0000 7f54d679f740 1 -- 172.21.15.1:0/3578184096 >> 172.21.15.1:0/3578184096 conn(0x55a5bd97d300 msgr2=0x55a5bdd32e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.154+0000 7f54d679f740 1 -- 172.21.15.1:0/3578184096 shutdown_connections 2024-05-04T03:23:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.154+0000 7f54d679f740 1 -- 172.21.15.1:0/3578184096 wait complete. 2024-05-04T03:23:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.154+0000 7f54d679f740 1 -- start start 2024-05-04T03:23:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.154+0000 7f54d679f740 1 Processor -- start 2024-05-04T03:23:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.154+0000 7f54d679f740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a5bd9602f0 0x55a5bdd37230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.154+0000 7f54d679f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5bd9684d0 0x55a5bdd37770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.154+0000 7f54d679f740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5bd9891b0 0x55a5bdda8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.154+0000 7f54d679f740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f54c81a9720 con 0x55a5bd9602f0 2024-05-04T03:23:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.154+0000 7f54d679f740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f54c81aab90 con 0x55a5bd9891b0 2024-05-04T03:23:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.154+0000 7f54d679f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f54c81a9df0 con 0x55a5bd9684d0 2024-05-04T03:23:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.154+0000 7f54d679f740 10 client.0 _ll_register_callbacks cb 0x7f54b8004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:23:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.155+0000 7f54cd68c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a5bd9602f0 0x55a5bdd37230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.155+0000 7f54cd68c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a5bd9602f0 0x55a5bdd37230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44742/0 (socket says 172.21.15.1:44742) 2024-05-04T03:23:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.155+0000 7f54cd68c640 1 -- 172.21.15.1:0/1522141606 learned_addr learned my addr 172.21.15.1:0/1522141606 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.155+0000 7f54cde8d640 1 --2- 172.21.15.1:0/1522141606 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5bd9891b0 0x55a5bdda8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.155+0000 7f54cce8b640 1 --2- 172.21.15.1:0/1522141606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5bd9684d0 0x55a5bdd37770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.155+0000 7f54cd68c640 1 -- 172.21.15.1:0/1522141606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5bd9684d0 msgr2=0x55a5bdd37770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.155+0000 7f54cd68c640 1 --2- 172.21.15.1:0/1522141606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5bd9684d0 0x55a5bdd37770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.155+0000 7f54cd68c640 1 -- 172.21.15.1:0/1522141606 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5bd9891b0 msgr2=0x55a5bdda8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.155+0000 7f54cd68c640 1 --2- 172.21.15.1:0/1522141606 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5bd9891b0 0x55a5bdda8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.155+0000 7f54cd68c640 1 -- 172.21.15.1:0/1522141606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5bdda8b70 con 0x55a5bd9602f0 2024-05-04T03:23:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.155+0000 7f54cd68c640 1 --2- 172.21.15.1:0/1522141606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a5bd9602f0 0x55a5bdd37230 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f54c40c5e70 tx=0x7f54c4003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79967c66272a2e18 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.155+0000 7f54b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:23:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.155+0000 7f54b0ff9640 1 -- 172.21.15.1:0/1522141606 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c4049040 con 0x55a5bd9602f0 2024-05-04T03:23:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.155+0000 7f54b0ff9640 1 -- 172.21.15.1:0/1522141606 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f54c40e4330 con 0x55a5bd9602f0 2024-05-04T03:23:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.155+0000 7f54d679f740 1 -- 172.21.15.1:0/1522141606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54b8022070 con 0x55a5bd9602f0 2024-05-04T03:23:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.155+0000 7f54b0ff9640 1 -- 172.21.15.1:0/1522141606 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f54c40d2ba0 con 0x55a5bd9602f0 2024-05-04T03:23:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.155+0000 7f54d679f740 10 client.15684 fetch_fsmap learned FSMap version 29 2024-05-04T03:23:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.156+0000 7f54d679f740 1 -- 172.21.15.1:0/1522141606 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a5bdda9c20 con 0x55a5bd9602f0 2024-05-04T03:23:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.156+0000 7f54b0ff9640 1 -- 172.21.15.1:0/1522141606 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54b8022070 con 0x55a5bd9602f0 2024-05-04T03:23:09.162 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.156+0000 7f54b0ff9640 1 -- 172.21.15.1:0/1522141606 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f54c403bc80 con 0x55a5bd9602f0 2024-05-04T03:23:09.163 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.156+0000 7f54d679f740 10 client.15684 fetch_fsmap finished waiting for FSMap version 29 2024-05-04T03:23:09.163 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.156+0000 7f54d679f740 10 client.15684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:23:09.163 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.156+0000 7f54d679f740 20 client.15684 populate_metadata read hostname 'smithi001' 2024-05-04T03:23:09.163 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.156+0000 7f54d679f740 1 --2- 172.21.15.1:0/1522141606 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a5bd91d1c0 0x55a5bddac640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:09.163 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.156+0000 7f54d679f740 4 client.15684 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:23:09.163 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.156+0000 7f54d679f740 1 -- 172.21.15.1:0/1522141606 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a5bdc97050 con 0x55a5bd91d1c0 2024-05-04T03:23:09.163 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.156+0000 7f54cce8b640 1 --2- 172.21.15.1:0/1522141606 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a5bd91d1c0 0x55a5bddac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:09.163 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.157+0000 7f54b0ff9640 10 client.15684 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:23:09.163 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.157+0000 7f54cce8b640 1 --2- 172.21.15.1:0/1522141606 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a5bd91d1c0 0x55a5bddac640 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54aa45029a8997aa server_cookie=a7a08a4b9b2f5575 in_seq=0 out_seq=0 2024-05-04T03:23:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.165+0000 7f54b0ff9640 1 -- 172.21.15.1:0/1522141606 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a5bdc97050 con 0x55a5bd91d1c0 2024-05-04T03:23:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.165+0000 7f54b0ff9640 10 client.15684 handle_command_reply: tid=0 2024-05-04T03:23:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.165+0000 7f54d679f740 1 client.15684 shutdown 2024-05-04T03:23:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.166+0000 7f54d679f740 1 -- 172.21.15.1:0/1522141606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a5bd9602f0 msgr2=0x55a5bdd37230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:09.168 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.166+0000 7f54d679f740 1 --2- 172.21.15.1:0/1522141606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a5bd9602f0 0x55a5bdd37230 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f54c40c5e70 tx=0x7f54c4003f90 comp rx=0 tx=0).stop 2024-05-04T03:23:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.166+0000 7f54d679f740 1 -- 172.21.15.1:0/1522141606 shutdown_connections 2024-05-04T03:23:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.166+0000 7f54d679f740 1 --2- 172.21.15.1:0/1522141606 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a5bd91d1c0 0x55a5bddac640 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.166+0000 7f54d679f740 1 --2- 172.21.15.1:0/1522141606 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5bd9891b0 0x55a5bdda8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.166+0000 7f54d679f740 1 --2- 172.21.15.1:0/1522141606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a5bd9684d0 0x55a5bdd37770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.166+0000 7f54d679f740 1 --2- 172.21.15.1:0/1522141606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a5bd9602f0 0x55a5bdd37230 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.166+0000 7f54b0ff9640 0 client.15684 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:23:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.166+0000 7f54d679f740 1 -- 172.21.15.1:0/1522141606 >> 172.21.15.1:0/1522141606 conn(0x55a5bd97d300 msgr2=0x55a5bdd25cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.166+0000 7f54d679f740 1 -- 172.21.15.1:0/1522141606 shutdown_connections 2024-05-04T03:23:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.166+0000 7f54d679f740 1 -- 172.21.15.1:0/1522141606 wait complete. 2024-05-04T03:23:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.166+0000 7f54d679f740 20 client.15684 trim_cache size 0 max 16384 2024-05-04T03:23:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:23:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:23:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:23:09.170 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:23:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.166+0000 7f54cf8f0640 1 -- 172.21.15.1:0/1364558587 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f54b40b28a0 msgr2=0x7f54b40b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.166+0000 7f54cf8f0640 1 --2- 172.21.15.1:0/1364558587 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f54b40b28a0 0x7f54b40b4db0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f54c001f000 tx=0x7f54c001b000 comp rx=0 tx=0).stop 2024-05-04T03:23:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.166+0000 7f54cf8f0640 1 -- 172.21.15.1:0/1364558587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c8102b30 msgr2=0x7f54c819a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.166+0000 7f54cf8f0640 1 --2- 172.21.15.1:0/1364558587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c8102b30 0x7f54c819a2e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f54c4006010 tx=0x7f54c4031a50 comp rx=0 tx=0).stop 2024-05-04T03:23:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.167+0000 7f54cf8f0640 1 -- 172.21.15.1:0/1364558587 shutdown_connections 2024-05-04T03:23:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.167+0000 7f54cf8f0640 1 --2- 172.21.15.1:0/1364558587 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f54b40b28a0 0x7f54b40b4db0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.167+0000 7f54cf8f0640 1 --2- 172.21.15.1:0/1364558587 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54c8108a70 0x7f54c81a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.167+0000 7f54cf8f0640 1 --2- 172.21.15.1:0/1364558587 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54c8103d70 0x7f54c819a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.167+0000 7f54cf8f0640 1 --2- 172.21.15.1:0/1364558587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c8102b30 0x7f54c819a2e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.167+0000 7f54cf8f0640 1 -- 172.21.15.1:0/1364558587 >> 172.21.15.1:0/1364558587 conn(0x7f54c80fdd60 msgr2=0x7f54c8104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.167+0000 7f54cf8f0640 1 -- 172.21.15.1:0/1364558587 shutdown_connections 2024-05-04T03:23:09.171 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:09.167+0000 7f54cf8f0640 1 -- 172.21.15.1:0/1364558587 wait complete. 2024-05-04T03:23:10.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:10 smithi077 ceph-mon[33531]: pgmap v511: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-05-04T03:23:10.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:10 smithi072 ceph-mon[33332]: pgmap v511: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-05-04T03:23:11.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:10 smithi001 ceph-mon[29731]: pgmap v511: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-05-04T03:23:11.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:11 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:23:11.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:11 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:23:12.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:11 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:23:12.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:12 smithi077 ceph-mon[33531]: pgmap v512: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-04T03:23:12.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:12 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:23:12.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:12 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:23:12.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:12 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:23:12.998 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:12 smithi072 ceph-mon[33332]: pgmap v512: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-04T03:23:12.998 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:12 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:23:12.998 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:12 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:23:12.998 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:12 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:23:13.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:12 smithi001 ceph-mon[29731]: pgmap v512: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 5 op/s 2024-05-04T03:23:13.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:12 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:23:13.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:12 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:23:13.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:12 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:23:14.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:14 smithi077 ceph-mon[33531]: pgmap v513: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-05-04T03:23:14.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:14 smithi072 ceph-mon[33332]: pgmap v513: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-05-04T03:23:15.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:14 smithi001 ceph-mon[29731]: pgmap v513: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-05-04T03:23:16.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:16 smithi077 ceph-mon[33531]: pgmap v514: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 47 KiB/s wr, 8 op/s 2024-05-04T03:23:16.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:16 smithi072 ceph-mon[33332]: pgmap v514: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 47 KiB/s wr, 8 op/s 2024-05-04T03:23:17.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:16 smithi001 ceph-mon[29731]: pgmap v514: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 47 KiB/s wr, 8 op/s 2024-05-04T03:23:18.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:18 smithi077 ceph-mon[33531]: pgmap v515: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 47 KiB/s wr, 7 op/s 2024-05-04T03:23:18.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:18 smithi072 ceph-mon[33332]: pgmap v515: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 47 KiB/s wr, 7 op/s 2024-05-04T03:23:19.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:18 smithi001 ceph-mon[29731]: pgmap v515: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 47 KiB/s wr, 7 op/s 2024-05-04T03:23:20.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:20 smithi077 ceph-mon[33531]: pgmap v516: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 47 KiB/s wr, 9 op/s 2024-05-04T03:23:20.931 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:20 smithi001 ceph-mon[29731]: pgmap v516: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 47 KiB/s wr, 9 op/s 2024-05-04T03:23:20.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:20 smithi072 ceph-mon[33332]: pgmap v516: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 47 KiB/s wr, 9 op/s 2024-05-04T03:23:21.749 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:23:22.011 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:23:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.267+0000 7fdac82b6640 1 -- 172.21.15.1:0/1738666435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdac0108a90 msgr2=0x7fdac010aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.267+0000 7fdac82b6640 1 --2- 172.21.15.1:0/1738666435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdac0108a90 0x7fdac010aeb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009ca0 tx=0x7fdab802a8c0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.267+0000 7fdac82b6640 1 -- 172.21.15.1:0/1738666435 shutdown_connections 2024-05-04T03:23:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.267+0000 7fdac82b6640 1 --2- 172.21.15.1:0/1738666435 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdac0108a90 0x7fdac010aeb0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.267+0000 7fdac82b6640 1 --2- 172.21.15.1:0/1738666435 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdac0103d90 0x7fdac01041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.267+0000 7fdac82b6640 1 --2- 172.21.15.1:0/1738666435 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdac0102b50 0x7fdac0103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.267+0000 7fdac82b6640 1 -- 172.21.15.1:0/1738666435 >> 172.21.15.1:0/1738666435 conn(0x7fdac00fdd60 msgr2=0x7fdac01001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.267+0000 7fdac82b6640 1 -- 172.21.15.1:0/1738666435 shutdown_connections 2024-05-04T03:23:22.270 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.267+0000 7fdac82b6640 1 -- 172.21.15.1:0/1738666435 wait complete. 2024-05-04T03:23:22.272 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.268+0000 7fdac82b6640 1 Processor -- start 2024-05-04T03:23:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.268+0000 7fdac82b6640 1 -- start start 2024-05-04T03:23:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.269+0000 7fdac82b6640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdac0102b50 0x7fdac019a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.269+0000 7fdac82b6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdac0103d90 0x7fdac019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.269+0000 7fdac82b6640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdac0108a90 0x7fdac01a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.269+0000 7fdac82b6640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdac010e6d0 con 0x7fdac0108a90 2024-05-04T03:23:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.269+0000 7fdac82b6640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdac010e550 con 0x7fdac0102b50 2024-05-04T03:23:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.269+0000 7fdac82b6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdac010e850 con 0x7fdac0103d90 2024-05-04T03:23:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.269+0000 7fdac6853640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdac0108a90 0x7fdac01a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.269+0000 7fdac6853640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdac0108a90 0x7fdac01a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55792/0 (socket says 172.21.15.1:55792) 2024-05-04T03:23:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.269+0000 7fdac6853640 1 -- 172.21.15.1:0/2640095544 learned_addr learned my addr 172.21.15.1:0/2640095544 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.269+0000 7fdac6052640 1 --2- 172.21.15.1:0/2640095544 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdac0102b50 0x7fdac019a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.269+0000 7fdac5851640 1 --2- 172.21.15.1:0/2640095544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdac0103d90 0x7fdac019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:22.273 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.269+0000 7fdac6853640 1 -- 172.21.15.1:0/2640095544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdac0103d90 msgr2=0x7fdac019a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.269+0000 7fdac6853640 1 --2- 172.21.15.1:0/2640095544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdac0103d90 0x7fdac019a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.269+0000 7fdac6853640 1 -- 172.21.15.1:0/2640095544 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdac0102b50 msgr2=0x7fdac019a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.269+0000 7fdac6853640 1 --2- 172.21.15.1:0/2640095544 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdac0102b50 0x7fdac019a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.269+0000 7fdac6853640 1 -- 172.21.15.1:0/2640095544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac01a1d70 con 0x7fdac0108a90 2024-05-04T03:23:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.269+0000 7fdac6853640 1 --2- 172.21.15.1:0/2640095544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdac0108a90 0x7fdac01a1700 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009430 tx=0x7fdab80317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.269+0000 7fdab77fe640 1 -- 172.21.15.1:0/2640095544 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab8003bf0 con 0x7fdac0108a90 2024-05-04T03:23:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.269+0000 7fdab77fe640 1 -- 172.21.15.1:0/2640095544 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdab8003d90 con 0x7fdac0108a90 2024-05-04T03:23:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.269+0000 7fdab77fe640 1 -- 172.21.15.1:0/2640095544 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab802fc80 con 0x7fdac0108a90 2024-05-04T03:23:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.270+0000 7fdac82b6640 1 -- 172.21.15.1:0/2640095544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac01a2000 con 0x7fdac0108a90 2024-05-04T03:23:22.274 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.270+0000 7fdac82b6640 1 -- 172.21.15.1:0/2640095544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac01a25a0 con 0x7fdac0108a90 2024-05-04T03:23:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.272+0000 7fdacf16b740 1 Processor -- start 2024-05-04T03:23:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.272+0000 7fdab77fe640 1 -- 172.21.15.1:0/2640095544 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7fdab8004710 con 0x7fdac0108a90 2024-05-04T03:23:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.273+0000 7fdacf16b740 1 -- start start 2024-05-04T03:23:22.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.273+0000 7fdacf16b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af823bc1b0 0x55af8275da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.273+0000 7fdacf16b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af823932f0 0x55af8275dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.273+0000 7fdacf16b740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55af8239b4d0 0x55af82764ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.273+0000 7fdacf16b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55af826c25d0 con 0x55af823932f0 2024-05-04T03:23:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.273+0000 7fdacf16b740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55af8267b890 con 0x55af8239b4d0 2024-05-04T03:23:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.273+0000 7fdacf16b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55af826be590 con 0x55af823bc1b0 2024-05-04T03:23:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.273+0000 7fdac6052640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af823932f0 0x55af8275dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.273+0000 7fdac6052640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af823932f0 0x55af8275dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55802/0 (socket says 172.21.15.1:55802) 2024-05-04T03:23:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.273+0000 7fdac6052640 1 -- 172.21.15.1:0/2689816135 learned_addr learned my addr 172.21.15.1:0/2689816135 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.273+0000 7fdac5851640 1 --2- 172.21.15.1:0/2689816135 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55af8239b4d0 0x55af82764ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.273+0000 7fdac5851640 1 --2- 172.21.15.1:0/2689816135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af823bc1b0 0x55af8275da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.273+0000 7fdab77fe640 1 --2- 172.21.15.1:0/2640095544 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fdaac0b2920 0x7fdaac0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.274+0000 7fdac6052640 1 -- 172.21.15.1:0/2689816135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af823bc1b0 msgr2=0x55af8275da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.274+0000 7fdac6052640 1 --2- 172.21.15.1:0/2689816135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af823bc1b0 0x55af8275da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.274+0000 7fdac6052640 1 -- 172.21.15.1:0/2689816135 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55af8239b4d0 msgr2=0x55af82764ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:22.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.274+0000 7fdac6052640 1 --2- 172.21.15.1:0/2689816135 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55af8239b4d0 0x55af82764ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.274+0000 7fdac6052640 1 -- 172.21.15.1:0/2689816135 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af823d4210 con 0x55af823932f0 2024-05-04T03:23:22.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.274+0000 7fdac6853640 1 --2- 172.21.15.1:0/2640095544 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fdaac0b2920 0x7fdaac0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:22.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.274+0000 7fdab77fe640 1 -- 172.21.15.1:0/2640095544 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fdab81037c0 con 0x7fdac0108a90 2024-05-04T03:23:22.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.274+0000 7fdac6052640 1 --2- 172.21.15.1:0/2689816135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af823932f0 0x55af8275dfe0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fdabc009c60 tx=0x7fdabc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea52a1b804a7a9ae server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:22.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.274+0000 7fdab4bfe640 1 -- 172.21.15.1:0/2689816135 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc009e10 con 0x55af823932f0 2024-05-04T03:23:22.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.274+0000 7fdab4bfe640 1 -- 172.21.15.1:0/2689816135 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdabc005020 con 0x55af823932f0 2024-05-04T03:23:22.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.274+0000 7fdab4bfe640 1 -- 172.21.15.1:0/2689816135 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc004a00 con 0x55af823932f0 2024-05-04T03:23:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.275+0000 7fdacf16b740 1 -- 172.21.15.1:0/2689816135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af823932f0 msgr2=0x55af8275dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.275+0000 7fdac6853640 1 --2- 172.21.15.1:0/2640095544 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fdaac0b2920 0x7fdaac0b4e30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdab80cae30 tx=0x7fdab8098ac0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:22.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.275+0000 7fdacf16b740 1 --2- 172.21.15.1:0/2689816135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af823932f0 0x55af8275dfe0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fdabc009c60 tx=0x7fdabc011810 comp rx=0 tx=0).stop 2024-05-04T03:23:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.275+0000 7fdacf16b740 1 -- 172.21.15.1:0/2689816135 shutdown_connections 2024-05-04T03:23:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.275+0000 7fdacf16b740 1 --2- 172.21.15.1:0/2689816135 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55af8239b4d0 0x55af82764ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.275+0000 7fdacf16b740 1 --2- 172.21.15.1:0/2689816135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af823932f0 0x55af8275dfe0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.275+0000 7fdacf16b740 1 --2- 172.21.15.1:0/2689816135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af823bc1b0 0x55af8275da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.275+0000 7fdacf16b740 1 -- 172.21.15.1:0/2689816135 >> 172.21.15.1:0/2689816135 conn(0x55af823b0300 msgr2=0x55af827593a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.275+0000 7fdacf16b740 1 -- 172.21.15.1:0/2689816135 shutdown_connections 2024-05-04T03:23:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.275+0000 7fdacf16b740 1 -- 172.21.15.1:0/2689816135 wait complete. 2024-05-04T03:23:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.275+0000 7fdacf16b740 1 -- start start 2024-05-04T03:23:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.276+0000 7fdacf16b740 1 Processor -- start 2024-05-04T03:23:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.276+0000 7fdacf16b740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55af823932f0 0x55af82767640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.276+0000 7fdacf16b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af8239b4d0 0x55af82767b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:22.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.276+0000 7fdacf16b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af823bc1b0 0x55af8276eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.276+0000 7fdacf16b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55af82765930 con 0x55af823bc1b0 2024-05-04T03:23:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.276+0000 7fdacf16b740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55af8267f370 con 0x55af823932f0 2024-05-04T03:23:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.276+0000 7fdacf16b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55af82765ab0 con 0x55af8239b4d0 2024-05-04T03:23:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.276+0000 7fdacf16b740 10 client.0 _ll_register_callbacks cb 0x55af823d3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:23:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.276+0000 7fdac5851640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af823bc1b0 0x55af8276eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.276+0000 7fdac5851640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af823bc1b0 0x55af8276eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55804/0 (socket says 172.21.15.1:55804) 2024-05-04T03:23:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.276+0000 7fdac5851640 1 -- 172.21.15.1:0/1762790617 learned_addr learned my addr 172.21.15.1:0/1762790617 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.276+0000 7fdac5851640 1 --2- 172.21.15.1:0/1762790617 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55af823932f0 0x55af82767640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.276+0000 7fdac6052640 1 --2- 172.21.15.1:0/1762790617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af8239b4d0 0x55af82767b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.276+0000 7fdac5851640 1 -- 172.21.15.1:0/1762790617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af8239b4d0 msgr2=0x55af82767b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.276+0000 7fdac5851640 1 --2- 172.21.15.1:0/1762790617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af8239b4d0 0x55af82767b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.276+0000 7fdac5851640 1 -- 172.21.15.1:0/1762790617 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55af823932f0 msgr2=0x55af82767640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.276+0000 7fdac5851640 1 --2- 172.21.15.1:0/1762790617 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55af823932f0 0x55af82767640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.276+0000 7fdac5851640 1 -- 172.21.15.1:0/1762790617 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af824a0620 con 0x55af823bc1b0 2024-05-04T03:23:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.276+0000 7fdac5851640 1 --2- 172.21.15.1:0/1762790617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af823bc1b0 0x55af8276eae0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fdab0014f40 tx=0x7fdab0016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214cfaa78484d060 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.276+0000 7fdaa9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:23:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.277+0000 7fdaa9ffb640 1 -- 172.21.15.1:0/1762790617 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab0014110 con 0x55af823bc1b0 2024-05-04T03:23:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.277+0000 7fdaa9ffb640 1 -- 172.21.15.1:0/1762790617 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fdab0011650 con 0x55af823bc1b0 2024-05-04T03:23:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.277+0000 7fdacf16b740 1 -- 172.21.15.1:0/1762790617 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdabc005020 con 0x55af823bc1b0 2024-05-04T03:23:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.277+0000 7fdaa9ffb640 1 -- 172.21.15.1:0/1762790617 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdab00142b0 con 0x55af823bc1b0 2024-05-04T03:23:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.277+0000 7fdacf16b740 10 client.15696 fetch_fsmap learned FSMap version 30 2024-05-04T03:23:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.277+0000 7fdacf16b740 1 -- 172.21.15.1:0/1762790617 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55af82771ae0 con 0x55af823bc1b0 2024-05-04T03:23:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.277+0000 7fdaa9ffb640 1 -- 172.21.15.1:0/1762790617 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdabc005020 con 0x55af823bc1b0 2024-05-04T03:23:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.277+0000 7fdaa9ffb640 1 -- 172.21.15.1:0/1762790617 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fdab0012610 con 0x55af823bc1b0 2024-05-04T03:23:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.277+0000 7fdacf16b740 10 client.15696 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:23:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.277+0000 7fdacf16b740 10 client.15696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:23:22.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.277+0000 7fdacf16b740 20 client.15696 populate_metadata read hostname 'smithi001' 2024-05-04T03:23:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.277+0000 7fdacf16b740 1 --2- 172.21.15.1:0/1762790617 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55af8237ad80 0x55af82775210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.277+0000 7fdacf16b740 4 client.15696 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:23:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.277+0000 7fdacf16b740 1 -- 172.21.15.1:0/1762790617 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af826d01e0 con 0x55af8237ad80 2024-05-04T03:23:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.278+0000 7fdac6052640 1 --2- 172.21.15.1:0/1762790617 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55af8237ad80 0x55af82775210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.278+0000 7fdaa9ffb640 10 client.15696 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:23:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.278+0000 7fdac6052640 1 --2- 172.21.15.1:0/1762790617 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55af8237ad80 0x55af82775210 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17b10239ed4eacb8 server_cookie=a18018fcbbd32156 in_seq=0 out_seq=0 2024-05-04T03:23:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.279+0000 7fdaa9ffb640 1 -- 172.21.15.1:0/1762790617 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55af826d01e0 con 0x55af8237ad80 2024-05-04T03:23:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.279+0000 7fdaa9ffb640 10 client.15696 handle_command_reply: tid=0 2024-05-04T03:23:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.279+0000 7fdacf16b740 1 client.15696 shutdown 2024-05-04T03:23:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.279+0000 7fdacf16b740 1 -- 172.21.15.1:0/1762790617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af823bc1b0 msgr2=0x55af8276eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.279+0000 7fdacf16b740 1 --2- 172.21.15.1:0/1762790617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af823bc1b0 0x55af8276eae0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fdab0014f40 tx=0x7fdab0016040 comp rx=0 tx=0).stop 2024-05-04T03:23:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.279+0000 7fdacf16b740 1 -- 172.21.15.1:0/1762790617 shutdown_connections 2024-05-04T03:23:22.287 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.279+0000 7fdacf16b740 1 --2- 172.21.15.1:0/1762790617 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55af8237ad80 0x55af82775210 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.279+0000 7fdacf16b740 1 --2- 172.21.15.1:0/1762790617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af823bc1b0 0x55af8276eae0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.279+0000 7fdacf16b740 1 --2- 172.21.15.1:0/1762790617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af8239b4d0 0x55af82767b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.279+0000 7fdacf16b740 1 --2- 172.21.15.1:0/1762790617 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55af823932f0 0x55af82767640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.279+0000 7fdacf16b740 1 -- 172.21.15.1:0/1762790617 >> 172.21.15.1:0/1762790617 conn(0x55af823b0300 msgr2=0x55af8275c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.279+0000 7fdaa9ffb640 0 client.15696 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:23:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.279+0000 7fdacf16b740 1 -- 172.21.15.1:0/1762790617 shutdown_connections 2024-05-04T03:23:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.279+0000 7fdacf16b740 1 -- 172.21.15.1:0/1762790617 wait complete. 2024-05-04T03:23:22.288 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.280+0000 7fdacf16b740 20 client.15696 trim_cache size 0 max 16384 2024-05-04T03:23:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.296+0000 7fdacf16b740 1 Processor -- start 2024-05-04T03:23:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.296+0000 7fdacf16b740 1 -- start start 2024-05-04T03:23:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.297+0000 7fdacf16b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af823932f0 0x55af827665f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.297+0000 7fdacf16b740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55af8239b4d0 0x55af8276ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.297+0000 7fdacf16b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af823bc1b0 0x55af8276efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.297+0000 7fdacf16b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01ad9f0 con 0x55af823bc1b0 2024-05-04T03:23:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.297+0000 7fdacf16b740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01ab290 con 0x55af8239b4d0 2024-05-04T03:23:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.297+0000 7fdacf16b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01a9100 con 0x55af823932f0 2024-05-04T03:23:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.297+0000 7fdac5851640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af823bc1b0 0x55af8276efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.297+0000 7fdac5851640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af823bc1b0 0x55af8276efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55820/0 (socket says 172.21.15.1:55820) 2024-05-04T03:23:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.297+0000 7fdac5851640 1 -- 172.21.15.1:0/3981113537 learned_addr learned my addr 172.21.15.1:0/3981113537 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.297+0000 7fdac5851640 1 --2- 172.21.15.1:0/3981113537 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af823932f0 0x55af827665f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.297+0000 7fdac6052640 1 --2- 172.21.15.1:0/3981113537 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55af8239b4d0 0x55af8276ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.297+0000 7fdac5851640 1 -- 172.21.15.1:0/3981113537 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af823932f0 msgr2=0x55af827665f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.297+0000 7fdac5851640 1 --2- 172.21.15.1:0/3981113537 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af823932f0 0x55af827665f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.297+0000 7fdac5851640 1 -- 172.21.15.1:0/3981113537 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55af8239b4d0 msgr2=0x55af8276ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.297+0000 7fdac5851640 1 --2- 172.21.15.1:0/3981113537 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55af8239b4d0 0x55af8276ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.297+0000 7fdac5851640 1 -- 172.21.15.1:0/3981113537 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af82764de0 con 0x55af823bc1b0 2024-05-04T03:23:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.297+0000 7fdac5851640 1 --2- 172.21.15.1:0/3981113537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af823bc1b0 0x55af8276efb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fdab00059f0 tx=0x7fdab00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b637a86d2c292 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.298+0000 7fdaabfff640 1 -- 172.21.15.1:0/3981113537 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab0012450 con 0x55af823bc1b0 2024-05-04T03:23:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.298+0000 7fdaabfff640 1 -- 172.21.15.1:0/3981113537 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdab00125f0 con 0x55af823bc1b0 2024-05-04T03:23:22.301 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.298+0000 7fdaabfff640 1 -- 172.21.15.1:0/3981113537 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab0021420 con 0x55af823bc1b0 2024-05-04T03:23:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.299+0000 7fdacf16b740 1 -- 172.21.15.1:0/3981113537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af823bc1b0 msgr2=0x55af8276efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.299+0000 7fdacf16b740 1 --2- 172.21.15.1:0/3981113537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af823bc1b0 0x55af8276efb0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fdab00059f0 tx=0x7fdab00057f0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.299+0000 7fdacf16b740 1 -- 172.21.15.1:0/3981113537 shutdown_connections 2024-05-04T03:23:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.299+0000 7fdacf16b740 1 --2- 172.21.15.1:0/3981113537 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55af8239b4d0 0x55af8276ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.299+0000 7fdacf16b740 1 --2- 172.21.15.1:0/3981113537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af823bc1b0 0x55af8276efb0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.299+0000 7fdacf16b740 1 --2- 172.21.15.1:0/3981113537 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af823932f0 0x55af827665f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.299+0000 7fdacf16b740 1 -- 172.21.15.1:0/3981113537 >> 172.21.15.1:0/3981113537 conn(0x55af823b0300 msgr2=0x55af82765ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.299+0000 7fdacf16b740 1 -- 172.21.15.1:0/3981113537 shutdown_connections 2024-05-04T03:23:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.299+0000 7fdacf16b740 1 -- 172.21.15.1:0/3981113537 wait complete. 2024-05-04T03:23:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.299+0000 7fdacf16b740 1 -- start start 2024-05-04T03:23:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.299+0000 7fdacf16b740 1 Processor -- start 2024-05-04T03:23:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.299+0000 7fdacf16b740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af823932f0 0x55af8276c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.299+0000 7fdacf16b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af8239b4d0 0x55af8276c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.299+0000 7fdacf16b740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55af823bc1b0 0x55af8276d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.299+0000 7fdacf16b740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01a9750 con 0x55af823932f0 2024-05-04T03:23:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.299+0000 7fdacf16b740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01aabc0 con 0x55af823bc1b0 2024-05-04T03:23:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.299+0000 7fdacf16b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01a9e20 con 0x55af8239b4d0 2024-05-04T03:23:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.299+0000 7fdacf16b740 10 client.0 _ll_register_callbacks cb 0x7fdab0014b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:23:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.299+0000 7fdac5851640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af823932f0 0x55af8276c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.299+0000 7fdac5851640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af823932f0 0x55af8276c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55828/0 (socket says 172.21.15.1:55828) 2024-05-04T03:23:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.299+0000 7fdac5851640 1 -- 172.21.15.1:0/1091425912 learned_addr learned my addr 172.21.15.1:0/1091425912 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.300+0000 7fdac6052640 1 --2- 172.21.15.1:0/1091425912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af8239b4d0 0x55af8276c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.300+0000 7fdac5851640 1 --2- 172.21.15.1:0/1091425912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55af823bc1b0 0x55af8276d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.300+0000 7fdac5851640 1 -- 172.21.15.1:0/1091425912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af8239b4d0 msgr2=0x55af8276c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.300+0000 7fdac5851640 1 --2- 172.21.15.1:0/1091425912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af8239b4d0 0x55af8276c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.300+0000 7fdac5851640 1 -- 172.21.15.1:0/1091425912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55af823bc1b0 msgr2=0x55af8276d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.300+0000 7fdac5851640 1 --2- 172.21.15.1:0/1091425912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55af823bc1b0 0x55af8276d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.300+0000 7fdac5851640 1 -- 172.21.15.1:0/1091425912 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af8276de00 con 0x55af823932f0 2024-05-04T03:23:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.300+0000 7fdac6052640 1 --2- 172.21.15.1:0/1091425912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af8239b4d0 0x55af8276c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:23:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.300+0000 7fdac5851640 1 --2- 172.21.15.1:0/1091425912 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af823932f0 0x55af8276c490 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fdab0006010 tx=0x7fdab0014c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4b5b121d65eb3f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.300+0000 7fdaa9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:23:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.300+0000 7fdaa9ffb640 1 -- 172.21.15.1:0/1091425912 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab0012450 con 0x55af823932f0 2024-05-04T03:23:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.300+0000 7fdaa9ffb640 1 -- 172.21.15.1:0/1091425912 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fdab0028020 con 0x55af823932f0 2024-05-04T03:23:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.300+0000 7fdacf16b740 1 -- 172.21.15.1:0/1091425912 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdab00125f0 con 0x55af823932f0 2024-05-04T03:23:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.300+0000 7fdaa9ffb640 1 -- 172.21.15.1:0/1091425912 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdab000cb50 con 0x55af823932f0 2024-05-04T03:23:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.300+0000 7fdacf16b740 10 client.15702 fetch_fsmap learned FSMap version 30 2024-05-04T03:23:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.300+0000 7fdacf16b740 1 -- 172.21.15.1:0/1091425912 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55af8276f090 con 0x55af823932f0 2024-05-04T03:23:22.308 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.300+0000 7fdaa9ffb640 1 -- 172.21.15.1:0/1091425912 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdab00125f0 con 0x55af823932f0 2024-05-04T03:23:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.301+0000 7fdaa9ffb640 1 -- 172.21.15.1:0/1091425912 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fdab00048e0 con 0x55af823932f0 2024-05-04T03:23:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.301+0000 7fdacf16b740 10 client.15702 fetch_fsmap finished waiting for FSMap version 30 2024-05-04T03:23:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.301+0000 7fdacf16b740 10 client.15702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:23:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.301+0000 7fdacf16b740 20 client.15702 populate_metadata read hostname 'smithi001' 2024-05-04T03:23:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.301+0000 7fdacf16b740 1 --2- 172.21.15.1:0/1091425912 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55af823501c0 0x55af827df840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.301+0000 7fdacf16b740 4 client.15702 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:23:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.301+0000 7fdacf16b740 1 -- 172.21.15.1:0/1091425912 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55af826ca050 con 0x55af823501c0 2024-05-04T03:23:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.301+0000 7fdac6052640 1 --2- 172.21.15.1:0/1091425912 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55af823501c0 0x55af827df840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.301+0000 7fdaa9ffb640 10 client.15702 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:23:22.309 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.301+0000 7fdac6052640 1 --2- 172.21.15.1:0/1091425912 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55af823501c0 0x55af827df840 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76be8bf7d4b61fbc server_cookie=7fc4d963aa26ce36 in_seq=0 out_seq=0 2024-05-04T03:23:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.306+0000 7fdaa9ffb640 1 -- 172.21.15.1:0/1091425912 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55af826ca050 con 0x55af823501c0 2024-05-04T03:23:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.306+0000 7fdaa9ffb640 10 client.15702 handle_command_reply: tid=0 2024-05-04T03:23:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.306+0000 7fdacf16b740 1 client.15702 shutdown 2024-05-04T03:23:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.306+0000 7fdacf16b740 1 -- 172.21.15.1:0/1091425912 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af823932f0 msgr2=0x55af8276c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.306+0000 7fdacf16b740 1 --2- 172.21.15.1:0/1091425912 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af823932f0 0x55af8276c490 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fdab0006010 tx=0x7fdab0014c20 comp rx=0 tx=0).stop 2024-05-04T03:23:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.306+0000 7fdacf16b740 1 -- 172.21.15.1:0/1091425912 shutdown_connections 2024-05-04T03:23:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.306+0000 7fdacf16b740 1 --2- 172.21.15.1:0/1091425912 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55af823501c0 0x55af827df840 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.306+0000 7fdacf16b740 1 --2- 172.21.15.1:0/1091425912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55af823bc1b0 0x55af8276d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.306+0000 7fdaa9ffb640 0 client.15702 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:23:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.306+0000 7fdacf16b740 1 --2- 172.21.15.1:0/1091425912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af8239b4d0 0x55af8276c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.306+0000 7fdacf16b740 1 --2- 172.21.15.1:0/1091425912 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af823932f0 0x55af8276c490 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.306+0000 7fdacf16b740 1 -- 172.21.15.1:0/1091425912 >> 172.21.15.1:0/1091425912 conn(0x55af823b0300 msgr2=0x55af8275c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.306+0000 7fdacf16b740 1 -- 172.21.15.1:0/1091425912 shutdown_connections 2024-05-04T03:23:22.312 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.306+0000 7fdacf16b740 1 -- 172.21.15.1:0/1091425912 wait complete. 2024-05-04T03:23:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.306+0000 7fdacf16b740 20 client.15702 trim_cache size 0 max 16384 2024-05-04T03:23:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:23:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:23:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:23:22.313 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:23:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.306+0000 7fdac82b6640 1 -- 172.21.15.1:0/2640095544 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fdaac0b2920 msgr2=0x7fdaac0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.306+0000 7fdac82b6640 1 --2- 172.21.15.1:0/2640095544 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fdaac0b2920 0x7fdaac0b4e30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdab80cae30 tx=0x7fdab8098ac0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.306+0000 7fdac82b6640 1 -- 172.21.15.1:0/2640095544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdac0108a90 msgr2=0x7fdac01a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.306+0000 7fdac82b6640 1 --2- 172.21.15.1:0/2640095544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdac0108a90 0x7fdac01a1700 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009430 tx=0x7fdab80317c0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.307+0000 7fdac82b6640 1 -- 172.21.15.1:0/2640095544 shutdown_connections 2024-05-04T03:23:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.307+0000 7fdac82b6640 1 --2- 172.21.15.1:0/2640095544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdac0108a90 0x7fdac01a1700 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.307+0000 7fdac82b6640 1 --2- 172.21.15.1:0/2640095544 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdac0103d90 0x7fdac019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.307+0000 7fdac82b6640 1 --2- 172.21.15.1:0/2640095544 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fdaac0b2920 0x7fdaac0b4e30 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.307+0000 7fdac82b6640 1 --2- 172.21.15.1:0/2640095544 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdac0102b50 0x7fdac019a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.307+0000 7fdac82b6640 1 -- 172.21.15.1:0/2640095544 >> 172.21.15.1:0/2640095544 conn(0x7fdac00fdd60 msgr2=0x7fdac0109e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.307+0000 7fdac82b6640 1 -- 172.21.15.1:0/2640095544 shutdown_connections 2024-05-04T03:23:22.315 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:22.307+0000 7fdac82b6640 1 -- 172.21.15.1:0/2640095544 wait complete. 2024-05-04T03:23:22.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:22 smithi077 ceph-mon[33531]: pgmap v517: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 47 KiB/s wr, 7 op/s 2024-05-04T03:23:22.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:22 smithi077 ceph-mon[33531]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:23:22.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:22 smithi072 ceph-mon[33332]: pgmap v517: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 47 KiB/s wr, 7 op/s 2024-05-04T03:23:22.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:22 smithi072 ceph-mon[33332]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:23:23.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:22 smithi001 ceph-mon[29731]: pgmap v517: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 47 KiB/s wr, 7 op/s 2024-05-04T03:23:23.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:22 smithi001 ceph-mon[29731]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:23:23.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.857+0000 7f9babb9e700 1 -- 172.21.15.1:0/2570072419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ba4108740 msgr2=0x7f9ba41100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:23.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.857+0000 7f9babb9e700 1 --2- 172.21.15.1:0/2570072419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ba4108740 0x7f9ba41100a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c00b0d0 tx=0x7f9b9c00b3e0 comp rx=0 tx=0).stop 2024-05-04T03:23:23.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.857+0000 7f9babb9e700 1 -- 172.21.15.1:0/2570072419 shutdown_connections 2024-05-04T03:23:23.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.857+0000 7f9babb9e700 1 --2- 172.21.15.1:0/2570072419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ba4108740 0x7f9ba41100a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:23.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.857+0000 7f9babb9e700 1 --2- 172.21.15.1:0/2570072419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ba41034c0 0x7f9ba41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:23.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.857+0000 7f9babb9e700 1 --2- 172.21.15.1:0/2570072419 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ba4107df0 0x7f9ba4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:23.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.857+0000 7f9babb9e700 1 -- 172.21.15.1:0/2570072419 >> 172.21.15.1:0/2570072419 conn(0x7f9ba40fedf0 msgr2=0x7f9ba4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:23.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.857+0000 7f9babb9e700 1 -- 172.21.15.1:0/2570072419 shutdown_connections 2024-05-04T03:23:23.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.857+0000 7f9babb9e700 1 -- 172.21.15.1:0/2570072419 wait complete. 2024-05-04T03:23:23.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.858+0000 7f9babb9e700 1 Processor -- start 2024-05-04T03:23:23.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.858+0000 7f9babb9e700 1 -- start start 2024-05-04T03:23:23.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.858+0000 7f9babb9e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ba41034c0 0x7f9ba419cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:23.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.858+0000 7f9babb9e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ba4107df0 0x7f9ba419d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:23.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.858+0000 7f9babb9e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ba4108740 0x7f9ba41a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:23.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.858+0000 7f9babb9e700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba4113370 con 0x7f9ba4108740 2024-05-04T03:23:23.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.858+0000 7f9babb9e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba41131f0 con 0x7f9ba4107df0 2024-05-04T03:23:23.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.858+0000 7f9babb9e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba41134f0 con 0x7f9ba41034c0 2024-05-04T03:23:23.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.858+0000 7f9baa13b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ba4108740 0x7f9ba41a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:23.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.858+0000 7f9baa13b700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ba4108740 0x7f9ba41a17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55850/0 (socket says 172.21.15.1:55850) 2024-05-04T03:23:23.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.858+0000 7f9baa13b700 1 -- 172.21.15.1:0/3056379527 learned_addr learned my addr 172.21.15.1:0/3056379527 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:23.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.858+0000 7f9ba993a700 1 --2- 172.21.15.1:0/3056379527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ba41034c0 0x7f9ba419cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:23.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.858+0000 7f9ba9139700 1 --2- 172.21.15.1:0/3056379527 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ba4107df0 0x7f9ba419d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:23.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.859+0000 7f9baa13b700 1 -- 172.21.15.1:0/3056379527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ba41034c0 msgr2=0x7f9ba419cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:23.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.859+0000 7f9baa13b700 1 --2- 172.21.15.1:0/3056379527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ba41034c0 0x7f9ba419cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:23.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.859+0000 7f9baa13b700 1 -- 172.21.15.1:0/3056379527 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ba4107df0 msgr2=0x7f9ba419d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:23.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.859+0000 7f9baa13b700 1 --2- 172.21.15.1:0/3056379527 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ba4107df0 0x7f9ba419d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:23.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.859+0000 7f9baa13b700 1 -- 172.21.15.1:0/3056379527 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba41a1e70 con 0x7f9ba4108740 2024-05-04T03:23:23.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.859+0000 7f9baa13b700 1 --2- 172.21.15.1:0/3056379527 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ba4108740 0x7f9ba41a17d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c009850 tx=0x7f9b9c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:23.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.859+0000 7f9b9affd700 1 -- 172.21.15.1:0/3056379527 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c01f070 con 0x7f9ba4108740 2024-05-04T03:23:23.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.859+0000 7f9b9affd700 1 -- 172.21.15.1:0/3056379527 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b9c004c50 con 0x7f9ba4108740 2024-05-04T03:23:23.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.859+0000 7f9babb9e700 1 -- 172.21.15.1:0/3056379527 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ba41a2100 con 0x7f9ba4108740 2024-05-04T03:23:23.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.859+0000 7f9b9affd700 1 -- 172.21.15.1:0/3056379527 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c01b690 con 0x7f9ba4108740 2024-05-04T03:23:23.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.859+0000 7f9babb9e700 1 -- 172.21.15.1:0/3056379527 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ba41a25e0 con 0x7f9ba4108740 2024-05-04T03:23:23.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.860+0000 7f9babb9e700 1 -- 172.21.15.1:0/3056379527 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ba40512c0 con 0x7f9ba4108740 2024-05-04T03:23:23.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.862+0000 7f9b9affd700 1 -- 172.21.15.1:0/3056379527 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f9b9c022070 con 0x7f9ba4108740 2024-05-04T03:23:23.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.863+0000 7f9b9affd700 1 --2- 172.21.15.1:0/3056379527 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9b900b2880 0x7f9b900b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:23.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.863+0000 7f9ba993a700 1 --2- 172.21.15.1:0/3056379527 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9b900b2880 0x7f9b900b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:23.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.863+0000 7f9b9affd700 1 -- 172.21.15.1:0/3056379527 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f9b9c00f020 con 0x7f9ba4108740 2024-05-04T03:23:23.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.864+0000 7f9ba993a700 1 --2- 172.21.15.1:0/3056379527 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9b900b2880 0x7f9b900b4d90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0005950 tx=0x7f9ba000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:23.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:23.867+0000 7f9b9affd700 1 -- 172.21.15.1:0/3056379527 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9b9c029020 con 0x7f9ba4108740 2024-05-04T03:23:24.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:24.131+0000 7f9babb9e700 1 -- 172.21.15.1:0/3056379527 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ba402e5f0 con 0x7f9ba4108740 2024-05-04T03:23:24.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:24.132+0000 7f9b9affd700 1 -- 172.21.15.1:0/3056379527 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11846 (secure 0 0 0) 0x7f9b9c0ab4b0 con 0x7f9ba4108740 2024-05-04T03:23:24.134 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:23:24.135 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2186681910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2186681910},{"type":"v1","addr":"172.21.15.1:6835","nonce":2186681910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2747191047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2747191047},{"type":"v1","addr":"172.21.15.1:6837","nonce":2747191047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4115120540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4115120540},{"type":"v1","addr":"172.21.15.1:6839","nonce":4115120540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3405709780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3405709780},{"type":"v1","addr":"172.21.15.1:6841","nonce":3405709780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/3816098761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":3816098761},{"type":"v1","addr":"172.21.15.72:6833","nonce":3816098761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/379026128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":379026128},{"type":"v1","addr":"172.21.15.77:6833","nonce":379026128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:15:49.823471+0000","modified":"2024-05-04T03:23:21.572875+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24472,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/3930984976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3930984976},{"type":"v1","addr":"172.21.15.72:6835","nonce":3930984976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/4279951305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4279951305},{"type":"v1","addr":"172.21.15.77:6835","nonce":4279951305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2003521417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2003521417},{"type":"v1","addr":"172.21.15.77:6837","nonce":2003521417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/1417614203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1417614203},{"type":"v1","addr":"172.21.15.72:6837","nonce":1417614203}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.77:6839/922723189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":922723189},{"type":"v1","addr":"172.21.15.77:6839","nonce":922723189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2976050650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2976050650},{"type":"v1","addr":"172.21.15.72:6839","nonce":2976050650}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24467,24472]},"id":1}]} 2024-05-04T03:23:24.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:24.134+0000 7f9babb9e700 1 -- 172.21.15.1:0/3056379527 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9b900b2880 msgr2=0x7f9b900b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:24.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:24.134+0000 7f9babb9e700 1 --2- 172.21.15.1:0/3056379527 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9b900b2880 0x7f9b900b4d90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0005950 tx=0x7f9ba000b410 comp rx=0 tx=0).stop 2024-05-04T03:23:24.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:24.134+0000 7f9babb9e700 1 -- 172.21.15.1:0/3056379527 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ba4108740 msgr2=0x7f9ba41a17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:24.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:24.134+0000 7f9babb9e700 1 --2- 172.21.15.1:0/3056379527 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ba4108740 0x7f9ba41a17d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c009850 tx=0x7f9b9c007a20 comp rx=0 tx=0).stop 2024-05-04T03:23:24.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:24.135+0000 7f9babb9e700 1 -- 172.21.15.1:0/3056379527 shutdown_connections 2024-05-04T03:23:24.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:24.135+0000 7f9babb9e700 1 --2- 172.21.15.1:0/3056379527 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f9b900b2880 0x7f9b900b4d90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:24.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:24.135+0000 7f9babb9e700 1 --2- 172.21.15.1:0/3056379527 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ba4108740 0x7f9ba41a17d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:24.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:24.135+0000 7f9babb9e700 1 --2- 172.21.15.1:0/3056379527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ba41034c0 0x7f9ba419cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:24.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:24.135+0000 7f9babb9e700 1 --2- 172.21.15.1:0/3056379527 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ba4107df0 0x7f9ba419d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:24.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:24.135+0000 7f9babb9e700 1 -- 172.21.15.1:0/3056379527 >> 172.21.15.1:0/3056379527 conn(0x7f9ba40fedf0 msgr2=0x7f9ba410ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:24.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:24.135+0000 7f9babb9e700 1 -- 172.21.15.1:0/3056379527 shutdown_connections 2024-05-04T03:23:24.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:24.135+0000 7f9babb9e700 1 -- 172.21.15.1:0/3056379527 wait complete. 2024-05-04T03:23:24.138 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 30 2024-05-04T03:23:24.851 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:24 smithi001 ceph-mon[29731]: pgmap v518: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 49 KiB/s wr, 9 op/s 2024-05-04T03:23:24.851 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:24 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3056379527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:23:24.881 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 88323539 --release --await' 2024-05-04T03:23:24.881 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24494 quiesce db --set-id 88323539 --release --await 2024-05-04T03:23:24.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:24 smithi077 ceph-mon[33531]: pgmap v518: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 49 KiB/s wr, 9 op/s 2024-05-04T03:23:24.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:24 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/3056379527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:23:24.998 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:24 smithi072 ceph-mon[33332]: pgmap v518: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 49 KiB/s wr, 9 op/s 2024-05-04T03:23:24.998 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:24 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/3056379527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:23:25.136 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:23:26.874 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:26 smithi001 ceph-mon[29731]: pgmap v519: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 47 KiB/s wr, 7 op/s 2024-05-04T03:23:26.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.872+0000 7f67a5f28700 1 -- 172.21.15.1:0/553166992 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a0107e00 msgr2=0x7f67a0108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:26.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.872+0000 7f67a5f28700 1 --2- 172.21.15.1:0/553166992 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a0107e00 0x7f67a0108210 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f67940099c0 tx=0x7f6794009cd0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.872+0000 7f67a5f28700 1 -- 172.21.15.1:0/553166992 shutdown_connections 2024-05-04T03:23:26.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.872+0000 7f67a5f28700 1 --2- 172.21.15.1:0/553166992 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67a0108750 0x7f67a01100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.872+0000 7f67a5f28700 1 --2- 172.21.15.1:0/553166992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67a0103520 0x7f67a0103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.872+0000 7f67a5f28700 1 --2- 172.21.15.1:0/553166992 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a0107e00 0x7f67a0108210 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.872+0000 7f67a5f28700 1 -- 172.21.15.1:0/553166992 >> 172.21.15.1:0/553166992 conn(0x7f67a00fef90 msgr2=0x7f67a01013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:26.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.872+0000 7f67a5f28700 1 -- 172.21.15.1:0/553166992 shutdown_connections 2024-05-04T03:23:26.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.872+0000 7f67a5f28700 1 -- 172.21.15.1:0/553166992 wait complete. 2024-05-04T03:23:26.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.872+0000 7f67a5f28700 1 Processor -- start 2024-05-04T03:23:26.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.873+0000 7f67a5f28700 1 -- start start 2024-05-04T03:23:26.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.873+0000 7f67a5f28700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67a0103520 0x7f67a019aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:26.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.873+0000 7f67a5f28700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67a0107e00 0x7f67a019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:26.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.873+0000 7f67a5f28700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a0108750 0x7f67a019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:26.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.873+0000 7f67a5f28700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f67a0113360 con 0x7f67a0108750 2024-05-04T03:23:26.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.873+0000 7f67a5f28700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f67a01131e0 con 0x7f67a0107e00 2024-05-04T03:23:26.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.873+0000 7f67a5f28700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f67a01134e0 con 0x7f67a0103520 2024-05-04T03:23:26.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.873+0000 7f679ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a0108750 0x7f67a019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:26.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.873+0000 7f679f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67a0103520 0x7f67a019aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:26.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.873+0000 7f679f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67a0103520 0x7f67a019aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.1:35266/0 (socket says 172.21.15.1:35266) 2024-05-04T03:23:26.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.873+0000 7f679ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a0108750 0x7f67a019f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55862/0 (socket says 172.21.15.1:55862) 2024-05-04T03:23:26.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.873+0000 7f679f7fe700 1 -- 172.21.15.1:0/1994110617 learned_addr learned my addr 172.21.15.1:0/1994110617 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:26.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.873+0000 7f679effd700 1 --2- 172.21.15.1:0/1994110617 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67a0107e00 0x7f67a019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:26.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.874+0000 7f679f7fe700 1 -- 172.21.15.1:0/1994110617 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67a0107e00 msgr2=0x7f67a019b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:26.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.874+0000 7f679f7fe700 1 --2- 172.21.15.1:0/1994110617 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67a0107e00 0x7f67a019b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.874+0000 7f679f7fe700 1 -- 172.21.15.1:0/1994110617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a0108750 msgr2=0x7f67a019f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:26.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.874+0000 7f679f7fe700 1 --2- 172.21.15.1:0/1994110617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a0108750 0x7f67a019f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.874+0000 7f679f7fe700 1 -- 172.21.15.1:0/1994110617 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a019fc20 con 0x7f67a0103520 2024-05-04T03:23:26.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.874+0000 7f679ffff700 1 --2- 172.21.15.1:0/1994110617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a0108750 0x7f67a019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:23:26.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.874+0000 7f679effd700 1 --2- 172.21.15.1:0/1994110617 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67a0107e00 0x7f67a019b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:23:26.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.874+0000 7f679f7fe700 1 --2- 172.21.15.1:0/1994110617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67a0103520 0x7f67a019aea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f679000d950 tx=0x7f679000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:26.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.874+0000 7f679cff9700 1 -- 172.21.15.1:0/1994110617 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6790016070 con 0x7f67a0103520 2024-05-04T03:23:26.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.874+0000 7f67a5f28700 1 -- 172.21.15.1:0/1994110617 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a019ff10 con 0x7f67a0103520 2024-05-04T03:23:26.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.874+0000 7f67a5f28700 1 -- 172.21.15.1:0/1994110617 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a01a0470 con 0x7f67a0103520 2024-05-04T03:23:26.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.874+0000 7f679cff9700 1 -- 172.21.15.1:0/1994110617 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6790009df0 con 0x7f67a0103520 2024-05-04T03:23:26.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.874+0000 7f679cff9700 1 -- 172.21.15.1:0/1994110617 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6790020060 con 0x7f67a0103520 2024-05-04T03:23:26.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.877+0000 7f679cff9700 1 -- 172.21.15.1:0/1994110617 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f6790019070 con 0x7f67a0103520 2024-05-04T03:23:26.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.878+0000 7f67b493cb80 1 Processor -- start 2024-05-04T03:23:26.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.878+0000 7f679cff9700 1 --2- 172.21.15.1:0/1994110617 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f67880b2880 0x7f67880b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:26.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.878+0000 7f67b493cb80 1 -- start start 2024-05-04T03:23:26.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.878+0000 7f679ffff700 1 --2- 172.21.15.1:0/1994110617 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f67880b2880 0x7f67880b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:26.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.878+0000 7f679cff9700 1 -- 172.21.15.1:0/1994110617 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f67900e1d60 con 0x7f67a0103520 2024-05-04T03:23:26.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.878+0000 7f67b493cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558fff1e3ad0 0x558fff1e3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:26.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.878+0000 7f67b493cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fff1e4420 0x558fff1ee700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:26.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.878+0000 7f67b493cb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558fff1eec40 0x558fff1f1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:26.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.878+0000 7f67b493cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558fff106a40 con 0x558fff1e4420 2024-05-04T03:23:26.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.878+0000 7f67b493cb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x558fff06bea0 con 0x558fff1eec40 2024-05-04T03:23:26.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.878+0000 7f67b493cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558fff102a00 con 0x558fff1e3ad0 2024-05-04T03:23:26.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.878+0000 7f679effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fff1e4420 0x558fff1ee700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:26.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.878+0000 7f679effd700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fff1e4420 0x558fff1ee700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55878/0 (socket says 172.21.15.1:55878) 2024-05-04T03:23:26.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.878+0000 7f679effd700 1 -- 172.21.15.1:0/2878688699 learned_addr learned my addr 172.21.15.1:0/2878688699 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:26.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.878+0000 7f679ffff700 1 --2- 172.21.15.1:0/1994110617 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f67880b2880 0x7f67880b4d90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6798007fd0 tx=0x7f6798007950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:26.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.879+0000 7f679f7fe700 1 --2- 172.21.15.1:0/2878688699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558fff1e3ad0 0x558fff1e3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:26.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.879+0000 7f679ffff700 1 --2- 172.21.15.1:0/2878688699 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558fff1eec40 0x558fff1f1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:26.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.879+0000 7f679effd700 1 -- 172.21.15.1:0/2878688699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558fff1e3ad0 msgr2=0x558fff1e3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:26.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.879+0000 7f679effd700 1 --2- 172.21.15.1:0/2878688699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558fff1e3ad0 0x558fff1e3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.879+0000 7f679effd700 1 -- 172.21.15.1:0/2878688699 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558fff1eec40 msgr2=0x558fff1f1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:26.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.879+0000 7f679effd700 1 --2- 172.21.15.1:0/2878688699 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558fff1eec40 0x558fff1f1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.879+0000 7f679effd700 1 -- 172.21.15.1:0/2878688699 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fff0539d0 con 0x558fff1e4420 2024-05-04T03:23:26.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.879+0000 7f679effd700 1 --2- 172.21.15.1:0/2878688699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fff1e4420 0x558fff1ee700 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f67940099c0 tx=0x7f67940036e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608a51692fa5ee9f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:26.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.879+0000 7f6785ffb700 1 -- 172.21.15.1:0/2878688699 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6794026070 con 0x558fff1e4420 2024-05-04T03:23:26.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.879+0000 7f6785ffb700 1 -- 172.21.15.1:0/2878688699 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6794003860 con 0x558fff1e4420 2024-05-04T03:23:26.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.879+0000 7f6785ffb700 1 -- 172.21.15.1:0/2878688699 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6794005530 con 0x558fff1e4420 2024-05-04T03:23:26.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.880+0000 7f67b493cb80 1 -- 172.21.15.1:0/2878688699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fff1e4420 msgr2=0x558fff1ee700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:26.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.880+0000 7f67b493cb80 1 --2- 172.21.15.1:0/2878688699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fff1e4420 0x558fff1ee700 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f67940099c0 tx=0x7f67940036e0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.880+0000 7f67b493cb80 1 -- 172.21.15.1:0/2878688699 shutdown_connections 2024-05-04T03:23:26.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.880+0000 7f67b493cb80 1 --2- 172.21.15.1:0/2878688699 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558fff1eec40 0x558fff1f1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.880+0000 7f67b493cb80 1 --2- 172.21.15.1:0/2878688699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558fff1e3ad0 0x558fff1e3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.880+0000 7f67b493cb80 1 --2- 172.21.15.1:0/2878688699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fff1e4420 0x558fff1ee700 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.880+0000 7f67b493cb80 1 -- 172.21.15.1:0/2878688699 >> 172.21.15.1:0/2878688699 conn(0x558fff066f40 msgr2=0x558fff1df9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:26.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.880+0000 7f67b493cb80 1 -- 172.21.15.1:0/2878688699 shutdown_connections 2024-05-04T03:23:26.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.880+0000 7f67b493cb80 1 -- 172.21.15.1:0/2878688699 wait complete. 2024-05-04T03:23:26.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.880+0000 7f67b493cb80 1 -- start start 2024-05-04T03:23:26.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.881+0000 7f67b493cb80 1 Processor -- start 2024-05-04T03:23:26.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.881+0000 7f67b493cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fff1e3ad0 0x558fff1ebd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:26.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.881+0000 7f67b493cb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558fff1e4420 0x558fff1ec2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:26.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.881+0000 7f67b493cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558fff1eec40 0x558fff1ec800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:26.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.881+0000 7f67b493cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558fff1f19b0 con 0x558fff1e3ad0 2024-05-04T03:23:26.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.881+0000 7f67b493cb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x558fff0bffd0 con 0x558fff1e4420 2024-05-04T03:23:26.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.881+0000 7f67b493cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558fff1f1b30 con 0x558fff1eec40 2024-05-04T03:23:26.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.881+0000 7f67b493cb80 10 client.0 _ll_register_callbacks cb 0x558fff09ea50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:23:26.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.881+0000 7f679f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fff1e3ad0 0x558fff1ebd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:26.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.881+0000 7f679f7fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fff1e3ad0 0x558fff1ebd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55888/0 (socket says 172.21.15.1:55888) 2024-05-04T03:23:26.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.881+0000 7f679f7fe700 1 -- 172.21.15.1:0/3902860126 learned_addr learned my addr 172.21.15.1:0/3902860126 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:26.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.881+0000 7f679effd700 1 --2- 172.21.15.1:0/3902860126 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558fff1e4420 0x558fff1ec2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:26.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.881+0000 7f679ffff700 1 --2- 172.21.15.1:0/3902860126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558fff1eec40 0x558fff1ec800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.882+0000 7f679f7fe700 1 -- 172.21.15.1:0/3902860126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558fff1eec40 msgr2=0x558fff1ec800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.882+0000 7f679f7fe700 1 --2- 172.21.15.1:0/3902860126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558fff1eec40 0x558fff1ec800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.882+0000 7f679f7fe700 1 -- 172.21.15.1:0/3902860126 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558fff1e4420 msgr2=0x558fff1ec2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.882+0000 7f679f7fe700 1 --2- 172.21.15.1:0/3902860126 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558fff1e4420 0x558fff1ec2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.882+0000 7f679f7fe700 1 -- 172.21.15.1:0/3902860126 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fff0bd860 con 0x558fff1e3ad0 2024-05-04T03:23:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.882+0000 7f679f7fe700 1 --2- 172.21.15.1:0/3902860126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fff1e3ad0 0x558fff1ebd80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f67900df8a0 tx=0x7f67900a9020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ec2266c3df6a78 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.882+0000 7f677effd700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:23:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.882+0000 7f677effd700 1 -- 172.21.15.1:0/3902860126 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67900b2280 con 0x558fff1e3ad0 2024-05-04T03:23:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.882+0000 7f677effd700 1 -- 172.21.15.1:0/3902860126 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f67900a9680 con 0x558fff1e3ad0 2024-05-04T03:23:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.882+0000 7f67b493cb80 1 -- 172.21.15.1:0/3902860126 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6794003860 con 0x558fff1e3ad0 2024-05-04T03:23:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.882+0000 7f677effd700 1 -- 172.21.15.1:0/3902860126 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f67900a9880 con 0x558fff1e3ad0 2024-05-04T03:23:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.882+0000 7f67b493cb80 10 client.15720 fetch_fsmap learned FSMap version 31 2024-05-04T03:23:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.882+0000 7f67b493cb80 1 -- 172.21.15.1:0/3902860126 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558fff1fc7d0 con 0x558fff1e3ad0 2024-05-04T03:23:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.882+0000 7f677effd700 1 -- 172.21.15.1:0/3902860126 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6794003860 con 0x558fff1e3ad0 2024-05-04T03:23:26.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.883+0000 7f677effd700 1 -- 172.21.15.1:0/3902860126 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 31) ==== 8342+0+0 (secure 0 0 0) 0x7f6790009940 con 0x558fff1e3ad0 2024-05-04T03:23:26.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.883+0000 7f67b493cb80 10 client.15720 fetch_fsmap finished waiting for FSMap version 31 2024-05-04T03:23:26.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.883+0000 7f67b493cb80 10 client.15720 resolve_mds: validated gid 24494 aka daemon mds.b 2024-05-04T03:23:26.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.883+0000 7f67b493cb80 20 client.15720 populate_metadata read hostname 'smithi001' 2024-05-04T03:23:26.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.883+0000 7f67b493cb80 1 --2- 172.21.15.1:0/3902860126 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x558fff1fee10 0x558fff201290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:26.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.883+0000 7f67b493cb80 4 client.15720 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:23:26.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.883+0000 7f67b493cb80 1 -- 172.21.15.1:0/3902860126 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fff114420 con 0x558fff1fee10 2024-05-04T03:23:26.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.883+0000 7f679effd700 1 --2- 172.21.15.1:0/3902860126 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x558fff1fee10 0x558fff201290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:26.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.883+0000 7f679effd700 1 --2- 172.21.15.1:0/3902860126 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x558fff1fee10 0x558fff201290 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59ccbdd9ba56d1ba server_cookie=48bda0e59027440d in_seq=0 out_seq=0 2024-05-04T03:23:26.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.883+0000 7f677effd700 10 client.15720 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:23:26.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.884+0000 7f677effd700 1 -- 172.21.15.1:0/3902860126 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558fff114420 con 0x558fff1fee10 2024-05-04T03:23:26.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.884+0000 7f677effd700 10 client.15720 handle_command_reply: tid=0 2024-05-04T03:23:26.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.884+0000 7f67b493cb80 1 client.15720 shutdown 2024-05-04T03:23:26.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.884+0000 7f67b493cb80 1 -- 172.21.15.1:0/3902860126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fff1e3ad0 msgr2=0x558fff1ebd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:26.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.884+0000 7f67b493cb80 1 --2- 172.21.15.1:0/3902860126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fff1e3ad0 0x558fff1ebd80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f67900df8a0 tx=0x7f67900a9020 comp rx=0 tx=0).stop 2024-05-04T03:23:26.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.884+0000 7f67b493cb80 1 -- 172.21.15.1:0/3902860126 shutdown_connections 2024-05-04T03:23:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.884+0000 7f67b493cb80 1 --2- 172.21.15.1:0/3902860126 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x558fff1fee10 0x558fff201290 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.884+0000 7f67b493cb80 1 --2- 172.21.15.1:0/3902860126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558fff1eec40 0x558fff1ec800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.884+0000 7f67b493cb80 1 --2- 172.21.15.1:0/3902860126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fff1e3ad0 0x558fff1ebd80 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.884+0000 7f67b493cb80 1 --2- 172.21.15.1:0/3902860126 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558fff1e4420 0x558fff1ec2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.884+0000 7f67b493cb80 1 -- 172.21.15.1:0/3902860126 >> 172.21.15.1:0/3902860126 conn(0x558fff066f40 msgr2=0x558fff1e54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.884+0000 7f677effd700 0 client.15720 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:23:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.885+0000 7f67b493cb80 1 -- 172.21.15.1:0/3902860126 shutdown_connections 2024-05-04T03:23:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.885+0000 7f67b493cb80 1 -- 172.21.15.1:0/3902860126 wait complete. 2024-05-04T03:23:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.885+0000 7f67b493cb80 20 client.15720 trim_cache size 0 max 16384 2024-05-04T03:23:26.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.904+0000 7f67b493cb80 1 Processor -- start 2024-05-04T03:23:26.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.905+0000 7f67b493cb80 1 -- start start 2024-05-04T03:23:26.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.905+0000 7f67b493cb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558fff1e3ad0 0x558fff1e7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:26.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.905+0000 7f67b493cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558fff1e4420 0x558fff1df840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:26.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.905+0000 7f67b493cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fff1eec40 0x558fff1ec640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:26.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.905+0000 7f67b493cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f67a01b10c0 con 0x558fff1eec40 2024-05-04T03:23:26.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.905+0000 7f67b493cb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f67a01ae950 con 0x558fff1e3ad0 2024-05-04T03:23:26.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.905+0000 7f67b493cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f67a01ac860 con 0x558fff1e4420 2024-05-04T03:23:26.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.905+0000 7f679ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fff1eec40 0x558fff1ec640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:26.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.905+0000 7f679ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fff1eec40 0x558fff1ec640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55894/0 (socket says 172.21.15.1:55894) 2024-05-04T03:23:26.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.905+0000 7f679ffff700 1 -- 172.21.15.1:0/1549542630 learned_addr learned my addr 172.21.15.1:0/1549542630 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:26.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.905+0000 7f679effd700 1 --2- 172.21.15.1:0/1549542630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558fff1e4420 0x558fff1df840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:26.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.905+0000 7f679f7fe700 1 --2- 172.21.15.1:0/1549542630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558fff1e3ad0 0x558fff1e7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:26.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.905+0000 7f679ffff700 1 -- 172.21.15.1:0/1549542630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558fff1e4420 msgr2=0x558fff1df840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:26.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.905+0000 7f679ffff700 1 --2- 172.21.15.1:0/1549542630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558fff1e4420 0x558fff1df840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.905+0000 7f679ffff700 1 -- 172.21.15.1:0/1549542630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558fff1e3ad0 msgr2=0x558fff1e7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:26.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.905+0000 7f679ffff700 1 --2- 172.21.15.1:0/1549542630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558fff1e3ad0 0x558fff1e7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.905+0000 7f679ffff700 1 -- 172.21.15.1:0/1549542630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fff1e4830 con 0x558fff1eec40 2024-05-04T03:23:26.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.905+0000 7f679effd700 1 --2- 172.21.15.1:0/1549542630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558fff1e4420 0x558fff1df840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:23:26.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.905+0000 7f679ffff700 1 --2- 172.21.15.1:0/1549542630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fff1eec40 0x558fff1ec640 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f67980149b0 tx=0x7f6798012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e331a8ea19a642 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:26.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.906+0000 7f67857fa700 1 -- 172.21.15.1:0/1549542630 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6798026070 con 0x558fff1eec40 2024-05-04T03:23:26.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.906+0000 7f67857fa700 1 -- 172.21.15.1:0/1549542630 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f679801c040 con 0x558fff1eec40 2024-05-04T03:23:26.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.906+0000 7f67857fa700 1 -- 172.21.15.1:0/1549542630 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679801b040 con 0x558fff1eec40 2024-05-04T03:23:26.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.906+0000 7f67b493cb80 1 -- 172.21.15.1:0/1549542630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fff1eec40 msgr2=0x558fff1ec640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:26.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.906+0000 7f67b493cb80 1 --2- 172.21.15.1:0/1549542630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fff1eec40 0x558fff1ec640 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f67980149b0 tx=0x7f6798012000 comp rx=0 tx=0).stop 2024-05-04T03:23:26.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.906+0000 7f67b493cb80 1 -- 172.21.15.1:0/1549542630 shutdown_connections 2024-05-04T03:23:26.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.906+0000 7f67b493cb80 1 --2- 172.21.15.1:0/1549542630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558fff1e3ad0 0x558fff1e7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.906+0000 7f67b493cb80 1 --2- 172.21.15.1:0/1549542630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fff1eec40 0x558fff1ec640 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.906+0000 7f67b493cb80 1 --2- 172.21.15.1:0/1549542630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558fff1e4420 0x558fff1df840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.906+0000 7f67b493cb80 1 -- 172.21.15.1:0/1549542630 >> 172.21.15.1:0/1549542630 conn(0x558fff066f40 msgr2=0x558fff1d9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:26.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.907+0000 7f67b493cb80 1 -- 172.21.15.1:0/1549542630 shutdown_connections 2024-05-04T03:23:26.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.907+0000 7f67b493cb80 1 -- 172.21.15.1:0/1549542630 wait complete. 2024-05-04T03:23:26.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.907+0000 7f67b493cb80 1 -- start start 2024-05-04T03:23:26.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.907+0000 7f67b493cb80 1 Processor -- start 2024-05-04T03:23:26.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.907+0000 7f67b493cb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558fff1e3ad0 0x558fff1f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:26.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.907+0000 7f67b493cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558fff1e4420 0x558fff1f72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:26.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.907+0000 7f67b493cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fff1eec40 0x558fff1f3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:26.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.907+0000 7f67b493cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f67a01aceb0 con 0x558fff1eec40 2024-05-04T03:23:26.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.907+0000 7f67b493cb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f67a01ae280 con 0x558fff1e3ad0 2024-05-04T03:23:26.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.907+0000 7f67b493cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f67a01ad500 con 0x558fff1e4420 2024-05-04T03:23:26.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.908+0000 7f67b493cb80 10 client.0 _ll_register_callbacks cb 0x558fff09ea50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:23:26.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.908+0000 7f679ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fff1eec40 0x558fff1f3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:26.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.908+0000 7f679f7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558fff1e3ad0 0x558fff1f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:26.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.908+0000 7f679ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fff1eec40 0x558fff1f3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55896/0 (socket says 172.21.15.1:55896) 2024-05-04T03:23:26.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.908+0000 7f679ffff700 1 -- 172.21.15.1:0/1899300290 learned_addr learned my addr 172.21.15.1:0/1899300290 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:26.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.908+0000 7f679effd700 1 --2- 172.21.15.1:0/1899300290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558fff1e4420 0x558fff1f72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:26.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.908+0000 7f679ffff700 1 -- 172.21.15.1:0/1899300290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558fff1e4420 msgr2=0x558fff1f72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:26.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.908+0000 7f679ffff700 1 --2- 172.21.15.1:0/1899300290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558fff1e4420 0x558fff1f72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.908+0000 7f679ffff700 1 -- 172.21.15.1:0/1899300290 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558fff1e3ad0 msgr2=0x558fff1f6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:26.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.908+0000 7f679ffff700 1 --2- 172.21.15.1:0/1899300290 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558fff1e3ad0 0x558fff1f6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.908+0000 7f679ffff700 1 -- 172.21.15.1:0/1899300290 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fff1e3720 con 0x558fff1eec40 2024-05-04T03:23:26.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.908+0000 7f679f7fe700 1 --2- 172.21.15.1:0/1899300290 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558fff1e3ad0 0x558fff1f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:23:26.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.908+0000 7f679ffff700 1 --2- 172.21.15.1:0/1899300290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fff1eec40 0x558fff1f3140 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6798014980 tx=0x7f6798022d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d82b62842ff7b607 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:26.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.908+0000 7f677effd700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:23:26.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.908+0000 7f677effd700 1 -- 172.21.15.1:0/1899300290 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6798026030 con 0x558fff1eec40 2024-05-04T03:23:26.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.908+0000 7f677effd700 1 -- 172.21.15.1:0/1899300290 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f6798004ce0 con 0x558fff1eec40 2024-05-04T03:23:26.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.908+0000 7f67b493cb80 1 -- 172.21.15.1:0/1899300290 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f679801c040 con 0x558fff1eec40 2024-05-04T03:23:26.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.909+0000 7f677effd700 1 -- 172.21.15.1:0/1899300290 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f679801b040 con 0x558fff1eec40 2024-05-04T03:23:26.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.909+0000 7f67b493cb80 10 client.15726 fetch_fsmap learned FSMap version 31 2024-05-04T03:23:26.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.909+0000 7f67b493cb80 1 -- 172.21.15.1:0/1899300290 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558fff269eb0 con 0x558fff1eec40 2024-05-04T03:23:26.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.909+0000 7f677effd700 1 -- 172.21.15.1:0/1899300290 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f679801c040 con 0x558fff1eec40 2024-05-04T03:23:26.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.909+0000 7f677effd700 1 -- 172.21.15.1:0/1899300290 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 31) ==== 8342+0+0 (secure 0 0 0) 0x7f6798021040 con 0x558fff1eec40 2024-05-04T03:23:26.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.909+0000 7f67b493cb80 10 client.15726 fetch_fsmap finished waiting for FSMap version 31 2024-05-04T03:23:26.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.909+0000 7f67b493cb80 10 client.15726 resolve_mds: validated gid 24494 aka daemon mds.b 2024-05-04T03:23:26.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.909+0000 7f67b493cb80 20 client.15726 populate_metadata read hostname 'smithi001' 2024-05-04T03:23:26.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.909+0000 7f67b493cb80 1 --2- 172.21.15.1:0/1899300290 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x558fff26a4b0 0x558fff26c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:26.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.909+0000 7f67b493cb80 4 client.15726 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "88323539", "release": true, "await": true}] 2024-05-04T03:23:26.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.909+0000 7f67b493cb80 1 -- 172.21.15.1:0/1899300290 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "quiesce db", "set_id": "88323539", "release": true, "await": true}) -- 0x558fff10e4c0 con 0x558fff26a4b0 2024-05-04T03:23:26.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.909+0000 7f679f7fe700 1 --2- 172.21.15.1:0/1899300290 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x558fff26a4b0 0x558fff26c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:26.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.910+0000 7f679f7fe700 1 --2- 172.21.15.1:0/1899300290 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x558fff26a4b0 0x558fff26c8d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f41304ba48f9e27 server_cookie=ef61b063af83aef1 in_seq=0 out_seq=0 2024-05-04T03:23:26.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.910+0000 7f677effd700 10 client.15726 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:23:26.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.917+0000 7f677effd700 1 -- 172.21.15.1:0/1899300290 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x558fff10e4c0 con 0x558fff26a4b0 2024-05-04T03:23:26.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.917+0000 7f677effd700 10 client.15726 handle_command_reply: tid=0 2024-05-04T03:23:26.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.917+0000 7f67b493cb80 1 client.15726 shutdown 2024-05-04T03:23:26.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.918+0000 7f67b493cb80 1 -- 172.21.15.1:0/1899300290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fff1eec40 msgr2=0x558fff1f3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:26.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.918+0000 7f67b493cb80 1 --2- 172.21.15.1:0/1899300290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fff1eec40 0x558fff1f3140 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6798014980 tx=0x7f6798022d20 comp rx=0 tx=0).stop 2024-05-04T03:23:26.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.918+0000 7f67b493cb80 1 -- 172.21.15.1:0/1899300290 shutdown_connections 2024-05-04T03:23:26.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.918+0000 7f67b493cb80 1 --2- 172.21.15.1:0/1899300290 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x558fff26a4b0 0x558fff26c8d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.918+0000 7f67b493cb80 1 --2- 172.21.15.1:0/1899300290 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558fff1e3ad0 0x558fff1f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.918+0000 7f67b493cb80 1 --2- 172.21.15.1:0/1899300290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fff1eec40 0x558fff1f3140 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.918+0000 7f677effd700 0 client.15726 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:23:26.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.918+0000 7f67b493cb80 1 --2- 172.21.15.1:0/1899300290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558fff1e4420 0x558fff1f72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.918+0000 7f67b493cb80 1 -- 172.21.15.1:0/1899300290 >> 172.21.15.1:0/1899300290 conn(0x558fff066f40 msgr2=0x558fff1e5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:26.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.918+0000 7f67b493cb80 1 -- 172.21.15.1:0/1899300290 shutdown_connections 2024-05-04T03:23:26.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.918+0000 7f67b493cb80 1 -- 172.21.15.1:0/1899300290 wait complete. 2024-05-04T03:23:26.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.918+0000 7f67b493cb80 20 client.15726 trim_cache size 0 max 16384 2024-05-04T03:23:26.921 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-04T03:23:26.922 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 31, 2024-05-04T03:23:26.922 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24494, 2024-05-04T03:23:26.922 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 12, 2024-05-04T03:23:26.922 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-05-04T03:23:26.922 INFO:teuthology.orchestra.run.smithi001.stdout: "88323539": { 2024-05-04T03:23:26.922 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 12, 2024-05-04T03:23:26.922 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-05-04T03:23:26.922 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-05-04T03:23:26.922 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-05-04T03:23:26.922 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-05-04T03:23:26.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:23:26.922 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 60.0, 2024-05-04T03:23:26.922 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 90.0, 2024-05-04T03:23:26.922 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-05-04T03:23:26.923 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-05-04T03:23:26.923 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-05-04T03:23:26.923 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-05-04T03:23:26.923 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-05-04T03:23:26.923 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-05-04T03:23:26.923 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:23:26.923 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:23:26.923 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:23:26.923 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:23:26.923 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:23:26.923 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-04T03:23:26.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.919+0000 7f67a5f28700 1 -- 172.21.15.1:0/1994110617 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f67880b2880 msgr2=0x7f67880b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:26.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.919+0000 7f67a5f28700 1 --2- 172.21.15.1:0/1994110617 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f67880b2880 0x7f67880b4d90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6798007fd0 tx=0x7f6798007950 comp rx=0 tx=0).stop 2024-05-04T03:23:26.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.919+0000 7f67a5f28700 1 -- 172.21.15.1:0/1994110617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67a0103520 msgr2=0x7f67a019aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:26.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.919+0000 7f67a5f28700 1 --2- 172.21.15.1:0/1994110617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67a0103520 0x7f67a019aea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f679000d950 tx=0x7f679000dc60 comp rx=0 tx=0).stop 2024-05-04T03:23:26.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.919+0000 7f67a5f28700 1 -- 172.21.15.1:0/1994110617 shutdown_connections 2024-05-04T03:23:26.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.919+0000 7f67a5f28700 1 --2- 172.21.15.1:0/1994110617 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f67880b2880 0x7f67880b4d90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.919+0000 7f67a5f28700 1 --2- 172.21.15.1:0/1994110617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67a0108750 0x7f67a019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.919+0000 7f67a5f28700 1 --2- 172.21.15.1:0/1994110617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67a0103520 0x7f67a019aea0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.919+0000 7f67a5f28700 1 --2- 172.21.15.1:0/1994110617 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67a0107e00 0x7f67a019b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:26.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.919+0000 7f67a5f28700 1 -- 172.21.15.1:0/1994110617 >> 172.21.15.1:0/1994110617 conn(0x7f67a00fef90 msgr2=0x7f67a006c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:26.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.919+0000 7f67a5f28700 1 -- 172.21.15.1:0/1994110617 shutdown_connections 2024-05-04T03:23:26.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:26.919+0000 7f67a5f28700 1 -- 172.21.15.1:0/1994110617 wait complete. 2024-05-04T03:23:26.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:26 smithi077 ceph-mon[33531]: pgmap v519: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 47 KiB/s wr, 7 op/s 2024-05-04T03:23:26.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:26 smithi072 ceph-mon[33332]: pgmap v519: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 47 KiB/s wr, 7 op/s 2024-05-04T03:23:27.801 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '88323539', seconds elapsed: 44.7 2024-05-04T03:23:27.801 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-05-04T03:23:27.802 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:23:27.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:27 smithi072 ceph-mon[33332]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:23:28.078 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:23:28.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:27 smithi001 ceph-mon[29731]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:23:28.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:27 smithi077 ceph-mon[33531]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:23:28.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:28 smithi001 ceph-mon[29731]: pgmap v520: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-05-04T03:23:28.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:28 smithi072 ceph-mon[33332]: pgmap v520: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-05-04T03:23:29.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:28 smithi077 ceph-mon[33531]: pgmap v520: 97 pgs: 97 active+clean; 91 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-05-04T03:23:29.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.922+0000 7f785ec86700 1 -- 172.21.15.1:0/366043804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7858108f20 msgr2=0x7f7858109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:29.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.922+0000 7f785ec86700 1 --2- 172.21.15.1:0/366043804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7858108f20 0x7f7858109330 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f78480099c0 tx=0x7f7848009cd0 comp rx=0 tx=0).stop 2024-05-04T03:23:29.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.922+0000 7f785ec86700 1 -- 172.21.15.1:0/366043804 shutdown_connections 2024-05-04T03:23:29.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.922+0000 7f785ec86700 1 --2- 172.21.15.1:0/366043804 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7858109870 0x7f78581111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:29.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.922+0000 7f785ec86700 1 --2- 172.21.15.1:0/366043804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78581045f0 0x7f7858104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:29.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.922+0000 7f785ec86700 1 --2- 172.21.15.1:0/366043804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7858108f20 0x7f7858109330 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:29.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.922+0000 7f785ec86700 1 -- 172.21.15.1:0/366043804 >> 172.21.15.1:0/366043804 conn(0x7f7858100040 msgr2=0x7f7858102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:29.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.922+0000 7f785ec86700 1 -- 172.21.15.1:0/366043804 shutdown_connections 2024-05-04T03:23:29.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.922+0000 7f785ec86700 1 -- 172.21.15.1:0/366043804 wait complete. 2024-05-04T03:23:29.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.923+0000 7f785ec86700 1 Processor -- start 2024-05-04T03:23:29.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.923+0000 7f785ec86700 1 -- start start 2024-05-04T03:23:29.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.923+0000 7f785ec86700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78581045f0 0x7f7858075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:29.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.923+0000 7f785ec86700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7858108f20 0x7f7858076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:29.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.923+0000 7f785ec86700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7858109870 0x7f7858076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:29.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.923+0000 7f785ec86700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f7858114480 con 0x7f7858108f20 2024-05-04T03:23:29.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.923+0000 7f785ec86700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7858114300 con 0x7f7858109870 2024-05-04T03:23:29.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.923+0000 7f785ec86700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7858114600 con 0x7f78581045f0 2024-05-04T03:23:29.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.923+0000 7f784ffff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7858108f20 0x7f7858076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:29.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.923+0000 7f785ca22700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78581045f0 0x7f7858075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:29.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.923+0000 7f785ca22700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78581045f0 0x7f7858075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.1:35324/0 (socket says 172.21.15.1:35324) 2024-05-04T03:23:29.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.923+0000 7f785ca22700 1 -- 172.21.15.1:0/4082901495 learned_addr learned my addr 172.21.15.1:0/4082901495 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:29.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.923+0000 7f785d223700 1 --2- 172.21.15.1:0/4082901495 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7858109870 0x7f7858076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:29.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.924+0000 7f785ca22700 1 -- 172.21.15.1:0/4082901495 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7858109870 msgr2=0x7f7858076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:29.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.924+0000 7f785ca22700 1 --2- 172.21.15.1:0/4082901495 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7858109870 0x7f7858076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:29.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.924+0000 7f785ca22700 1 -- 172.21.15.1:0/4082901495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7858108f20 msgr2=0x7f7858076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:29.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.924+0000 7f785ca22700 1 --2- 172.21.15.1:0/4082901495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7858108f20 0x7f7858076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:29.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.924+0000 7f785ca22700 1 -- 172.21.15.1:0/4082901495 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78581a91b0 con 0x7f78581045f0 2024-05-04T03:23:29.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.924+0000 7f785ca22700 1 --2- 172.21.15.1:0/4082901495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78581045f0 0x7f7858075ac0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f785400e9a0 tx=0x7f785400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:29.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.924+0000 7f784dffb700 1 -- 172.21.15.1:0/4082901495 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785400cbd0 con 0x7f78581045f0 2024-05-04T03:23:29.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.924+0000 7f785ec86700 1 -- 172.21.15.1:0/4082901495 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78581a94a0 con 0x7f78581045f0 2024-05-04T03:23:29.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.924+0000 7f784dffb700 1 -- 172.21.15.1:0/4082901495 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f785400cd70 con 0x7f78581045f0 2024-05-04T03:23:29.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.924+0000 7f784dffb700 1 -- 172.21.15.1:0/4082901495 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7854018680 con 0x7f78581045f0 2024-05-04T03:23:29.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.924+0000 7f785ec86700 1 -- 172.21.15.1:0/4082901495 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78581a99b0 con 0x7f78581045f0 2024-05-04T03:23:29.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.925+0000 7f785ec86700 1 -- 172.21.15.1:0/4082901495 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78580512c0 con 0x7f78581045f0 2024-05-04T03:23:29.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.927+0000 7f784dffb700 1 -- 172.21.15.1:0/4082901495 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f7854010430 con 0x7f78581045f0 2024-05-04T03:23:29.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.928+0000 7f784dffb700 1 --2- 172.21.15.1:0/4082901495 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f78400b2880 0x7f78400b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:29.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.928+0000 7f784ffff700 1 --2- 172.21.15.1:0/4082901495 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f78400b2880 0x7f78400b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:29.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.928+0000 7f784dffb700 1 -- 172.21.15.1:0/4082901495 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f78540e2920 con 0x7f78581045f0 2024-05-04T03:23:29.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.928+0000 7f784ffff700 1 --2- 172.21.15.1:0/4082901495 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f78400b2880 0x7f78400b4d90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7848000c00 tx=0x7f784801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:29.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:29.932+0000 7f784dffb700 1 -- 172.21.15.1:0/4082901495 <== mon.1 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f78540aa660 con 0x7f78581045f0 2024-05-04T03:23:30.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:30.240+0000 7f785ec86700 1 -- 172.21.15.1:0/4082901495 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f785802e5f0 con 0x7f78581045f0 2024-05-04T03:23:30.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:30.240+0000 7f784dffb700 1 -- 172.21.15.1:0/4082901495 <== mon.1 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11844 (secure 0 0 0) 0x7f78540a9f60 con 0x7f78581045f0 2024-05-04T03:23:30.243 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:23:30.243 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2186681910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2186681910},{"type":"v1","addr":"172.21.15.1:6835","nonce":2186681910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2747191047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2747191047},{"type":"v1","addr":"172.21.15.1:6837","nonce":2747191047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4115120540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4115120540},{"type":"v1","addr":"172.21.15.1:6839","nonce":4115120540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3405709780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3405709780},{"type":"v1","addr":"172.21.15.1:6841","nonce":3405709780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/3816098761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":3816098761},{"type":"v1","addr":"172.21.15.72:6833","nonce":3816098761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/379026128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":379026128},{"type":"v1","addr":"172.21.15.77:6833","nonce":379026128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:15:49.823471+0000","modified":"2024-05-04T03:23:29.384311+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24472,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/3930984976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3930984976},{"type":"v1","addr":"172.21.15.72:6835","nonce":3930984976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/4279951305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4279951305},{"type":"v1","addr":"172.21.15.77:6835","nonce":4279951305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2003521417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2003521417},{"type":"v1","addr":"172.21.15.77:6837","nonce":2003521417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/1417614203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1417614203},{"type":"v1","addr":"172.21.15.72:6837","nonce":1417614203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.77:6839/922723189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":922723189},{"type":"v1","addr":"172.21.15.77:6839","nonce":922723189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2976050650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2976050650},{"type":"v1","addr":"172.21.15.72:6839","nonce":2976050650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24467,24472,24494]},"id":1}]} 2024-05-04T03:23:30.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:30.243+0000 7f785ec86700 1 -- 172.21.15.1:0/4082901495 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f78400b2880 msgr2=0x7f78400b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:30.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:30.243+0000 7f785ec86700 1 --2- 172.21.15.1:0/4082901495 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f78400b2880 0x7f78400b4d90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7848000c00 tx=0x7f784801a040 comp rx=0 tx=0).stop 2024-05-04T03:23:30.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:30.243+0000 7f785ec86700 1 -- 172.21.15.1:0/4082901495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78581045f0 msgr2=0x7f7858075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:30.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:30.243+0000 7f785ec86700 1 --2- 172.21.15.1:0/4082901495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78581045f0 0x7f7858075ac0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f785400e9a0 tx=0x7f785400ecb0 comp rx=0 tx=0).stop 2024-05-04T03:23:30.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:30.243+0000 7f785ec86700 1 -- 172.21.15.1:0/4082901495 shutdown_connections 2024-05-04T03:23:30.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:30.243+0000 7f785ec86700 1 --2- 172.21.15.1:0/4082901495 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7858109870 0x7f7858076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:30.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:30.243+0000 7f785ec86700 1 --2- 172.21.15.1:0/4082901495 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f78400b2880 0x7f78400b4d90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:30.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:30.243+0000 7f785ec86700 1 --2- 172.21.15.1:0/4082901495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f78581045f0 0x7f7858075ac0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:30.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:30.243+0000 7f785ec86700 1 --2- 172.21.15.1:0/4082901495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7858108f20 0x7f7858076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:30.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:30.243+0000 7f785ec86700 1 -- 172.21.15.1:0/4082901495 >> 172.21.15.1:0/4082901495 conn(0x7f7858100040 msgr2=0x7f785810a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:30.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:30.244+0000 7f785ec86700 1 -- 172.21.15.1:0/4082901495 shutdown_connections 2024-05-04T03:23:30.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:30.244+0000 7f785ec86700 1 -- 172.21.15.1:0/4082901495 wait complete. 2024-05-04T03:23:30.247 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 32 2024-05-04T03:23:30.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:30 smithi072 ceph-mon[33332]: pgmap v521: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 212 KiB/s wr, 38 op/s 2024-05-04T03:23:30.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:30 smithi072 ceph-mon[33332]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:23:30.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:30 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/4082901495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:23:30.998 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:30 smithi001 ceph-mon[29731]: pgmap v521: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 212 KiB/s wr, 38 op/s 2024-05-04T03:23:30.998 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:30 smithi001 ceph-mon[29731]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:23:30.998 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:30 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/4082901495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:23:31.030 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0' 2024-05-04T03:23:31.030 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24494 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0 2024-05-04T03:23:31.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:30 smithi077 ceph-mon[33531]: pgmap v521: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 212 KiB/s wr, 38 op/s 2024-05-04T03:23:31.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:30 smithi077 ceph-mon[33531]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:23:31.179 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:30 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/4082901495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:23:31.313 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:23:32.652 INFO:tasks.workunit.client.0.smithi001.stdout:50000 write 0x5304f4 thru 0x53fa11 (0xf51e bytes) 2024-05-04T03:23:32.652 INFO:tasks.workunit.client.0.smithi001.stdout:All 50000 operations completed A-OK! 2024-05-04T03:23:32.656 INFO:tasks.workunit.client.0.smithi001.stderr:+ ./fsx -z 100MB -N 50000 -p 10000 -l 104857600 2024-05-04T03:23:32.658 INFO:tasks.workunit.client.0.smithi001.stdout:Seed set to 1 2024-05-04T03:23:32.815 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:32 smithi001 ceph-mon[29731]: pgmap v522: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 672 KiB/s wr, 90 op/s 2024-05-04T03:23:32.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:32 smithi072 ceph-mon[33332]: pgmap v522: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 672 KiB/s wr, 90 op/s 2024-05-04T03:23:33.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:32 smithi077 ceph-mon[33531]: pgmap v522: 97 pgs: 97 active+clean; 90 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 672 KiB/s wr, 90 op/s 2024-05-04T03:23:33.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.215+0000 7f0cd35b8700 1 -- 172.21.15.1:0/138606317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ccc107df0 msgr2=0x7f0ccc108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:33.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.215+0000 7f0cd35b8700 1 --2- 172.21.15.1:0/138606317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ccc107df0 0x7f0ccc108200 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0cc00099c0 tx=0x7f0cc0009cd0 comp rx=0 tx=0).stop 2024-05-04T03:23:33.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.215+0000 7f0cd35b8700 1 -- 172.21.15.1:0/138606317 shutdown_connections 2024-05-04T03:23:33.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.215+0000 7f0cd35b8700 1 --2- 172.21.15.1:0/138606317 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0ccc108740 0x7f0ccc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:33.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.215+0000 7f0cd35b8700 1 --2- 172.21.15.1:0/138606317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ccc1034c0 0x7f0ccc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:33.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.215+0000 7f0cd35b8700 1 --2- 172.21.15.1:0/138606317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ccc107df0 0x7f0ccc108200 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:33.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.215+0000 7f0cd35b8700 1 -- 172.21.15.1:0/138606317 >> 172.21.15.1:0/138606317 conn(0x7f0ccc0fedc0 msgr2=0x7f0ccc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:33.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.216+0000 7f0cd35b8700 1 -- 172.21.15.1:0/138606317 shutdown_connections 2024-05-04T03:23:33.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.216+0000 7f0cd35b8700 1 -- 172.21.15.1:0/138606317 wait complete. 2024-05-04T03:23:33.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.216+0000 7f0cd35b8700 1 Processor -- start 2024-05-04T03:23:33.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.216+0000 7f0cd35b8700 1 -- start start 2024-05-04T03:23:33.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.217+0000 7f0cd35b8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ccc1034c0 0x7f0ccc19f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:33.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.217+0000 7f0cd35b8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ccc107df0 0x7f0ccc19ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:33.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.217+0000 7f0cd35b8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0ccc108740 0x7f0ccc199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:33.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.217+0000 7f0cd35b8700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc113350 con 0x7f0ccc107df0 2024-05-04T03:23:33.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.217+0000 7f0cd35b8700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1131d0 con 0x7f0ccc108740 2024-05-04T03:23:33.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.217+0000 7f0cd35b8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1134d0 con 0x7f0ccc1034c0 2024-05-04T03:23:33.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.217+0000 7f0cd0b53700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ccc107df0 0x7f0ccc19ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:33.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.217+0000 7f0cd0b53700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ccc107df0 0x7f0ccc19ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46306/0 (socket says 172.21.15.1:46306) 2024-05-04T03:23:33.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.217+0000 7f0cd0b53700 1 -- 172.21.15.1:0/2275144227 learned_addr learned my addr 172.21.15.1:0/2275144227 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:33.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.217+0000 7f0cd1354700 1 --2- 172.21.15.1:0/2275144227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ccc1034c0 0x7f0ccc19f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:33.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.217+0000 7f0cd1b55700 1 --2- 172.21.15.1:0/2275144227 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0ccc108740 0x7f0ccc199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:33.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.217+0000 7f0cd0b53700 1 -- 172.21.15.1:0/2275144227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ccc1034c0 msgr2=0x7f0ccc19f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:33.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.217+0000 7f0cd0b53700 1 --2- 172.21.15.1:0/2275144227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ccc1034c0 0x7f0ccc19f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:33.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.217+0000 7f0cd0b53700 1 -- 172.21.15.1:0/2275144227 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0ccc108740 msgr2=0x7f0ccc199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:33.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.217+0000 7f0cd0b53700 1 --2- 172.21.15.1:0/2275144227 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0ccc108740 0x7f0ccc199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:33.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.217+0000 7f0cd0b53700 1 -- 172.21.15.1:0/2275144227 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ccc19a280 con 0x7f0ccc107df0 2024-05-04T03:23:33.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.217+0000 7f0cd1b55700 1 --2- 172.21.15.1:0/2275144227 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0ccc108740 0x7f0ccc199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:23:33.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.218+0000 7f0cd0b53700 1 --2- 172.21.15.1:0/2275144227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ccc107df0 0x7f0ccc19ff00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0cc00045e0 tx=0x7f0cc0004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:33.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.218+0000 7f0cca7fc700 1 -- 172.21.15.1:0/2275144227 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0026070 con 0x7f0ccc107df0 2024-05-04T03:23:33.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.218+0000 7f0cca7fc700 1 -- 172.21.15.1:0/2275144227 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cc000f460 con 0x7f0ccc107df0 2024-05-04T03:23:33.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.218+0000 7f0cca7fc700 1 -- 172.21.15.1:0/2275144227 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0009e60 con 0x7f0ccc107df0 2024-05-04T03:23:33.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.218+0000 7f0cd35b8700 1 -- 172.21.15.1:0/2275144227 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ccc19a510 con 0x7f0ccc107df0 2024-05-04T03:23:33.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.218+0000 7f0cd35b8700 1 -- 172.21.15.1:0/2275144227 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ccc19a9a0 con 0x7f0ccc107df0 2024-05-04T03:23:33.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.221+0000 7f0cca7fc700 1 -- 172.21.15.1:0/2275144227 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f0cc0029070 con 0x7f0ccc107df0 2024-05-04T03:23:33.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.222+0000 7f0cca7fc700 1 --2- 172.21.15.1:0/2275144227 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0cb40b2880 0x7f0cb40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:33.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.222+0000 7f0cd1354700 1 --2- 172.21.15.1:0/2275144227 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0cb40b2880 0x7f0cb40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:33.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.222+0000 7f0cca7fc700 1 -- 172.21.15.1:0/2275144227 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f0cc00e2a80 con 0x7f0ccc107df0 2024-05-04T03:23:33.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.222+0000 7f0cd1354700 1 --2- 172.21.15.1:0/2275144227 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0cb40b2880 0x7f0cb40b4d90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc00f540 tx=0x7f0cbc00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:33.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.223+0000 7f0ce1fccb80 1 Processor -- start 2024-05-04T03:23:33.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.223+0000 7f0ce1fccb80 1 -- start start 2024-05-04T03:23:33.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.223+0000 7f0ce1fccb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a6d14660 0x55a1a6d14a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:33.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.223+0000 7f0ce1fccb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1a6d14fb0 0x55a1a6d1f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:33.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.223+0000 7f0ce1fccb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a6d1f970 0x55a1a6d21d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:33.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.223+0000 7f0ce1fccb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a1a6c379f0 con 0x55a1a6d14660 2024-05-04T03:23:33.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.223+0000 7f0ce1fccb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55a1a6bd6950 con 0x55a1a6d14fb0 2024-05-04T03:23:33.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.223+0000 7f0ce1fccb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a1a6c339b0 con 0x55a1a6d1f970 2024-05-04T03:23:33.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.223+0000 7f0cd1354700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a6d14660 0x55a1a6d14a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:33.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.224+0000 7f0cd1354700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a6d14660 0x55a1a6d14a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46322/0 (socket says 172.21.15.1:46322) 2024-05-04T03:23:33.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.224+0000 7f0cd1354700 1 -- 172.21.15.1:0/2897905790 learned_addr learned my addr 172.21.15.1:0/2897905790 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:33.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.224+0000 7f0cd0b53700 1 --2- 172.21.15.1:0/2897905790 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1a6d14fb0 0x55a1a6d1f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:33.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.224+0000 7f0cd1b55700 1 --2- 172.21.15.1:0/2897905790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a6d1f970 0x55a1a6d21d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:33.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.224+0000 7f0cd1354700 1 -- 172.21.15.1:0/2897905790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a6d1f970 msgr2=0x55a1a6d21d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:33.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.224+0000 7f0cd1354700 1 --2- 172.21.15.1:0/2897905790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a6d1f970 0x55a1a6d21d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:33.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.224+0000 7f0cd1354700 1 -- 172.21.15.1:0/2897905790 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1a6d14fb0 msgr2=0x55a1a6d1f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:33.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.224+0000 7f0cd1354700 1 --2- 172.21.15.1:0/2897905790 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1a6d14fb0 0x55a1a6d1f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:33.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.224+0000 7f0cd1354700 1 -- 172.21.15.1:0/2897905790 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1a6b84a60 con 0x55a1a6d14660 2024-05-04T03:23:33.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.224+0000 7f0cd1354700 1 --2- 172.21.15.1:0/2897905790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a6d14660 0x55a1a6d14a70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc003d20 tx=0x7f0cbc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbeb14ea3dab2b4c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:33.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.224+0000 7f0cb37fe700 1 -- 172.21.15.1:0/2897905790 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc027070 con 0x55a1a6d14660 2024-05-04T03:23:33.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.224+0000 7f0cb37fe700 1 -- 172.21.15.1:0/2897905790 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cbc01e040 con 0x55a1a6d14660 2024-05-04T03:23:33.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.224+0000 7f0cb37fe700 1 -- 172.21.15.1:0/2897905790 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc01d050 con 0x55a1a6d14660 2024-05-04T03:23:33.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.225+0000 7f0ce1fccb80 1 -- 172.21.15.1:0/2897905790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a6d14660 msgr2=0x55a1a6d14a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:33.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.225+0000 7f0ce1fccb80 1 --2- 172.21.15.1:0/2897905790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a6d14660 0x55a1a6d14a70 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc003d20 tx=0x7f0cbc014000 comp rx=0 tx=0).stop 2024-05-04T03:23:33.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.226+0000 7f0ce1fccb80 1 -- 172.21.15.1:0/2897905790 shutdown_connections 2024-05-04T03:23:33.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.226+0000 7f0ce1fccb80 1 --2- 172.21.15.1:0/2897905790 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a6d1f970 0x55a1a6d21d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:33.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.226+0000 7f0ce1fccb80 1 --2- 172.21.15.1:0/2897905790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a6d14660 0x55a1a6d14a70 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:33.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.226+0000 7f0ce1fccb80 1 --2- 172.21.15.1:0/2897905790 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1a6d14fb0 0x55a1a6d1f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:33.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.226+0000 7f0ce1fccb80 1 -- 172.21.15.1:0/2897905790 >> 172.21.15.1:0/2897905790 conn(0x55a1a6b97fd0 msgr2=0x55a1a6d10690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:33.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.226+0000 7f0ce1fccb80 1 -- 172.21.15.1:0/2897905790 shutdown_connections 2024-05-04T03:23:33.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.226+0000 7f0ce1fccb80 1 -- 172.21.15.1:0/2897905790 wait complete. 2024-05-04T03:23:33.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.226+0000 7f0ce1fccb80 1 -- start start 2024-05-04T03:23:33.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.226+0000 7f0ce1fccb80 1 Processor -- start 2024-05-04T03:23:33.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.227+0000 7f0ce1fccb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a6d14660 0x55a1a6d1c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:33.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.227+0000 7f0ce1fccb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a6d14fb0 0x55a1a6d1ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:33.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.227+0000 7f0ce1fccb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1a6d1f970 0x55a1a6d1d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:33.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.227+0000 7f0ce1fccb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55a1a6d22700 con 0x55a1a6d14fb0 2024-05-04T03:23:33.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.227+0000 7f0ce1fccb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55a1a6b9ddc0 con 0x55a1a6d1f970 2024-05-04T03:23:33.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.227+0000 7f0ce1fccb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a1a6d22880 con 0x55a1a6d14660 2024-05-04T03:23:33.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.227+0000 7f0ce1fccb80 10 client.0 _ll_register_callbacks cb 0x55a1a6bcfab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:23:33.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.227+0000 7f0cd0b53700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a6d14fb0 0x55a1a6d1ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:33.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.227+0000 7f0cd0b53700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a6d14fb0 0x55a1a6d1ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46334/0 (socket says 172.21.15.1:46334) 2024-05-04T03:23:33.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.227+0000 7f0cd0b53700 1 -- 172.21.15.1:0/1738865767 learned_addr learned my addr 172.21.15.1:0/1738865767 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:33.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.227+0000 7f0cd1354700 1 --2- 172.21.15.1:0/1738865767 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a6d14660 0x55a1a6d1c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:33.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.227+0000 7f0cd1b55700 1 --2- 172.21.15.1:0/1738865767 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1a6d1f970 0x55a1a6d1d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:33.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.227+0000 7f0cd0b53700 1 -- 172.21.15.1:0/1738865767 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a6d14660 msgr2=0x55a1a6d1c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:33.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.227+0000 7f0cd0b53700 1 --2- 172.21.15.1:0/1738865767 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a6d14660 0x55a1a6d1c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:33.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.227+0000 7f0cd0b53700 1 -- 172.21.15.1:0/1738865767 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1a6d1f970 msgr2=0x55a1a6d1d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:33.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.227+0000 7f0cd0b53700 1 --2- 172.21.15.1:0/1738865767 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1a6d1f970 0x55a1a6d1d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:33.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.227+0000 7f0cd0b53700 1 -- 172.21.15.1:0/1738865767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1a6bd3820 con 0x55a1a6d14fb0 2024-05-04T03:23:33.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.227+0000 7f0cd1b55700 1 --2- 172.21.15.1:0/1738865767 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1a6d1f970 0x55a1a6d1d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:23:33.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.227+0000 7f0cd1354700 1 --2- 172.21.15.1:0/1738865767 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a6d14660 0x55a1a6d1c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:23:33.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.227+0000 7f0cd0b53700 1 --2- 172.21.15.1:0/1738865767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a6d14fb0 0x55a1a6d1ce90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000c010 tx=0x7f0cc000fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113d444f55aac0e server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:33.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.227+0000 7f0cb0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:23:33.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.227+0000 7f0cb0ff9700 1 -- 172.21.15.1:0/1738865767 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc00aa850 con 0x55a1a6d14fb0 2024-05-04T03:23:33.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.228+0000 7f0ce1fccb80 1 -- 172.21.15.1:0/1738865767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cbc01e040 con 0x55a1a6d14fb0 2024-05-04T03:23:33.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.228+0000 7f0cb0ff9700 1 -- 172.21.15.1:0/1738865767 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f0cc0055be0 con 0x55a1a6d14fb0 2024-05-04T03:23:33.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.228+0000 7f0cb0ff9700 1 -- 172.21.15.1:0/1738865767 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0cc00aa150 con 0x55a1a6d14fb0 2024-05-04T03:23:33.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.228+0000 7f0ce1fccb80 10 client.15744 fetch_fsmap learned FSMap version 32 2024-05-04T03:23:33.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.228+0000 7f0ce1fccb80 1 -- 172.21.15.1:0/1738865767 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a1a6d2d350 con 0x55a1a6d14fb0 2024-05-04T03:23:33.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.228+0000 7f0cb0ff9700 1 -- 172.21.15.1:0/1738865767 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cc00ae500 con 0x55a1a6d14fb0 2024-05-04T03:23:33.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.228+0000 7f0cb0ff9700 1 -- 172.21.15.1:0/1738865767 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f0cc00aa3b0 con 0x55a1a6d14fb0 2024-05-04T03:23:33.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.228+0000 7f0ce1fccb80 10 client.15744 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:23:33.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.228+0000 7f0ce1fccb80 10 client.15744 resolve_mds: validated gid 24494 aka daemon mds.b 2024-05-04T03:23:33.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.228+0000 7f0ce1fccb80 20 client.15744 populate_metadata read hostname 'smithi001' 2024-05-04T03:23:33.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.228+0000 7f0ce1fccb80 1 --2- 172.21.15.1:0/1738865767 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a1a6d2fa80 0x55a1a6d31ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:33.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.228+0000 7f0ce1fccb80 4 client.15744 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:23:33.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.228+0000 7f0ce1fccb80 1 -- 172.21.15.1:0/1738865767 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0cbc01e040 con 0x55a1a6d2fa80 2024-05-04T03:23:33.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.229+0000 7f0cd1354700 1 --2- 172.21.15.1:0/1738865767 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a1a6d2fa80 0x55a1a6d31ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:33.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.229+0000 7f0cb0ff9700 10 client.15744 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:23:33.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.229+0000 7f0cd1354700 1 --2- 172.21.15.1:0/1738865767 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a1a6d2fa80 0x55a1a6d31ef0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbc38dbc9815356 server_cookie=bc44da36a7940735 in_seq=0 out_seq=0 2024-05-04T03:23:33.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.230+0000 7f0cb0ff9700 1 -- 172.21.15.1:0/1738865767 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f0cbc01e040 con 0x55a1a6d2fa80 2024-05-04T03:23:33.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.230+0000 7f0cb0ff9700 10 client.15744 handle_command_reply: tid=0 2024-05-04T03:23:33.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.230+0000 7f0ce1fccb80 1 client.15744 shutdown 2024-05-04T03:23:33.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.230+0000 7f0ce1fccb80 1 -- 172.21.15.1:0/1738865767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a6d14fb0 msgr2=0x55a1a6d1ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:33.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.230+0000 7f0ce1fccb80 1 --2- 172.21.15.1:0/1738865767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a6d14fb0 0x55a1a6d1ce90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000c010 tx=0x7f0cc000fc10 comp rx=0 tx=0).stop 2024-05-04T03:23:33.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.230+0000 7f0ce1fccb80 1 -- 172.21.15.1:0/1738865767 shutdown_connections 2024-05-04T03:23:33.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.230+0000 7f0ce1fccb80 1 --2- 172.21.15.1:0/1738865767 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a1a6d2fa80 0x55a1a6d31ef0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:33.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.230+0000 7f0ce1fccb80 1 --2- 172.21.15.1:0/1738865767 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1a6d1f970 0x55a1a6d1d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:33.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.230+0000 7f0ce1fccb80 1 --2- 172.21.15.1:0/1738865767 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a6d14660 0x55a1a6d1c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:33.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.230+0000 7f0ce1fccb80 1 --2- 172.21.15.1:0/1738865767 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a6d14fb0 0x55a1a6d1ce90 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:33.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.230+0000 7f0cb0ff9700 0 client.15744 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:23:33.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.230+0000 7f0ce1fccb80 1 -- 172.21.15.1:0/1738865767 >> 172.21.15.1:0/1738865767 conn(0x55a1a6b97fd0 msgr2=0x55a1a6d16230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:33.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.230+0000 7f0ce1fccb80 1 -- 172.21.15.1:0/1738865767 shutdown_connections 2024-05-04T03:23:33.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.230+0000 7f0ce1fccb80 1 -- 172.21.15.1:0/1738865767 wait complete. 2024-05-04T03:23:33.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.230+0000 7f0ce1fccb80 20 client.15744 trim_cache size 0 max 16384 2024-05-04T03:23:33.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.266+0000 7f0ce1fccb80 1 Processor -- start 2024-05-04T03:23:33.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.266+0000 7f0ce1fccb80 1 -- start start 2024-05-04T03:23:33.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.266+0000 7f0ce1fccb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1a6d14660 0x55a1a6d18380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:33.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.266+0000 7f0ce1fccb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a6d14fb0 0x55a1a6d2b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:33.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.266+0000 7f0ce1fccb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a6d1f970 0x55a1a6d376f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:33.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.266+0000 7f0ce1fccb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1aefd0 con 0x55a1a6d1f970 2024-05-04T03:23:33.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.266+0000 7f0ce1fccb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1ac860 con 0x55a1a6d14660 2024-05-04T03:23:33.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.266+0000 7f0ce1fccb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1aa6d0 con 0x55a1a6d14fb0 2024-05-04T03:23:33.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.267+0000 7f0cd1b55700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a6d1f970 0x55a1a6d376f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:33.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.267+0000 7f0cd1b55700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a6d1f970 0x55a1a6d376f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46344/0 (socket says 172.21.15.1:46344) 2024-05-04T03:23:33.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.267+0000 7f0cd1354700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1a6d14660 0x55a1a6d18380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:33.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.267+0000 7f0cd1b55700 1 -- 172.21.15.1:0/3327470229 learned_addr learned my addr 172.21.15.1:0/3327470229 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:33.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.267+0000 7f0cd0b53700 1 --2- 172.21.15.1:0/3327470229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a6d14fb0 0x55a1a6d2b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:33.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.267+0000 7f0cd1354700 1 -- 172.21.15.1:0/3327470229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a6d14fb0 msgr2=0x55a1a6d2b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:33.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.267+0000 7f0cd1354700 1 --2- 172.21.15.1:0/3327470229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a6d14fb0 0x55a1a6d2b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:33.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.267+0000 7f0cd1354700 1 -- 172.21.15.1:0/3327470229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a6d1f970 msgr2=0x55a1a6d376f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:33.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.267+0000 7f0cd1354700 1 --2- 172.21.15.1:0/3327470229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a6d1f970 0x55a1a6d376f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:33.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.267+0000 7f0cd1354700 1 -- 172.21.15.1:0/3327470229 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1a6d0c340 con 0x55a1a6d14660 2024-05-04T03:23:33.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.267+0000 7f0cd1b55700 1 --2- 172.21.15.1:0/3327470229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a6d1f970 0x55a1a6d376f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:23:33.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.267+0000 7f0cd1354700 1 --2- 172.21.15.1:0/3327470229 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1a6d14660 0x55a1a6d18380 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc010010 tx=0x7f0cbc0049a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39a519ff260800c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:33.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.267+0000 7f0cb2ffd700 1 -- 172.21.15.1:0/3327470229 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc01d040 con 0x55a1a6d14660 2024-05-04T03:23:33.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.268+0000 7f0cb2ffd700 1 -- 172.21.15.1:0/3327470229 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cbc028040 con 0x55a1a6d14660 2024-05-04T03:23:33.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.268+0000 7f0cb2ffd700 1 -- 172.21.15.1:0/3327470229 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc027040 con 0x55a1a6d14660 2024-05-04T03:23:33.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.268+0000 7f0ce1fccb80 1 -- 172.21.15.1:0/3327470229 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1a6d14660 msgr2=0x55a1a6d18380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:33.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.268+0000 7f0ce1fccb80 1 --2- 172.21.15.1:0/3327470229 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1a6d14660 0x55a1a6d18380 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc010010 tx=0x7f0cbc0049a0 comp rx=0 tx=0).stop 2024-05-04T03:23:33.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.268+0000 7f0ce1fccb80 1 -- 172.21.15.1:0/3327470229 shutdown_connections 2024-05-04T03:23:33.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.268+0000 7f0ce1fccb80 1 --2- 172.21.15.1:0/3327470229 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1a6d14660 0x55a1a6d18380 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:33.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.268+0000 7f0ce1fccb80 1 --2- 172.21.15.1:0/3327470229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a6d1f970 0x55a1a6d376f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:33.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.268+0000 7f0ce1fccb80 1 --2- 172.21.15.1:0/3327470229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a6d14fb0 0x55a1a6d2b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:33.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.268+0000 7f0ce1fccb80 1 -- 172.21.15.1:0/3327470229 >> 172.21.15.1:0/3327470229 conn(0x55a1a6b97fd0 msgr2=0x55a1a6d103e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:33.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.269+0000 7f0ce1fccb80 1 -- 172.21.15.1:0/3327470229 shutdown_connections 2024-05-04T03:23:33.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.269+0000 7f0ce1fccb80 1 -- 172.21.15.1:0/3327470229 wait complete. 2024-05-04T03:23:33.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.269+0000 7f0ce1fccb80 1 -- start start 2024-05-04T03:23:33.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.269+0000 7f0ce1fccb80 1 Processor -- start 2024-05-04T03:23:33.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.270+0000 7f0ce1fccb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a6d14660 0x55a1a6d24ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:33.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.270+0000 7f0ce1fccb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a6d14fb0 0x55a1a6d25020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:33.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.270+0000 7f0ce1fccb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1a6d1f970 0x55a1a6d25560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:33.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.270+0000 7f0ce1fccb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1aad20 con 0x55a1a6d14fb0 2024-05-04T03:23:33.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.270+0000 7f0ce1fccb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1ac190 con 0x55a1a6d1f970 2024-05-04T03:23:33.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.270+0000 7f0ce1fccb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1ab3f0 con 0x55a1a6d14660 2024-05-04T03:23:33.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.270+0000 7f0cd0b53700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a6d14fb0 0x55a1a6d25020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:33.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.270+0000 7f0cd1354700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a6d14660 0x55a1a6d24ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:33.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.270+0000 7f0cd0b53700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a6d14fb0 0x55a1a6d25020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46346/0 (socket says 172.21.15.1:46346) 2024-05-04T03:23:33.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.270+0000 7f0cd0b53700 1 -- 172.21.15.1:0/881890216 learned_addr learned my addr 172.21.15.1:0/881890216 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:33.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.270+0000 7f0ce1fccb80 10 client.0 _ll_register_callbacks cb 0x55a1a6bcfab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:23:33.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.270+0000 7f0cd1b55700 1 --2- 172.21.15.1:0/881890216 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1a6d1f970 0x55a1a6d25560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:33.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.270+0000 7f0cd0b53700 1 -- 172.21.15.1:0/881890216 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a6d14660 msgr2=0x55a1a6d24ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:33.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.270+0000 7f0cd0b53700 1 --2- 172.21.15.1:0/881890216 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a6d14660 0x55a1a6d24ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:33.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.270+0000 7f0cd0b53700 1 -- 172.21.15.1:0/881890216 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1a6d1f970 msgr2=0x55a1a6d25560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:33.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.270+0000 7f0cd0b53700 1 --2- 172.21.15.1:0/881890216 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1a6d1f970 0x55a1a6d25560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:33.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.270+0000 7f0cd0b53700 1 -- 172.21.15.1:0/881890216 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1a6d9e6c0 con 0x55a1a6d14fb0 2024-05-04T03:23:33.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.270+0000 7f0cd1354700 1 --2- 172.21.15.1:0/881890216 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a6d14660 0x55a1a6d24ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:23:33.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.270+0000 7f0cd0b53700 1 --2- 172.21.15.1:0/881890216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a6d14fb0 0x55a1a6d25020 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0003760 tx=0x7f0cc00a9da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586397b5836d1c3a server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:33.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.270+0000 7f0cb0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:23:33.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.270+0000 7f0cb0ff9700 1 -- 172.21.15.1:0/881890216 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc00aa850 con 0x55a1a6d14fb0 2024-05-04T03:23:33.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.271+0000 7f0ce1fccb80 1 -- 172.21.15.1:0/881890216 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cbc028040 con 0x55a1a6d14fb0 2024-05-04T03:23:33.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.271+0000 7f0cb0ff9700 1 -- 172.21.15.1:0/881890216 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f0cc00aa3b0 con 0x55a1a6d14fb0 2024-05-04T03:23:33.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.271+0000 7f0cb0ff9700 1 -- 172.21.15.1:0/881890216 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0cc00aa5b0 con 0x55a1a6d14fb0 2024-05-04T03:23:33.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.271+0000 7f0ce1fccb80 10 client.15750 fetch_fsmap learned FSMap version 32 2024-05-04T03:23:33.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.271+0000 7f0ce1fccb80 1 -- 172.21.15.1:0/881890216 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a1a6d92350 con 0x55a1a6d14fb0 2024-05-04T03:23:33.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.271+0000 7f0cb0ff9700 1 -- 172.21.15.1:0/881890216 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cbc028040 con 0x55a1a6d14fb0 2024-05-04T03:23:33.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.271+0000 7f0cb0ff9700 1 -- 172.21.15.1:0/881890216 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f0cc0069bc0 con 0x55a1a6d14fb0 2024-05-04T03:23:33.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.271+0000 7f0ce1fccb80 10 client.15750 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:23:33.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.271+0000 7f0ce1fccb80 10 client.15750 resolve_mds: validated gid 24494 aka daemon mds.b 2024-05-04T03:23:33.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.271+0000 7f0ce1fccb80 20 client.15750 populate_metadata read hostname 'smithi001' 2024-05-04T03:23:33.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.271+0000 7f0ce1fccb80 1 --2- 172.21.15.1:0/881890216 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a1a6d92940 0x55a1a6d94d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:33.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.271+0000 7f0ce1fccb80 4 client.15750 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}] 2024-05-04T03:23:33.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.271+0000 7f0ce1fccb80 1 -- 172.21.15.1:0/881890216 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}) -- 0x55a1a6c453d0 con 0x55a1a6d92940 2024-05-04T03:23:33.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.271+0000 7f0cd1354700 1 --2- 172.21.15.1:0/881890216 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a1a6d92940 0x55a1a6d94d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:33.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.272+0000 7f0cb0ff9700 10 client.15750 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:23:33.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:33.272+0000 7f0cd1354700 1 --2- 172.21.15.1:0/881890216 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a1a6d92940 0x55a1a6d94d60 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4c42fc6452b7b8e server_cookie=76b6c5b9d806e36 in_seq=0 out_seq=0 2024-05-04T03:23:33.884 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-05-04T03:23:33.884 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-05-04T03:23:34.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:34.890+0000 7f0cb0ff9700 1 -- 172.21.15.1:0/881890216 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55a1a6c453d0 con 0x55a1a6d92940 2024-05-04T03:23:34.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:34.890+0000 7f0cb0ff9700 10 client.15750 handle_command_reply: tid=0 2024-05-04T03:23:34.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:34.890+0000 7f0ce1fccb80 1 client.15750 shutdown 2024-05-04T03:23:34.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:34.891+0000 7f0ce1fccb80 1 -- 172.21.15.1:0/881890216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a6d14fb0 msgr2=0x55a1a6d25020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:34.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:34.891+0000 7f0ce1fccb80 1 --2- 172.21.15.1:0/881890216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a6d14fb0 0x55a1a6d25020 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0003760 tx=0x7f0cc00a9da0 comp rx=0 tx=0).stop 2024-05-04T03:23:34.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:34.891+0000 7f0ce1fccb80 1 -- 172.21.15.1:0/881890216 shutdown_connections 2024-05-04T03:23:34.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:34.891+0000 7f0ce1fccb80 1 --2- 172.21.15.1:0/881890216 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55a1a6d92940 0x55a1a6d94d60 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:34.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:34.891+0000 7f0ce1fccb80 1 --2- 172.21.15.1:0/881890216 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1a6d1f970 0x55a1a6d25560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:34.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:34.891+0000 7f0ce1fccb80 1 --2- 172.21.15.1:0/881890216 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1a6d14660 0x55a1a6d24ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:34.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:34.891+0000 7f0ce1fccb80 1 --2- 172.21.15.1:0/881890216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a6d14fb0 0x55a1a6d25020 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:34.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:34.891+0000 7f0ce1fccb80 1 -- 172.21.15.1:0/881890216 >> 172.21.15.1:0/881890216 conn(0x55a1a6b97fd0 msgr2=0x55a1a6d17370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:34.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:34.891+0000 7f0cb0ff9700 0 client.15750 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:23:34.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:34.891+0000 7f0ce1fccb80 1 -- 172.21.15.1:0/881890216 shutdown_connections 2024-05-04T03:23:34.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:34.891+0000 7f0ce1fccb80 1 -- 172.21.15.1:0/881890216 wait complete. 2024-05-04T03:23:34.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:34.891+0000 7f0ce1fccb80 20 client.15750 trim_cache size 0 max 16384 2024-05-04T03:23:34.895 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-04T03:23:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 32, 2024-05-04T03:23:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24494, 2024-05-04T03:23:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 14, 2024-05-04T03:23:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-05-04T03:23:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: "961ff620": { 2024-05-04T03:23:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 14, 2024-05-04T03:23:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-05-04T03:23:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-05-04T03:23:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-05-04T03:23:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-05-04T03:23:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:23:34.895 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 60.0, 2024-05-04T03:23:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 90.0, 2024-05-04T03:23:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-05-04T03:23:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-05-04T03:23:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-05-04T03:23:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-05-04T03:23:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "QUIESCED", 2024-05-04T03:23:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-05-04T03:23:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:23:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:23:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:23:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:23:34.896 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:23:34.896 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-04T03:23:34.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:34.892+0000 7f0cd35b8700 1 -- 172.21.15.1:0/2275144227 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0cb40b2880 msgr2=0x7f0cb40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:34.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:34.892+0000 7f0cd35b8700 1 --2- 172.21.15.1:0/2275144227 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0cb40b2880 0x7f0cb40b4d90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc00f540 tx=0x7f0cbc00b040 comp rx=0 tx=0).stop 2024-05-04T03:23:34.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:34.892+0000 7f0cd35b8700 1 -- 172.21.15.1:0/2275144227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ccc107df0 msgr2=0x7f0ccc19ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:34.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:34.892+0000 7f0cd35b8700 1 --2- 172.21.15.1:0/2275144227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ccc107df0 0x7f0ccc19ff00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0cc00045e0 tx=0x7f0cc0004610 comp rx=0 tx=0).stop 2024-05-04T03:23:34.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:34.892+0000 7f0cd35b8700 1 -- 172.21.15.1:0/2275144227 shutdown_connections 2024-05-04T03:23:34.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:34.892+0000 7f0cd35b8700 1 --2- 172.21.15.1:0/2275144227 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0ccc108740 0x7f0ccc199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:34.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:34.892+0000 7f0cd35b8700 1 --2- 172.21.15.1:0/2275144227 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0cb40b2880 0x7f0cb40b4d90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:34.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:34.892+0000 7f0cd35b8700 1 --2- 172.21.15.1:0/2275144227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ccc1034c0 0x7f0ccc19f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:34.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:34.892+0000 7f0cd35b8700 1 --2- 172.21.15.1:0/2275144227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ccc107df0 0x7f0ccc19ff00 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:34.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:34.892+0000 7f0cd35b8700 1 -- 172.21.15.1:0/2275144227 >> 172.21.15.1:0/2275144227 conn(0x7f0ccc0fedc0 msgr2=0x7f0ccc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:34.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:34.892+0000 7f0cd35b8700 1 -- 172.21.15.1:0/2275144227 shutdown_connections 2024-05-04T03:23:34.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:23:34.892+0000 7f0cd35b8700 1 -- 172.21.15.1:0/2275144227 wait complete. 2024-05-04T03:23:34.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:34 smithi072 ceph-mon[33332]: pgmap v523: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 895 KiB/s wr, 166 op/s 2024-05-04T03:23:35.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:34 smithi001 ceph-mon[29731]: pgmap v523: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 895 KiB/s wr, 166 op/s 2024-05-04T03:23:35.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:34 smithi077 ceph-mon[33531]: pgmap v523: 97 pgs: 97 active+clean; 91 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 895 KiB/s wr, 166 op/s 2024-05-04T03:23:35.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.419+0000 7f7823ea8640 1 -- 172.21.15.1:0/132113516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f781c109ba0 msgr2=0x7f781c10bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:35.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.419+0000 7f7823ea8640 1 --2- 172.21.15.1:0/132113516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f781c109ba0 0x7f781c10bfc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7814009ca0 tx=0x7f781402a8c0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.419+0000 7f7823ea8640 1 -- 172.21.15.1:0/132113516 shutdown_connections 2024-05-04T03:23:35.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.419+0000 7f7823ea8640 1 --2- 172.21.15.1:0/132113516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f781c109ba0 0x7f781c10bfc0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.419+0000 7f7823ea8640 1 --2- 172.21.15.1:0/132113516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f781c104ea0 0x7f781c1052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.419+0000 7f7823ea8640 1 --2- 172.21.15.1:0/132113516 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f781c103c60 0x7f781c104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.419+0000 7f7823ea8640 1 -- 172.21.15.1:0/132113516 >> 172.21.15.1:0/132113516 conn(0x7f781c0fee70 msgr2=0x7f781c101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:35.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.419+0000 7f7823ea8640 1 -- 172.21.15.1:0/132113516 shutdown_connections 2024-05-04T03:23:35.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.420+0000 7f7823ea8640 1 -- 172.21.15.1:0/132113516 wait complete. 2024-05-04T03:23:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.420+0000 7f7823ea8640 1 Processor -- start 2024-05-04T03:23:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.420+0000 7f7823ea8640 1 -- start start 2024-05-04T03:23:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.420+0000 7f7823ea8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f781c103c60 0x7f781c19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.420+0000 7f7823ea8640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f781c104ea0 0x7f781c19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.420+0000 7f7823ea8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f781c109ba0 0x7f781c1a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.420+0000 7f7823ea8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f781c10f790 con 0x7f781c103c60 2024-05-04T03:23:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.420+0000 7f7823ea8640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f781c10f610 con 0x7f781c104ea0 2024-05-04T03:23:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.420+0000 7f7823ea8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f781c10f910 con 0x7f781c109ba0 2024-05-04T03:23:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.421+0000 7f7821c44640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f781c103c60 0x7f781c19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.421+0000 7f7821c44640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f781c103c60 0x7f781c19a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46364/0 (socket says 172.21.15.1:46364) 2024-05-04T03:23:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.421+0000 7f7821c44640 1 -- 172.21.15.1:0/2721818400 learned_addr learned my addr 172.21.15.1:0/2721818400 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.421+0000 7f7821443640 1 --2- 172.21.15.1:0/2721818400 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f781c104ea0 0x7f781c19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.421+0000 7f7822445640 1 --2- 172.21.15.1:0/2721818400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f781c109ba0 0x7f781c1a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.421+0000 7f7821c44640 1 -- 172.21.15.1:0/2721818400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f781c109ba0 msgr2=0x7f781c1a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.421+0000 7f7821c44640 1 --2- 172.21.15.1:0/2721818400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f781c109ba0 0x7f781c1a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.421+0000 7f7821c44640 1 -- 172.21.15.1:0/2721818400 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f781c104ea0 msgr2=0x7f781c19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.421+0000 7f7821c44640 1 --2- 172.21.15.1:0/2721818400 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f781c104ea0 0x7f781c19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.421+0000 7f7821c44640 1 -- 172.21.15.1:0/2721818400 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781c1a1d80 con 0x7f781c103c60 2024-05-04T03:23:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.421+0000 7f7821443640 1 --2- 172.21.15.1:0/2721818400 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f781c104ea0 0x7f781c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:23:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.421+0000 7f7821c44640 1 --2- 172.21.15.1:0/2721818400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f781c103c60 0x7f781c19a240 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7818005fd0 tx=0x7f781800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.421+0000 7f7812ffd640 1 -- 172.21.15.1:0/2721818400 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818017070 con 0x7f781c103c60 2024-05-04T03:23:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.421+0000 7f7812ffd640 1 -- 172.21.15.1:0/2721818400 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7818009c30 con 0x7f781c103c60 2024-05-04T03:23:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.421+0000 7f7823ea8640 1 -- 172.21.15.1:0/2721818400 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f781c1a2070 con 0x7f781c103c60 2024-05-04T03:23:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.421+0000 7f7812ffd640 1 -- 172.21.15.1:0/2721818400 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818003d10 con 0x7f781c103c60 2024-05-04T03:23:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.421+0000 7f7823ea8640 1 -- 172.21.15.1:0/2721818400 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f781c1a25c0 con 0x7f781c103c60 2024-05-04T03:23:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.424+0000 7f782ae3a740 1 Processor -- start 2024-05-04T03:23:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.425+0000 7f782ae3a740 1 -- start start 2024-05-04T03:23:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.425+0000 7f7812ffd640 1 -- 172.21.15.1:0/2721818400 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f7818003eb0 con 0x7f781c103c60 2024-05-04T03:23:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.425+0000 7f782ae3a740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8120491b0 0x55f8123eaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.425+0000 7f782ae3a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8120202f0 0x55f8123eafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.425+0000 7f782ae3a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8120284d0 0x55f8123f1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.425+0000 7f782ae3a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f81234f5d0 con 0x55f8120284d0 2024-05-04T03:23:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.425+0000 7f782ae3a740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55f812308890 con 0x55f8120491b0 2024-05-04T03:23:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.425+0000 7f782ae3a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f81234b590 con 0x55f8120202f0 2024-05-04T03:23:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.425+0000 7f7821443640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8120284d0 0x55f8123f1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.425+0000 7f7822445640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8120491b0 0x55f8123eaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.425+0000 7f7822445640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8120491b0 0x55f8123eaa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:52966/0 (socket says 172.21.15.1:52966) 2024-05-04T03:23:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.425+0000 7f7821443640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8120284d0 0x55f8123f1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46366/0 (socket says 172.21.15.1:46366) 2024-05-04T03:23:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.425+0000 7f7822445640 1 -- 172.21.15.1:0/2865909915 learned_addr learned my addr 172.21.15.1:0/2865909915 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.425+0000 7f7821c44640 1 --2- 172.21.15.1:0/2865909915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8120202f0 0x55f8123eafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.425+0000 7f7822445640 1 -- 172.21.15.1:0/2865909915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8120202f0 msgr2=0x55f8123eafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.425+0000 7f7822445640 1 --2- 172.21.15.1:0/2865909915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8120202f0 0x55f8123eafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.425+0000 7f7822445640 1 -- 172.21.15.1:0/2865909915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8120284d0 msgr2=0x55f8123f1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.425+0000 7f7822445640 1 --2- 172.21.15.1:0/2865909915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8120284d0 0x55f8123f1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.425+0000 7f7822445640 1 -- 172.21.15.1:0/2865909915 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f812061210 con 0x55f8120491b0 2024-05-04T03:23:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.426+0000 7f7822445640 1 --2- 172.21.15.1:0/2865909915 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8120491b0 0x55f8123eaa60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7814035000 tx=0x7f78140098b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d643cfaca2e715b server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.426+0000 7f7812ffd640 1 --2- 172.21.15.1:0/2721818400 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f78080b28a0 0x7f78080b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.426+0000 7f7807fff640 1 -- 172.21.15.1:0/2865909915 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7814031710 con 0x55f8120491b0 2024-05-04T03:23:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.426+0000 7f7821c44640 1 --2- 172.21.15.1:0/2721818400 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f78080b28a0 0x7f78080b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.426+0000 7f7807fff640 1 -- 172.21.15.1:0/2865909915 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7814036030 con 0x55f8120491b0 2024-05-04T03:23:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.426+0000 7f7807fff640 1 -- 172.21.15.1:0/2865909915 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f781402f660 con 0x55f8120491b0 2024-05-04T03:23:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.426+0000 7f7812ffd640 1 -- 172.21.15.1:0/2721818400 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f78180e1d80 con 0x7f781c103c60 2024-05-04T03:23:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.427+0000 7f782ae3a740 1 -- 172.21.15.1:0/2865909915 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8120491b0 msgr2=0x55f8123eaa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.427+0000 7f7821c44640 1 --2- 172.21.15.1:0/2721818400 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f78080b28a0 0x7f78080b4db0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f78180a93f0 tx=0x7f78180e58a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.427+0000 7f782ae3a740 1 --2- 172.21.15.1:0/2865909915 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8120491b0 0x55f8123eaa60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7814035000 tx=0x7f78140098b0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.427+0000 7f782ae3a740 1 -- 172.21.15.1:0/2865909915 shutdown_connections 2024-05-04T03:23:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.427+0000 7f782ae3a740 1 --2- 172.21.15.1:0/2865909915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8120284d0 0x55f8123f1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.427+0000 7f782ae3a740 1 --2- 172.21.15.1:0/2865909915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8120202f0 0x55f8123eafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.427+0000 7f782ae3a740 1 --2- 172.21.15.1:0/2865909915 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8120491b0 0x55f8123eaa60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.427+0000 7f782ae3a740 1 -- 172.21.15.1:0/2865909915 >> 172.21.15.1:0/2865909915 conn(0x55f81203d300 msgr2=0x55f8123e63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.427+0000 7f782ae3a740 1 -- 172.21.15.1:0/2865909915 shutdown_connections 2024-05-04T03:23:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.427+0000 7f782ae3a740 1 -- 172.21.15.1:0/2865909915 wait complete. 2024-05-04T03:23:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.427+0000 7f782ae3a740 1 -- start start 2024-05-04T03:23:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.428+0000 7f782ae3a740 1 Processor -- start 2024-05-04T03:23:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.428+0000 7f782ae3a740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8120202f0 0x55f8123f44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.428+0000 7f782ae3a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8120284d0 0x55f8123f4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.428+0000 7f782ae3a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8120491b0 0x55f8123fb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.428+0000 7f782ae3a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55f8123f2750 con 0x55f8120491b0 2024-05-04T03:23:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.428+0000 7f782ae3a740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55f81230c370 con 0x55f8120202f0 2024-05-04T03:23:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.428+0000 7f782ae3a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f8123f28d0 con 0x55f8120284d0 2024-05-04T03:23:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.428+0000 7f782ae3a740 10 client.0 _ll_register_callbacks cb 0x55f812060a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:23:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.428+0000 7f7822445640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8120491b0 0x55f8123fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.428+0000 7f7822445640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8120491b0 0x55f8123fb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46370/0 (socket says 172.21.15.1:46370) 2024-05-04T03:23:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.428+0000 7f7822445640 1 -- 172.21.15.1:0/272818004 learned_addr learned my addr 172.21.15.1:0/272818004 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.428+0000 7f7822445640 1 --2- 172.21.15.1:0/272818004 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8120202f0 0x55f8123f44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.428+0000 7f7821443640 1 --2- 172.21.15.1:0/272818004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8120284d0 0x55f8123f4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.428+0000 7f7822445640 1 -- 172.21.15.1:0/272818004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8120284d0 msgr2=0x55f8123f4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.428+0000 7f7822445640 1 --2- 172.21.15.1:0/272818004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8120284d0 0x55f8123f4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.428+0000 7f7822445640 1 -- 172.21.15.1:0/272818004 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8120202f0 msgr2=0x55f8123f44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.428+0000 7f7822445640 1 --2- 172.21.15.1:0/272818004 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8120202f0 0x55f8123f44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.429+0000 7f7822445640 1 -- 172.21.15.1:0/272818004 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f81212d620 con 0x55f8120491b0 2024-05-04T03:23:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.429+0000 7f7822445640 1 --2- 172.21.15.1:0/272818004 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8120491b0 0x55f8123fb990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7814009ca0 tx=0x7f7814031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1fd26f49e2f5c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.429+0000 7f78057fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:23:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.429+0000 7f78057fa640 1 -- 172.21.15.1:0/272818004 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7814059060 con 0x55f8120491b0 2024-05-04T03:23:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.429+0000 7f78057fa640 1 -- 172.21.15.1:0/272818004 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f7814003d80 con 0x55f8120491b0 2024-05-04T03:23:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.429+0000 7f782ae3a740 1 -- 172.21.15.1:0/272818004 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7814036030 con 0x55f8120491b0 2024-05-04T03:23:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.429+0000 7f78057fa640 1 -- 172.21.15.1:0/272818004 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f78140317c0 con 0x55f8120491b0 2024-05-04T03:23:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.429+0000 7f782ae3a740 10 client.15762 fetch_fsmap learned FSMap version 32 2024-05-04T03:23:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.429+0000 7f782ae3a740 1 -- 172.21.15.1:0/272818004 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f8123f2750 con 0x55f8120491b0 2024-05-04T03:23:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.429+0000 7f78057fa640 1 -- 172.21.15.1:0/272818004 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7814036030 con 0x55f8120491b0 2024-05-04T03:23:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.429+0000 7f78057fa640 1 -- 172.21.15.1:0/272818004 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f7814043070 con 0x55f8120491b0 2024-05-04T03:23:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.430+0000 7f782ae3a740 10 client.15762 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:23:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.430+0000 7f782ae3a740 10 client.15762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:23:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.430+0000 7f782ae3a740 20 client.15762 populate_metadata read hostname 'smithi001' 2024-05-04T03:23:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.430+0000 7f782ae3a740 1 --2- 172.21.15.1:0/272818004 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55f812007d80 0x55f812401c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.430+0000 7f782ae3a740 4 client.15762 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:23:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.430+0000 7f782ae3a740 1 -- 172.21.15.1:0/272818004 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f81235d1e0 con 0x55f812007d80 2024-05-04T03:23:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.430+0000 7f7821443640 1 --2- 172.21.15.1:0/272818004 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55f812007d80 0x55f812401c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.430+0000 7f7821443640 1 --2- 172.21.15.1:0/272818004 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55f812007d80 0x55f812401c60 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdf4ab039c4051ca server_cookie=f23be0d3a95f023e in_seq=0 out_seq=0 2024-05-04T03:23:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.430+0000 7f78057fa640 10 client.15762 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:23:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.431+0000 7f78057fa640 1 -- 172.21.15.1:0/272818004 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f81235d1e0 con 0x55f812007d80 2024-05-04T03:23:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.431+0000 7f78057fa640 10 client.15762 handle_command_reply: tid=0 2024-05-04T03:23:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.431+0000 7f782ae3a740 1 client.15762 shutdown 2024-05-04T03:23:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.431+0000 7f782ae3a740 1 -- 172.21.15.1:0/272818004 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8120491b0 msgr2=0x55f8123fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.431+0000 7f782ae3a740 1 --2- 172.21.15.1:0/272818004 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8120491b0 0x55f8123fb990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7814009ca0 tx=0x7f7814031b10 comp rx=0 tx=0).stop 2024-05-04T03:23:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.431+0000 7f782ae3a740 1 -- 172.21.15.1:0/272818004 shutdown_connections 2024-05-04T03:23:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.431+0000 7f782ae3a740 1 --2- 172.21.15.1:0/272818004 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55f812007d80 0x55f812401c60 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.431+0000 7f782ae3a740 1 --2- 172.21.15.1:0/272818004 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8120491b0 0x55f8123fb990 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.431+0000 7f782ae3a740 1 --2- 172.21.15.1:0/272818004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8120284d0 0x55f8123f4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.431+0000 7f782ae3a740 1 --2- 172.21.15.1:0/272818004 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8120202f0 0x55f8123f44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.431+0000 7f78057fa640 0 client.15762 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:23:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.431+0000 7f782ae3a740 1 -- 172.21.15.1:0/272818004 >> 172.21.15.1:0/272818004 conn(0x55f81203d300 msgr2=0x55f8123e9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.431+0000 7f782ae3a740 1 -- 172.21.15.1:0/272818004 shutdown_connections 2024-05-04T03:23:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.431+0000 7f782ae3a740 1 -- 172.21.15.1:0/272818004 wait complete. 2024-05-04T03:23:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.431+0000 7f782ae3a740 20 client.15762 trim_cache size 0 max 16384 2024-05-04T03:23:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.447+0000 7f782ae3a740 1 Processor -- start 2024-05-04T03:23:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.447+0000 7f782ae3a740 1 -- start start 2024-05-04T03:23:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.447+0000 7f782ae3a740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8120202f0 0x55f8123ef130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.447+0000 7f782ae3a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8120284d0 0x55f8123e9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.447+0000 7f782ae3a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8120491b0 0x55f8123fbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.447+0000 7f782ae3a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1ad9b0 con 0x55f8120284d0 2024-05-04T03:23:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.447+0000 7f782ae3a740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1ab280 con 0x55f8120202f0 2024-05-04T03:23:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.447+0000 7f782ae3a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1a90f0 con 0x55f8120491b0 2024-05-04T03:23:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.447+0000 7f7821443640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8120284d0 0x55f8123e9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.447+0000 7f7821443640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8120284d0 0x55f8123e9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46374/0 (socket says 172.21.15.1:46374) 2024-05-04T03:23:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.447+0000 7f7821443640 1 -- 172.21.15.1:0/27010974 learned_addr learned my addr 172.21.15.1:0/27010974 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.447+0000 7f7822445640 1 --2- 172.21.15.1:0/27010974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8120202f0 0x55f8123ef130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.447+0000 7f7822445640 1 --2- 172.21.15.1:0/27010974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8120491b0 0x55f8123fbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.448+0000 7f7821443640 1 -- 172.21.15.1:0/27010974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8120491b0 msgr2=0x55f8123fbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.448+0000 7f7821443640 1 --2- 172.21.15.1:0/27010974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8120491b0 0x55f8123fbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.448+0000 7f7821443640 1 -- 172.21.15.1:0/27010974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8120202f0 msgr2=0x55f8123ef130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.448+0000 7f7821443640 1 --2- 172.21.15.1:0/27010974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8120202f0 0x55f8123ef130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.448+0000 7f7821443640 1 -- 172.21.15.1:0/27010974 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8123f5560 con 0x55f8120284d0 2024-05-04T03:23:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.448+0000 7f7822445640 1 --2- 172.21.15.1:0/27010974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8120202f0 0x55f8123ef130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:23:35.451 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.448+0000 7f7821443640 1 --2- 172.21.15.1:0/27010974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8120284d0 0x55f8123e9980 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x55f8123e7180 tx=0x7f780c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fab5762754064c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.448+0000 7f78077fe640 1 -- 172.21.15.1:0/27010974 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c005220 con 0x55f8120284d0 2024-05-04T03:23:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.448+0000 7f78077fe640 1 -- 172.21.15.1:0/27010974 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f780c024030 con 0x55f8120284d0 2024-05-04T03:23:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.448+0000 7f78077fe640 1 -- 172.21.15.1:0/27010974 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c020500 con 0x55f8120284d0 2024-05-04T03:23:35.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.449+0000 7f782ae3a740 1 -- 172.21.15.1:0/27010974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8120284d0 msgr2=0x55f8123e9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.449+0000 7f782ae3a740 1 --2- 172.21.15.1:0/27010974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8120284d0 0x55f8123e9980 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x55f8123e7180 tx=0x7f780c00b390 comp rx=0 tx=0).stop 2024-05-04T03:23:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.449+0000 7f782ae3a740 1 -- 172.21.15.1:0/27010974 shutdown_connections 2024-05-04T03:23:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.449+0000 7f782ae3a740 1 --2- 172.21.15.1:0/27010974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8120491b0 0x55f8123fbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.449+0000 7f782ae3a740 1 --2- 172.21.15.1:0/27010974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8120284d0 0x55f8123e9980 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.449+0000 7f782ae3a740 1 --2- 172.21.15.1:0/27010974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8120202f0 0x55f8123ef130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.449+0000 7f782ae3a740 1 -- 172.21.15.1:0/27010974 >> 172.21.15.1:0/27010974 conn(0x55f81203d300 msgr2=0x55f8123e9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.449+0000 7f782ae3a740 1 -- 172.21.15.1:0/27010974 shutdown_connections 2024-05-04T03:23:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.449+0000 7f782ae3a740 1 -- 172.21.15.1:0/27010974 wait complete. 2024-05-04T03:23:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.449+0000 7f782ae3a740 1 -- start start 2024-05-04T03:23:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.449+0000 7f782ae3a740 1 Processor -- start 2024-05-04T03:23:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.450+0000 7f782ae3a740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8120202f0 0x55f8124655a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.450+0000 7f782ae3a740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8120284d0 0x55f812465ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.450+0000 7f782ae3a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8120491b0 0x55f81246ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.450+0000 7f782ae3a740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1a9740 con 0x55f8120202f0 2024-05-04T03:23:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.450+0000 7f782ae3a740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1aabb0 con 0x55f8120284d0 2024-05-04T03:23:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.450+0000 7f782ae3a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f781c1a9e10 con 0x55f8120491b0 2024-05-04T03:23:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.450+0000 7f782ae3a740 10 client.0 _ll_register_callbacks cb 0x7f780c00ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:23:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.450+0000 7f7822445640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8120202f0 0x55f8124655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.450+0000 7f7822445640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8120202f0 0x55f8124655a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46386/0 (socket says 172.21.15.1:46386) 2024-05-04T03:23:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.450+0000 7f7822445640 1 -- 172.21.15.1:0/4149491302 learned_addr learned my addr 172.21.15.1:0/4149491302 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.450+0000 7f7821443640 1 --2- 172.21.15.1:0/4149491302 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8120284d0 0x55f812465ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.450+0000 7f7822445640 1 --2- 172.21.15.1:0/4149491302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8120491b0 0x55f81246ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.450+0000 7f7822445640 1 -- 172.21.15.1:0/4149491302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8120491b0 msgr2=0x55f81246ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:35.458 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.450+0000 7f7822445640 1 --2- 172.21.15.1:0/4149491302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8120491b0 0x55f81246ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.450+0000 7f7822445640 1 -- 172.21.15.1:0/4149491302 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8120284d0 msgr2=0x55f812465ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.450+0000 7f7822445640 1 --2- 172.21.15.1:0/4149491302 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8120284d0 0x55f812465ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.450+0000 7f7822445640 1 -- 172.21.15.1:0/4149491302 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f81246d000 con 0x55f8120202f0 2024-05-04T03:23:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.450+0000 7f7821443640 1 --2- 172.21.15.1:0/4149491302 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8120284d0 0x55f812465ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:23:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.450+0000 7f7822445640 1 --2- 172.21.15.1:0/4149491302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8120202f0 0x55f8124655a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7814005d50 tx=0x7f7814031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad61abe69935632a server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.450+0000 7f78057fa640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:23:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.451+0000 7f78057fa640 1 -- 172.21.15.1:0/4149491302 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7814004000 con 0x55f8120202f0 2024-05-04T03:23:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.451+0000 7f782ae3a740 1 -- 172.21.15.1:0/4149491302 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f780c024030 con 0x55f8120202f0 2024-05-04T03:23:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.451+0000 7f78057fa640 1 -- 172.21.15.1:0/4149491302 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f7814004400 con 0x55f8120202f0 2024-05-04T03:23:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.451+0000 7f78057fa640 1 -- 172.21.15.1:0/4149491302 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7814070050 con 0x55f8120202f0 2024-05-04T03:23:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.451+0000 7f782ae3a740 10 client.15768 fetch_fsmap learned FSMap version 32 2024-05-04T03:23:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.451+0000 7f782ae3a740 1 -- 172.21.15.1:0/4149491302 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f81246e2c0 con 0x55f8120202f0 2024-05-04T03:23:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.451+0000 7f78057fa640 1 -- 172.21.15.1:0/4149491302 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f780c024030 con 0x55f8120202f0 2024-05-04T03:23:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.451+0000 7f78057fa640 1 -- 172.21.15.1:0/4149491302 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f7814043040 con 0x55f8120202f0 2024-05-04T03:23:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.451+0000 7f782ae3a740 10 client.15768 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:23:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.451+0000 7f782ae3a740 10 client.15768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:23:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.451+0000 7f782ae3a740 20 client.15768 populate_metadata read hostname 'smithi001' 2024-05-04T03:23:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.451+0000 7f782ae3a740 1 --2- 172.21.15.1:0/4149491302 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55f811fdd1c0 0x55f812470c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.451+0000 7f782ae3a740 4 client.15768 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:23:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.451+0000 7f782ae3a740 1 -- 172.21.15.1:0/4149491302 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f812357050 con 0x55f811fdd1c0 2024-05-04T03:23:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.451+0000 7f7821443640 1 --2- 172.21.15.1:0/4149491302 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55f811fdd1c0 0x55f812470c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.452+0000 7f7821443640 1 --2- 172.21.15.1:0/4149491302 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55f811fdd1c0 0x55f812470c90 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b488afe6a1889280 server_cookie=21fae54ab5dc0d30 in_seq=0 out_seq=0 2024-05-04T03:23:35.460 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.452+0000 7f78057fa640 10 client.15768 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:23:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.460+0000 7f78057fa640 1 -- 172.21.15.1:0/4149491302 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f812357050 con 0x55f811fdd1c0 2024-05-04T03:23:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.460+0000 7f78057fa640 10 client.15768 handle_command_reply: tid=0 2024-05-04T03:23:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.460+0000 7f782ae3a740 1 client.15768 shutdown 2024-05-04T03:23:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.460+0000 7f782ae3a740 1 -- 172.21.15.1:0/4149491302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8120202f0 msgr2=0x55f8124655a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.460+0000 7f782ae3a740 1 --2- 172.21.15.1:0/4149491302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8120202f0 0x55f8124655a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7814005d50 tx=0x7f7814031b10 comp rx=0 tx=0).stop 2024-05-04T03:23:35.463 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.461+0000 7f782ae3a740 1 -- 172.21.15.1:0/4149491302 shutdown_connections 2024-05-04T03:23:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.461+0000 7f782ae3a740 1 --2- 172.21.15.1:0/4149491302 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55f811fdd1c0 0x55f812470c90 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.461+0000 7f782ae3a740 1 --2- 172.21.15.1:0/4149491302 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f8120284d0 0x55f812465ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.461+0000 7f782ae3a740 1 --2- 172.21.15.1:0/4149491302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f8120491b0 0x55f81246ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.461+0000 7f78057fa640 0 client.15768 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:23:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.461+0000 7f782ae3a740 1 --2- 172.21.15.1:0/4149491302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8120202f0 0x55f8124655a0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.461+0000 7f782ae3a740 1 -- 172.21.15.1:0/4149491302 >> 172.21.15.1:0/4149491302 conn(0x55f81203d300 msgr2=0x55f8123f2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.461+0000 7f782ae3a740 1 -- 172.21.15.1:0/4149491302 shutdown_connections 2024-05-04T03:23:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.461+0000 7f782ae3a740 1 -- 172.21.15.1:0/4149491302 wait complete. 2024-05-04T03:23:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.461+0000 7f782ae3a740 20 client.15768 trim_cache size 0 max 16384 2024-05-04T03:23:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:23:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:23:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:23:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:23:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.461+0000 7f7823ea8640 1 -- 172.21.15.1:0/2721818400 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f78080b28a0 msgr2=0x7f78080b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:35.465 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.461+0000 7f7823ea8640 1 --2- 172.21.15.1:0/2721818400 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f78080b28a0 0x7f78080b4db0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f78180a93f0 tx=0x7f78180e58a0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.461+0000 7f7823ea8640 1 -- 172.21.15.1:0/2721818400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f781c103c60 msgr2=0x7f781c19a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.461+0000 7f7823ea8640 1 --2- 172.21.15.1:0/2721818400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f781c103c60 0x7f781c19a240 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7818005fd0 tx=0x7f781800dde0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.461+0000 7f7823ea8640 1 -- 172.21.15.1:0/2721818400 shutdown_connections 2024-05-04T03:23:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.461+0000 7f7823ea8640 1 --2- 172.21.15.1:0/2721818400 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f78080b28a0 0x7f78080b4db0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.461+0000 7f7823ea8640 1 --2- 172.21.15.1:0/2721818400 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f781c104ea0 0x7f781c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.461+0000 7f7823ea8640 1 --2- 172.21.15.1:0/2721818400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f781c109ba0 0x7f781c1a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.461+0000 7f7823ea8640 1 --2- 172.21.15.1:0/2721818400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f781c103c60 0x7f781c19a240 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.461+0000 7f7823ea8640 1 -- 172.21.15.1:0/2721818400 >> 172.21.15.1:0/2721818400 conn(0x7f781c0fee70 msgr2=0x7f781c105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.462+0000 7f7823ea8640 1 -- 172.21.15.1:0/2721818400 shutdown_connections 2024-05-04T03:23:35.466 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:35.462+0000 7f7823ea8640 1 -- 172.21.15.1:0/2721818400 wait complete. 2024-05-04T03:23:35.698 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '961ff620', quiesce took 7.9 seconds. Will release after: 30.0 2024-05-04T03:23:36.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:36 smithi072 ceph-mon[33332]: pgmap v524: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.6 MiB/s wr, 276 op/s 2024-05-04T03:23:37.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:36 smithi001 ceph-mon[29731]: pgmap v524: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.6 MiB/s wr, 276 op/s 2024-05-04T03:23:37.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:36 smithi077 ceph-mon[33531]: pgmap v524: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.6 MiB/s wr, 276 op/s 2024-05-04T03:23:38.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:38 smithi072 ceph-mon[33332]: pgmap v525: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.6 MiB/s wr, 275 op/s 2024-05-04T03:23:39.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:38 smithi001 ceph-mon[29731]: pgmap v525: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.6 MiB/s wr, 275 op/s 2024-05-04T03:23:39.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:38 smithi077 ceph-mon[33531]: pgmap v525: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.6 MiB/s wr, 275 op/s 2024-05-04T03:23:40.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:40 smithi072 ceph-mon[33332]: pgmap v526: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.6 MiB/s wr, 282 op/s 2024-05-04T03:23:41.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:40 smithi001 ceph-mon[29731]: pgmap v526: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.6 MiB/s wr, 282 op/s 2024-05-04T03:23:41.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:40 smithi077 ceph-mon[33531]: pgmap v526: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.6 MiB/s wr, 282 op/s 2024-05-04T03:23:42.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:42 smithi072 ceph-mon[33332]: pgmap v527: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.4 MiB/s wr, 249 op/s 2024-05-04T03:23:43.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:42 smithi001 ceph-mon[29731]: pgmap v527: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.4 MiB/s wr, 249 op/s 2024-05-04T03:23:43.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:42 smithi077 ceph-mon[33531]: pgmap v527: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.4 MiB/s wr, 249 op/s 2024-05-04T03:23:44.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:44 smithi072 ceph-mon[33332]: pgmap v528: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 973 KiB/s wr, 197 op/s 2024-05-04T03:23:45.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:44 smithi001 ceph-mon[29731]: pgmap v528: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 973 KiB/s wr, 197 op/s 2024-05-04T03:23:45.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:44 smithi077 ceph-mon[33531]: pgmap v528: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 973 KiB/s wr, 197 op/s 2024-05-04T03:23:46.980 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:46 smithi077 ceph-mon[33531]: pgmap v529: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 751 KiB/s wr, 122 op/s 2024-05-04T03:23:46.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:46 smithi072 ceph-mon[33332]: pgmap v529: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 751 KiB/s wr, 122 op/s 2024-05-04T03:23:47.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:46 smithi001 ceph-mon[29731]: pgmap v529: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 751 KiB/s wr, 122 op/s 2024-05-04T03:23:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.579+0000 7f5975327640 1 -- 172.21.15.1:0/3321036344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5970103c90 msgr2=0x7f5970104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.579+0000 7f5975327640 1 --2- 172.21.15.1:0/3321036344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5970103c90 0x7f5970104990 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5960009900 tx=0x7f596002a3f0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.579+0000 7f5975327640 1 -- 172.21.15.1:0/3321036344 shutdown_connections 2024-05-04T03:23:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.579+0000 7f5975327640 1 --2- 172.21.15.1:0/3321036344 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5970104ed0 0x7f59701052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.579+0000 7f5975327640 1 --2- 172.21.15.1:0/3321036344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5970109bd0 0x7f597010bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.579+0000 7f5975327640 1 --2- 172.21.15.1:0/3321036344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5970103c90 0x7f5970104990 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.579+0000 7f5975327640 1 -- 172.21.15.1:0/3321036344 >> 172.21.15.1:0/3321036344 conn(0x7f59700feea0 msgr2=0x7f5970101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.579+0000 7f5975327640 1 -- 172.21.15.1:0/3321036344 shutdown_connections 2024-05-04T03:23:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.579+0000 7f5975327640 1 -- 172.21.15.1:0/3321036344 wait complete. 2024-05-04T03:23:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.580+0000 7f5975327640 1 Processor -- start 2024-05-04T03:23:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.580+0000 7f5975327640 1 -- start start 2024-05-04T03:23:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.580+0000 7f5975327640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5970103c90 0x7f597019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.580+0000 7f5975327640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5970104ed0 0x7f597019a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.580+0000 7f5975327640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5970109bd0 0x7f59701a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.580+0000 7f5975327640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f597010f770 con 0x7f5970109bd0 2024-05-04T03:23:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.580+0000 7f5975327640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f597010f5f0 con 0x7f5970104ed0 2024-05-04T03:23:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.580+0000 7f5975327640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f597010f8f0 con 0x7f5970103c90 2024-05-04T03:23:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.580+0000 7f596f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5970109bd0 0x7f59701a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.580+0000 7f596f7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5970109bd0 0x7f59701a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47940/0 (socket says 172.21.15.1:47940) 2024-05-04T03:23:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.580+0000 7f596f7fe640 1 -- 172.21.15.1:0/1374067530 learned_addr learned my addr 172.21.15.1:0/1374067530 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.581+0000 7f596e7fc640 1 --2- 172.21.15.1:0/1374067530 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5970104ed0 0x7f597019a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.581+0000 7f596effd640 1 --2- 172.21.15.1:0/1374067530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5970103c90 0x7f597019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.581+0000 7f596f7fe640 1 -- 172.21.15.1:0/1374067530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5970103c90 msgr2=0x7f597019a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.581+0000 7f596f7fe640 1 --2- 172.21.15.1:0/1374067530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5970103c90 0x7f597019a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.581+0000 7f596f7fe640 1 -- 172.21.15.1:0/1374067530 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5970104ed0 msgr2=0x7f597019a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.581+0000 7f596f7fe640 1 --2- 172.21.15.1:0/1374067530 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5970104ed0 0x7f597019a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.581+0000 7f596f7fe640 1 -- 172.21.15.1:0/1374067530 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59701a1e10 con 0x7f5970109bd0 2024-05-04T03:23:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.581+0000 7f596f7fe640 1 --2- 172.21.15.1:0/1374067530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5970109bd0 0x7f59701a1850 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f596800cce0 tx=0x7f596800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.581+0000 7f5957fff640 1 -- 172.21.15.1:0/1374067530 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596800f450 con 0x7f5970109bd0 2024-05-04T03:23:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.581+0000 7f5957fff640 1 -- 172.21.15.1:0/1374067530 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f596800faa0 con 0x7f5970109bd0 2024-05-04T03:23:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.581+0000 7f5975327640 1 -- 172.21.15.1:0/1374067530 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59701a2100 con 0x7f5970109bd0 2024-05-04T03:23:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.581+0000 7f5957fff640 1 -- 172.21.15.1:0/1374067530 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5968018b70 con 0x7f5970109bd0 2024-05-04T03:23:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.581+0000 7f5975327640 1 -- 172.21.15.1:0/1374067530 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59701a2600 con 0x7f5970109bd0 2024-05-04T03:23:48.591 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.584+0000 7f597c1d7740 1 Processor -- start 2024-05-04T03:23:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.584+0000 7f5957fff640 1 -- 172.21.15.1:0/1374067530 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f5968018d10 con 0x7f5970109bd0 2024-05-04T03:23:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.584+0000 7f597c1d7740 1 -- start start 2024-05-04T03:23:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.584+0000 7f597c1d7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f1b1351b0 0x555f1b4dae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.584+0000 7f597c1d7740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f1b10c2f0 0x555f1b4db3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.584+0000 7f597c1d7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f1b1144d0 0x555f1b4e22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.584+0000 7f597c1d7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555f1b43b5d0 con 0x555f1b1351b0 2024-05-04T03:23:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.584+0000 7f597c1d7740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x555f1b3f4890 con 0x555f1b10c2f0 2024-05-04T03:23:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.584+0000 7f597c1d7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555f1b437590 con 0x555f1b1144d0 2024-05-04T03:23:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.584+0000 7f596e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f1b1351b0 0x555f1b4dae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.584+0000 7f596e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f1b1351b0 0x555f1b4dae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47956/0 (socket says 172.21.15.1:47956) 2024-05-04T03:23:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.584+0000 7f596e7fc640 1 -- 172.21.15.1:0/4203083304 learned_addr learned my addr 172.21.15.1:0/4203083304 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.584+0000 7f596e7fc640 1 --2- 172.21.15.1:0/4203083304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f1b1144d0 0x555f1b4e22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.584+0000 7f596effd640 1 --2- 172.21.15.1:0/4203083304 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f1b10c2f0 0x555f1b4db3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.585+0000 7f5957fff640 1 --2- 172.21.15.1:0/1374067530 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f59580b28a0 0x7f59580b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.585+0000 7f596e7fc640 1 -- 172.21.15.1:0/4203083304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f1b1144d0 msgr2=0x555f1b4e22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.585+0000 7f596e7fc640 1 --2- 172.21.15.1:0/4203083304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f1b1144d0 0x555f1b4e22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.585+0000 7f596e7fc640 1 -- 172.21.15.1:0/4203083304 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f1b10c2f0 msgr2=0x555f1b4db3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.585+0000 7f596e7fc640 1 --2- 172.21.15.1:0/4203083304 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f1b10c2f0 0x555f1b4db3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.585+0000 7f596e7fc640 1 -- 172.21.15.1:0/4203083304 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f1b14d210 con 0x555f1b1351b0 2024-05-04T03:23:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.585+0000 7f596effd640 1 --2- 172.21.15.1:0/4203083304 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f1b10c2f0 0x555f1b4db3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:23:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.585+0000 7f596f7fe640 1 --2- 172.21.15.1:0/1374067530 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f59580b28a0 0x7f59580b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.585+0000 7f596e7fc640 1 --2- 172.21.15.1:0/4203083304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f1b1351b0 0x555f1b4dae70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5964012840 tx=0x7f596400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47fdcc87af07c61c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.585+0000 7f5957fff640 1 -- 172.21.15.1:0/1374067530 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f59680e2520 con 0x7f5970109bd0 2024-05-04T03:23:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.585+0000 7f59557fa640 1 -- 172.21.15.1:0/4203083304 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59640120d0 con 0x555f1b1351b0 2024-05-04T03:23:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.585+0000 7f59557fa640 1 -- 172.21.15.1:0/4203083304 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5964004c50 con 0x555f1b1351b0 2024-05-04T03:23:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.585+0000 7f59557fa640 1 -- 172.21.15.1:0/4203083304 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59640055e0 con 0x555f1b1351b0 2024-05-04T03:23:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.586+0000 7f597c1d7740 1 -- 172.21.15.1:0/4203083304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f1b1351b0 msgr2=0x555f1b4dae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.586+0000 7f597c1d7740 1 --2- 172.21.15.1:0/4203083304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f1b1351b0 0x555f1b4dae70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5964012840 tx=0x7f596400b8c0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.586+0000 7f596f7fe640 1 --2- 172.21.15.1:0/1374067530 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f59580b28a0 0x7f59580b4db0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f59680a5c80 tx=0x7f59680a5b60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.586+0000 7f597c1d7740 1 -- 172.21.15.1:0/4203083304 shutdown_connections 2024-05-04T03:23:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.586+0000 7f597c1d7740 1 --2- 172.21.15.1:0/4203083304 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f1b10c2f0 0x555f1b4db3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.586+0000 7f597c1d7740 1 --2- 172.21.15.1:0/4203083304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f1b1144d0 0x555f1b4e22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.586+0000 7f597c1d7740 1 --2- 172.21.15.1:0/4203083304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f1b1351b0 0x555f1b4dae70 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.586+0000 7f597c1d7740 1 -- 172.21.15.1:0/4203083304 >> 172.21.15.1:0/4203083304 conn(0x555f1b129300 msgr2=0x555f1b4d23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.586+0000 7f597c1d7740 1 -- 172.21.15.1:0/4203083304 shutdown_connections 2024-05-04T03:23:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.586+0000 7f597c1d7740 1 -- 172.21.15.1:0/4203083304 wait complete. 2024-05-04T03:23:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.587+0000 7f597c1d7740 1 -- start start 2024-05-04T03:23:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.587+0000 7f597c1d7740 1 Processor -- start 2024-05-04T03:23:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.587+0000 7f597c1d7740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f1b10c2f0 0x555f1b4e49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.587+0000 7f597c1d7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f1b1144d0 0x555f1b4e4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.587+0000 7f597c1d7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f1b1351b0 0x555f1b4ebe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.587+0000 7f597c1d7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x555f1b4e2ce0 con 0x555f1b1351b0 2024-05-04T03:23:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.587+0000 7f597c1d7740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x555f1b3f8370 con 0x555f1b10c2f0 2024-05-04T03:23:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.587+0000 7f597c1d7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555f1b4e2e60 con 0x555f1b1144d0 2024-05-04T03:23:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.587+0000 7f597c1d7740 10 client.0 _ll_register_callbacks cb 0x555f1b14ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:23:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.587+0000 7f596e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f1b1351b0 0x555f1b4ebe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.587+0000 7f596e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f1b1351b0 0x555f1b4ebe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47960/0 (socket says 172.21.15.1:47960) 2024-05-04T03:23:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.587+0000 7f596e7fc640 1 -- 172.21.15.1:0/3908047108 learned_addr learned my addr 172.21.15.1:0/3908047108 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.587+0000 7f596effd640 1 --2- 172.21.15.1:0/3908047108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f1b1144d0 0x555f1b4e4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.587+0000 7f596e7fc640 1 --2- 172.21.15.1:0/3908047108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f1b10c2f0 0x555f1b4e49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.587+0000 7f596e7fc640 1 -- 172.21.15.1:0/3908047108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f1b1144d0 msgr2=0x555f1b4e4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.587+0000 7f596e7fc640 1 --2- 172.21.15.1:0/3908047108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f1b1144d0 0x555f1b4e4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.587+0000 7f596e7fc640 1 -- 172.21.15.1:0/3908047108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f1b10c2f0 msgr2=0x555f1b4e49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.587+0000 7f596e7fc640 1 --2- 172.21.15.1:0/3908047108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f1b10c2f0 0x555f1b4e49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.587+0000 7f596e7fc640 1 -- 172.21.15.1:0/3908047108 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f1b219620 con 0x555f1b1351b0 2024-05-04T03:23:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.588+0000 7f596e7fc640 1 --2- 172.21.15.1:0/3908047108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f1b1351b0 0x555f1b4ebe60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f596401c040 tx=0x7f596400b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43714219f391b54b server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.588+0000 7f5952ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:23:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.588+0000 7f5952ff5640 1 -- 172.21.15.1:0/3908047108 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964005560 con 0x555f1b1351b0 2024-05-04T03:23:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.588+0000 7f5952ff5640 1 -- 172.21.15.1:0/3908047108 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f5964005700 con 0x555f1b1351b0 2024-05-04T03:23:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.588+0000 7f597c1d7740 1 -- 172.21.15.1:0/3908047108 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5964004c50 con 0x555f1b1351b0 2024-05-04T03:23:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.588+0000 7f5952ff5640 1 -- 172.21.15.1:0/3908047108 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f596402d070 con 0x555f1b1351b0 2024-05-04T03:23:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.588+0000 7f597c1d7740 10 client.15780 fetch_fsmap learned FSMap version 32 2024-05-04T03:23:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.588+0000 7f597c1d7740 1 -- 172.21.15.1:0/3908047108 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555f1b4eed90 con 0x555f1b1351b0 2024-05-04T03:23:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.588+0000 7f5952ff5640 1 -- 172.21.15.1:0/3908047108 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5964004c50 con 0x555f1b1351b0 2024-05-04T03:23:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.588+0000 7f5952ff5640 1 -- 172.21.15.1:0/3908047108 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f5964010580 con 0x555f1b1351b0 2024-05-04T03:23:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.589+0000 7f597c1d7740 10 client.15780 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:23:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.589+0000 7f597c1d7740 10 client.15780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:23:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.589+0000 7f597c1d7740 20 client.15780 populate_metadata read hostname 'smithi001' 2024-05-04T03:23:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.589+0000 7f597c1d7740 1 --2- 172.21.15.1:0/3908047108 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x555f1b0f3d80 0x555f1b4f21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.589+0000 7f597c1d7740 4 client.15780 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:23:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.589+0000 7f597c1d7740 1 -- 172.21.15.1:0/3908047108 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f1b4491e0 con 0x555f1b0f3d80 2024-05-04T03:23:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.589+0000 7f596effd640 1 --2- 172.21.15.1:0/3908047108 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x555f1b0f3d80 0x555f1b4f21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.589+0000 7f596effd640 1 --2- 172.21.15.1:0/3908047108 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x555f1b0f3d80 0x555f1b4f21b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e7970d6b3da6d20 server_cookie=9edf65653d78ae59 in_seq=0 out_seq=0 2024-05-04T03:23:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.589+0000 7f5952ff5640 10 client.15780 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:23:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.590+0000 7f5952ff5640 1 -- 172.21.15.1:0/3908047108 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555f1b4491e0 con 0x555f1b0f3d80 2024-05-04T03:23:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.590+0000 7f5952ff5640 10 client.15780 handle_command_reply: tid=0 2024-05-04T03:23:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.590+0000 7f597c1d7740 1 client.15780 shutdown 2024-05-04T03:23:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.590+0000 7f597c1d7740 1 -- 172.21.15.1:0/3908047108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f1b1351b0 msgr2=0x555f1b4ebe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.590+0000 7f597c1d7740 1 --2- 172.21.15.1:0/3908047108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f1b1351b0 0x555f1b4ebe60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f596401c040 tx=0x7f596400b630 comp rx=0 tx=0).stop 2024-05-04T03:23:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.590+0000 7f597c1d7740 1 -- 172.21.15.1:0/3908047108 shutdown_connections 2024-05-04T03:23:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.590+0000 7f597c1d7740 1 --2- 172.21.15.1:0/3908047108 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x555f1b0f3d80 0x555f1b4f21b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.590+0000 7f597c1d7740 1 --2- 172.21.15.1:0/3908047108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f1b1351b0 0x555f1b4ebe60 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.590+0000 7f597c1d7740 1 --2- 172.21.15.1:0/3908047108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f1b1144d0 0x555f1b4e4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.590+0000 7f5952ff5640 0 client.15780 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:23:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.590+0000 7f597c1d7740 1 --2- 172.21.15.1:0/3908047108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f1b10c2f0 0x555f1b4e49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.590+0000 7f597c1d7740 1 -- 172.21.15.1:0/3908047108 >> 172.21.15.1:0/3908047108 conn(0x555f1b129300 msgr2=0x555f1b4d5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.590+0000 7f597c1d7740 1 -- 172.21.15.1:0/3908047108 shutdown_connections 2024-05-04T03:23:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.590+0000 7f597c1d7740 1 -- 172.21.15.1:0/3908047108 wait complete. 2024-05-04T03:23:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.591+0000 7f597c1d7740 20 client.15780 trim_cache size 0 max 16384 2024-05-04T03:23:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.606+0000 7f597c1d7740 1 Processor -- start 2024-05-04T03:23:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.606+0000 7f597c1d7740 1 -- start start 2024-05-04T03:23:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.606+0000 7f597c1d7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f1b10c2f0 0x555f1b4de5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.606+0000 7f597c1d7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f1b1144d0 0x555f1b4ebd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.606+0000 7f597c1d7740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f1b1351b0 0x555f1b4ec2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.606+0000 7f597c1d7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f59701ab780 con 0x555f1b1144d0 2024-05-04T03:23:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.606+0000 7f597c1d7740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f59701a9050 con 0x555f1b1351b0 2024-05-04T03:23:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.606+0000 7f597c1d7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f59701a6ec0 con 0x555f1b10c2f0 2024-05-04T03:23:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.606+0000 7f596effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f1b1144d0 0x555f1b4ebd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.606+0000 7f596effd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f1b1144d0 0x555f1b4ebd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47964/0 (socket says 172.21.15.1:47964) 2024-05-04T03:23:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.606+0000 7f596effd640 1 -- 172.21.15.1:0/1942335751 learned_addr learned my addr 172.21.15.1:0/1942335751 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.606+0000 7f596e7fc640 1 --2- 172.21.15.1:0/1942335751 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f1b1351b0 0x555f1b4ec2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.606+0000 7f596e7fc640 1 --2- 172.21.15.1:0/1942335751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f1b10c2f0 0x555f1b4de5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.606+0000 7f596effd640 1 -- 172.21.15.1:0/1942335751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f1b10c2f0 msgr2=0x555f1b4de5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.606+0000 7f596effd640 1 --2- 172.21.15.1:0/1942335751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f1b10c2f0 0x555f1b4de5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.606+0000 7f596effd640 1 -- 172.21.15.1:0/1942335751 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f1b1351b0 msgr2=0x555f1b4ec2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.607+0000 7f596effd640 1 --2- 172.21.15.1:0/1942335751 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f1b1351b0 0x555f1b4ec2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.607+0000 7f596effd640 1 -- 172.21.15.1:0/1942335751 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f1b4df8a0 con 0x555f1b1144d0 2024-05-04T03:23:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.607+0000 7f596effd640 1 --2- 172.21.15.1:0/1942335751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f1b1144d0 0x555f1b4ebd80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5960009230 tx=0x7f5960031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c3b7733d763989 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.607+0000 7f5954ff9640 1 -- 172.21.15.1:0/1942335751 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5960035040 con 0x555f1b1144d0 2024-05-04T03:23:48.611 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.607+0000 7f5954ff9640 1 -- 172.21.15.1:0/1942335751 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5960049070 con 0x555f1b1144d0 2024-05-04T03:23:48.611 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.607+0000 7f5954ff9640 1 -- 172.21.15.1:0/1942335751 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5960004920 con 0x555f1b1144d0 2024-05-04T03:23:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.608+0000 7f597c1d7740 1 -- 172.21.15.1:0/1942335751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f1b1144d0 msgr2=0x555f1b4ebd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.608+0000 7f597c1d7740 1 --2- 172.21.15.1:0/1942335751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f1b1144d0 0x555f1b4ebd80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5960009230 tx=0x7f5960031400 comp rx=0 tx=0).stop 2024-05-04T03:23:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.608+0000 7f597c1d7740 1 -- 172.21.15.1:0/1942335751 shutdown_connections 2024-05-04T03:23:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.608+0000 7f597c1d7740 1 --2- 172.21.15.1:0/1942335751 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f1b1351b0 0x555f1b4ec2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.608+0000 7f597c1d7740 1 --2- 172.21.15.1:0/1942335751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f1b1144d0 0x555f1b4ebd80 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.608+0000 7f597c1d7740 1 --2- 172.21.15.1:0/1942335751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f1b10c2f0 0x555f1b4de5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.608+0000 7f597c1d7740 1 -- 172.21.15.1:0/1942335751 >> 172.21.15.1:0/1942335751 conn(0x555f1b129300 msgr2=0x555f1b4ddee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.608+0000 7f597c1d7740 1 -- 172.21.15.1:0/1942335751 shutdown_connections 2024-05-04T03:23:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.608+0000 7f597c1d7740 1 -- 172.21.15.1:0/1942335751 wait complete. 2024-05-04T03:23:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.608+0000 7f597c1d7740 1 -- start start 2024-05-04T03:23:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.608+0000 7f597c1d7740 1 Processor -- start 2024-05-04T03:23:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.608+0000 7f597c1d7740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f1b10c2f0 0x555f1b4e9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.608+0000 7f597c1d7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f1b1144d0 0x555f1b4e9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.608+0000 7f597c1d7740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f1b1351b0 0x555f1b4eae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.608+0000 7f597c1d7740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f59701a7510 con 0x555f1b10c2f0 2024-05-04T03:23:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.608+0000 7f597c1d7740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f59701a8980 con 0x555f1b1351b0 2024-05-04T03:23:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.608+0000 7f597c1d7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f59701a7be0 con 0x555f1b1144d0 2024-05-04T03:23:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.609+0000 7f597c1d7740 10 client.0 _ll_register_callbacks cb 0x7f596400be40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:23:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.609+0000 7f596e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f1b10c2f0 0x555f1b4e9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.609+0000 7f596e7fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f1b10c2f0 0x555f1b4e9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47978/0 (socket says 172.21.15.1:47978) 2024-05-04T03:23:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.609+0000 7f596e7fc640 1 -- 172.21.15.1:0/787723471 learned_addr learned my addr 172.21.15.1:0/787723471 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:23:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.609+0000 7f596e7fc640 1 --2- 172.21.15.1:0/787723471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f1b1351b0 0x555f1b4eae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.609+0000 7f596effd640 1 --2- 172.21.15.1:0/787723471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f1b1144d0 0x555f1b4e9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.609+0000 7f596e7fc640 1 -- 172.21.15.1:0/787723471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f1b1144d0 msgr2=0x555f1b4e9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.609+0000 7f596e7fc640 1 --2- 172.21.15.1:0/787723471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f1b1144d0 0x555f1b4e9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.609+0000 7f596e7fc640 1 -- 172.21.15.1:0/787723471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f1b1351b0 msgr2=0x555f1b4eae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:23:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.609+0000 7f596e7fc640 1 --2- 172.21.15.1:0/787723471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f1b1351b0 0x555f1b4eae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.609+0000 7f596e7fc640 1 -- 172.21.15.1:0/787723471 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f1b4eb420 con 0x555f1b10c2f0 2024-05-04T03:23:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.609+0000 7f596effd640 1 --2- 172.21.15.1:0/787723471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f1b1144d0 0x555f1b4e9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:23:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.609+0000 7f596e7fc640 1 --2- 172.21.15.1:0/787723471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f1b10c2f0 0x555f1b4e9a60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5964012110 tx=0x7f5964023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b15ede16d46cba server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:23:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.609+0000 7f5952ff5640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:23:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.609+0000 7f5952ff5640 1 -- 172.21.15.1:0/787723471 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964004e90 con 0x555f1b10c2f0 2024-05-04T03:23:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.609+0000 7f5952ff5640 1 -- 172.21.15.1:0/787723471 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f5964004020 con 0x555f1b10c2f0 2024-05-04T03:23:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.609+0000 7f597c1d7740 1 -- 172.21.15.1:0/787723471 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5960049070 con 0x555f1b10c2f0 2024-05-04T03:23:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.609+0000 7f5952ff5640 1 -- 172.21.15.1:0/787723471 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5964004220 con 0x555f1b10c2f0 2024-05-04T03:23:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.609+0000 7f597c1d7740 10 client.15786 fetch_fsmap learned FSMap version 32 2024-05-04T03:23:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.609+0000 7f597c1d7740 1 -- 172.21.15.1:0/787723471 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555f1b4ec6b0 con 0x555f1b10c2f0 2024-05-04T03:23:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.610+0000 7f5952ff5640 1 -- 172.21.15.1:0/787723471 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5960049070 con 0x555f1b10c2f0 2024-05-04T03:23:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.610+0000 7f5952ff5640 1 -- 172.21.15.1:0/787723471 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f596403e050 con 0x555f1b10c2f0 2024-05-04T03:23:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.610+0000 7f597c1d7740 10 client.15786 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:23:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.610+0000 7f597c1d7740 10 client.15786 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:23:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.610+0000 7f597c1d7740 20 client.15786 populate_metadata read hostname 'smithi001' 2024-05-04T03:23:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.610+0000 7f597c1d7740 1 --2- 172.21.15.1:0/787723471 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x555f1b0c91c0 0x555f1b55cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:23:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.610+0000 7f597c1d7740 4 client.15786 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:23:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.610+0000 7f597c1d7740 1 -- 172.21.15.1:0/787723471 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555f1b443050 con 0x555f1b0c91c0 2024-05-04T03:23:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.610+0000 7f596effd640 1 --2- 172.21.15.1:0/787723471 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x555f1b0c91c0 0x555f1b55cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:23:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.610+0000 7f5952ff5640 10 client.15786 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:23:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.610+0000 7f596effd640 1 --2- 172.21.15.1:0/787723471 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x555f1b0c91c0 0x555f1b55cad0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaded878ed333a70 server_cookie=599a07fa389ce06b in_seq=0 out_seq=0 2024-05-04T03:23:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.620+0000 7f5952ff5640 1 -- 172.21.15.1:0/787723471 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555f1b443050 con 0x555f1b0c91c0 2024-05-04T03:23:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.620+0000 7f5952ff5640 10 client.15786 handle_command_reply: tid=0 2024-05-04T03:23:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.620+0000 7f597c1d7740 1 client.15786 shutdown 2024-05-04T03:23:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.620+0000 7f597c1d7740 1 -- 172.21.15.1:0/787723471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f1b10c2f0 msgr2=0x555f1b4e9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.620+0000 7f597c1d7740 1 --2- 172.21.15.1:0/787723471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f1b10c2f0 0x555f1b4e9a60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5964012110 tx=0x7f5964023040 comp rx=0 tx=0).stop 2024-05-04T03:23:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.620+0000 7f597c1d7740 1 -- 172.21.15.1:0/787723471 shutdown_connections 2024-05-04T03:23:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.620+0000 7f597c1d7740 1 --2- 172.21.15.1:0/787723471 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x555f1b0c91c0 0x555f1b55cad0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.620+0000 7f597c1d7740 1 --2- 172.21.15.1:0/787723471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555f1b1351b0 0x555f1b4eae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.620+0000 7f597c1d7740 1 --2- 172.21.15.1:0/787723471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f1b1144d0 0x555f1b4e9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.620+0000 7f5952ff5640 0 client.15786 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:23:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.620+0000 7f597c1d7740 1 --2- 172.21.15.1:0/787723471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f1b10c2f0 0x555f1b4e9a60 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.620+0000 7f597c1d7740 1 -- 172.21.15.1:0/787723471 >> 172.21.15.1:0/787723471 conn(0x555f1b129300 msgr2=0x555f1b4d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.620+0000 7f597c1d7740 1 -- 172.21.15.1:0/787723471 shutdown_connections 2024-05-04T03:23:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.620+0000 7f597c1d7740 1 -- 172.21.15.1:0/787723471 wait complete. 2024-05-04T03:23:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.620+0000 7f597c1d7740 20 client.15786 trim_cache size 0 max 16384 2024-05-04T03:23:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:23:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:23:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:23:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:23:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.621+0000 7f5975327640 1 -- 172.21.15.1:0/1374067530 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f59580b28a0 msgr2=0x7f59580b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.621+0000 7f5975327640 1 --2- 172.21.15.1:0/1374067530 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f59580b28a0 0x7f59580b4db0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f59680a5c80 tx=0x7f59680a5b60 comp rx=0 tx=0).stop 2024-05-04T03:23:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.621+0000 7f5975327640 1 -- 172.21.15.1:0/1374067530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5970109bd0 msgr2=0x7f59701a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:23:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.621+0000 7f5975327640 1 --2- 172.21.15.1:0/1374067530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5970109bd0 0x7f59701a1850 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f596800cce0 tx=0x7f596800a620 comp rx=0 tx=0).stop 2024-05-04T03:23:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.621+0000 7f5975327640 1 -- 172.21.15.1:0/1374067530 shutdown_connections 2024-05-04T03:23:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.621+0000 7f5975327640 1 --2- 172.21.15.1:0/1374067530 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f59580b28a0 0x7f59580b4db0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.621+0000 7f5975327640 1 --2- 172.21.15.1:0/1374067530 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5970104ed0 0x7f597019a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.621+0000 7f5975327640 1 --2- 172.21.15.1:0/1374067530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5970109bd0 0x7f59701a1850 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.621+0000 7f5975327640 1 --2- 172.21.15.1:0/1374067530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5970103c90 0x7f597019a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:23:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.621+0000 7f5975327640 1 -- 172.21.15.1:0/1374067530 >> 172.21.15.1:0/1374067530 conn(0x7f59700feea0 msgr2=0x7f5970105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:23:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.621+0000 7f5975327640 1 -- 172.21.15.1:0/1374067530 shutdown_connections 2024-05-04T03:23:48.626 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:23:48.621+0000 7f5975327640 1 -- 172.21.15.1:0/1374067530 wait complete. 2024-05-04T03:23:48.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:48 smithi072 ceph-mon[33332]: pgmap v530: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-04T03:23:49.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:48 smithi001 ceph-mon[29731]: pgmap v530: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-04T03:23:49.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:48 smithi077 ceph-mon[33531]: pgmap v530: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-04T03:23:50.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:50 smithi072 ceph-mon[33332]: pgmap v531: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-05-04T03:23:51.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:50 smithi001 ceph-mon[29731]: pgmap v531: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-05-04T03:23:51.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:50 smithi077 ceph-mon[33531]: pgmap v531: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-05-04T03:23:52.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:52 smithi072 ceph-mon[33332]: pgmap v532: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-04T03:23:53.130 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:52 smithi001 ceph-mon[29731]: pgmap v532: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-04T03:23:53.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:52 smithi077 ceph-mon[33531]: pgmap v532: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 6 op/s 2024-05-04T03:23:54.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:54 smithi072 ceph-mon[33332]: pgmap v533: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-04T03:23:55.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:54 smithi001 ceph-mon[29731]: pgmap v533: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-04T03:23:55.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:54 smithi077 ceph-mon[33531]: pgmap v533: 97 pgs: 97 active+clean; 93 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-05-04T03:23:56.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:56 smithi072 ceph-mon[33332]: pgmap v534: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 49 KiB/s wr, 8 op/s 2024-05-04T03:23:57.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:56 smithi001 ceph-mon[29731]: pgmap v534: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 49 KiB/s wr, 8 op/s 2024-05-04T03:23:57.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:56 smithi077 ceph-mon[33531]: pgmap v534: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 49 KiB/s wr, 8 op/s 2024-05-04T03:23:58.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:58 smithi072 ceph-mon[33332]: pgmap v535: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 49 KiB/s wr, 7 op/s 2024-05-04T03:23:59.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:58 smithi001 ceph-mon[29731]: pgmap v535: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 49 KiB/s wr, 7 op/s 2024-05-04T03:23:59.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:58 smithi077 ceph-mon[33531]: pgmap v535: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 49 KiB/s wr, 7 op/s 2024-05-04T03:23:59.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:23:59 smithi072 ceph-mon[33332]: pgmap v536: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-05-04T03:24:00.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:23:59 smithi001 ceph-mon[29731]: pgmap v536: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-05-04T03:24:00.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:23:59 smithi077 ceph-mon[33531]: pgmap v536: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 50 KiB/s wr, 10 op/s 2024-05-04T03:24:01.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.733+0000 7fa0e5ed8640 1 -- 172.21.15.1:0/2460447085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0e0102080 msgr2=0x7fa0e0106cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:01.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.733+0000 7fa0e5ed8640 1 --2- 172.21.15.1:0/2460447085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0e0102080 0x7fa0e0106cf0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0009900 tx=0x7fa0d002a410 comp rx=0 tx=0).stop 2024-05-04T03:24:01.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.734+0000 7fa0e5ed8640 1 -- 172.21.15.1:0/2460447085 shutdown_connections 2024-05-04T03:24:01.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.734+0000 7fa0e5ed8640 1 --2- 172.21.15.1:0/2460447085 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa0e0107230 0x7fa0e0109650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.734+0000 7fa0e5ed8640 1 --2- 172.21.15.1:0/2460447085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0e0109b90 0x7fa0e010bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.734+0000 7fa0e5ed8640 1 --2- 172.21.15.1:0/2460447085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0e0102080 0x7fa0e0106cf0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.734+0000 7fa0e5ed8640 1 -- 172.21.15.1:0/2460447085 >> 172.21.15.1:0/2460447085 conn(0x7fa0e00fd4e0 msgr2=0x7fa0e00ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:01.736 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.734+0000 7fa0e5ed8640 1 -- 172.21.15.1:0/2460447085 shutdown_connections 2024-05-04T03:24:01.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.734+0000 7fa0e5ed8640 1 -- 172.21.15.1:0/2460447085 wait complete. 2024-05-04T03:24:01.737 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.734+0000 7fa0e5ed8640 1 Processor -- start 2024-05-04T03:24:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.734+0000 7fa0e5ed8640 1 -- start start 2024-05-04T03:24:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.735+0000 7fa0e5ed8640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa0e0102080 0x7fa0e019a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.735+0000 7fa0e5ed8640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0e0107230 0x7fa0e019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.735+0000 7fa0e5ed8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0e0109b90 0x7fa0e01a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.735+0000 7fa0e5ed8640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e010f7e0 con 0x7fa0e0107230 2024-05-04T03:24:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.735+0000 7fa0e5ed8640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e010f660 con 0x7fa0e0102080 2024-05-04T03:24:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.735+0000 7fa0e5ed8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e010f960 con 0x7fa0e0109b90 2024-05-04T03:24:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.735+0000 7fa0deffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0e0107230 0x7fa0e019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.735+0000 7fa0deffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0e0107230 0x7fa0e019a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41184/0 (socket says 172.21.15.1:41184) 2024-05-04T03:24:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.735+0000 7fa0deffd640 1 -- 172.21.15.1:0/2396630523 learned_addr learned my addr 172.21.15.1:0/2396630523 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.735+0000 7fa0df7fe640 1 --2- 172.21.15.1:0/2396630523 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa0e0102080 0x7fa0e019a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.735+0000 7fa0dffff640 1 --2- 172.21.15.1:0/2396630523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0e0109b90 0x7fa0e01a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.735+0000 7fa0deffd640 1 -- 172.21.15.1:0/2396630523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0e0109b90 msgr2=0x7fa0e01a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.735+0000 7fa0deffd640 1 --2- 172.21.15.1:0/2396630523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0e0109b90 0x7fa0e01a15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.735+0000 7fa0deffd640 1 -- 172.21.15.1:0/2396630523 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa0e0102080 msgr2=0x7fa0e019a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.735+0000 7fa0deffd640 1 --2- 172.21.15.1:0/2396630523 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa0e0102080 0x7fa0e019a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.735+0000 7fa0deffd640 1 -- 172.21.15.1:0/2396630523 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e01a1c50 con 0x7fa0e0107230 2024-05-04T03:24:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.735+0000 7fa0deffd640 1 --2- 172.21.15.1:0/2396630523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0e0107230 0x7fa0e019a650 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400d950 tx=0x7fa0d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.735+0000 7fa0dcff9640 1 -- 172.21.15.1:0/2396630523 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4017070 con 0x7fa0e0107230 2024-05-04T03:24:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.736+0000 7fa0dcff9640 1 -- 172.21.15.1:0/2396630523 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0d4009c30 con 0x7fa0e0107230 2024-05-04T03:24:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.736+0000 7fa0e5ed8640 1 -- 172.21.15.1:0/2396630523 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0e01a1f40 con 0x7fa0e0107230 2024-05-04T03:24:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.736+0000 7fa0dcff9640 1 -- 172.21.15.1:0/2396630523 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d400f620 con 0x7fa0e0107230 2024-05-04T03:24:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.736+0000 7fa0e5ed8640 1 -- 172.21.15.1:0/2396630523 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0e01a23a0 con 0x7fa0e0107230 2024-05-04T03:24:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.738+0000 7fa0ecd88740 1 Processor -- start 2024-05-04T03:24:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.738+0000 7fa0dcff9640 1 -- 172.21.15.1:0/2396630523 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7fa0d4010460 con 0x7fa0e0107230 2024-05-04T03:24:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.738+0000 7fa0ecd88740 1 -- start start 2024-05-04T03:24:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.738+0000 7fa0ecd88740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ba71b1b0 0x55d7baabca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.738+0000 7fa0ecd88740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d7ba6f22f0 0x55d7baabcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.738+0000 7fa0ecd88740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7ba6fa4d0 0x55d7baac3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.738+0000 7fa0ecd88740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d7baa215d0 con 0x55d7ba6fa4d0 2024-05-04T03:24:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.738+0000 7fa0ecd88740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55d7ba9da890 con 0x55d7ba6f22f0 2024-05-04T03:24:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.738+0000 7fa0ecd88740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d7baa1d590 con 0x55d7ba71b1b0 2024-05-04T03:24:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.739+0000 7fa0deffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7ba6fa4d0 0x55d7baac3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.739+0000 7fa0deffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7ba6fa4d0 0x55d7baac3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41200/0 (socket says 172.21.15.1:41200) 2024-05-04T03:24:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.739+0000 7fa0deffd640 1 -- 172.21.15.1:0/2716551770 learned_addr learned my addr 172.21.15.1:0/2716551770 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.739+0000 7fa0dffff640 1 --2- 172.21.15.1:0/2716551770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ba71b1b0 0x55d7baabca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.739+0000 7fa0df7fe640 1 --2- 172.21.15.1:0/2716551770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d7ba6f22f0 0x55d7baabcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.739+0000 7fa0deffd640 1 -- 172.21.15.1:0/2716551770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ba71b1b0 msgr2=0x55d7baabca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.739+0000 7fa0deffd640 1 --2- 172.21.15.1:0/2716551770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ba71b1b0 0x55d7baabca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.739+0000 7fa0deffd640 1 -- 172.21.15.1:0/2716551770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d7ba6f22f0 msgr2=0x55d7baabcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.739+0000 7fa0deffd640 1 --2- 172.21.15.1:0/2716551770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d7ba6f22f0 0x55d7baabcfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.739+0000 7fa0dcff9640 1 --2- 172.21.15.1:0/2396630523 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa0c80b28a0 0x7fa0c80b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.739+0000 7fa0deffd640 1 -- 172.21.15.1:0/2716551770 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7ba733210 con 0x55d7ba6fa4d0 2024-05-04T03:24:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.739+0000 7fa0dffff640 1 --2- 172.21.15.1:0/2396630523 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa0c80b28a0 0x7fa0c80b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.739+0000 7fa0deffd640 1 --2- 172.21.15.1:0/2716551770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7ba6fa4d0 0x55d7baac3ee0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40a54e0 tx=0x7fa0d40a5ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0df167028fa7e94 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.739+0000 7fa0dcff9640 1 -- 172.21.15.1:0/2396630523 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fa0d40e1d80 con 0x7fa0e0107230 2024-05-04T03:24:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.739+0000 7fa0c5ffb640 1 -- 172.21.15.1:0/2716551770 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4020020 con 0x55d7ba6fa4d0 2024-05-04T03:24:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.740+0000 7fa0c5ffb640 1 -- 172.21.15.1:0/2716551770 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0d40a8f70 con 0x55d7ba6fa4d0 2024-05-04T03:24:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.740+0000 7fa0c5ffb640 1 -- 172.21.15.1:0/2716551770 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d40a8db0 con 0x55d7ba6fa4d0 2024-05-04T03:24:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.740+0000 7fa0ecd88740 1 -- 172.21.15.1:0/2716551770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7ba6fa4d0 msgr2=0x55d7baac3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.740+0000 7fa0ecd88740 1 --2- 172.21.15.1:0/2716551770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7ba6fa4d0 0x55d7baac3ee0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40a54e0 tx=0x7fa0d40a5ab0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.740+0000 7fa0dffff640 1 --2- 172.21.15.1:0/2396630523 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa0c80b28a0 0x7fa0c80b4db0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55d7baabdc30 tx=0x7fa0d80085f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.740+0000 7fa0ecd88740 1 -- 172.21.15.1:0/2716551770 shutdown_connections 2024-05-04T03:24:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.740+0000 7fa0ecd88740 1 --2- 172.21.15.1:0/2716551770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d7ba6f22f0 0x55d7baabcfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.740+0000 7fa0ecd88740 1 --2- 172.21.15.1:0/2716551770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7ba6fa4d0 0x55d7baac3ee0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.740+0000 7fa0ecd88740 1 --2- 172.21.15.1:0/2716551770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ba71b1b0 0x55d7baabca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.740+0000 7fa0ecd88740 1 -- 172.21.15.1:0/2716551770 >> 172.21.15.1:0/2716551770 conn(0x55d7ba70f300 msgr2=0x55d7baab83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.740+0000 7fa0ecd88740 1 -- 172.21.15.1:0/2716551770 shutdown_connections 2024-05-04T03:24:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.741+0000 7fa0ecd88740 1 -- 172.21.15.1:0/2716551770 wait complete. 2024-05-04T03:24:01.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.741+0000 7fa0ecd88740 1 -- start start 2024-05-04T03:24:01.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.741+0000 7fa0ecd88740 1 Processor -- start 2024-05-04T03:24:01.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.741+0000 7fa0ecd88740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ba6f22f0 0x55d7baac6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:01.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.741+0000 7fa0ecd88740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d7ba6fa4d0 0x55d7baac6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:01.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.741+0000 7fa0ecd88740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7ba71b1b0 0x55d7baacdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:01.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.741+0000 7fa0ecd88740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55d7baac4930 con 0x55d7ba71b1b0 2024-05-04T03:24:01.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.741+0000 7fa0ecd88740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55d7ba9de370 con 0x55d7ba6fa4d0 2024-05-04T03:24:01.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.741+0000 7fa0ecd88740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d7baac4ab0 con 0x55d7ba6f22f0 2024-05-04T03:24:01.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.741+0000 7fa0ecd88740 10 client.0 _ll_register_callbacks cb 0x55d7ba732a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:24:01.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.742+0000 7fa0dffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7ba71b1b0 0x55d7baacdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:01.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.742+0000 7fa0dffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7ba71b1b0 0x55d7baacdb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41216/0 (socket says 172.21.15.1:41216) 2024-05-04T03:24:01.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.742+0000 7fa0dffff640 1 -- 172.21.15.1:0/1877511837 learned_addr learned my addr 172.21.15.1:0/1877511837 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:01.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.742+0000 7fa0df7fe640 1 --2- 172.21.15.1:0/1877511837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ba6f22f0 0x55d7baac6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.742+0000 7fa0deffd640 1 --2- 172.21.15.1:0/1877511837 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d7ba6fa4d0 0x55d7baac6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.742+0000 7fa0dffff640 1 -- 172.21.15.1:0/1877511837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ba6f22f0 msgr2=0x55d7baac6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.742+0000 7fa0dffff640 1 --2- 172.21.15.1:0/1877511837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ba6f22f0 0x55d7baac6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.742+0000 7fa0dffff640 1 -- 172.21.15.1:0/1877511837 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d7ba6fa4d0 msgr2=0x55d7baac6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.742+0000 7fa0dffff640 1 --2- 172.21.15.1:0/1877511837 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d7ba6fa4d0 0x55d7baac6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.742+0000 7fa0dffff640 1 -- 172.21.15.1:0/1877511837 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7ba7ff620 con 0x55d7ba71b1b0 2024-05-04T03:24:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.742+0000 7fa0dffff640 1 --2- 172.21.15.1:0/1877511837 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7ba71b1b0 0x55d7baacdb10 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800eda0 tx=0x7fa0d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6cdeaa6b69d1892 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.742+0000 7fa0beffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:24:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.742+0000 7fa0beffd640 1 -- 172.21.15.1:0/1877511837 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d801cb20 con 0x55d7ba71b1b0 2024-05-04T03:24:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.742+0000 7fa0beffd640 1 -- 172.21.15.1:0/1877511837 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fa0d801ccc0 con 0x55d7ba71b1b0 2024-05-04T03:24:01.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.742+0000 7fa0ecd88740 1 -- 172.21.15.1:0/1877511837 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0d40a8f70 con 0x55d7ba71b1b0 2024-05-04T03:24:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.742+0000 7fa0beffd640 1 -- 172.21.15.1:0/1877511837 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa0d8028480 con 0x55d7ba71b1b0 2024-05-04T03:24:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.743+0000 7fa0ecd88740 10 client.15798 fetch_fsmap learned FSMap version 32 2024-05-04T03:24:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.743+0000 7fa0ecd88740 1 -- 172.21.15.1:0/1877511837 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d7baad0b60 con 0x55d7ba71b1b0 2024-05-04T03:24:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.743+0000 7fa0beffd640 1 -- 172.21.15.1:0/1877511837 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0d40a8f70 con 0x55d7ba71b1b0 2024-05-04T03:24:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.743+0000 7fa0beffd640 1 -- 172.21.15.1:0/1877511837 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fa0d8032020 con 0x55d7ba71b1b0 2024-05-04T03:24:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.743+0000 7fa0ecd88740 10 client.15798 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:24:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.743+0000 7fa0ecd88740 10 client.15798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:24:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.743+0000 7fa0ecd88740 20 client.15798 populate_metadata read hostname 'smithi001' 2024-05-04T03:24:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.743+0000 7fa0ecd88740 1 --2- 172.21.15.1:0/1877511837 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55d7ba6d9d80 0x55d7baad4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.743+0000 7fa0ecd88740 4 client.15798 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:24:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.743+0000 7fa0ecd88740 1 -- 172.21.15.1:0/1877511837 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7baa2f1e0 con 0x55d7ba6d9d80 2024-05-04T03:24:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.743+0000 7fa0df7fe640 1 --2- 172.21.15.1:0/1877511837 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55d7ba6d9d80 0x55d7baad4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.744+0000 7fa0df7fe640 1 --2- 172.21.15.1:0/1877511837 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55d7ba6d9d80 0x55d7baad4280 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6097d08c9b9bd970 server_cookie=1a22043bf60049cd in_seq=0 out_seq=0 2024-05-04T03:24:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.744+0000 7fa0beffd640 10 client.15798 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:24:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.744+0000 7fa0beffd640 1 -- 172.21.15.1:0/1877511837 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d7baa2f1e0 con 0x55d7ba6d9d80 2024-05-04T03:24:01.751 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.744+0000 7fa0beffd640 10 client.15798 handle_command_reply: tid=0 2024-05-04T03:24:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.744+0000 7fa0ecd88740 1 client.15798 shutdown 2024-05-04T03:24:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.745+0000 7fa0ecd88740 1 -- 172.21.15.1:0/1877511837 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7ba71b1b0 msgr2=0x55d7baacdb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.745+0000 7fa0ecd88740 1 --2- 172.21.15.1:0/1877511837 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7ba71b1b0 0x55d7baacdb10 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800eda0 tx=0x7fa0d8004a40 comp rx=0 tx=0).stop 2024-05-04T03:24:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.745+0000 7fa0ecd88740 1 -- 172.21.15.1:0/1877511837 shutdown_connections 2024-05-04T03:24:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.745+0000 7fa0ecd88740 1 --2- 172.21.15.1:0/1877511837 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55d7ba6d9d80 0x55d7baad4280 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.745+0000 7fa0ecd88740 1 --2- 172.21.15.1:0/1877511837 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d7ba6fa4d0 0x55d7baac6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.745+0000 7fa0ecd88740 1 --2- 172.21.15.1:0/1877511837 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7ba71b1b0 0x55d7baacdb10 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.745+0000 7fa0ecd88740 1 --2- 172.21.15.1:0/1877511837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ba6f22f0 0x55d7baac6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.745+0000 7fa0ecd88740 1 -- 172.21.15.1:0/1877511837 >> 172.21.15.1:0/1877511837 conn(0x55d7ba70f300 msgr2=0x55d7baab73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.745+0000 7fa0beffd640 0 client.15798 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:24:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.745+0000 7fa0ecd88740 1 -- 172.21.15.1:0/1877511837 shutdown_connections 2024-05-04T03:24:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.745+0000 7fa0ecd88740 1 -- 172.21.15.1:0/1877511837 wait complete. 2024-05-04T03:24:01.752 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.745+0000 7fa0ecd88740 20 client.15798 trim_cache size 0 max 16384 2024-05-04T03:24:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.760+0000 7fa0ecd88740 1 Processor -- start 2024-05-04T03:24:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.760+0000 7fa0ecd88740 1 -- start start 2024-05-04T03:24:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.760+0000 7fa0ecd88740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7ba6f22f0 0x55d7baab7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.761+0000 7fa0ecd88740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d7ba6fa4d0 0x55d7baab8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.761+0000 7fa0ecd88740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ba71b1b0 0x55d7baab6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.761+0000 7fa0ecd88740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e01ad790 con 0x55d7ba6f22f0 2024-05-04T03:24:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.761+0000 7fa0ecd88740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e01ab060 con 0x55d7ba6fa4d0 2024-05-04T03:24:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.761+0000 7fa0ecd88740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e01a8ed0 con 0x55d7ba71b1b0 2024-05-04T03:24:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.761+0000 7fa0df7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7ba6f22f0 0x55d7baab7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.761+0000 7fa0df7fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7ba6f22f0 0x55d7baab7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41220/0 (socket says 172.21.15.1:41220) 2024-05-04T03:24:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.761+0000 7fa0df7fe640 1 -- 172.21.15.1:0/1132909594 learned_addr learned my addr 172.21.15.1:0/1132909594 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.761+0000 7fa0df7fe640 1 -- 172.21.15.1:0/1132909594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ba71b1b0 msgr2=0x55d7baab6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.761+0000 7fa0df7fe640 1 --2- 172.21.15.1:0/1132909594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ba71b1b0 0x55d7baab6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.761+0000 7fa0deffd640 1 --2- 172.21.15.1:0/1132909594 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d7ba6fa4d0 0x55d7baab8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.761+0000 7fa0df7fe640 1 -- 172.21.15.1:0/1132909594 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d7ba6fa4d0 msgr2=0x55d7baab8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.761+0000 7fa0df7fe640 1 --2- 172.21.15.1:0/1132909594 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d7ba6fa4d0 0x55d7baab8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.761+0000 7fa0df7fe640 1 -- 172.21.15.1:0/1132909594 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7baabd750 con 0x55d7ba6f22f0 2024-05-04T03:24:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.761+0000 7fa0deffd640 1 --2- 172.21.15.1:0/1132909594 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d7ba6fa4d0 0x55d7baab8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:24:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.761+0000 7fa0df7fe640 1 --2- 172.21.15.1:0/1132909594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7ba6f22f0 0x55d7baab7cf0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0104c70 tx=0x7fa0d0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab51787e366d624 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.761+0000 7fa0c57fa640 1 -- 172.21.15.1:0/1132909594 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d004a030 con 0x55d7ba6f22f0 2024-05-04T03:24:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.761+0000 7fa0c57fa640 1 -- 172.21.15.1:0/1132909594 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0d0005020 con 0x55d7ba6f22f0 2024-05-04T03:24:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.761+0000 7fa0c57fa640 1 -- 172.21.15.1:0/1132909594 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d0035040 con 0x55d7ba6f22f0 2024-05-04T03:24:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.762+0000 7fa0ecd88740 1 -- 172.21.15.1:0/1132909594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7ba6f22f0 msgr2=0x55d7baab7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.762+0000 7fa0ecd88740 1 --2- 172.21.15.1:0/1132909594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7ba6f22f0 0x55d7baab7cf0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0104c70 tx=0x7fa0d0000bc0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.762+0000 7fa0ecd88740 1 -- 172.21.15.1:0/1132909594 shutdown_connections 2024-05-04T03:24:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.762+0000 7fa0ecd88740 1 --2- 172.21.15.1:0/1132909594 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d7ba6fa4d0 0x55d7baab8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.762+0000 7fa0ecd88740 1 --2- 172.21.15.1:0/1132909594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ba71b1b0 0x55d7baab6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.762+0000 7fa0ecd88740 1 --2- 172.21.15.1:0/1132909594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7ba6f22f0 0x55d7baab7cf0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.762+0000 7fa0ecd88740 1 -- 172.21.15.1:0/1132909594 >> 172.21.15.1:0/1132909594 conn(0x55d7ba70f300 msgr2=0x55d7baac0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.762+0000 7fa0ecd88740 1 -- 172.21.15.1:0/1132909594 shutdown_connections 2024-05-04T03:24:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.762+0000 7fa0ecd88740 1 -- 172.21.15.1:0/1132909594 wait complete. 2024-05-04T03:24:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.762+0000 7fa0ecd88740 1 -- start start 2024-05-04T03:24:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.763+0000 7fa0ecd88740 1 Processor -- start 2024-05-04T03:24:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.763+0000 7fa0ecd88740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ba6f22f0 0x55d7baaca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.763+0000 7fa0ecd88740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d7ba6fa4d0 0x55d7baacab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.763+0000 7fa0ecd88740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7ba71b1b0 0x55d7baacba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.763+0000 7fa0ecd88740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e01a9520 con 0x55d7ba71b1b0 2024-05-04T03:24:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.763+0000 7fa0ecd88740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e01aa990 con 0x55d7ba6fa4d0 2024-05-04T03:24:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.763+0000 7fa0ecd88740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e01a9bf0 con 0x55d7ba6f22f0 2024-05-04T03:24:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.763+0000 7fa0ecd88740 10 client.0 _ll_register_callbacks cb 0x7fa0d8008890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:24:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.763+0000 7fa0dffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7ba71b1b0 0x55d7baacba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.763+0000 7fa0dffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7ba71b1b0 0x55d7baacba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41230/0 (socket says 172.21.15.1:41230) 2024-05-04T03:24:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.763+0000 7fa0dffff640 1 -- 172.21.15.1:0/1031985599 learned_addr learned my addr 172.21.15.1:0/1031985599 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.763+0000 7fa0df7fe640 1 --2- 172.21.15.1:0/1031985599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ba6f22f0 0x55d7baaca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.763+0000 7fa0deffd640 1 --2- 172.21.15.1:0/1031985599 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d7ba6fa4d0 0x55d7baacab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.763+0000 7fa0dffff640 1 -- 172.21.15.1:0/1031985599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ba6f22f0 msgr2=0x55d7baaca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.763+0000 7fa0dffff640 1 --2- 172.21.15.1:0/1031985599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ba6f22f0 0x55d7baaca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.763+0000 7fa0dffff640 1 -- 172.21.15.1:0/1031985599 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d7ba6fa4d0 msgr2=0x55d7baacab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.763+0000 7fa0dffff640 1 --2- 172.21.15.1:0/1031985599 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d7ba6fa4d0 0x55d7baacab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.763+0000 7fa0dffff640 1 -- 172.21.15.1:0/1031985599 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7baacc020 con 0x55d7ba71b1b0 2024-05-04T03:24:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.764+0000 7fa0dffff640 1 --2- 172.21.15.1:0/1031985599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7ba71b1b0 0x55d7baacba60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8011ca0 tx=0x7fa0d801a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ecb7e97c89b23a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.764+0000 7fa0beffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:24:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.764+0000 7fa0beffd640 1 -- 172.21.15.1:0/1031985599 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8004e50 con 0x55d7ba71b1b0 2024-05-04T03:24:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.764+0000 7fa0beffd640 1 -- 172.21.15.1:0/1031985599 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fa0d801c770 con 0x55d7ba71b1b0 2024-05-04T03:24:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.764+0000 7fa0ecd88740 1 -- 172.21.15.1:0/1031985599 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0d0005020 con 0x55d7ba71b1b0 2024-05-04T03:24:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.764+0000 7fa0beffd640 1 -- 172.21.15.1:0/1031985599 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa0d801c970 con 0x55d7ba71b1b0 2024-05-04T03:24:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.764+0000 7fa0ecd88740 10 client.15804 fetch_fsmap learned FSMap version 32 2024-05-04T03:24:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.764+0000 7fa0ecd88740 1 -- 172.21.15.1:0/1031985599 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d7baacd220 con 0x55d7ba71b1b0 2024-05-04T03:24:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.764+0000 7fa0beffd640 1 -- 172.21.15.1:0/1031985599 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0d0005020 con 0x55d7ba71b1b0 2024-05-04T03:24:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.764+0000 7fa0beffd640 1 -- 172.21.15.1:0/1031985599 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fa0d8044020 con 0x55d7ba71b1b0 2024-05-04T03:24:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.764+0000 7fa0ecd88740 10 client.15804 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:24:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.764+0000 7fa0ecd88740 10 client.15804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:24:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.765+0000 7fa0ecd88740 20 client.15804 populate_metadata read hostname 'smithi001' 2024-05-04T03:24:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.765+0000 7fa0ecd88740 1 --2- 172.21.15.1:0/1031985599 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55d7ba6af1c0 0x55d7bab3ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.765+0000 7fa0ecd88740 4 client.15804 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:24:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.765+0000 7fa0ecd88740 1 -- 172.21.15.1:0/1031985599 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d7baa29050 con 0x55d7ba6af1c0 2024-05-04T03:24:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.765+0000 7fa0df7fe640 1 --2- 172.21.15.1:0/1031985599 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55d7ba6af1c0 0x55d7bab3ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.765+0000 7fa0df7fe640 1 --2- 172.21.15.1:0/1031985599 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55d7ba6af1c0 0x55d7bab3ebd0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcb9cb96a0e6f405 server_cookie=8cacc05d9061cf5e in_seq=0 out_seq=0 2024-05-04T03:24:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.765+0000 7fa0beffd640 10 client.15804 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:24:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.774+0000 7fa0beffd640 1 -- 172.21.15.1:0/1031985599 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d7baa29050 con 0x55d7ba6af1c0 2024-05-04T03:24:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.774+0000 7fa0beffd640 10 client.15804 handle_command_reply: tid=0 2024-05-04T03:24:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.774+0000 7fa0ecd88740 1 client.15804 shutdown 2024-05-04T03:24:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.774+0000 7fa0ecd88740 1 -- 172.21.15.1:0/1031985599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7ba71b1b0 msgr2=0x55d7baacba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.774+0000 7fa0ecd88740 1 --2- 172.21.15.1:0/1031985599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7ba71b1b0 0x55d7baacba60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8011ca0 tx=0x7fa0d801a720 comp rx=0 tx=0).stop 2024-05-04T03:24:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.774+0000 7fa0ecd88740 1 -- 172.21.15.1:0/1031985599 shutdown_connections 2024-05-04T03:24:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.774+0000 7fa0ecd88740 1 --2- 172.21.15.1:0/1031985599 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55d7ba6af1c0 0x55d7bab3ebd0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.774+0000 7fa0ecd88740 1 --2- 172.21.15.1:0/1031985599 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d7ba6fa4d0 0x55d7baacab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.774+0000 7fa0ecd88740 1 --2- 172.21.15.1:0/1031985599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7ba71b1b0 0x55d7baacba60 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.774+0000 7fa0ecd88740 1 --2- 172.21.15.1:0/1031985599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7ba6f22f0 0x55d7baaca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.774+0000 7fa0beffd640 0 client.15804 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:24:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.774+0000 7fa0ecd88740 1 -- 172.21.15.1:0/1031985599 >> 172.21.15.1:0/1031985599 conn(0x55d7ba70f300 msgr2=0x55d7baac0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.774+0000 7fa0ecd88740 1 -- 172.21.15.1:0/1031985599 shutdown_connections 2024-05-04T03:24:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.774+0000 7fa0ecd88740 1 -- 172.21.15.1:0/1031985599 wait complete. 2024-05-04T03:24:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.775+0000 7fa0ecd88740 20 client.15804 trim_cache size 0 max 16384 2024-05-04T03:24:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:24:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:24:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:24:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:24:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.775+0000 7fa0e5ed8640 1 -- 172.21.15.1:0/2396630523 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa0c80b28a0 msgr2=0x7fa0c80b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.775+0000 7fa0e5ed8640 1 --2- 172.21.15.1:0/2396630523 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa0c80b28a0 0x7fa0c80b4db0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55d7baabdc30 tx=0x7fa0d80085f0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.775+0000 7fa0e5ed8640 1 -- 172.21.15.1:0/2396630523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0e0107230 msgr2=0x7fa0e019a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.775+0000 7fa0e5ed8640 1 --2- 172.21.15.1:0/2396630523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0e0107230 0x7fa0e019a650 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400d950 tx=0x7fa0d400de10 comp rx=0 tx=0).stop 2024-05-04T03:24:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.775+0000 7fa0e5ed8640 1 -- 172.21.15.1:0/2396630523 shutdown_connections 2024-05-04T03:24:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.775+0000 7fa0e5ed8640 1 --2- 172.21.15.1:0/2396630523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0e0109b90 0x7fa0e01a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.775+0000 7fa0e5ed8640 1 --2- 172.21.15.1:0/2396630523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa0e0107230 0x7fa0e019a650 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.775+0000 7fa0e5ed8640 1 --2- 172.21.15.1:0/2396630523 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fa0c80b28a0 0x7fa0c80b4db0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.775+0000 7fa0e5ed8640 1 --2- 172.21.15.1:0/2396630523 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa0e0102080 0x7fa0e019a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.775+0000 7fa0e5ed8640 1 -- 172.21.15.1:0/2396630523 >> 172.21.15.1:0/2396630523 conn(0x7fa0e00fd4e0 msgr2=0x7fa0e01052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:01.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.775+0000 7fa0e5ed8640 1 -- 172.21.15.1:0/2396630523 shutdown_connections 2024-05-04T03:24:01.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:01.775+0000 7fa0e5ed8640 1 -- 172.21.15.1:0/2396630523 wait complete. 2024-05-04T03:24:02.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:02 smithi001 ceph-mon[29731]: pgmap v537: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 50 KiB/s wr, 8 op/s 2024-05-04T03:24:02.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:02 smithi077 ceph-mon[33531]: pgmap v537: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 50 KiB/s wr, 8 op/s 2024-05-04T03:24:02.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:02 smithi072 ceph-mon[33332]: pgmap v537: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 50 KiB/s wr, 8 op/s 2024-05-04T03:24:04.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:04 smithi001 ceph-mon[29731]: pgmap v538: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-05-04T03:24:04.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:04 smithi077 ceph-mon[33531]: pgmap v538: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-05-04T03:24:04.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:04 smithi072 ceph-mon[33332]: pgmap v538: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-05-04T03:24:05.699 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:24:05.963 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:24:06.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:06 smithi001 ceph-mon[29731]: pgmap v539: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 50 KiB/s wr, 9 op/s 2024-05-04T03:24:06.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:06 smithi077 ceph-mon[33531]: pgmap v539: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 50 KiB/s wr, 9 op/s 2024-05-04T03:24:06.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:06 smithi072 ceph-mon[33332]: pgmap v539: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 50 KiB/s wr, 9 op/s 2024-05-04T03:24:07.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.716+0000 7f125065e700 1 -- 172.21.15.1:0/3420242019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1248109870 msgr2=0x7f12481111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:07.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.716+0000 7f125065e700 1 --2- 172.21.15.1:0/3420242019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1248109870 0x7f12481111d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f124000b0d0 tx=0x7f124000b3e0 comp rx=0 tx=0).stop 2024-05-04T03:24:07.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.716+0000 7f125065e700 1 -- 172.21.15.1:0/3420242019 shutdown_connections 2024-05-04T03:24:07.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.716+0000 7f125065e700 1 --2- 172.21.15.1:0/3420242019 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f12481045f0 0x7f1248104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:07.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.716+0000 7f125065e700 1 --2- 172.21.15.1:0/3420242019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1248109870 0x7f12481111d0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:07.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.716+0000 7f125065e700 1 --2- 172.21.15.1:0/3420242019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1248108f20 0x7f1248109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:07.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.716+0000 7f125065e700 1 -- 172.21.15.1:0/3420242019 >> 172.21.15.1:0/3420242019 conn(0x7f1248100040 msgr2=0x7f1248102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:07.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.716+0000 7f125065e700 1 -- 172.21.15.1:0/3420242019 shutdown_connections 2024-05-04T03:24:07.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.716+0000 7f125065e700 1 -- 172.21.15.1:0/3420242019 wait complete. 2024-05-04T03:24:07.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.717+0000 7f125065e700 1 Processor -- start 2024-05-04T03:24:07.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.717+0000 7f125065e700 1 -- start start 2024-05-04T03:24:07.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.717+0000 7f125065e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12481045f0 0x7f1248075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:07.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.717+0000 7f125065e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1248108f20 0x7f1248076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:07.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.717+0000 7f125065e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1248109870 0x7f1248076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:07.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.717+0000 7f125065e700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f12481144f0 con 0x7f12481045f0 2024-05-04T03:24:07.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.717+0000 7f125065e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1248114370 con 0x7f1248109870 2024-05-04T03:24:07.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.717+0000 7f125065e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1248114670 con 0x7f1248108f20 2024-05-04T03:24:07.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.717+0000 7f124e3fa700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12481045f0 0x7f1248075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:07.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.717+0000 7f124e3fa700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12481045f0 0x7f1248075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41258/0 (socket says 172.21.15.1:41258) 2024-05-04T03:24:07.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.717+0000 7f124e3fa700 1 -- 172.21.15.1:0/371539258 learned_addr learned my addr 172.21.15.1:0/371539258 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:07.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.717+0000 7f124dbf9700 1 --2- 172.21.15.1:0/371539258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1248108f20 0x7f1248076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:07.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.717+0000 7f124ebfb700 1 --2- 172.21.15.1:0/371539258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1248109870 0x7f1248076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:07.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.718+0000 7f124e3fa700 1 -- 172.21.15.1:0/371539258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1248108f20 msgr2=0x7f1248076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:07.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.718+0000 7f124e3fa700 1 --2- 172.21.15.1:0/371539258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1248108f20 0x7f1248076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:07.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.718+0000 7f124e3fa700 1 -- 172.21.15.1:0/371539258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1248109870 msgr2=0x7f1248076540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:07.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.718+0000 7f124e3fa700 1 --2- 172.21.15.1:0/371539258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1248109870 0x7f1248076540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:07.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.718+0000 7f124e3fa700 1 -- 172.21.15.1:0/371539258 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12481a91a0 con 0x7f12481045f0 2024-05-04T03:24:07.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.718+0000 7f124ebfb700 1 --2- 172.21.15.1:0/371539258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1248109870 0x7f1248076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:24:07.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.718+0000 7f124e3fa700 1 --2- 172.21.15.1:0/371539258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12481045f0 0x7f1248075ac0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f123800e9a0 tx=0x7f123800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:07.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.718+0000 7f12477fe700 1 -- 172.21.15.1:0/371539258 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123800cbd0 con 0x7f12481045f0 2024-05-04T03:24:07.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.718+0000 7f125065e700 1 -- 172.21.15.1:0/371539258 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12481a9490 con 0x7f12481045f0 2024-05-04T03:24:07.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.718+0000 7f12477fe700 1 -- 172.21.15.1:0/371539258 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f123800cd70 con 0x7f12481045f0 2024-05-04T03:24:07.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.718+0000 7f12477fe700 1 -- 172.21.15.1:0/371539258 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1238018680 con 0x7f12481045f0 2024-05-04T03:24:07.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.718+0000 7f125065e700 1 -- 172.21.15.1:0/371539258 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12481a99d0 con 0x7f12481045f0 2024-05-04T03:24:07.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.719+0000 7f125065e700 1 -- 172.21.15.1:0/371539258 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12480512c0 con 0x7f12481045f0 2024-05-04T03:24:07.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.722+0000 7f12477fe700 1 -- 172.21.15.1:0/371539258 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f1238010970 con 0x7f12481045f0 2024-05-04T03:24:07.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.723+0000 7f12477fe700 1 --2- 172.21.15.1:0/371539258 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f12300b2880 0x7f12300b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:07.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.723+0000 7f124dbf9700 1 --2- 172.21.15.1:0/371539258 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f12300b2880 0x7f12300b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:07.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.723+0000 7f12477fe700 1 -- 172.21.15.1:0/371539258 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f12380e2970 con 0x7f12481045f0 2024-05-04T03:24:07.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.723+0000 7f124dbf9700 1 --2- 172.21.15.1:0/371539258 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f12300b2880 0x7f12300b4d90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f12481077f0 tx=0x7f123c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:07.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.727+0000 7f12477fe700 1 -- 172.21.15.1:0/371539258 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f12380aa750 con 0x7f12481045f0 2024-05-04T03:24:07.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.986+0000 7f125065e700 1 -- 172.21.15.1:0/371539258 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f124802e5f0 con 0x7f12481045f0 2024-05-04T03:24:07.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.987+0000 7f12477fe700 1 -- 172.21.15.1:0/371539258 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11844 (secure 0 0 0) 0x7f12380aa050 con 0x7f12481045f0 2024-05-04T03:24:07.990 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:24:07.990 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2186681910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2186681910},{"type":"v1","addr":"172.21.15.1:6835","nonce":2186681910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2747191047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2747191047},{"type":"v1","addr":"172.21.15.1:6837","nonce":2747191047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4115120540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4115120540},{"type":"v1","addr":"172.21.15.1:6839","nonce":4115120540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3405709780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3405709780},{"type":"v1","addr":"172.21.15.1:6841","nonce":3405709780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/3816098761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":3816098761},{"type":"v1","addr":"172.21.15.72:6833","nonce":3816098761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/379026128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":379026128},{"type":"v1","addr":"172.21.15.77:6833","nonce":379026128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:15:49.823471+0000","modified":"2024-05-04T03:23:29.384311+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24472,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/3930984976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3930984976},{"type":"v1","addr":"172.21.15.72:6835","nonce":3930984976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/4279951305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4279951305},{"type":"v1","addr":"172.21.15.77:6835","nonce":4279951305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2003521417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2003521417},{"type":"v1","addr":"172.21.15.77:6837","nonce":2003521417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/1417614203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1417614203},{"type":"v1","addr":"172.21.15.72:6837","nonce":1417614203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.77:6839/922723189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":922723189},{"type":"v1","addr":"172.21.15.77:6839","nonce":922723189}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2976050650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2976050650},{"type":"v1","addr":"172.21.15.72:6839","nonce":2976050650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24467,24472,24494]},"id":1}]} 2024-05-04T03:24:07.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.989+0000 7f125065e700 1 -- 172.21.15.1:0/371539258 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f12300b2880 msgr2=0x7f12300b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:07.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.989+0000 7f125065e700 1 --2- 172.21.15.1:0/371539258 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f12300b2880 0x7f12300b4d90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f12481077f0 tx=0x7f123c009260 comp rx=0 tx=0).stop 2024-05-04T03:24:07.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.989+0000 7f125065e700 1 -- 172.21.15.1:0/371539258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12481045f0 msgr2=0x7f1248075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:07.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.989+0000 7f125065e700 1 --2- 172.21.15.1:0/371539258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12481045f0 0x7f1248075ac0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f123800e9a0 tx=0x7f123800ed60 comp rx=0 tx=0).stop 2024-05-04T03:24:07.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.989+0000 7f125065e700 1 -- 172.21.15.1:0/371539258 shutdown_connections 2024-05-04T03:24:07.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.989+0000 7f125065e700 1 --2- 172.21.15.1:0/371539258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1248109870 0x7f1248076540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:07.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.989+0000 7f125065e700 1 --2- 172.21.15.1:0/371539258 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f12300b2880 0x7f12300b4d90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:07.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.989+0000 7f125065e700 1 --2- 172.21.15.1:0/371539258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12481045f0 0x7f1248075ac0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:07.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.989+0000 7f125065e700 1 --2- 172.21.15.1:0/371539258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1248108f20 0x7f1248076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:07.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.989+0000 7f125065e700 1 -- 172.21.15.1:0/371539258 >> 172.21.15.1:0/371539258 conn(0x7f1248100040 msgr2=0x7f1248108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:07.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.989+0000 7f125065e700 1 -- 172.21.15.1:0/371539258 shutdown_connections 2024-05-04T03:24:07.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:07.989+0000 7f125065e700 1 -- 172.21.15.1:0/371539258 wait complete. 2024-05-04T03:24:07.993 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 32 2024-05-04T03:24:08.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:08 smithi001 ceph-mon[29731]: pgmap v540: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2024-05-04T03:24:08.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:08 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/371539258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:24:08.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:08 smithi077 ceph-mon[33531]: pgmap v540: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2024-05-04T03:24:08.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:08 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/371539258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:24:08.696 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 961ff620 --release --await' 2024-05-04T03:24:08.697 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24494 quiesce db --set-id 961ff620 --release --await 2024-05-04T03:24:08.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:08 smithi072 ceph-mon[33332]: pgmap v540: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2024-05-04T03:24:08.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:08 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/371539258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:24:08.948 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:24:10.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:10 smithi001 ceph-mon[29731]: pgmap v541: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-05-04T03:24:10.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.633+0000 7f15f7e43700 1 -- 172.21.15.1:0/1564514614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15f01045d0 msgr2=0x7f15f01049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:10.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.633+0000 7f15f7e43700 1 --2- 172.21.15.1:0/1564514614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15f01045d0 0x7f15f01049e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f15e800b0d0 tx=0x7f15e800b3e0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.634+0000 7f15f7e43700 1 -- 172.21.15.1:0/1564514614 shutdown_connections 2024-05-04T03:24:10.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.634+0000 7f15f7e43700 1 --2- 172.21.15.1:0/1564514614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15f0109850 0x7f15f01111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.634+0000 7f15f7e43700 1 --2- 172.21.15.1:0/1564514614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15f01045d0 0x7f15f01049e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.634+0000 7f15f7e43700 1 --2- 172.21.15.1:0/1564514614 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f15f0108f00 0x7f15f0109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.634+0000 7f15f7e43700 1 -- 172.21.15.1:0/1564514614 >> 172.21.15.1:0/1564514614 conn(0x7f15f00ffe80 msgr2=0x7f15f01022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:10.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.634+0000 7f15f7e43700 1 -- 172.21.15.1:0/1564514614 shutdown_connections 2024-05-04T03:24:10.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.634+0000 7f15f7e43700 1 -- 172.21.15.1:0/1564514614 wait complete. 2024-05-04T03:24:10.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.635+0000 7f15f7e43700 1 Processor -- start 2024-05-04T03:24:10.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.635+0000 7f15f7e43700 1 -- start start 2024-05-04T03:24:10.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.635+0000 7f15f7e43700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f15f01045d0 0x7f15f019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:10.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.635+0000 7f15f7e43700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15f0108f00 0x7f15f019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:10.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.635+0000 7f15f7e43700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15f0109850 0x7f15f019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:10.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.635+0000 7f15f7e43700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f15f0114480 con 0x7f15f0108f00 2024-05-04T03:24:10.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.635+0000 7f15f7e43700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f15f0114300 con 0x7f15f01045d0 2024-05-04T03:24:10.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.635+0000 7f15f7e43700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f15f0114600 con 0x7f15f0109850 2024-05-04T03:24:10.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.635+0000 7f15f53de700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15f0108f00 0x7f15f019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:10.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.635+0000 7f15f53de700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15f0108f00 0x7f15f019b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41276/0 (socket says 172.21.15.1:41276) 2024-05-04T03:24:10.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.635+0000 7f15f53de700 1 -- 172.21.15.1:0/1457388709 learned_addr learned my addr 172.21.15.1:0/1457388709 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:10.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.635+0000 7f15f5bdf700 1 --2- 172.21.15.1:0/1457388709 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f15f01045d0 0x7f15f019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:10.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.635+0000 7f15f63e0700 1 --2- 172.21.15.1:0/1457388709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15f0109850 0x7f15f019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:10.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.635+0000 7f15f53de700 1 -- 172.21.15.1:0/1457388709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15f0109850 msgr2=0x7f15f019f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:10.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.635+0000 7f15f53de700 1 --2- 172.21.15.1:0/1457388709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15f0109850 0x7f15f019f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.635+0000 7f15f53de700 1 -- 172.21.15.1:0/1457388709 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f15f01045d0 msgr2=0x7f15f019add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:10.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.635+0000 7f15f53de700 1 --2- 172.21.15.1:0/1457388709 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f15f01045d0 0x7f15f019add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.635+0000 7f15f53de700 1 -- 172.21.15.1:0/1457388709 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f019fca0 con 0x7f15f0108f00 2024-05-04T03:24:10.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.636+0000 7f15f53de700 1 --2- 172.21.15.1:0/1457388709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15f0108f00 0x7f15f019b370 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f15e400d980 tx=0x7f15e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:10.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.636+0000 7f15eeffd700 1 -- 172.21.15.1:0/1457388709 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e4017070 con 0x7f15f0108f00 2024-05-04T03:24:10.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.636+0000 7f15eeffd700 1 -- 172.21.15.1:0/1457388709 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15e400fd40 con 0x7f15f0108f00 2024-05-04T03:24:10.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.636+0000 7f15f7e43700 1 -- 172.21.15.1:0/1457388709 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f019ff90 con 0x7f15f0108f00 2024-05-04T03:24:10.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.636+0000 7f15eeffd700 1 -- 172.21.15.1:0/1457388709 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e400e3b0 con 0x7f15f0108f00 2024-05-04T03:24:10.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.636+0000 7f15f7e43700 1 -- 172.21.15.1:0/1457388709 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f0076b00 con 0x7f15f0108f00 2024-05-04T03:24:10.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.640+0000 7f15eeffd700 1 -- 172.21.15.1:0/1457388709 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f15e4010610 con 0x7f15f0108f00 2024-05-04T03:24:10.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.640+0000 7f15eeffd700 1 --2- 172.21.15.1:0/1457388709 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f15d80b2880 0x7f15d80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:10.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.640+0000 7f15f5bdf700 1 --2- 172.21.15.1:0/1457388709 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f15d80b2880 0x7f15d80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:10.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.640+0000 7f15eeffd700 1 -- 172.21.15.1:0/1457388709 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f15e40e1c60 con 0x7f15f0108f00 2024-05-04T03:24:10.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.641+0000 7f1606857b80 1 Processor -- start 2024-05-04T03:24:10.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.641+0000 7f1606857b80 1 -- start start 2024-05-04T03:24:10.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.641+0000 7f15f5bdf700 1 --2- 172.21.15.1:0/1457388709 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f15d80b2880 0x7f15d80b4d90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009900 tx=0x7f15e800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:10.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.641+0000 7f1606857b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5586b030cad0 0x5586b030cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:10.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.641+0000 7f1606857b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586b030d420 0x5586b0317700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:10.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.641+0000 7f1606857b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586b0317c40 0x5586b031a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:10.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.641+0000 7f1606857b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5586b022fa40 con 0x5586b0317c40 2024-05-04T03:24:10.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.641+0000 7f1606857b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5586b0194ea0 con 0x5586b030cad0 2024-05-04T03:24:10.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.641+0000 7f1606857b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5586b022ba00 con 0x5586b030d420 2024-05-04T03:24:10.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.641+0000 7f15f63e0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586b0317c40 0x5586b031a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:10.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.641+0000 7f15f5bdf700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5586b030cad0 0x5586b030cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:10.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.641+0000 7f15f63e0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586b0317c40 0x5586b031a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41288/0 (socket says 172.21.15.1:41288) 2024-05-04T03:24:10.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.641+0000 7f15f63e0700 1 -- 172.21.15.1:0/3620960735 learned_addr learned my addr 172.21.15.1:0/3620960735 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:10.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.641+0000 7f15f53de700 1 --2- 172.21.15.1:0/3620960735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586b030d420 0x5586b0317700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:10.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.642+0000 7f15f63e0700 1 -- 172.21.15.1:0/3620960735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586b030d420 msgr2=0x5586b0317700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:10.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.642+0000 7f15f63e0700 1 --2- 172.21.15.1:0/3620960735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586b030d420 0x5586b0317700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.642+0000 7f15f63e0700 1 -- 172.21.15.1:0/3620960735 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5586b030cad0 msgr2=0x5586b030cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:10.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.642+0000 7f15f63e0700 1 --2- 172.21.15.1:0/3620960735 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5586b030cad0 0x5586b030cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.642+0000 7f15f63e0700 1 -- 172.21.15.1:0/3620960735 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586b017c9d0 con 0x5586b0317c40 2024-05-04T03:24:10.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.642+0000 7f15f53de700 1 --2- 172.21.15.1:0/3620960735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586b030d420 0x5586b0317700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:24:10.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.642+0000 7f15f63e0700 1 --2- 172.21.15.1:0/3620960735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586b0317c40 0x5586b031a060 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f15e00040c0 tx=0x7f15e00043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=605a5d5b97e67a07 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:10.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.642+0000 7f15d7fff700 1 -- 172.21.15.1:0/3620960735 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e0004560 con 0x5586b0317c40 2024-05-04T03:24:10.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.642+0000 7f15d7fff700 1 -- 172.21.15.1:0/3620960735 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15e0004ce0 con 0x5586b0317c40 2024-05-04T03:24:10.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.642+0000 7f15d7fff700 1 -- 172.21.15.1:0/3620960735 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e0005550 con 0x5586b0317c40 2024-05-04T03:24:10.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.643+0000 7f1606857b80 1 -- 172.21.15.1:0/3620960735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586b0317c40 msgr2=0x5586b031a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:10.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.643+0000 7f1606857b80 1 --2- 172.21.15.1:0/3620960735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586b0317c40 0x5586b031a060 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f15e00040c0 tx=0x7f15e00043d0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.643+0000 7f1606857b80 1 -- 172.21.15.1:0/3620960735 shutdown_connections 2024-05-04T03:24:10.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.643+0000 7f1606857b80 1 --2- 172.21.15.1:0/3620960735 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5586b030cad0 0x5586b030cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.643+0000 7f1606857b80 1 --2- 172.21.15.1:0/3620960735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586b0317c40 0x5586b031a060 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.643+0000 7f1606857b80 1 --2- 172.21.15.1:0/3620960735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586b030d420 0x5586b0317700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.643+0000 7f1606857b80 1 -- 172.21.15.1:0/3620960735 >> 172.21.15.1:0/3620960735 conn(0x5586b018ff40 msgr2=0x5586b03089d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:10.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.643+0000 7f1606857b80 1 -- 172.21.15.1:0/3620960735 shutdown_connections 2024-05-04T03:24:10.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.643+0000 7f1606857b80 1 -- 172.21.15.1:0/3620960735 wait complete. 2024-05-04T03:24:10.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.644+0000 7f1606857b80 1 -- start start 2024-05-04T03:24:10.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.644+0000 7f1606857b80 1 Processor -- start 2024-05-04T03:24:10.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.644+0000 7f1606857b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5586b030cad0 0x5586b0314bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.644+0000 7f1606857b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586b030d420 0x5586b0315130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.644+0000 7f1606857b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586b0317c40 0x5586b0315670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.644+0000 7f1606857b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5586b031aa10 con 0x5586b0317c40 2024-05-04T03:24:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.644+0000 7f1606857b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5586b01e8fd0 con 0x5586b030cad0 2024-05-04T03:24:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.644+0000 7f1606857b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5586b031ab90 con 0x5586b030d420 2024-05-04T03:24:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.644+0000 7f1606857b80 10 client.0 _ll_register_callbacks cb 0x5586b01c7a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:24:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.644+0000 7f15f63e0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586b0317c40 0x5586b0315670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.644+0000 7f15f63e0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586b0317c40 0x5586b0315670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41298/0 (socket says 172.21.15.1:41298) 2024-05-04T03:24:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.644+0000 7f15f63e0700 1 -- 172.21.15.1:0/2773705773 learned_addr learned my addr 172.21.15.1:0/2773705773 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.644+0000 7f15f5bdf700 1 --2- 172.21.15.1:0/2773705773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5586b030cad0 0x5586b0314bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.645+0000 7f15f53de700 1 --2- 172.21.15.1:0/2773705773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586b030d420 0x5586b0315130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.645+0000 7f15f63e0700 1 -- 172.21.15.1:0/2773705773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586b030d420 msgr2=0x5586b0315130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:10.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.645+0000 7f15f63e0700 1 --2- 172.21.15.1:0/2773705773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586b030d420 0x5586b0315130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.645+0000 7f15f63e0700 1 -- 172.21.15.1:0/2773705773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5586b030cad0 msgr2=0x5586b0314bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.645+0000 7f15f63e0700 1 --2- 172.21.15.1:0/2773705773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5586b030cad0 0x5586b0314bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.645+0000 7f15f63e0700 1 -- 172.21.15.1:0/2773705773 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586b01e6860 con 0x5586b0317c40 2024-05-04T03:24:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.645+0000 7f15f63e0700 1 --2- 172.21.15.1:0/2773705773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586b0317c40 0x5586b0315670 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f15e000fcd0 tx=0x7f15e00045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4767eac7512f025d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.645+0000 7f15d57fa700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:24:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.645+0000 7f15d57fa700 1 -- 172.21.15.1:0/2773705773 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e0027070 con 0x5586b0317c40 2024-05-04T03:24:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.645+0000 7f15d57fa700 1 -- 172.21.15.1:0/2773705773 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f15e00051d0 con 0x5586b0317c40 2024-05-04T03:24:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.645+0000 7f1606857b80 1 -- 172.21.15.1:0/2773705773 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15e0004ce0 con 0x5586b0317c40 2024-05-04T03:24:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.645+0000 7f15d57fa700 1 -- 172.21.15.1:0/2773705773 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f15e0033050 con 0x5586b0317c40 2024-05-04T03:24:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.645+0000 7f1606857b80 10 client.15822 fetch_fsmap learned FSMap version 32 2024-05-04T03:24:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.645+0000 7f1606857b80 1 -- 172.21.15.1:0/2773705773 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5586b0325630 con 0x5586b0317c40 2024-05-04T03:24:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.646+0000 7f15d57fa700 1 -- 172.21.15.1:0/2773705773 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15e0004ce0 con 0x5586b0317c40 2024-05-04T03:24:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.646+0000 7f15d57fa700 1 -- 172.21.15.1:0/2773705773 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f15e0023c70 con 0x5586b0317c40 2024-05-04T03:24:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.646+0000 7f1606857b80 10 client.15822 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:24:10.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.646+0000 7f1606857b80 10 client.15822 resolve_mds: validated gid 24494 aka daemon mds.b 2024-05-04T03:24:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.646+0000 7f1606857b80 20 client.15822 populate_metadata read hostname 'smithi001' 2024-05-04T03:24:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.646+0000 7f1606857b80 1 --2- 172.21.15.1:0/2773705773 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5586b0327b00 0x5586b0329f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.646+0000 7f1606857b80 4 client.15822 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:24:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.646+0000 7f1606857b80 1 -- 172.21.15.1:0/2773705773 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586b023d420 con 0x5586b0327b00 2024-05-04T03:24:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.646+0000 7f15f5bdf700 1 --2- 172.21.15.1:0/2773705773 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5586b0327b00 0x5586b0329f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.647+0000 7f15f5bdf700 1 --2- 172.21.15.1:0/2773705773 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5586b0327b00 0x5586b0329f20 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d87e2c312540425 server_cookie=1f52ab735ba20bab in_seq=0 out_seq=0 2024-05-04T03:24:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.647+0000 7f15d57fa700 10 client.15822 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:24:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.648+0000 7f15d57fa700 1 -- 172.21.15.1:0/2773705773 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5586b023d420 con 0x5586b0327b00 2024-05-04T03:24:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.648+0000 7f15d57fa700 10 client.15822 handle_command_reply: tid=0 2024-05-04T03:24:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.648+0000 7f1606857b80 1 client.15822 shutdown 2024-05-04T03:24:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.648+0000 7f1606857b80 1 -- 172.21.15.1:0/2773705773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586b0317c40 msgr2=0x5586b0315670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.648+0000 7f1606857b80 1 --2- 172.21.15.1:0/2773705773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586b0317c40 0x5586b0315670 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f15e000fcd0 tx=0x7f15e00045c0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.648+0000 7f1606857b80 1 -- 172.21.15.1:0/2773705773 shutdown_connections 2024-05-04T03:24:10.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.648+0000 7f1606857b80 1 --2- 172.21.15.1:0/2773705773 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5586b0327b00 0x5586b0329f20 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.648+0000 7f1606857b80 1 --2- 172.21.15.1:0/2773705773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5586b030cad0 0x5586b0314bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.648+0000 7f1606857b80 1 --2- 172.21.15.1:0/2773705773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586b0317c40 0x5586b0315670 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.648+0000 7f1606857b80 1 --2- 172.21.15.1:0/2773705773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586b030d420 0x5586b0315130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.648+0000 7f15d57fa700 0 client.15822 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:24:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.648+0000 7f1606857b80 1 -- 172.21.15.1:0/2773705773 >> 172.21.15.1:0/2773705773 conn(0x5586b018ff40 msgr2=0x5586b030e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.648+0000 7f1606857b80 1 -- 172.21.15.1:0/2773705773 shutdown_connections 2024-05-04T03:24:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.648+0000 7f1606857b80 1 -- 172.21.15.1:0/2773705773 wait complete. 2024-05-04T03:24:10.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.648+0000 7f1606857b80 20 client.15822 trim_cache size 0 max 16384 2024-05-04T03:24:10.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.668+0000 7f1606857b80 1 Processor -- start 2024-05-04T03:24:10.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.669+0000 7f1606857b80 1 -- start start 2024-05-04T03:24:10.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.669+0000 7f1606857b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586b018ff40 0x5586b0310630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:10.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.669+0000 7f1606857b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586b030cad0 0x5586b03169b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:10.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.669+0000 7f1606857b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5586b030d420 0x5586b0314170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:10.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.669+0000 7f1606857b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f15f01af1d0 con 0x5586b030cad0 2024-05-04T03:24:10.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.669+0000 7f1606857b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f15f01aca60 con 0x5586b030d420 2024-05-04T03:24:10.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.669+0000 7f1606857b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f15f01aa910 con 0x5586b018ff40 2024-05-04T03:24:10.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.669+0000 7f15f53de700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586b030cad0 0x5586b03169b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:10.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.669+0000 7f15f53de700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586b030cad0 0x5586b03169b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41304/0 (socket says 172.21.15.1:41304) 2024-05-04T03:24:10.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.669+0000 7f15f53de700 1 -- 172.21.15.1:0/2035404045 learned_addr learned my addr 172.21.15.1:0/2035404045 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:10.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.669+0000 7f15f5bdf700 1 --2- 172.21.15.1:0/2035404045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586b018ff40 0x5586b0310630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:10.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.669+0000 7f15f63e0700 1 --2- 172.21.15.1:0/2035404045 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5586b030d420 0x5586b0314170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:10.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.669+0000 7f15f53de700 1 -- 172.21.15.1:0/2035404045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586b018ff40 msgr2=0x5586b0310630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:10.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.669+0000 7f15f53de700 1 --2- 172.21.15.1:0/2035404045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586b018ff40 0x5586b0310630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.669+0000 7f15f53de700 1 -- 172.21.15.1:0/2035404045 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5586b030d420 msgr2=0x5586b0314170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:10.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.669+0000 7f15f53de700 1 --2- 172.21.15.1:0/2035404045 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5586b030d420 0x5586b0314170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.669+0000 7f15f53de700 1 -- 172.21.15.1:0/2035404045 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586b0308ef0 con 0x5586b030cad0 2024-05-04T03:24:10.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.669+0000 7f15f5bdf700 1 --2- 172.21.15.1:0/2035404045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586b018ff40 0x5586b0310630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:24:10.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.669+0000 7f15f63e0700 1 --2- 172.21.15.1:0/2035404045 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5586b030d420 0x5586b0314170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:24:10.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.670+0000 7f15f53de700 1 --2- 172.21.15.1:0/2035404045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586b030cad0 0x5586b03169b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f15e40a5320 tx=0x7f15e40a8ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20dc74eccf8f76de server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:10.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.670+0000 7f15d77fe700 1 -- 172.21.15.1:0/2035404045 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e4020040 con 0x5586b030cad0 2024-05-04T03:24:10.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.670+0000 7f15d77fe700 1 -- 172.21.15.1:0/2035404045 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15e40ad650 con 0x5586b030cad0 2024-05-04T03:24:10.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.670+0000 7f15d77fe700 1 -- 172.21.15.1:0/2035404045 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e40ad170 con 0x5586b030cad0 2024-05-04T03:24:10.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.670+0000 7f1606857b80 1 -- 172.21.15.1:0/2035404045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586b030cad0 msgr2=0x5586b03169b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:10.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.670+0000 7f1606857b80 1 --2- 172.21.15.1:0/2035404045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586b030cad0 0x5586b03169b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f15e40a5320 tx=0x7f15e40a8ee0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.671+0000 7f1606857b80 1 -- 172.21.15.1:0/2035404045 shutdown_connections 2024-05-04T03:24:10.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.671+0000 7f1606857b80 1 --2- 172.21.15.1:0/2035404045 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5586b030d420 0x5586b0314170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.671+0000 7f1606857b80 1 --2- 172.21.15.1:0/2035404045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586b018ff40 0x5586b0310630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.671+0000 7f1606857b80 1 --2- 172.21.15.1:0/2035404045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586b030cad0 0x5586b03169b0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.671+0000 7f1606857b80 1 -- 172.21.15.1:0/2035404045 >> 172.21.15.1:0/2035404045 conn(0x5586b0301b60 msgr2=0x5586b03162a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:10.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.671+0000 7f1606857b80 1 -- 172.21.15.1:0/2035404045 shutdown_connections 2024-05-04T03:24:10.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.671+0000 7f1606857b80 1 -- 172.21.15.1:0/2035404045 wait complete. 2024-05-04T03:24:10.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.671+0000 7f1606857b80 1 -- start start 2024-05-04T03:24:10.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.671+0000 7f1606857b80 1 Processor -- start 2024-05-04T03:24:10.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.672+0000 7f1606857b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586b018ff40 0x5586b032d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:10.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.672+0000 7f1606857b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586b030cad0 0x5586b032b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:10.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.672+0000 7f1606857b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5586b030d420 0x5586b032bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:10.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.672+0000 7f1606857b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f15f01aaf60 con 0x5586b030cad0 2024-05-04T03:24:10.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.672+0000 7f1606857b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f15f01ac390 con 0x5586b030d420 2024-05-04T03:24:10.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.672+0000 7f1606857b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f15f01ab5f0 con 0x5586b018ff40 2024-05-04T03:24:10.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.672+0000 7f1606857b80 10 client.0 _ll_register_callbacks cb 0x5586b01c7a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:24:10.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.672+0000 7f15f53de700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586b030cad0 0x5586b032b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:10.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.672+0000 7f15f53de700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586b030cad0 0x5586b032b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41316/0 (socket says 172.21.15.1:41316) 2024-05-04T03:24:10.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.672+0000 7f15f53de700 1 -- 172.21.15.1:0/1900993493 learned_addr learned my addr 172.21.15.1:0/1900993493 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:10.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.672+0000 7f15f63e0700 1 --2- 172.21.15.1:0/1900993493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5586b030d420 0x5586b032bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:10.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.672+0000 7f15f5bdf700 1 --2- 172.21.15.1:0/1900993493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586b018ff40 0x5586b032d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:10.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.672+0000 7f15f53de700 1 -- 172.21.15.1:0/1900993493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586b018ff40 msgr2=0x5586b032d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:10.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.672+0000 7f15f53de700 1 --2- 172.21.15.1:0/1900993493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586b018ff40 0x5586b032d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.672+0000 7f15f53de700 1 -- 172.21.15.1:0/1900993493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5586b030d420 msgr2=0x5586b032bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:10.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.672+0000 7f15f53de700 1 --2- 172.21.15.1:0/1900993493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5586b030d420 0x5586b032bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.672+0000 7f15f53de700 1 -- 172.21.15.1:0/1900993493 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586b032c360 con 0x5586b030cad0 2024-05-04T03:24:10.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.672+0000 7f15f53de700 1 --2- 172.21.15.1:0/1900993493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586b030cad0 0x5586b032b740 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f15e40a8fa0 tx=0x7f15e40b1f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929ac8aa156f6110 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:10.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.672+0000 7f15d57fa700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:24:10.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.672+0000 7f15d57fa700 1 -- 172.21.15.1:0/1900993493 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e4020030 con 0x5586b030cad0 2024-05-04T03:24:10.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.672+0000 7f15d57fa700 1 -- 172.21.15.1:0/1900993493 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f15e40ad370 con 0x5586b030cad0 2024-05-04T03:24:10.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.672+0000 7f1606857b80 1 -- 172.21.15.1:0/1900993493 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15e40ad650 con 0x5586b030cad0 2024-05-04T03:24:10.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.673+0000 7f15d57fa700 1 -- 172.21.15.1:0/1900993493 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f15e40c3b90 con 0x5586b030cad0 2024-05-04T03:24:10.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.673+0000 7f1606857b80 10 client.15828 fetch_fsmap learned FSMap version 32 2024-05-04T03:24:10.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.673+0000 7f1606857b80 1 -- 172.21.15.1:0/1900993493 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5586b0392dd0 con 0x5586b030cad0 2024-05-04T03:24:10.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.673+0000 7f15d57fa700 1 -- 172.21.15.1:0/1900993493 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15e40ad650 con 0x5586b030cad0 2024-05-04T03:24:10.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.673+0000 7f15d57fa700 1 -- 172.21.15.1:0/1900993493 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f15e40a4c10 con 0x5586b030cad0 2024-05-04T03:24:10.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.673+0000 7f1606857b80 10 client.15828 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:24:10.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.673+0000 7f1606857b80 10 client.15828 resolve_mds: validated gid 24494 aka daemon mds.b 2024-05-04T03:24:10.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.673+0000 7f1606857b80 20 client.15828 populate_metadata read hostname 'smithi001' 2024-05-04T03:24:10.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.673+0000 7f1606857b80 1 --2- 172.21.15.1:0/1900993493 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5586b0317c40 0x5586b0395510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:10.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.673+0000 7f1606857b80 4 client.15828 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "961ff620", "release": true, "await": true}] 2024-05-04T03:24:10.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.673+0000 7f1606857b80 1 -- 172.21.15.1:0/1900993493 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "quiesce db", "set_id": "961ff620", "release": true, "await": true}) -- 0x5586b02374c0 con 0x5586b0317c40 2024-05-04T03:24:10.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.673+0000 7f15f5bdf700 1 --2- 172.21.15.1:0/1900993493 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5586b0317c40 0x5586b0395510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:10.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.674+0000 7f15f5bdf700 1 --2- 172.21.15.1:0/1900993493 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5586b0317c40 0x5586b0395510 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7d5e5beb8684209 server_cookie=7fdc3864a685c798 in_seq=0 out_seq=0 2024-05-04T03:24:10.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.674+0000 7f15d57fa700 10 client.15828 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:24:10.681 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:10 smithi077 ceph-mon[33531]: pgmap v541: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-05-04T03:24:10.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.688+0000 7f15d57fa700 1 -- 172.21.15.1:0/1900993493 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5586b02374c0 con 0x5586b0317c40 2024-05-04T03:24:10.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.688+0000 7f15d57fa700 10 client.15828 handle_command_reply: tid=0 2024-05-04T03:24:10.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.688+0000 7f1606857b80 1 client.15828 shutdown 2024-05-04T03:24:10.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.688+0000 7f1606857b80 1 -- 172.21.15.1:0/1900993493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586b030cad0 msgr2=0x5586b032b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:10.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.688+0000 7f1606857b80 1 --2- 172.21.15.1:0/1900993493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586b030cad0 0x5586b032b740 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f15e40a8fa0 tx=0x7f15e40b1f80 comp rx=0 tx=0).stop 2024-05-04T03:24:10.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.688+0000 7f1606857b80 1 -- 172.21.15.1:0/1900993493 shutdown_connections 2024-05-04T03:24:10.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.688+0000 7f1606857b80 1 --2- 172.21.15.1:0/1900993493 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5586b0317c40 0x5586b0395510 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.688+0000 7f1606857b80 1 --2- 172.21.15.1:0/1900993493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5586b030d420 0x5586b032bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.688+0000 7f1606857b80 1 --2- 172.21.15.1:0/1900993493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5586b018ff40 0x5586b032d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.688+0000 7f1606857b80 1 --2- 172.21.15.1:0/1900993493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5586b030cad0 0x5586b032b740 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.688+0000 7f15d57fa700 0 client.15828 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:24:10.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.688+0000 7f1606857b80 1 -- 172.21.15.1:0/1900993493 >> 172.21.15.1:0/1900993493 conn(0x5586b0301b60 msgr2=0x5586b0316a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:10.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.688+0000 7f1606857b80 1 -- 172.21.15.1:0/1900993493 shutdown_connections 2024-05-04T03:24:10.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.688+0000 7f1606857b80 1 -- 172.21.15.1:0/1900993493 wait complete. 2024-05-04T03:24:10.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.688+0000 7f1606857b80 20 client.15828 trim_cache size 0 max 16384 2024-05-04T03:24:10.692 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-04T03:24:10.692 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 32, 2024-05-04T03:24:10.692 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24494, 2024-05-04T03:24:10.692 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 16, 2024-05-04T03:24:10.692 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": { 2024-05-04T03:24:10.692 INFO:teuthology.orchestra.run.smithi001.stdout: "961ff620": { 2024-05-04T03:24:10.692 INFO:teuthology.orchestra.run.smithi001.stdout: "version": 16, 2024-05-04T03:24:10.692 INFO:teuthology.orchestra.run.smithi001.stdout: "age_ref": 0.0, 2024-05-04T03:24:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-05-04T03:24:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-05-04T03:24:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-05-04T03:24:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:24:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: "timeout": 60.0, 2024-05-04T03:24:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: "expiration": 90.0, 2024-05-04T03:24:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: "members": { 2024-05-04T03:24:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: "file:/": { 2024-05-04T03:24:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: "excluded": false, 2024-05-04T03:24:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: "state": { 2024-05-04T03:24:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RELEASED", 2024-05-04T03:24:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: "age": 0.0 2024-05-04T03:24:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:24:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:24:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:24:10.693 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:24:10.694 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:24:10.694 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-04T03:24:10.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.688+0000 7f15f7e43700 1 -- 172.21.15.1:0/1457388709 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f15d80b2880 msgr2=0x7f15d80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:10.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.688+0000 7f15f7e43700 1 --2- 172.21.15.1:0/1457388709 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f15d80b2880 0x7f15d80b4d90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009900 tx=0x7f15e800b040 comp rx=0 tx=0).stop 2024-05-04T03:24:10.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.688+0000 7f15f7e43700 1 -- 172.21.15.1:0/1457388709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15f0108f00 msgr2=0x7f15f019b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:10.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.688+0000 7f15f7e43700 1 --2- 172.21.15.1:0/1457388709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15f0108f00 0x7f15f019b370 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f15e400d980 tx=0x7f15e400dc90 comp rx=0 tx=0).stop 2024-05-04T03:24:10.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.688+0000 7f15f7e43700 1 -- 172.21.15.1:0/1457388709 shutdown_connections 2024-05-04T03:24:10.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.688+0000 7f15f7e43700 1 --2- 172.21.15.1:0/1457388709 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f15f01045d0 0x7f15f019add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.688+0000 7f15f7e43700 1 --2- 172.21.15.1:0/1457388709 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f15d80b2880 0x7f15d80b4d90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.688+0000 7f15f7e43700 1 --2- 172.21.15.1:0/1457388709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15f0109850 0x7f15f019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.688+0000 7f15f7e43700 1 --2- 172.21.15.1:0/1457388709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15f0108f00 0x7f15f019b370 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:10.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.688+0000 7f15f7e43700 1 -- 172.21.15.1:0/1457388709 >> 172.21.15.1:0/1457388709 conn(0x7f15f00ffe80 msgr2=0x7f15f01022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:10.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.688+0000 7f15f7e43700 1 -- 172.21.15.1:0/1457388709 shutdown_connections 2024-05-04T03:24:10.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:24:10.689+0000 7f15f7e43700 1 -- 172.21.15.1:0/1457388709 wait complete. 2024-05-04T03:24:10.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:10 smithi072 ceph-mon[33332]: pgmap v541: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-05-04T03:24:10.900 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-05-04T03:24:10.900 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-05-04T03:24:10.900 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support clone range, disabling! 2024-05-04T03:24:10.903 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support dedupe range, disabling! 2024-05-04T03:24:10.906 INFO:tasks.workunit.client.0.smithi001.stdout:skipping zero size read 2024-05-04T03:24:10.906 INFO:tasks.workunit.client.0.smithi001.stderr:main: filesystem does not support exchange range, disabling! 2024-05-04T03:24:10.918 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0x1c37b90 2024-05-04T03:24:10.925 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0x5bd330d 2024-05-04T03:24:10.941 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0x637bf99 2024-05-04T03:24:11.114 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0x63d9b9c 2024-05-04T03:24:11.643 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '961ff620', seconds elapsed: 43.8 2024-05-04T03:24:12.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:12 smithi001 ceph-mon[29731]: pgmap v542: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-05-04T03:24:12.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:12 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:24:12.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:12 smithi077 ceph-mon[33531]: pgmap v542: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-05-04T03:24:12.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:12 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:24:12.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:12 smithi072 ceph-mon[33332]: pgmap v542: 97 pgs: 97 active+clean; 97 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-05-04T03:24:12.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:12 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:24:13.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:13 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:24:13.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:13 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:24:13.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:13 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:24:13.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:13 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:24:13.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:13 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:24:13.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:13 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:24:13.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:13 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:24:13.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:13 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:24:13.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:13 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:24:13.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:13 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:24:13.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:13 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:24:13.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:13 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:24:13.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:13 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:24:13.679 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:13 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:24:13.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:13 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:24:13.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:13 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:24:13.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:13 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:24:13.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:13 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:24:13.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:13 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:24:13.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:13 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:24:13.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:13 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:24:14.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:14 smithi001 ceph-mon[29731]: Adjusting osd_memory_target on smithi001 to 177.2M 2024-05-04T03:24:14.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:14 smithi001 ceph-mon[29731]: Unable to set osd_memory_target on smithi001 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-05-04T03:24:14.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:14 smithi001 ceph-mon[29731]: pgmap v543: 97 pgs: 97 active+clean; 98 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 87 KiB/s wr, 43 op/s 2024-05-04T03:24:14.647 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:14 smithi072 ceph-mon[33332]: Adjusting osd_memory_target on smithi001 to 177.2M 2024-05-04T03:24:14.647 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:14 smithi072 ceph-mon[33332]: Unable to set osd_memory_target on smithi001 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-05-04T03:24:14.647 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:14 smithi072 ceph-mon[33332]: pgmap v543: 97 pgs: 97 active+clean; 98 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 87 KiB/s wr, 43 op/s 2024-05-04T03:24:14.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:14 smithi077 ceph-mon[33531]: Adjusting osd_memory_target on smithi001 to 177.2M 2024-05-04T03:24:14.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:14 smithi077 ceph-mon[33531]: Unable to set osd_memory_target on smithi001 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-05-04T03:24:14.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:14 smithi077 ceph-mon[33531]: pgmap v543: 97 pgs: 97 active+clean; 98 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 87 KiB/s wr, 43 op/s 2024-05-04T03:24:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.888+0000 7f67b8a9f640 1 -- 172.21.15.1:0/3785024750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67b4108ab0 msgr2=0x7f67b410aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.888+0000 7f67b8a9f640 1 --2- 172.21.15.1:0/3785024750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67b4108ab0 0x7f67b410aed0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009ca0 tx=0x7f67ac02a8c0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.889+0000 7f67b8a9f640 1 -- 172.21.15.1:0/3785024750 shutdown_connections 2024-05-04T03:24:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.889+0000 7f67b8a9f640 1 --2- 172.21.15.1:0/3785024750 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67b4103db0 0x7f67b41041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.889+0000 7f67b8a9f640 1 --2- 172.21.15.1:0/3785024750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67b4108ab0 0x7f67b410aed0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.889+0000 7f67b8a9f640 1 --2- 172.21.15.1:0/3785024750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67b4102b70 0x7f67b4103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.889+0000 7f67b8a9f640 1 -- 172.21.15.1:0/3785024750 >> 172.21.15.1:0/3785024750 conn(0x7f67b40fdd60 msgr2=0x7f67b4100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.889+0000 7f67b8a9f640 1 -- 172.21.15.1:0/3785024750 shutdown_connections 2024-05-04T03:24:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.889+0000 7f67b8a9f640 1 -- 172.21.15.1:0/3785024750 wait complete. 2024-05-04T03:24:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.889+0000 7f67b8a9f640 1 Processor -- start 2024-05-04T03:24:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.889+0000 7f67b8a9f640 1 -- start start 2024-05-04T03:24:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.890+0000 7f67b8a9f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67b4102b70 0x7f67b419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.890+0000 7f67b8a9f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67b4103db0 0x7f67b419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.890+0000 7f67b8a9f640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67b4108ab0 0x7f67b41a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.890+0000 7f67b8a9f640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f67b410e680 con 0x7f67b4108ab0 2024-05-04T03:24:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.890+0000 7f67b8a9f640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f67b410e500 con 0x7f67b4102b70 2024-05-04T03:24:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.890+0000 7f67b8a9f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f67b410e800 con 0x7f67b4103db0 2024-05-04T03:24:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.890+0000 7f67b2d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67b4108ab0 0x7f67b41a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.890+0000 7f67b2d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67b4108ab0 0x7f67b41a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39792/0 (socket says 172.21.15.1:39792) 2024-05-04T03:24:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.890+0000 7f67b2d9d640 1 -- 172.21.15.1:0/3067179062 learned_addr learned my addr 172.21.15.1:0/3067179062 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.890+0000 7f67b259c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67b4102b70 0x7f67b419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.890+0000 7f67b1d9b640 1 --2- 172.21.15.1:0/3067179062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67b4103db0 0x7f67b419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.890+0000 7f67b2d9d640 1 -- 172.21.15.1:0/3067179062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67b4103db0 msgr2=0x7f67b419a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.890+0000 7f67b2d9d640 1 --2- 172.21.15.1:0/3067179062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67b4103db0 0x7f67b419a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.890+0000 7f67b2d9d640 1 -- 172.21.15.1:0/3067179062 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67b4102b70 msgr2=0x7f67b419a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.890+0000 7f67b2d9d640 1 --2- 172.21.15.1:0/3067179062 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67b4102b70 0x7f67b419a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.890+0000 7f67b2d9d640 1 -- 172.21.15.1:0/3067179062 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b41a1d70 con 0x7f67b4108ab0 2024-05-04T03:24:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.890+0000 7f67b259c640 1 --2- 172.21.15.1:0/3067179062 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67b4102b70 0x7f67b419a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:24:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.890+0000 7f67b2d9d640 1 --2- 172.21.15.1:0/3067179062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67b4108ab0 0x7f67b41a1700 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f67ac0097b0 tx=0x7f67ac031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.891+0000 7f67a37fe640 1 -- 172.21.15.1:0/3067179062 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac003bf0 con 0x7f67b4108ab0 2024-05-04T03:24:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.891+0000 7f67a37fe640 1 -- 172.21.15.1:0/3067179062 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67ac003d90 con 0x7f67b4108ab0 2024-05-04T03:24:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.891+0000 7f67b8a9f640 1 -- 172.21.15.1:0/3067179062 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b41a2000 con 0x7f67b4108ab0 2024-05-04T03:24:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.891+0000 7f67a37fe640 1 -- 172.21.15.1:0/3067179062 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac02fc30 con 0x7f67b4108ab0 2024-05-04T03:24:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.891+0000 7f67b8a9f640 1 -- 172.21.15.1:0/3067179062 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b41a2550 con 0x7f67b4108ab0 2024-05-04T03:24:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.894+0000 7f67bfa2c740 1 Processor -- start 2024-05-04T03:24:14.897 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.894+0000 7f67a37fe640 1 -- 172.21.15.1:0/3067179062 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f67ac0305c0 con 0x7f67b4108ab0 2024-05-04T03:24:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.894+0000 7f67bfa2c740 1 -- start start 2024-05-04T03:24:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.895+0000 7f67bfa2c740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5580326df1b0 0x558032a80a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.895+0000 7f67bfa2c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580326b62f0 0x558032a80fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.895+0000 7f67bfa2c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580326be4d0 0x558032a87ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.895+0000 7f67bfa2c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5580329e55d0 con 0x5580326b62f0 2024-05-04T03:24:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.895+0000 7f67bfa2c740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55803299e890 con 0x5580326df1b0 2024-05-04T03:24:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.895+0000 7f67bfa2c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5580329e1590 con 0x5580326be4d0 2024-05-04T03:24:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.895+0000 7f67b259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580326b62f0 0x558032a80fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.895+0000 7f67b259c640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580326b62f0 0x558032a80fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39808/0 (socket says 172.21.15.1:39808) 2024-05-04T03:24:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.895+0000 7f67b259c640 1 -- 172.21.15.1:0/3548175024 learned_addr learned my addr 172.21.15.1:0/3548175024 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.895+0000 7f67b1d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580326be4d0 0x558032a87ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.895+0000 7f67b1d9b640 1 --2- 172.21.15.1:0/3548175024 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5580326df1b0 0x558032a80a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.895+0000 7f67b1d9b640 1 -- 172.21.15.1:0/3548175024 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5580326df1b0 msgr2=0x558032a80a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.895+0000 7f67b1d9b640 1 --2- 172.21.15.1:0/3548175024 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5580326df1b0 0x558032a80a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.895+0000 7f67b1d9b640 1 -- 172.21.15.1:0/3548175024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580326b62f0 msgr2=0x558032a80fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.895+0000 7f67b1d9b640 1 --2- 172.21.15.1:0/3548175024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580326b62f0 0x558032a80fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.895+0000 7f67b1d9b640 1 -- 172.21.15.1:0/3548175024 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580326f7210 con 0x5580326be4d0 2024-05-04T03:24:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.895+0000 7f67b259c640 1 --2- 172.21.15.1:0/3548175024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580326b62f0 0x558032a80fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:24:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.896+0000 7f67a37fe640 1 --2- 172.21.15.1:0/3067179062 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f679c0b28a0 0x7f679c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.896+0000 7f67b1d9b640 1 --2- 172.21.15.1:0/3548175024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580326be4d0 0x558032a87ee0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x558032a81c30 tx=0x7f67a8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=150cd2da259ad842 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.896+0000 7f67b259c640 1 --2- 172.21.15.1:0/3067179062 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f679c0b28a0 0x7f679c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.896+0000 7f67a37fe640 1 -- 172.21.15.1:0/3067179062 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f67ac1037c0 con 0x7f67b4108ab0 2024-05-04T03:24:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.896+0000 7f67a0ff9640 1 -- 172.21.15.1:0/3548175024 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a8012ca0 con 0x5580326be4d0 2024-05-04T03:24:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.896+0000 7f67a0ff9640 1 -- 172.21.15.1:0/3548175024 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67a8012e40 con 0x5580326be4d0 2024-05-04T03:24:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.896+0000 7f67a0ff9640 1 -- 172.21.15.1:0/3548175024 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a8004a00 con 0x5580326be4d0 2024-05-04T03:24:14.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.898+0000 7f67bfa2c740 1 -- 172.21.15.1:0/3548175024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580326be4d0 msgr2=0x558032a87ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:14.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.898+0000 7f67bfa2c740 1 --2- 172.21.15.1:0/3548175024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580326be4d0 0x558032a87ee0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x558032a81c30 tx=0x7f67a8004730 comp rx=0 tx=0).stop 2024-05-04T03:24:14.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.898+0000 7f67b259c640 1 --2- 172.21.15.1:0/3067179062 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f679c0b28a0 0x7f679c0b4db0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009fd0 tx=0x7f67a4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:14.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.898+0000 7f67bfa2c740 1 -- 172.21.15.1:0/3548175024 shutdown_connections 2024-05-04T03:24:14.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.898+0000 7f67bfa2c740 1 --2- 172.21.15.1:0/3548175024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580326be4d0 0x558032a87ee0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.904 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.898+0000 7f67bfa2c740 1 --2- 172.21.15.1:0/3548175024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580326b62f0 0x558032a80fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.898+0000 7f67bfa2c740 1 --2- 172.21.15.1:0/3548175024 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5580326df1b0 0x558032a80a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.898+0000 7f67bfa2c740 1 -- 172.21.15.1:0/3548175024 >> 172.21.15.1:0/3548175024 conn(0x5580326d3300 msgr2=0x558032a7c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:14.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.898+0000 7f67bfa2c740 1 -- 172.21.15.1:0/3548175024 shutdown_connections 2024-05-04T03:24:14.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.898+0000 7f67bfa2c740 1 -- 172.21.15.1:0/3548175024 wait complete. 2024-05-04T03:24:14.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.899+0000 7f67bfa2c740 1 -- start start 2024-05-04T03:24:14.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.899+0000 7f67bfa2c740 1 Processor -- start 2024-05-04T03:24:14.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.899+0000 7f67bfa2c740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5580326b62f0 0x558032a8a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:14.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.899+0000 7f67bfa2c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580326be4d0 0x558032a8ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:14.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.899+0000 7f67bfa2c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580326df1b0 0x558032a91c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:14.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.899+0000 7f67bfa2c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x558032a88930 con 0x5580326be4d0 2024-05-04T03:24:14.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.899+0000 7f67bfa2c740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5580329a2370 con 0x5580326b62f0 2024-05-04T03:24:14.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.899+0000 7f67bfa2c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558032a88ab0 con 0x5580326df1b0 2024-05-04T03:24:14.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.899+0000 7f67bfa2c740 10 client.0 _ll_register_callbacks cb 0x5580326f6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:24:14.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.899+0000 7f67b2d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580326df1b0 0x558032a91c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:14.905 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.899+0000 7f67b259c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5580326b62f0 0x558032a8a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.899+0000 7f67b2d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580326df1b0 0x558032a91c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.1:48806/0 (socket says 172.21.15.1:48806) 2024-05-04T03:24:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.899+0000 7f67b1d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580326be4d0 0x558032a8ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.899+0000 7f67b2d9d640 1 -- 172.21.15.1:0/4150218092 learned_addr learned my addr 172.21.15.1:0/4150218092 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.900+0000 7f67b2d9d640 1 -- 172.21.15.1:0/4150218092 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5580326b62f0 msgr2=0x558032a8a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.900+0000 7f67b2d9d640 1 --2- 172.21.15.1:0/4150218092 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5580326b62f0 0x558032a8a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.900+0000 7f67b2d9d640 1 -- 172.21.15.1:0/4150218092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580326be4d0 msgr2=0x558032a8ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.900+0000 7f67b2d9d640 1 --2- 172.21.15.1:0/4150218092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580326be4d0 0x558032a8ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.900+0000 7f67b2d9d640 1 -- 172.21.15.1:0/4150218092 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580327c3620 con 0x5580326df1b0 2024-05-04T03:24:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.900+0000 7f67b2d9d640 1 --2- 172.21.15.1:0/4150218092 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580326df1b0 0x558032a91c70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f67ac02f550 tx=0x7f67ac0099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=984087e88835aa6c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.900+0000 7f6799ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-04T03:24:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.900+0000 7f6799ffb640 1 -- 172.21.15.1:0/4150218092 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac0cf250 con 0x5580326df1b0 2024-05-04T03:24:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.900+0000 7f6799ffb640 1 -- 172.21.15.1:0/4150218092 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f67ac0c7490 con 0x5580326df1b0 2024-05-04T03:24:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.900+0000 7f67bfa2c740 1 -- 172.21.15.1:0/4150218092 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5580329f31e0 con 0x5580326df1b0 2024-05-04T03:24:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.900+0000 7f6799ffb640 1 -- 172.21.15.1:0/4150218092 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f67ac0d3d00 con 0x5580326df1b0 2024-05-04T03:24:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.900+0000 7f67bfa2c740 10 client.25513 fetch_fsmap learned FSMap version 32 2024-05-04T03:24:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.900+0000 7f67bfa2c740 1 -- 172.21.15.1:0/4150218092 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558032a88930 con 0x5580326df1b0 2024-05-04T03:24:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.901+0000 7f6799ffb640 1 -- 172.21.15.1:0/4150218092 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5580329f31e0 con 0x5580326df1b0 2024-05-04T03:24:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.901+0000 7f6799ffb640 1 -- 172.21.15.1:0/4150218092 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f67ac0e1030 con 0x5580326df1b0 2024-05-04T03:24:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.901+0000 7f67bfa2c740 10 client.25513 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:24:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.901+0000 7f67bfa2c740 10 client.25513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:24:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.901+0000 7f67bfa2c740 20 client.25513 populate_metadata read hostname 'smithi001' 2024-05-04T03:24:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.901+0000 7f67bfa2c740 1 --2- 172.21.15.1:0/4150218092 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55803269dd80 0x558032a98310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.901+0000 7f67bfa2c740 4 client.25513 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:24:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.901+0000 7f67bfa2c740 1 -- 172.21.15.1:0/4150218092 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580329ed050 con 0x55803269dd80 2024-05-04T03:24:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.901+0000 7f67b259c640 1 --2- 172.21.15.1:0/4150218092 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55803269dd80 0x558032a98310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.902+0000 7f6799ffb640 10 client.25513 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:24:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.902+0000 7f67b259c640 1 --2- 172.21.15.1:0/4150218092 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55803269dd80 0x558032a98310 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd0978599d55f92d server_cookie=da2a7d844e868613 in_seq=0 out_seq=0 2024-05-04T03:24:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.903+0000 7f6799ffb640 1 -- 172.21.15.1:0/4150218092 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5580329ed050 con 0x55803269dd80 2024-05-04T03:24:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.903+0000 7f6799ffb640 10 client.25513 handle_command_reply: tid=0 2024-05-04T03:24:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.903+0000 7f67bfa2c740 1 client.25513 shutdown 2024-05-04T03:24:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.903+0000 7f67bfa2c740 1 -- 172.21.15.1:0/4150218092 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580326df1b0 msgr2=0x558032a91c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.903+0000 7f67bfa2c740 1 --2- 172.21.15.1:0/4150218092 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580326df1b0 0x558032a91c70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f67ac02f550 tx=0x7f67ac0099e0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.903+0000 7f67bfa2c740 1 -- 172.21.15.1:0/4150218092 shutdown_connections 2024-05-04T03:24:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.903+0000 7f67bfa2c740 1 --2- 172.21.15.1:0/4150218092 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55803269dd80 0x558032a98310 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.903+0000 7f67bfa2c740 1 --2- 172.21.15.1:0/4150218092 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580326df1b0 0x558032a91c70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.903+0000 7f67bfa2c740 1 --2- 172.21.15.1:0/4150218092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580326be4d0 0x558032a8ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.903+0000 7f6799ffb640 0 client.25513 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:24:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.903+0000 7f67bfa2c740 1 --2- 172.21.15.1:0/4150218092 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5580326b62f0 0x558032a8a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.903+0000 7f67bfa2c740 1 -- 172.21.15.1:0/4150218092 >> 172.21.15.1:0/4150218092 conn(0x5580326d3300 msgr2=0x558032a7f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.903+0000 7f67bfa2c740 1 -- 172.21.15.1:0/4150218092 shutdown_connections 2024-05-04T03:24:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.903+0000 7f67bfa2c740 1 -- 172.21.15.1:0/4150218092 wait complete. 2024-05-04T03:24:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.903+0000 7f67bfa2c740 20 client.25513 trim_cache size 0 max 16384 2024-05-04T03:24:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.923+0000 7f67bfa2c740 1 Processor -- start 2024-05-04T03:24:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.923+0000 7f67bfa2c740 1 -- start start 2024-05-04T03:24:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.923+0000 7f67bfa2c740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55803269dd80 0x558032a809e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.923+0000 7f67bfa2c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580326b62f0 0x558032a80f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.923+0000 7f67bfa2c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580326be4d0 0x558032a84ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.923+0000 7f67bfa2c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f67b41afb40 con 0x5580326b62f0 2024-05-04T03:24:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.923+0000 7f67bfa2c740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f67b41ad410 con 0x55803269dd80 2024-05-04T03:24:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.923+0000 7f67bfa2c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f67b41ab280 con 0x5580326be4d0 2024-05-04T03:24:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.923+0000 7f67b1d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580326b62f0 0x558032a80f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.924+0000 7f67b1d9b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580326b62f0 0x558032a80f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39816/0 (socket says 172.21.15.1:39816) 2024-05-04T03:24:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.924+0000 7f67b1d9b640 1 -- 172.21.15.1:0/1208807305 learned_addr learned my addr 172.21.15.1:0/1208807305 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.924+0000 7f67b259c640 1 --2- 172.21.15.1:0/1208807305 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55803269dd80 0x558032a809e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.924+0000 7f67b2d9d640 1 --2- 172.21.15.1:0/1208807305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580326be4d0 0x558032a84ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.924+0000 7f67b1d9b640 1 -- 172.21.15.1:0/1208807305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580326be4d0 msgr2=0x558032a84ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.924+0000 7f67b1d9b640 1 --2- 172.21.15.1:0/1208807305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580326be4d0 0x558032a84ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.924+0000 7f67b1d9b640 1 -- 172.21.15.1:0/1208807305 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55803269dd80 msgr2=0x558032a809e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.924+0000 7f67b1d9b640 1 --2- 172.21.15.1:0/1208807305 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55803269dd80 0x558032a809e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.924+0000 7f67b1d9b640 1 -- 172.21.15.1:0/1208807305 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558032a7cd00 con 0x5580326b62f0 2024-05-04T03:24:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.924+0000 7f67b259c640 1 --2- 172.21.15.1:0/1208807305 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55803269dd80 0x558032a809e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:24:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.924+0000 7f67b1d9b640 1 --2- 172.21.15.1:0/1208807305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580326b62f0 0x558032a80f20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f67a80058c0 tx=0x7f67a801b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c56870a385d3f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.924+0000 7f679bfff640 1 -- 172.21.15.1:0/1208807305 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a8004220 con 0x5580326b62f0 2024-05-04T03:24:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.924+0000 7f679bfff640 1 -- 172.21.15.1:0/1208807305 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67a8016040 con 0x5580326b62f0 2024-05-04T03:24:14.929 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.924+0000 7f679bfff640 1 -- 172.21.15.1:0/1208807305 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a8003830 con 0x5580326b62f0 2024-05-04T03:24:14.933 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.925+0000 7f67bfa2c740 1 -- 172.21.15.1:0/1208807305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580326b62f0 msgr2=0x558032a80f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.925+0000 7f67bfa2c740 1 --2- 172.21.15.1:0/1208807305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580326b62f0 0x558032a80f20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f67a80058c0 tx=0x7f67a801b040 comp rx=0 tx=0).stop 2024-05-04T03:24:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.925+0000 7f67bfa2c740 1 -- 172.21.15.1:0/1208807305 shutdown_connections 2024-05-04T03:24:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.925+0000 7f67bfa2c740 1 --2- 172.21.15.1:0/1208807305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5580326be4d0 0x558032a84ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.925+0000 7f67bfa2c740 1 --2- 172.21.15.1:0/1208807305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580326b62f0 0x558032a80f20 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.925+0000 7f67bfa2c740 1 --2- 172.21.15.1:0/1208807305 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55803269dd80 0x558032a809e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.925+0000 7f67bfa2c740 1 -- 172.21.15.1:0/1208807305 >> 172.21.15.1:0/1208807305 conn(0x5580326d3300 msgr2=0x558032a88ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.925+0000 7f67bfa2c740 1 -- 172.21.15.1:0/1208807305 shutdown_connections 2024-05-04T03:24:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.925+0000 7f67bfa2c740 1 -- 172.21.15.1:0/1208807305 wait complete. 2024-05-04T03:24:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.926+0000 7f67bfa2c740 1 -- start start 2024-05-04T03:24:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.926+0000 7f67bfa2c740 1 Processor -- start 2024-05-04T03:24:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.926+0000 7f67bfa2c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55803269dd80 0x558032a8e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.926+0000 7f67bfa2c740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5580326b62f0 0x558032a8e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.926+0000 7f67bfa2c740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580326be4d0 0x558032afe6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.926+0000 7f67bfa2c740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f67b41ab8d0 con 0x5580326be4d0 2024-05-04T03:24:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.926+0000 7f67bfa2c740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f67b41acd40 con 0x5580326b62f0 2024-05-04T03:24:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.926+0000 7f67bfa2c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f67b41abfa0 con 0x55803269dd80 2024-05-04T03:24:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.926+0000 7f67bfa2c740 10 client.0 _ll_register_callbacks cb 0x7f67ac0c6d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:24:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.926+0000 7f67b1d9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5580326b62f0 0x558032a8e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.926+0000 7f67b2d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580326be4d0 0x558032afe6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.926+0000 7f67b1d9b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5580326b62f0 0x558032a8e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:50978/0 (socket says 172.21.15.1:50978) 2024-05-04T03:24:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.926+0000 7f67b2d9d640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580326be4d0 0x558032afe6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39818/0 (socket says 172.21.15.1:39818) 2024-05-04T03:24:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.926+0000 7f67b1d9b640 1 -- 172.21.15.1:0/3505987974 learned_addr learned my addr 172.21.15.1:0/3505987974 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.926+0000 7f67b259c640 1 --2- 172.21.15.1:0/3505987974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55803269dd80 0x558032a8e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.927+0000 7f67b1d9b640 1 -- 172.21.15.1:0/3505987974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55803269dd80 msgr2=0x558032a8e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.927+0000 7f67b1d9b640 1 --2- 172.21.15.1:0/3505987974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55803269dd80 0x558032a8e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.927+0000 7f67b1d9b640 1 -- 172.21.15.1:0/3505987974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580326be4d0 msgr2=0x558032afe6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.927+0000 7f67b1d9b640 1 --2- 172.21.15.1:0/3505987974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580326be4d0 0x558032afe6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.927+0000 7f67b1d9b640 1 -- 172.21.15.1:0/3505987974 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558032afebf0 con 0x5580326b62f0 2024-05-04T03:24:14.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.927+0000 7f67b259c640 1 --2- 172.21.15.1:0/3505987974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55803269dd80 0x558032a8e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:24:14.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.927+0000 7f67b2d9d640 1 --2- 172.21.15.1:0/3505987974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580326be4d0 0x558032afe6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:24:14.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.927+0000 7f67b1d9b640 1 --2- 172.21.15.1:0/3505987974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5580326b62f0 0x558032a8e7e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f67a8005890 tx=0x7f67a80042d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc35f928261cdf85 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:14.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.927+0000 7f6799ffb640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-04T03:24:14.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.927+0000 7f6799ffb640 1 -- 172.21.15.1:0/3505987974 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a8010b50 con 0x5580326b62f0 2024-05-04T03:24:14.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.927+0000 7f6799ffb640 1 -- 172.21.15.1:0/3505987974 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f67a8005030 con 0x5580326b62f0 2024-05-04T03:24:14.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.927+0000 7f67bfa2c740 1 -- 172.21.15.1:0/3505987974 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67a8016040 con 0x5580326b62f0 2024-05-04T03:24:14.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.927+0000 7f6799ffb640 1 -- 172.21.15.1:0/3505987974 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f67a800bbb0 con 0x5580326b62f0 2024-05-04T03:24:14.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.927+0000 7f67bfa2c740 10 client.25562 fetch_fsmap learned FSMap version 32 2024-05-04T03:24:14.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.927+0000 7f67bfa2c740 1 -- 172.21.15.1:0/3505987974 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558032affe20 con 0x5580326b62f0 2024-05-04T03:24:14.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.928+0000 7f6799ffb640 1 -- 172.21.15.1:0/3505987974 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67a80046e0 con 0x5580326b62f0 2024-05-04T03:24:14.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.928+0000 7f6799ffb640 1 -- 172.21.15.1:0/3505987974 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f67a80035f0 con 0x5580326b62f0 2024-05-04T03:24:14.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.928+0000 7f67bfa2c740 10 client.25562 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:24:14.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.928+0000 7f67bfa2c740 10 client.25562 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:24:14.936 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.928+0000 7f67bfa2c740 20 client.25562 populate_metadata read hostname 'smithi001' 2024-05-04T03:24:14.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.928+0000 7f67bfa2c740 1 --2- 172.21.15.1:0/3505987974 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5580326df1b0 0x558032b02600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:14.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.928+0000 7f67bfa2c740 4 client.25562 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:24:14.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.928+0000 7f67bfa2c740 1 -- 172.21.15.1:0/3505987974 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f67a8016040 con 0x5580326df1b0 2024-05-04T03:24:14.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.928+0000 7f67b259c640 1 --2- 172.21.15.1:0/3505987974 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5580326df1b0 0x558032b02600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:14.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.928+0000 7f6799ffb640 10 client.25562 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:24:14.937 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.929+0000 7f67b259c640 1 --2- 172.21.15.1:0/3505987974 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5580326df1b0 0x558032b02600 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a58663b59e06990 server_cookie=ec803908fa43dc54 in_seq=0 out_seq=0 2024-05-04T03:24:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.936+0000 7f6799ffb640 1 -- 172.21.15.1:0/3505987974 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f67a8016040 con 0x5580326df1b0 2024-05-04T03:24:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.936+0000 7f6799ffb640 10 client.25562 handle_command_reply: tid=0 2024-05-04T03:24:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.936+0000 7f67bfa2c740 1 client.25562 shutdown 2024-05-04T03:24:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.936+0000 7f67bfa2c740 1 -- 172.21.15.1:0/3505987974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5580326b62f0 msgr2=0x558032a8e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.936+0000 7f67bfa2c740 1 --2- 172.21.15.1:0/3505987974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5580326b62f0 0x558032a8e7e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f67a8005890 tx=0x7f67a80042d0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.937+0000 7f67bfa2c740 1 -- 172.21.15.1:0/3505987974 shutdown_connections 2024-05-04T03:24:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.937+0000 7f67bfa2c740 1 --2- 172.21.15.1:0/3505987974 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5580326df1b0 0x558032b02600 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.937+0000 7f67bfa2c740 1 --2- 172.21.15.1:0/3505987974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5580326b62f0 0x558032a8e7e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.937+0000 7f67bfa2c740 1 --2- 172.21.15.1:0/3505987974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5580326be4d0 0x558032afe6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.937+0000 7f6799ffb640 0 client.25562 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:24:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.937+0000 7f67bfa2c740 1 --2- 172.21.15.1:0/3505987974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55803269dd80 0x558032a8e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.937+0000 7f67bfa2c740 1 -- 172.21.15.1:0/3505987974 >> 172.21.15.1:0/3505987974 conn(0x5580326d3300 msgr2=0x558032a84ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.937+0000 7f67bfa2c740 1 -- 172.21.15.1:0/3505987974 shutdown_connections 2024-05-04T03:24:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.937+0000 7f67bfa2c740 1 -- 172.21.15.1:0/3505987974 wait complete. 2024-05-04T03:24:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.937+0000 7f67bfa2c740 20 client.25562 trim_cache size 0 max 16384 2024-05-04T03:24:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:24:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:24:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:24:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:24:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.938+0000 7f67b8a9f640 1 -- 172.21.15.1:0/3067179062 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f679c0b28a0 msgr2=0x7f679c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.938+0000 7f67b8a9f640 1 --2- 172.21.15.1:0/3067179062 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f679c0b28a0 0x7f679c0b4db0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009fd0 tx=0x7f67a4005f90 comp rx=0 tx=0).stop 2024-05-04T03:24:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.938+0000 7f67b8a9f640 1 -- 172.21.15.1:0/3067179062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67b4108ab0 msgr2=0x7f67b41a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.938+0000 7f67b8a9f640 1 --2- 172.21.15.1:0/3067179062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67b4108ab0 0x7f67b41a1700 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f67ac0097b0 tx=0x7f67ac031840 comp rx=0 tx=0).stop 2024-05-04T03:24:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.938+0000 7f67b8a9f640 1 -- 172.21.15.1:0/3067179062 shutdown_connections 2024-05-04T03:24:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.938+0000 7f67b8a9f640 1 --2- 172.21.15.1:0/3067179062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67b4108ab0 0x7f67b41a1700 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.938+0000 7f67b8a9f640 1 --2- 172.21.15.1:0/3067179062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67b4103db0 0x7f67b419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.938+0000 7f67b8a9f640 1 --2- 172.21.15.1:0/3067179062 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f679c0b28a0 0x7f679c0b4db0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.938+0000 7f67b8a9f640 1 --2- 172.21.15.1:0/3067179062 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f67b4102b70 0x7f67b419a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.938+0000 7f67b8a9f640 1 -- 172.21.15.1:0/3067179062 >> 172.21.15.1:0/3067179062 conn(0x7f67b40fdd60 msgr2=0x7f67b406bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.938+0000 7f67b8a9f640 1 -- 172.21.15.1:0/3067179062 shutdown_connections 2024-05-04T03:24:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:14.938+0000 7f67b8a9f640 1 -- 172.21.15.1:0/3067179062 wait complete. 2024-05-04T03:24:16.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:16 smithi001 ceph-mon[29731]: pgmap v544: 97 pgs: 97 active+clean; 101 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 543 KiB/s wr, 206 op/s 2024-05-04T03:24:16.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:16 smithi077 ceph-mon[33531]: pgmap v544: 97 pgs: 97 active+clean; 101 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 543 KiB/s wr, 206 op/s 2024-05-04T03:24:16.750 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:16 smithi072 ceph-mon[33332]: pgmap v544: 97 pgs: 97 active+clean; 101 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 543 KiB/s wr, 206 op/s 2024-05-04T03:24:18.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:18 smithi001 ceph-mon[29731]: pgmap v545: 97 pgs: 97 active+clean; 102 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 822 KiB/s wr, 328 op/s 2024-05-04T03:24:18.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:18 smithi077 ceph-mon[33531]: pgmap v545: 97 pgs: 97 active+clean; 102 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 822 KiB/s wr, 328 op/s 2024-05-04T03:24:18.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:18 smithi072 ceph-mon[33332]: pgmap v545: 97 pgs: 97 active+clean; 102 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 822 KiB/s wr, 328 op/s 2024-05-04T03:24:20.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:20 smithi001 ceph-mon[29731]: pgmap v546: 97 pgs: 97 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 559 op/s 2024-05-04T03:24:20.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:20 smithi077 ceph-mon[33531]: pgmap v546: 97 pgs: 97 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 559 op/s 2024-05-04T03:24:20.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:20 smithi072 ceph-mon[33332]: pgmap v546: 97 pgs: 97 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 559 op/s 2024-05-04T03:24:21.401 INFO:tasks.workunit.client.0.smithi001.stdout:10000 mapread 0x33cc4e3 thru 0x33cec16 (0x2734 bytes) 2024-05-04T03:24:22.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:22 smithi001 ceph-mon[29731]: pgmap v547: 97 pgs: 97 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.6 MiB/s wr, 660 op/s 2024-05-04T03:24:22.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:22 smithi077 ceph-mon[33531]: pgmap v547: 97 pgs: 97 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.6 MiB/s wr, 660 op/s 2024-05-04T03:24:22.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:22 smithi072 ceph-mon[33332]: pgmap v547: 97 pgs: 97 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.6 MiB/s wr, 660 op/s 2024-05-04T03:24:24.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:24 smithi001 ceph-mon[29731]: pgmap v548: 97 pgs: 97 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.1 MiB/s wr, 842 op/s 2024-05-04T03:24:24.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:24 smithi077 ceph-mon[33531]: pgmap v548: 97 pgs: 97 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.1 MiB/s wr, 842 op/s 2024-05-04T03:24:24.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:24 smithi072 ceph-mon[33332]: pgmap v548: 97 pgs: 97 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.1 MiB/s wr, 842 op/s 2024-05-04T03:24:26.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:26 smithi001 ceph-mon[29731]: pgmap v549: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 2.7 MiB/s wr, 1.05k op/s 2024-05-04T03:24:26.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:26 smithi077 ceph-mon[33531]: pgmap v549: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 2.7 MiB/s wr, 1.05k op/s 2024-05-04T03:24:26.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:26 smithi072 ceph-mon[33332]: pgmap v549: 97 pgs: 97 active+clean; 103 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 2.7 MiB/s wr, 1.05k op/s 2024-05-04T03:24:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.063+0000 7f68df0bd640 1 -- 172.21.15.1:0/845556872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68d8103d90 msgr2=0x7f68d81041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.063+0000 7f68df0bd640 1 --2- 172.21.15.1:0/845556872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68d8103d90 0x7f68d81041a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f68c8009900 tx=0x7f68c802a420 comp rx=0 tx=0).stop 2024-05-04T03:24:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.063+0000 7f68df0bd640 1 -- 172.21.15.1:0/845556872 shutdown_connections 2024-05-04T03:24:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.063+0000 7f68df0bd640 1 --2- 172.21.15.1:0/845556872 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68d8108a90 0x7f68d810aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.063+0000 7f68df0bd640 1 --2- 172.21.15.1:0/845556872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68d8103d90 0x7f68d81041a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.063+0000 7f68df0bd640 1 --2- 172.21.15.1:0/845556872 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68d8102b50 0x7f68d8103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.063+0000 7f68df0bd640 1 -- 172.21.15.1:0/845556872 >> 172.21.15.1:0/845556872 conn(0x7f68d80fdd40 msgr2=0x7f68d81001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.063+0000 7f68df0bd640 1 -- 172.21.15.1:0/845556872 shutdown_connections 2024-05-04T03:24:28.066 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.063+0000 7f68df0bd640 1 -- 172.21.15.1:0/845556872 wait complete. 2024-05-04T03:24:28.068 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.064+0000 7f68df0bd640 1 Processor -- start 2024-05-04T03:24:28.068 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.064+0000 7f68df0bd640 1 -- start start 2024-05-04T03:24:28.068 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.065+0000 7f68df0bd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68d8102b50 0x7f68d8199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.065+0000 7f68df0bd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68d8103d90 0x7f68d81996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.065+0000 7f68df0bd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68d8108a90 0x7f68d81a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.065+0000 7f68df0bd640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f68d810e650 con 0x7f68d8103d90 2024-05-04T03:24:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.065+0000 7f68df0bd640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f68d810e4d0 con 0x7f68d8108a90 2024-05-04T03:24:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.065+0000 7f68df0bd640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f68d810e7d0 con 0x7f68d8102b50 2024-05-04T03:24:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.065+0000 7f68cffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68d8103d90 0x7f68d81996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.065+0000 7f68dce59640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68d8102b50 0x7f68d8199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.065+0000 7f68dce59640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68d8102b50 0x7f68d8199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.1:52642/0 (socket says 172.21.15.1:52642) 2024-05-04T03:24:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.065+0000 7f68cffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68d8103d90 0x7f68d81996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57628/0 (socket says 172.21.15.1:57628) 2024-05-04T03:24:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.065+0000 7f68dce59640 1 -- 172.21.15.1:0/312955481 learned_addr learned my addr 172.21.15.1:0/312955481 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.065+0000 7f68dd65a640 1 --2- 172.21.15.1:0/312955481 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68d8108a90 0x7f68d81a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.065+0000 7f68dce59640 1 -- 172.21.15.1:0/312955481 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68d8108a90 msgr2=0x7f68d81a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.065+0000 7f68dce59640 1 --2- 172.21.15.1:0/312955481 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68d8108a90 0x7f68d81a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.065+0000 7f68dce59640 1 -- 172.21.15.1:0/312955481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68d8103d90 msgr2=0x7f68d81996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.065+0000 7f68dce59640 1 --2- 172.21.15.1:0/312955481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68d8103d90 0x7f68d81996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.065+0000 7f68dce59640 1 -- 172.21.15.1:0/312955481 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d81a0cd0 con 0x7f68d8102b50 2024-05-04T03:24:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.065+0000 7f68dd65a640 1 --2- 172.21.15.1:0/312955481 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68d8108a90 0x7f68d81a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:24:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.066+0000 7f68dce59640 1 --2- 172.21.15.1:0/312955481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68d8102b50 0x7f68d8199190 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f68d400e980 tx=0x7f68d400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.066+0000 7f68cdffb640 1 -- 172.21.15.1:0/312955481 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d400cd80 con 0x7f68d8102b50 2024-05-04T03:24:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.066+0000 7f68cdffb640 1 -- 172.21.15.1:0/312955481 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68d40108e0 con 0x7f68d8102b50 2024-05-04T03:24:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.066+0000 7f68df0bd640 1 -- 172.21.15.1:0/312955481 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68d81a0fc0 con 0x7f68d8102b50 2024-05-04T03:24:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.066+0000 7f68cdffb640 1 -- 172.21.15.1:0/312955481 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d4018990 con 0x7f68d8102b50 2024-05-04T03:24:28.070 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.066+0000 7f68df0bd640 1 -- 172.21.15.1:0/312955481 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68d81a13c0 con 0x7f68d8102b50 2024-05-04T03:24:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.069+0000 7f68cdffb640 1 -- 172.21.15.1:0/312955481 <== mon.1 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f68d401f020 con 0x7f68d8102b50 2024-05-04T03:24:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.070+0000 7f68e5f73740 1 Processor -- start 2024-05-04T03:24:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.070+0000 7f68e5f73740 1 -- start start 2024-05-04T03:24:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.070+0000 7f68e5f73740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5628e0c471b0 0x5628e0feac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:28.073 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.070+0000 7f68e5f73740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e0c1e2f0 0x5628e0feb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.070+0000 7f68e5f73740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628e0c264d0 0x5628e0ff20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.070+0000 7f68e5f73740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5628e0f4d5d0 con 0x5628e0c264d0 2024-05-04T03:24:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.070+0000 7f68e5f73740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5628e0f06890 con 0x5628e0c471b0 2024-05-04T03:24:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.070+0000 7f68e5f73740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5628e0f49590 con 0x5628e0c1e2f0 2024-05-04T03:24:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.070+0000 7f68cffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628e0c264d0 0x5628e0ff20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.070+0000 7f68dce59640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e0c1e2f0 0x5628e0feb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.070+0000 7f68cffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628e0c264d0 0x5628e0ff20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57644/0 (socket says 172.21.15.1:57644) 2024-05-04T03:24:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.070+0000 7f68dd65a640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5628e0c471b0 0x5628e0feac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.070+0000 7f68cffff640 1 -- 172.21.15.1:0/763889824 learned_addr learned my addr 172.21.15.1:0/763889824 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.070+0000 7f68dce59640 1 --2- 172.21.15.1:0/763889824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e0c1e2f0 0x5628e0feb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.1:52652/0 (socket says 172.21.15.1:52652) 2024-05-04T03:24:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.071+0000 7f68dd65a640 1 -- 172.21.15.1:0/763889824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e0c1e2f0 msgr2=0x5628e0feb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.071+0000 7f68dd65a640 1 --2- 172.21.15.1:0/763889824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e0c1e2f0 0x5628e0feb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.071+0000 7f68dd65a640 1 -- 172.21.15.1:0/763889824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628e0c264d0 msgr2=0x5628e0ff20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.071+0000 7f68dd65a640 1 --2- 172.21.15.1:0/763889824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628e0c264d0 0x5628e0ff20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.071+0000 7f68dd65a640 1 -- 172.21.15.1:0/763889824 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628e0c5f210 con 0x5628e0c471b0 2024-05-04T03:24:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.071+0000 7f68cdffb640 1 --2- 172.21.15.1:0/312955481 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f68c00b2920 0x7f68c00b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.071+0000 7f68dd65a640 1 --2- 172.21.15.1:0/763889824 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5628e0c471b0 0x5628e0feac70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009f40 tx=0x7f68d000f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=454d58fe9ce742c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.071+0000 7f68dce59640 1 --2- 172.21.15.1:0/312955481 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f68c00b2920 0x7f68c00b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.071+0000 7f68cdffb640 1 -- 172.21.15.1:0/312955481 <== mon.1 v2:172.21.15.77:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f68d40e23a0 con 0x7f68d8102b50 2024-05-04T03:24:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.071+0000 7f68beffd640 1 -- 172.21.15.1:0/763889824 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0011640 con 0x5628e0c471b0 2024-05-04T03:24:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.071+0000 7f68beffd640 1 -- 172.21.15.1:0/763889824 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68d0017070 con 0x5628e0c471b0 2024-05-04T03:24:28.075 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.071+0000 7f68beffd640 1 -- 172.21.15.1:0/763889824 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0010bb0 con 0x5628e0c471b0 2024-05-04T03:24:28.078 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.072+0000 7f68e5f73740 1 -- 172.21.15.1:0/763889824 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5628e0c471b0 msgr2=0x5628e0feac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.072+0000 7f68dce59640 1 --2- 172.21.15.1:0/312955481 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f68c00b2920 0x7f68c00b4e30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x5628e0fec050 tx=0x7f68d40e5ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.072+0000 7f68e5f73740 1 --2- 172.21.15.1:0/763889824 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5628e0c471b0 0x5628e0feac70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009f40 tx=0x7f68d000f7f0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.072+0000 7f68e5f73740 1 -- 172.21.15.1:0/763889824 shutdown_connections 2024-05-04T03:24:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.072+0000 7f68e5f73740 1 --2- 172.21.15.1:0/763889824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628e0c264d0 0x5628e0ff20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.072+0000 7f68e5f73740 1 --2- 172.21.15.1:0/763889824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e0c1e2f0 0x5628e0feb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.072+0000 7f68e5f73740 1 --2- 172.21.15.1:0/763889824 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5628e0c471b0 0x5628e0feac70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.072+0000 7f68e5f73740 1 -- 172.21.15.1:0/763889824 >> 172.21.15.1:0/763889824 conn(0x5628e0c3b300 msgr2=0x5628e0fe43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.072+0000 7f68e5f73740 1 -- 172.21.15.1:0/763889824 shutdown_connections 2024-05-04T03:24:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.072+0000 7f68e5f73740 1 -- 172.21.15.1:0/763889824 wait complete. 2024-05-04T03:24:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.073+0000 7f68e5f73740 1 -- start start 2024-05-04T03:24:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.073+0000 7f68e5f73740 1 Processor -- start 2024-05-04T03:24:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.073+0000 7f68e5f73740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628e0c1e2f0 0x5628e0ff4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:28.079 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.073+0000 7f68e5f73740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e0c264d0 0x5628e0ff4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.073+0000 7f68e5f73740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5628e0c471b0 0x5628e0ffbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.073+0000 7f68e5f73740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5628e0ff2b40 con 0x5628e0c1e2f0 2024-05-04T03:24:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.073+0000 7f68e5f73740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5628e0f0a370 con 0x5628e0c471b0 2024-05-04T03:24:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.073+0000 7f68e5f73740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5628e0ff2cc0 con 0x5628e0c264d0 2024-05-04T03:24:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.073+0000 7f68dd65a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628e0c1e2f0 0x5628e0ff4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.073+0000 7f68e5f73740 10 client.0 _ll_register_callbacks cb 0x5628e0c5ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:24:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.073+0000 7f68dd65a640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628e0c1e2f0 0x5628e0ff4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57652/0 (socket says 172.21.15.1:57652) 2024-05-04T03:24:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.073+0000 7f68dd65a640 1 -- 172.21.15.1:0/1684871278 learned_addr learned my addr 172.21.15.1:0/1684871278 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.073+0000 7f68cffff640 1 --2- 172.21.15.1:0/1684871278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e0c264d0 0x5628e0ff4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.073+0000 7f68dd65a640 1 --2- 172.21.15.1:0/1684871278 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5628e0c471b0 0x5628e0ffbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.074+0000 7f68dd65a640 1 -- 172.21.15.1:0/1684871278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e0c264d0 msgr2=0x5628e0ff4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.074+0000 7f68dd65a640 1 --2- 172.21.15.1:0/1684871278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e0c264d0 0x5628e0ff4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.074+0000 7f68dd65a640 1 -- 172.21.15.1:0/1684871278 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5628e0c471b0 msgr2=0x5628e0ffbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.074+0000 7f68dd65a640 1 --2- 172.21.15.1:0/1684871278 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5628e0c471b0 0x5628e0ffbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.074+0000 7f68dd65a640 1 -- 172.21.15.1:0/1684871278 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628e0d2b620 con 0x5628e0c1e2f0 2024-05-04T03:24:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.074+0000 7f68cffff640 1 --2- 172.21.15.1:0/1684871278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e0c264d0 0x5628e0ff4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:24:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.074+0000 7f68dd65a640 1 --2- 172.21.15.1:0/1684871278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628e0c1e2f0 0x5628e0ff4850 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009d80 tx=0x7f68d00092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be3fc4459ffb037 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.074+0000 7f68b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:24:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.074+0000 7f68b7fff640 1 -- 172.21.15.1:0/1684871278 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0026070 con 0x5628e0c1e2f0 2024-05-04T03:24:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.074+0000 7f68e5f73740 1 -- 172.21.15.1:0/1684871278 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68d0017070 con 0x5628e0c1e2f0 2024-05-04T03:24:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.074+0000 7f68b7fff640 1 -- 172.21.15.1:0/1684871278 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f68d0010d60 con 0x5628e0c1e2f0 2024-05-04T03:24:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.075+0000 7f68b7fff640 1 -- 172.21.15.1:0/1684871278 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f68d0004b20 con 0x5628e0c1e2f0 2024-05-04T03:24:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.075+0000 7f68e5f73740 10 client.15858 fetch_fsmap learned FSMap version 32 2024-05-04T03:24:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.075+0000 7f68e5f73740 1 -- 172.21.15.1:0/1684871278 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5628e0fe8680 con 0x5628e0c1e2f0 2024-05-04T03:24:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.075+0000 7f68b7fff640 1 -- 172.21.15.1:0/1684871278 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68d0017070 con 0x5628e0c1e2f0 2024-05-04T03:24:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.075+0000 7f68b7fff640 1 -- 172.21.15.1:0/1684871278 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f68d0004040 con 0x5628e0c1e2f0 2024-05-04T03:24:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.075+0000 7f68e5f73740 10 client.15858 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:24:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.075+0000 7f68e5f73740 10 client.15858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:24:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.075+0000 7f68e5f73740 20 client.15858 populate_metadata read hostname 'smithi001' 2024-05-04T03:24:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.075+0000 7f68e5f73740 1 --2- 172.21.15.1:0/1684871278 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5628e0c05d80 0x5628e0fe9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.075+0000 7f68e5f73740 4 client.15858 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:24:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.075+0000 7f68e5f73740 1 -- 172.21.15.1:0/1684871278 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628e0f5b1e0 con 0x5628e0c05d80 2024-05-04T03:24:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.075+0000 7f68cffff640 1 --2- 172.21.15.1:0/1684871278 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5628e0c05d80 0x5628e0fe9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.076+0000 7f68cffff640 1 --2- 172.21.15.1:0/1684871278 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5628e0c05d80 0x5628e0fe9af0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d317ef00c43c024f server_cookie=cb1724bf0c81bd05 in_seq=0 out_seq=0 2024-05-04T03:24:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.076+0000 7f68b7fff640 10 client.15858 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:24:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.077+0000 7f68b7fff640 1 -- 172.21.15.1:0/1684871278 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5628e0f5b1e0 con 0x5628e0c05d80 2024-05-04T03:24:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.077+0000 7f68b7fff640 10 client.15858 handle_command_reply: tid=0 2024-05-04T03:24:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.077+0000 7f68e5f73740 1 client.15858 shutdown 2024-05-04T03:24:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.077+0000 7f68e5f73740 1 -- 172.21.15.1:0/1684871278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628e0c1e2f0 msgr2=0x5628e0ff4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:28.083 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.077+0000 7f68e5f73740 1 --2- 172.21.15.1:0/1684871278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628e0c1e2f0 0x5628e0ff4850 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009d80 tx=0x7f68d00092a0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.077+0000 7f68e5f73740 1 -- 172.21.15.1:0/1684871278 shutdown_connections 2024-05-04T03:24:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.077+0000 7f68e5f73740 1 --2- 172.21.15.1:0/1684871278 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5628e0c05d80 0x5628e0fe9af0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.077+0000 7f68e5f73740 1 --2- 172.21.15.1:0/1684871278 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5628e0c471b0 0x5628e0ffbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.077+0000 7f68e5f73740 1 --2- 172.21.15.1:0/1684871278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e0c264d0 0x5628e0ff4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.077+0000 7f68e5f73740 1 --2- 172.21.15.1:0/1684871278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628e0c1e2f0 0x5628e0ff4850 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.077+0000 7f68e5f73740 1 -- 172.21.15.1:0/1684871278 >> 172.21.15.1:0/1684871278 conn(0x5628e0c3b300 msgr2=0x5628e0fe7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.077+0000 7f68b7fff640 0 client.15858 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:24:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.078+0000 7f68e5f73740 1 -- 172.21.15.1:0/1684871278 shutdown_connections 2024-05-04T03:24:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.078+0000 7f68e5f73740 1 -- 172.21.15.1:0/1684871278 wait complete. 2024-05-04T03:24:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.078+0000 7f68e5f73740 20 client.15858 trim_cache size 0 max 16384 2024-05-04T03:24:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.098+0000 7f68e5f73740 1 Processor -- start 2024-05-04T03:24:28.100 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.098+0000 7f68e5f73740 1 -- start start 2024-05-04T03:24:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.099+0000 7f68e5f73740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e0c1e2f0 0x5628e0fe7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.099+0000 7f68e5f73740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628e0c264d0 0x5628e0ffbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.099+0000 7f68e5f73740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5628e0c471b0 0x5628e0ffc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.099+0000 7f68cffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628e0c264d0 0x5628e0ffbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.099+0000 7f68cffff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628e0c264d0 0x5628e0ffbc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57666/0 (socket says 172.21.15.1:57666) 2024-05-04T03:24:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.099+0000 7f68cffff640 1 -- 172.21.15.1:0/4248941459 learned_addr learned my addr 172.21.15.1:0/4248941459 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.099+0000 7f68dd65a640 1 --2- 172.21.15.1:0/4248941459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e0c1e2f0 0x5628e0fe7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.099+0000 7f68e5f73740 1 -- 172.21.15.1:0/4248941459 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f68d81ab8c0 con 0x5628e0c264d0 2024-05-04T03:24:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.099+0000 7f68e5f73740 1 -- 172.21.15.1:0/4248941459 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f68d81a9190 con 0x5628e0c471b0 2024-05-04T03:24:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.099+0000 7f68e5f73740 1 -- 172.21.15.1:0/4248941459 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f68d81a7020 con 0x5628e0c1e2f0 2024-05-04T03:24:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.099+0000 7f68dd65a640 1 -- 172.21.15.1:0/4248941459 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5628e0c471b0 msgr2=0x5628e0ffc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.099+0000 7f68dd65a640 1 --2- 172.21.15.1:0/4248941459 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5628e0c471b0 0x5628e0ffc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.099+0000 7f68dd65a640 1 -- 172.21.15.1:0/4248941459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628e0c264d0 msgr2=0x5628e0ffbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.100+0000 7f68dd65a640 1 --2- 172.21.15.1:0/4248941459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628e0c264d0 0x5628e0ffbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.100+0000 7f68dd65a640 1 -- 172.21.15.1:0/4248941459 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628e0fea830 con 0x5628e0c1e2f0 2024-05-04T03:24:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.100+0000 7f68cffff640 1 --2- 172.21.15.1:0/4248941459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628e0c264d0 0x5628e0ffbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:24:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.100+0000 7f68dd65a640 1 --2- 172.21.15.1:0/4248941459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e0c1e2f0 0x5628e0fe7900 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f68d000fd70 tx=0x7f68d0009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b43b33bae4986b1e server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.100+0000 7f68be7fc640 1 -- 172.21.15.1:0/4248941459 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0026040 con 0x5628e0c1e2f0 2024-05-04T03:24:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.100+0000 7f68be7fc640 1 -- 172.21.15.1:0/4248941459 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68d0017040 con 0x5628e0c1e2f0 2024-05-04T03:24:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.100+0000 7f68be7fc640 1 -- 172.21.15.1:0/4248941459 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d002d680 con 0x5628e0c1e2f0 2024-05-04T03:24:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.101+0000 7f68e5f73740 1 -- 172.21.15.1:0/4248941459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e0c1e2f0 msgr2=0x5628e0fe7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.102+0000 7f68e5f73740 1 --2- 172.21.15.1:0/4248941459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e0c1e2f0 0x5628e0fe7900 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f68d000fd70 tx=0x7f68d0009fd0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.102+0000 7f68e5f73740 1 -- 172.21.15.1:0/4248941459 shutdown_connections 2024-05-04T03:24:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.102+0000 7f68e5f73740 1 --2- 172.21.15.1:0/4248941459 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5628e0c471b0 0x5628e0ffc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.102+0000 7f68e5f73740 1 --2- 172.21.15.1:0/4248941459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628e0c264d0 0x5628e0ffbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.102+0000 7f68e5f73740 1 --2- 172.21.15.1:0/4248941459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e0c1e2f0 0x5628e0fe7900 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.102+0000 7f68e5f73740 1 -- 172.21.15.1:0/4248941459 >> 172.21.15.1:0/4248941459 conn(0x5628e0c3b300 msgr2=0x5628e0fe71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.102+0000 7f68e5f73740 1 -- 172.21.15.1:0/4248941459 shutdown_connections 2024-05-04T03:24:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.102+0000 7f68e5f73740 1 -- 172.21.15.1:0/4248941459 wait complete. 2024-05-04T03:24:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.102+0000 7f68e5f73740 1 -- start start 2024-05-04T03:24:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.102+0000 7f68e5f73740 1 Processor -- start 2024-05-04T03:24:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.103+0000 7f68e5f73740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628e0c1e2f0 0x5628e0ffbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.103+0000 7f68e5f73740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5628e0c264d0 0x5628e0ffc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.103+0000 7f68e5f73740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e0c471b0 0x5628e0ff8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.103+0000 7f68e5f73740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f68d81a7670 con 0x5628e0c1e2f0 2024-05-04T03:24:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.103+0000 7f68e5f73740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f68d81a8ac0 con 0x5628e0c264d0 2024-05-04T03:24:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.103+0000 7f68e5f73740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f68d81a7d20 con 0x5628e0c471b0 2024-05-04T03:24:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.103+0000 7f68e5f73740 10 client.0 _ll_register_callbacks cb 0x7f68d0015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:24:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.103+0000 7f68cffff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5628e0c264d0 0x5628e0ffc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.103+0000 7f68cffff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5628e0c264d0 0x5628e0ffc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:56496/0 (socket says 172.21.15.1:56496) 2024-05-04T03:24:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.103+0000 7f68cffff640 1 -- 172.21.15.1:0/2202984092 learned_addr learned my addr 172.21.15.1:0/2202984092 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.103+0000 7f68dd65a640 1 --2- 172.21.15.1:0/2202984092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628e0c1e2f0 0x5628e0ffbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.103+0000 7f68dd65a640 1 --2- 172.21.15.1:0/2202984092 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e0c471b0 0x5628e0ff8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.103+0000 7f68cffff640 1 -- 172.21.15.1:0/2202984092 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e0c471b0 msgr2=0x5628e0ff8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.103+0000 7f68cffff640 1 --2- 172.21.15.1:0/2202984092 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e0c471b0 0x5628e0ff8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.103+0000 7f68cffff640 1 -- 172.21.15.1:0/2202984092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628e0c1e2f0 msgr2=0x5628e0ffbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.103+0000 7f68cffff640 1 --2- 172.21.15.1:0/2202984092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628e0c1e2f0 0x5628e0ffbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.103+0000 7f68cffff640 1 -- 172.21.15.1:0/2202984092 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628e0ff8dc0 con 0x5628e0c264d0 2024-05-04T03:24:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.104+0000 7f68cffff640 1 --2- 172.21.15.1:0/2202984092 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5628e0c264d0 0x5628e0ffc0e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f68c8000c00 tx=0x7f68c8031b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da3cca8531384b70 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.104+0000 7f68b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-04T03:24:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.104+0000 7f68b7fff640 1 -- 172.21.15.1:0/2202984092 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8035040 con 0x5628e0c264d0 2024-05-04T03:24:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.104+0000 7f68b7fff640 1 -- 172.21.15.1:0/2202984092 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f68c80463a0 con 0x5628e0c264d0 2024-05-04T03:24:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.104+0000 7f68e5f73740 1 -- 172.21.15.1:0/2202984092 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68d0017040 con 0x5628e0c264d0 2024-05-04T03:24:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.104+0000 7f68b7fff640 1 -- 172.21.15.1:0/2202984092 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f68c8055050 con 0x5628e0c264d0 2024-05-04T03:24:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.104+0000 7f68e5f73740 10 client.25574 fetch_fsmap learned FSMap version 32 2024-05-04T03:24:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.104+0000 7f68e5f73740 1 -- 172.21.15.1:0/2202984092 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5628e1067470 con 0x5628e0c264d0 2024-05-04T03:24:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.104+0000 7f68b7fff640 1 -- 172.21.15.1:0/2202984092 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68c8046a50 con 0x5628e0c264d0 2024-05-04T03:24:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.105+0000 7f68b7fff640 1 -- 172.21.15.1:0/2202984092 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f68c803cc20 con 0x5628e0c264d0 2024-05-04T03:24:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.105+0000 7f68e5f73740 10 client.25574 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:24:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.105+0000 7f68e5f73740 10 client.25574 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:24:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.105+0000 7f68e5f73740 20 client.25574 populate_metadata read hostname 'smithi001' 2024-05-04T03:24:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.105+0000 7f68e5f73740 1 --2- 172.21.15.1:0/2202984092 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5628e0bdb1c0 0x5628e1069e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.105+0000 7f68e5f73740 4 client.25574 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:24:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.105+0000 7f68e5f73740 1 -- 172.21.15.1:0/2202984092 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f68d0017040 con 0x5628e0bdb1c0 2024-05-04T03:24:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.105+0000 7f68dd65a640 1 --2- 172.21.15.1:0/2202984092 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5628e0bdb1c0 0x5628e1069e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.105+0000 7f68b7fff640 10 client.25574 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:24:28.110 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.105+0000 7f68dd65a640 1 --2- 172.21.15.1:0/2202984092 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5628e0bdb1c0 0x5628e1069e40 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cb9cdb041f7da2c server_cookie=14e2a30d82f3617a in_seq=0 out_seq=0 2024-05-04T03:24:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.114+0000 7f68b7fff640 1 -- 172.21.15.1:0/2202984092 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f68d0017040 con 0x5628e0bdb1c0 2024-05-04T03:24:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.114+0000 7f68b7fff640 10 client.25574 handle_command_reply: tid=0 2024-05-04T03:24:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.114+0000 7f68e5f73740 1 client.25574 shutdown 2024-05-04T03:24:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.114+0000 7f68e5f73740 1 -- 172.21.15.1:0/2202984092 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5628e0c264d0 msgr2=0x5628e0ffc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.114+0000 7f68e5f73740 1 --2- 172.21.15.1:0/2202984092 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5628e0c264d0 0x5628e0ffc0e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f68c8000c00 tx=0x7f68c8031b70 comp rx=0 tx=0).stop 2024-05-04T03:24:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.114+0000 7f68e5f73740 1 -- 172.21.15.1:0/2202984092 shutdown_connections 2024-05-04T03:24:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.114+0000 7f68e5f73740 1 --2- 172.21.15.1:0/2202984092 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5628e0bdb1c0 0x5628e1069e40 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.114+0000 7f68e5f73740 1 --2- 172.21.15.1:0/2202984092 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5628e0c264d0 0x5628e0ffc0e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.114+0000 7f68e5f73740 1 --2- 172.21.15.1:0/2202984092 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5628e0c471b0 0x5628e0ff8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.114+0000 7f68b7fff640 0 client.25574 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:24:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.114+0000 7f68e5f73740 1 --2- 172.21.15.1:0/2202984092 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5628e0c1e2f0 0x5628e0ffbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.114+0000 7f68e5f73740 1 -- 172.21.15.1:0/2202984092 >> 172.21.15.1:0/2202984092 conn(0x5628e0c3b300 msgr2=0x5628e0ff4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.114+0000 7f68e5f73740 1 -- 172.21.15.1:0/2202984092 shutdown_connections 2024-05-04T03:24:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.114+0000 7f68e5f73740 1 -- 172.21.15.1:0/2202984092 wait complete. 2024-05-04T03:24:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.114+0000 7f68e5f73740 20 client.25574 trim_cache size 0 max 16384 2024-05-04T03:24:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:24:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:24:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:24:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:24:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.115+0000 7f68df0bd640 1 -- 172.21.15.1:0/312955481 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f68c00b2920 msgr2=0x7f68c00b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.115+0000 7f68df0bd640 1 --2- 172.21.15.1:0/312955481 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f68c00b2920 0x7f68c00b4e30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x5628e0fec050 tx=0x7f68d40e5ec0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.115+0000 7f68df0bd640 1 -- 172.21.15.1:0/312955481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68d8102b50 msgr2=0x7f68d8199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.115+0000 7f68df0bd640 1 --2- 172.21.15.1:0/312955481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68d8102b50 0x7f68d8199190 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f68d400e980 tx=0x7f68d400ee40 comp rx=0 tx=0).stop 2024-05-04T03:24:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.115+0000 7f68df0bd640 1 -- 172.21.15.1:0/312955481 shutdown_connections 2024-05-04T03:24:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.115+0000 7f68df0bd640 1 --2- 172.21.15.1:0/312955481 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f68c00b2920 0x7f68c00b4e30 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.115+0000 7f68df0bd640 1 --2- 172.21.15.1:0/312955481 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68d8108a90 0x7f68d81a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.115+0000 7f68df0bd640 1 --2- 172.21.15.1:0/312955481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f68d8103d90 0x7f68d81996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.115+0000 7f68df0bd640 1 --2- 172.21.15.1:0/312955481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68d8102b50 0x7f68d8199190 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.115+0000 7f68df0bd640 1 -- 172.21.15.1:0/312955481 >> 172.21.15.1:0/312955481 conn(0x7f68d80fdd40 msgr2=0x7f68d8104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.115+0000 7f68df0bd640 1 -- 172.21.15.1:0/312955481 shutdown_connections 2024-05-04T03:24:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:28.115+0000 7f68df0bd640 1 -- 172.21.15.1:0/312955481 wait complete. 2024-05-04T03:24:28.364 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0x63e5d2a 2024-05-04T03:24:28.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:28 smithi001 ceph-mon[29731]: pgmap v550: 97 pgs: 97 active+clean; 104 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.5 MiB/s wr, 981 op/s 2024-05-04T03:24:28.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:28 smithi077 ceph-mon[33531]: pgmap v550: 97 pgs: 97 active+clean; 104 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.5 MiB/s wr, 981 op/s 2024-05-04T03:24:28.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:28 smithi072 ceph-mon[33332]: pgmap v550: 97 pgs: 97 active+clean; 104 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.5 MiB/s wr, 981 op/s 2024-05-04T03:24:30.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:30 smithi001 ceph-mon[29731]: pgmap v551: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 2.8 MiB/s wr, 1.09k op/s 2024-05-04T03:24:30.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:30 smithi077 ceph-mon[33531]: pgmap v551: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 2.8 MiB/s wr, 1.09k op/s 2024-05-04T03:24:30.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:30 smithi072 ceph-mon[33332]: pgmap v551: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 2.8 MiB/s wr, 1.09k op/s 2024-05-04T03:24:31.024 INFO:tasks.workunit.client.0.smithi001.stdout:truncating to largest ever: 0x63e9b52 2024-05-04T03:24:32.379 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:32 smithi001 ceph-mon[29731]: pgmap v552: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.6 MiB/s wr, 959 op/s 2024-05-04T03:24:32.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:32 smithi077 ceph-mon[33531]: pgmap v552: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.6 MiB/s wr, 959 op/s 2024-05-04T03:24:32.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:32 smithi072 ceph-mon[33332]: pgmap v552: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.6 MiB/s wr, 959 op/s 2024-05-04T03:24:33.120 INFO:tasks.workunit.client.0.smithi001.stdout:20000 read 0xa6cbf1 thru 0xa6e358 (0x1768 bytes) 2024-05-04T03:24:34.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:34 smithi001 ceph-mon[29731]: pgmap v553: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.5 MiB/s wr, 962 op/s 2024-05-04T03:24:34.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:34 smithi077 ceph-mon[33531]: pgmap v553: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.5 MiB/s wr, 962 op/s 2024-05-04T03:24:34.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:34 smithi072 ceph-mon[33332]: pgmap v553: 97 pgs: 97 active+clean; 116 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.5 MiB/s wr, 962 op/s 2024-05-04T03:24:35.102 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0x63ed6b9 2024-05-04T03:24:36.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:36 smithi001 ceph-mon[29731]: pgmap v554: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 2.8 MiB/s wr, 1.07k op/s 2024-05-04T03:24:36.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:36 smithi077 ceph-mon[33531]: pgmap v554: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 2.8 MiB/s wr, 1.07k op/s 2024-05-04T03:24:36.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:36 smithi072 ceph-mon[33332]: pgmap v554: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 2.8 MiB/s wr, 1.07k op/s 2024-05-04T03:24:38.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:38 smithi001 ceph-mon[29731]: pgmap v555: 97 pgs: 97 active+clean; 114 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.2 MiB/s wr, 881 op/s 2024-05-04T03:24:38.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:38 smithi077 ceph-mon[33531]: pgmap v555: 97 pgs: 97 active+clean; 114 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.2 MiB/s wr, 881 op/s 2024-05-04T03:24:38.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:38 smithi072 ceph-mon[33332]: pgmap v555: 97 pgs: 97 active+clean; 114 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.2 MiB/s wr, 881 op/s 2024-05-04T03:24:40.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:40 smithi001 ceph-mon[29731]: pgmap v556: 97 pgs: 97 active+clean; 120 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.8 MiB/s wr, 1.03k op/s 2024-05-04T03:24:40.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:40 smithi077 ceph-mon[33531]: pgmap v556: 97 pgs: 97 active+clean; 120 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.8 MiB/s wr, 1.03k op/s 2024-05-04T03:24:40.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:40 smithi072 ceph-mon[33332]: pgmap v556: 97 pgs: 97 active+clean; 120 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.8 MiB/s wr, 1.03k op/s 2024-05-04T03:24:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.240+0000 7f70ada3b640 1 -- 172.21.15.1:0/1324489656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70a8109ba0 msgr2=0x7f70a810bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.240+0000 7f70ada3b640 1 --2- 172.21.15.1:0/1324489656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70a8109ba0 0x7f70a810bfc0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f70a0009ca0 tx=0x7f70a002a8c0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.240+0000 7f70ada3b640 1 -- 172.21.15.1:0/1324489656 shutdown_connections 2024-05-04T03:24:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.240+0000 7f70ada3b640 1 --2- 172.21.15.1:0/1324489656 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70a8109ba0 0x7f70a810bfc0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.240+0000 7f70ada3b640 1 --2- 172.21.15.1:0/1324489656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70a8104ea0 0x7f70a81052b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.240+0000 7f70ada3b640 1 --2- 172.21.15.1:0/1324489656 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f70a8103c60 0x7f70a8104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.243 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.240+0000 7f70ada3b640 1 -- 172.21.15.1:0/1324489656 >> 172.21.15.1:0/1324489656 conn(0x7f70a80fee70 msgr2=0x7f70a8101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.241+0000 7f70ada3b640 1 -- 172.21.15.1:0/1324489656 shutdown_connections 2024-05-04T03:24:41.244 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.241+0000 7f70ada3b640 1 -- 172.21.15.1:0/1324489656 wait complete. 2024-05-04T03:24:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.241+0000 7f70ada3b640 1 Processor -- start 2024-05-04T03:24:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.242+0000 7f70ada3b640 1 -- start start 2024-05-04T03:24:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.242+0000 7f70ada3b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f70a8103c60 0x7f70a819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.242+0000 7f70ada3b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70a8104ea0 0x7f70a819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.242+0000 7f70ada3b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70a8109ba0 0x7f70a81a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.242+0000 7f70ada3b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f70a810f790 con 0x7f70a8104ea0 2024-05-04T03:24:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.242+0000 7f70ada3b640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f70a810f610 con 0x7f70a8103c60 2024-05-04T03:24:41.246 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.242+0000 7f70ada3b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f70a810f910 con 0x7f70a8109ba0 2024-05-04T03:24:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.242+0000 7f70a67fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70a8104ea0 0x7f70a819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.242+0000 7f70a67fc640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70a8104ea0 0x7f70a819a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54672/0 (socket says 172.21.15.1:54672) 2024-05-04T03:24:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.242+0000 7f70a77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70a8109ba0 0x7f70a81a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.242+0000 7f70a67fc640 1 -- 172.21.15.1:0/1047632966 learned_addr learned my addr 172.21.15.1:0/1047632966 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.242+0000 7f70a6ffd640 1 --2- 172.21.15.1:0/1047632966 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f70a8103c60 0x7f70a819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.242+0000 7f70a6ffd640 1 -- 172.21.15.1:0/1047632966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70a8109ba0 msgr2=0x7f70a81a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.242+0000 7f70a6ffd640 1 --2- 172.21.15.1:0/1047632966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70a8109ba0 0x7f70a81a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.242+0000 7f70a6ffd640 1 -- 172.21.15.1:0/1047632966 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70a8104ea0 msgr2=0x7f70a819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.242+0000 7f70a6ffd640 1 --2- 172.21.15.1:0/1047632966 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70a8104ea0 0x7f70a819a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.242+0000 7f70a6ffd640 1 -- 172.21.15.1:0/1047632966 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a81a1d80 con 0x7f70a8103c60 2024-05-04T03:24:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.242+0000 7f70a67fc640 1 --2- 172.21.15.1:0/1047632966 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70a8104ea0 0x7f70a819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:24:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.243+0000 7f70a6ffd640 1 --2- 172.21.15.1:0/1047632966 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f70a8103c60 0x7f70a819a240 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f70a8103530 tx=0x7f709c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.243+0000 7f70aca39640 1 -- 172.21.15.1:0/1047632966 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c00cc40 con 0x7f70a8103c60 2024-05-04T03:24:41.247 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.243+0000 7f70ada3b640 1 -- 172.21.15.1:0/1047632966 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70a81a2070 con 0x7f70a8103c60 2024-05-04T03:24:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.243+0000 7f70aca39640 1 -- 172.21.15.1:0/1047632966 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f709c00cde0 con 0x7f70a8103c60 2024-05-04T03:24:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.243+0000 7f70aca39640 1 -- 172.21.15.1:0/1047632966 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c010430 con 0x7f70a8103c60 2024-05-04T03:24:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.243+0000 7f70ada3b640 1 -- 172.21.15.1:0/1047632966 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70a81a25c0 con 0x7f70a8103c60 2024-05-04T03:24:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.246+0000 7f70b49c6740 1 Processor -- start 2024-05-04T03:24:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.246+0000 7f70b49c6740 1 -- start start 2024-05-04T03:24:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.246+0000 7f70b49c6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622ae7fe1b0 0x5622aeba3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.246+0000 7f70b49c6740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622ae7d52f0 0x5622aeba43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.246+0000 7f70b49c6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ae7dd4d0 0x5622aebab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.246+0000 7f70b49c6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5622aeb045d0 con 0x5622ae7fe1b0 2024-05-04T03:24:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.246+0000 7f70b49c6740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5622aeabd890 con 0x5622ae7d52f0 2024-05-04T03:24:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.246+0000 7f70b49c6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5622aeb00590 con 0x5622ae7dd4d0 2024-05-04T03:24:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.246+0000 7f70aca39640 1 -- 172.21.15.1:0/1047632966 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f709c0105d0 con 0x7f70a8103c60 2024-05-04T03:24:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.246+0000 7f70a77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622ae7fe1b0 0x5622aeba3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.246+0000 7f70a77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622ae7fe1b0 0x5622aeba3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54678/0 (socket says 172.21.15.1:54678) 2024-05-04T03:24:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.246+0000 7f70a77fe640 1 -- 172.21.15.1:0/1420033032 learned_addr learned my addr 172.21.15.1:0/1420033032 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.246+0000 7f70a67fc640 1 --2- 172.21.15.1:0/1420033032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ae7dd4d0 0x5622aebab2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.247+0000 7f70a6ffd640 1 --2- 172.21.15.1:0/1420033032 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622ae7d52f0 0x5622aeba43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.247+0000 7f70a77fe640 1 -- 172.21.15.1:0/1420033032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ae7dd4d0 msgr2=0x5622aebab2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.247+0000 7f70a77fe640 1 --2- 172.21.15.1:0/1420033032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ae7dd4d0 0x5622aebab2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.247+0000 7f70a77fe640 1 -- 172.21.15.1:0/1420033032 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622ae7d52f0 msgr2=0x5622aeba43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:41.250 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.247+0000 7f70a77fe640 1 --2- 172.21.15.1:0/1420033032 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622ae7d52f0 0x5622aeba43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.247+0000 7f70a77fe640 1 -- 172.21.15.1:0/1420033032 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622ae816210 con 0x5622ae7fe1b0 2024-05-04T03:24:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.247+0000 7f70a77fe640 1 --2- 172.21.15.1:0/1420033032 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622ae7fe1b0 0x5622aeba3e70 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f70a0035000 tx=0x7f70a002f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940017fe54508e6c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.247+0000 7f70aca39640 1 --2- 172.21.15.1:0/1047632966 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f70900b28a0 0x7f70900b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.248+0000 7f708dffb640 1 -- 172.21.15.1:0/1420033032 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a0031650 con 0x5622ae7fe1b0 2024-05-04T03:24:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.248+0000 7f708dffb640 1 -- 172.21.15.1:0/1420033032 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70a0036030 con 0x5622ae7fe1b0 2024-05-04T03:24:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.248+0000 7f70aca39640 1 -- 172.21.15.1:0/1047632966 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f709c0e2560 con 0x7f70a8103c60 2024-05-04T03:24:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.248+0000 7f708dffb640 1 -- 172.21.15.1:0/1420033032 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a00047a0 con 0x5622ae7fe1b0 2024-05-04T03:24:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.248+0000 7f70a6ffd640 1 --2- 172.21.15.1:0/1047632966 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f70900b28a0 0x7f70900b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.249+0000 7f70b49c6740 1 -- 172.21.15.1:0/1420033032 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622ae7fe1b0 msgr2=0x5622aeba3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.249+0000 7f70a6ffd640 1 --2- 172.21.15.1:0/1047632966 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f70900b28a0 0x7f70900b4db0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f709c000c00 tx=0x7f709c0e80e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.249+0000 7f70b49c6740 1 --2- 172.21.15.1:0/1420033032 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622ae7fe1b0 0x5622aeba3e70 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f70a0035000 tx=0x7f70a002f410 comp rx=0 tx=0).stop 2024-05-04T03:24:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.250+0000 7f70b49c6740 1 -- 172.21.15.1:0/1420033032 shutdown_connections 2024-05-04T03:24:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.250+0000 7f70b49c6740 1 --2- 172.21.15.1:0/1420033032 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622ae7d52f0 0x5622aeba43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.250+0000 7f70b49c6740 1 --2- 172.21.15.1:0/1420033032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ae7dd4d0 0x5622aebab2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.250+0000 7f70b49c6740 1 --2- 172.21.15.1:0/1420033032 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622ae7fe1b0 0x5622aeba3e70 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.250+0000 7f70b49c6740 1 -- 172.21.15.1:0/1420033032 >> 172.21.15.1:0/1420033032 conn(0x5622ae7f2300 msgr2=0x5622aeb9b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.250+0000 7f70b49c6740 1 -- 172.21.15.1:0/1420033032 shutdown_connections 2024-05-04T03:24:41.253 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.250+0000 7f70b49c6740 1 -- 172.21.15.1:0/1420033032 wait complete. 2024-05-04T03:24:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.250+0000 7f70b49c6740 1 -- start start 2024-05-04T03:24:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.250+0000 7f70b49c6740 1 Processor -- start 2024-05-04T03:24:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.251+0000 7f70b49c6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622ae7d52f0 0x5622aebad930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.251+0000 7f70b49c6740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622ae7dd4d0 0x5622aebade70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.251+0000 7f70b49c6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ae7fe1b0 0x5622aebb4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.251+0000 7f70b49c6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5622aebabb60 con 0x5622ae7d52f0 2024-05-04T03:24:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.251+0000 7f70b49c6740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5622aeac1370 con 0x5622ae7dd4d0 2024-05-04T03:24:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.251+0000 7f70b49c6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5622aebabce0 con 0x5622ae7fe1b0 2024-05-04T03:24:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.251+0000 7f70b49c6740 10 client.0 _ll_register_callbacks cb 0x5622ae815a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.251+0000 7f70a77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622ae7d52f0 0x5622aebad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.251+0000 7f70a77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622ae7d52f0 0x5622aebad930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54682/0 (socket says 172.21.15.1:54682) 2024-05-04T03:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.251+0000 7f70a77fe640 1 -- 172.21.15.1:0/354322175 learned_addr learned my addr 172.21.15.1:0/354322175 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.251+0000 7f70a77fe640 1 --2- 172.21.15.1:0/354322175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ae7fe1b0 0x5622aebb4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.251+0000 7f70a67fc640 1 --2- 172.21.15.1:0/354322175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622ae7dd4d0 0x5622aebade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.251+0000 7f70a77fe640 1 -- 172.21.15.1:0/354322175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ae7fe1b0 msgr2=0x5622aebb4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.251+0000 7f70a77fe640 1 --2- 172.21.15.1:0/354322175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ae7fe1b0 0x5622aebb4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.251+0000 7f70a77fe640 1 -- 172.21.15.1:0/354322175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622ae7dd4d0 msgr2=0x5622aebade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.251+0000 7f70a77fe640 1 --2- 172.21.15.1:0/354322175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622ae7dd4d0 0x5622aebade70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.251+0000 7f70a77fe640 1 -- 172.21.15.1:0/354322175 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622ae8e2620 con 0x5622ae7d52f0 2024-05-04T03:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.252+0000 7f70a77fe640 1 --2- 172.21.15.1:0/354322175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622ae7d52f0 0x5622aebad930 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f70a0004af0 tx=0x7f70a0031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9adde57650926227 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.252+0000 7f7086ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:24:41.257 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.252+0000 7f7086ffd640 1 -- 172.21.15.1:0/354322175 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a0059060 con 0x5622ae7d52f0 2024-05-04T03:24:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.252+0000 7f7086ffd640 1 -- 172.21.15.1:0/354322175 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f70a0003b70 con 0x5622ae7d52f0 2024-05-04T03:24:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.252+0000 7f70b49c6740 1 -- 172.21.15.1:0/354322175 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70a0036030 con 0x5622ae7d52f0 2024-05-04T03:24:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.252+0000 7f7086ffd640 1 -- 172.21.15.1:0/354322175 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f70a0003d70 con 0x5622ae7d52f0 2024-05-04T03:24:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.252+0000 7f70b49c6740 10 client.15876 fetch_fsmap learned FSMap version 32 2024-05-04T03:24:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.252+0000 7f70b49c6740 1 -- 172.21.15.1:0/354322175 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5622aebabb60 con 0x5622ae7d52f0 2024-05-04T03:24:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.253+0000 7f7086ffd640 1 -- 172.21.15.1:0/354322175 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70a0036030 con 0x5622ae7d52f0 2024-05-04T03:24:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.253+0000 7f7086ffd640 1 -- 172.21.15.1:0/354322175 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f70a0043070 con 0x5622ae7d52f0 2024-05-04T03:24:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.253+0000 7f70b49c6740 10 client.15876 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:24:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.253+0000 7f70b49c6740 10 client.15876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:24:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.253+0000 7f70b49c6740 20 client.15876 populate_metadata read hostname 'smithi001' 2024-05-04T03:24:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.253+0000 7f70b49c6740 1 --2- 172.21.15.1:0/354322175 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5622ae7bcd80 0x5622aebbb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.253+0000 7f70b49c6740 4 client.15876 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:24:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.253+0000 7f70b49c6740 1 -- 172.21.15.1:0/354322175 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622aeb121e0 con 0x5622ae7bcd80 2024-05-04T03:24:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.253+0000 7f70a67fc640 1 --2- 172.21.15.1:0/354322175 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5622ae7bcd80 0x5622aebbb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.253+0000 7f7086ffd640 10 client.15876 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:24:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.254+0000 7f70a67fc640 1 --2- 172.21.15.1:0/354322175 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5622ae7bcd80 0x5622aebbb050 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6186be8c247729c server_cookie=d873915ab4cd03d2 in_seq=0 out_seq=0 2024-05-04T03:24:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.254+0000 7f7086ffd640 1 -- 172.21.15.1:0/354322175 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5622aeb121e0 con 0x5622ae7bcd80 2024-05-04T03:24:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.254+0000 7f7086ffd640 10 client.15876 handle_command_reply: tid=0 2024-05-04T03:24:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.255+0000 7f70b49c6740 1 client.15876 shutdown 2024-05-04T03:24:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.255+0000 7f70b49c6740 1 -- 172.21.15.1:0/354322175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622ae7d52f0 msgr2=0x5622aebad930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.255+0000 7f70b49c6740 1 --2- 172.21.15.1:0/354322175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622ae7d52f0 0x5622aebad930 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f70a0004af0 tx=0x7f70a0031af0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.255+0000 7f70b49c6740 1 -- 172.21.15.1:0/354322175 shutdown_connections 2024-05-04T03:24:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.255+0000 7f70b49c6740 1 --2- 172.21.15.1:0/354322175 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5622ae7bcd80 0x5622aebbb050 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.255+0000 7f70b49c6740 1 --2- 172.21.15.1:0/354322175 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622ae7dd4d0 0x5622aebade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.255+0000 7f70b49c6740 1 --2- 172.21.15.1:0/354322175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ae7fe1b0 0x5622aebb4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.255+0000 7f70b49c6740 1 --2- 172.21.15.1:0/354322175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622ae7d52f0 0x5622aebad930 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.255+0000 7f70b49c6740 1 -- 172.21.15.1:0/354322175 >> 172.21.15.1:0/354322175 conn(0x5622ae7f2300 msgr2=0x5622aeb9a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.255+0000 7f7086ffd640 0 client.15876 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:24:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.255+0000 7f70b49c6740 1 -- 172.21.15.1:0/354322175 shutdown_connections 2024-05-04T03:24:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.255+0000 7f70b49c6740 1 -- 172.21.15.1:0/354322175 wait complete. 2024-05-04T03:24:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.255+0000 7f70b49c6740 20 client.15876 trim_cache size 0 max 16384 2024-05-04T03:24:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.272+0000 7f70b49c6740 1 Processor -- start 2024-05-04T03:24:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.272+0000 7f70b49c6740 1 -- start start 2024-05-04T03:24:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.272+0000 7f70b49c6740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622ae7d52f0 0x5622aeb9b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.272+0000 7f70b49c6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ae7dd4d0 0x5622aebb4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.272+0000 7f70b49c6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622ae7fe1b0 0x5622aebb52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.272+0000 7f70b49c6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f70a81ab790 con 0x5622ae7fe1b0 2024-05-04T03:24:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.272+0000 7f70b49c6740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f70a81a9060 con 0x5622ae7d52f0 2024-05-04T03:24:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.272+0000 7f70b49c6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f70a81a6eb0 con 0x5622ae7dd4d0 2024-05-04T03:24:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.272+0000 7f70a67fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ae7dd4d0 0x5622aebb4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.272+0000 7f70a77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622ae7fe1b0 0x5622aebb52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.272+0000 7f70a77fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622ae7d52f0 0x5622aeb9b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.272+0000 7f70a67fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ae7dd4d0 0x5622aebb4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.1:52386/0 (socket says 172.21.15.1:52386) 2024-05-04T03:24:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.272+0000 7f70a67fc640 1 -- 172.21.15.1:0/3057358172 learned_addr learned my addr 172.21.15.1:0/3057358172 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.273+0000 7f70a67fc640 1 -- 172.21.15.1:0/3057358172 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622ae7d52f0 msgr2=0x5622aeb9b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.273+0000 7f70a67fc640 1 --2- 172.21.15.1:0/3057358172 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622ae7d52f0 0x5622aeb9b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.273+0000 7f70a67fc640 1 -- 172.21.15.1:0/3057358172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622ae7fe1b0 msgr2=0x5622aebb52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.273+0000 7f70a67fc640 1 --2- 172.21.15.1:0/3057358172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622ae7fe1b0 0x5622aebb52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.273+0000 7f70a67fc640 1 -- 172.21.15.1:0/3057358172 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622aeb9b940 con 0x5622ae7dd4d0 2024-05-04T03:24:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.273+0000 7f70a77fe640 1 --2- 172.21.15.1:0/3057358172 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622ae7d52f0 0x5622aeb9b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:24:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.273+0000 7f70a67fc640 1 --2- 172.21.15.1:0/3057358172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ae7dd4d0 0x5622aebb4d60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x5622aeb9c180 tx=0x7f7098005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93c4632e3587da0f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.273+0000 7f708d7fa640 1 -- 172.21.15.1:0/3057358172 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709800f890 con 0x5622ae7dd4d0 2024-05-04T03:24:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.273+0000 7f708d7fa640 1 -- 172.21.15.1:0/3057358172 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7098026030 con 0x5622ae7dd4d0 2024-05-04T03:24:41.277 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.273+0000 7f708d7fa640 1 -- 172.21.15.1:0/3057358172 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70980047a0 con 0x5622ae7dd4d0 2024-05-04T03:24:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.274+0000 7f70b49c6740 1 -- 172.21.15.1:0/3057358172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ae7dd4d0 msgr2=0x5622aebb4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.274+0000 7f70b49c6740 1 --2- 172.21.15.1:0/3057358172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ae7dd4d0 0x5622aebb4d60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x5622aeb9c180 tx=0x7f7098005f90 comp rx=0 tx=0).stop 2024-05-04T03:24:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.275+0000 7f70b49c6740 1 -- 172.21.15.1:0/3057358172 shutdown_connections 2024-05-04T03:24:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.275+0000 7f70b49c6740 1 --2- 172.21.15.1:0/3057358172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622ae7fe1b0 0x5622aebb52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.275+0000 7f70b49c6740 1 --2- 172.21.15.1:0/3057358172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ae7dd4d0 0x5622aebb4d60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.275+0000 7f70b49c6740 1 --2- 172.21.15.1:0/3057358172 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622ae7d52f0 0x5622aeb9b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.278 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.275+0000 7f70b49c6740 1 -- 172.21.15.1:0/3057358172 >> 172.21.15.1:0/3057358172 conn(0x5622ae7f2300 msgr2=0x5622aeb9acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.277+0000 7f70b49c6740 1 -- 172.21.15.1:0/3057358172 shutdown_connections 2024-05-04T03:24:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.277+0000 7f70b49c6740 1 -- 172.21.15.1:0/3057358172 wait complete. 2024-05-04T03:24:41.279 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.277+0000 7f70b49c6740 1 -- start start 2024-05-04T03:24:41.280 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.278+0000 7f70b49c6740 1 Processor -- start 2024-05-04T03:24:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.278+0000 7f70b49c6740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622ae7d52f0 0x5622aebb09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.278+0000 7f70b49c6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ae7dd4d0 0x5622aebb0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.278+0000 7f70b49c6740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622ae7fe1b0 0x5622aebb3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.278+0000 7f70b49c6740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f70a81a7520 con 0x5622ae7d52f0 2024-05-04T03:24:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.278+0000 7f70b49c6740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f70a81a8990 con 0x5622ae7fe1b0 2024-05-04T03:24:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.278+0000 7f70b49c6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f70a81a7bf0 con 0x5622ae7dd4d0 2024-05-04T03:24:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.278+0000 7f70b49c6740 10 client.0 _ll_register_callbacks cb 0x7f709c0e5f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:24:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.278+0000 7f70a77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622ae7d52f0 0x5622aebb09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.278+0000 7f70a67fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ae7dd4d0 0x5622aebb0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.278+0000 7f70a67fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ae7dd4d0 0x5622aebb0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.1:52402/0 (socket says 172.21.15.1:52402) 2024-05-04T03:24:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.279+0000 7f70a67fc640 1 -- 172.21.15.1:0/2235967075 learned_addr learned my addr 172.21.15.1:0/2235967075 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.279+0000 7f70a77fe640 1 --2- 172.21.15.1:0/2235967075 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622ae7fe1b0 0x5622aebb3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.279+0000 7f70a67fc640 1 -- 172.21.15.1:0/2235967075 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622ae7fe1b0 msgr2=0x5622aebb3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.279+0000 7f70a67fc640 1 --2- 172.21.15.1:0/2235967075 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622ae7fe1b0 0x5622aebb3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.279+0000 7f70a67fc640 1 -- 172.21.15.1:0/2235967075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622ae7d52f0 msgr2=0x5622aebb09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.279+0000 7f70a67fc640 1 --2- 172.21.15.1:0/2235967075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622ae7d52f0 0x5622aebb09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.279+0000 7f70a67fc640 1 -- 172.21.15.1:0/2235967075 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622aebb4420 con 0x5622ae7dd4d0 2024-05-04T03:24:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.279+0000 7f70a77fe640 1 --2- 172.21.15.1:0/2235967075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622ae7d52f0 0x5622aebb09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:24:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.279+0000 7f70a67fc640 1 --2- 172.21.15.1:0/2235967075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ae7dd4d0 0x5622aebb0f20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7098000c00 tx=0x7f70980056f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57c48ad6f1dd87c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.279+0000 7f7086ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-04T03:24:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.279+0000 7f7086ffd640 1 -- 172.21.15.1:0/2235967075 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709802e020 con 0x5622ae7dd4d0 2024-05-04T03:24:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.279+0000 7f7086ffd640 1 -- 172.21.15.1:0/2235967075 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f709800f890 con 0x5622ae7dd4d0 2024-05-04T03:24:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.279+0000 7f70b49c6740 1 -- 172.21.15.1:0/2235967075 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7098026030 con 0x5622ae7dd4d0 2024-05-04T03:24:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.280+0000 7f7086ffd640 1 -- 172.21.15.1:0/2235967075 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f709800fa90 con 0x5622ae7dd4d0 2024-05-04T03:24:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.280+0000 7f70b49c6740 10 client.25555 fetch_fsmap learned FSMap version 32 2024-05-04T03:24:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.280+0000 7f70b49c6740 1 -- 172.21.15.1:0/2235967075 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5622aebb56e0 con 0x5622ae7dd4d0 2024-05-04T03:24:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.280+0000 7f7086ffd640 1 -- 172.21.15.1:0/2235967075 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7098026030 con 0x5622ae7dd4d0 2024-05-04T03:24:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.280+0000 7f7086ffd640 1 -- 172.21.15.1:0/2235967075 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f7098010430 con 0x5622ae7dd4d0 2024-05-04T03:24:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.280+0000 7f70b49c6740 10 client.25555 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:24:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.280+0000 7f70b49c6740 10 client.25555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:24:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.280+0000 7f70b49c6740 20 client.25555 populate_metadata read hostname 'smithi001' 2024-05-04T03:24:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.280+0000 7f70b49c6740 1 --2- 172.21.15.1:0/2235967075 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5622ae7921c0 0x5622aec259d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.280+0000 7f70b49c6740 4 client.25555 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:24:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.280+0000 7f70b49c6740 1 -- 172.21.15.1:0/2235967075 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5622aeb0c050 con 0x5622ae7921c0 2024-05-04T03:24:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.280+0000 7f70a77fe640 1 --2- 172.21.15.1:0/2235967075 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5622ae7921c0 0x5622aec259d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.281+0000 7f7086ffd640 10 client.25555 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:24:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.281+0000 7f70a77fe640 1 --2- 172.21.15.1:0/2235967075 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5622ae7921c0 0x5622aec259d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28ee051e216c81f4 server_cookie=6ed000e9f899bb8d in_seq=0 out_seq=0 2024-05-04T03:24:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.289+0000 7f7086ffd640 1 -- 172.21.15.1:0/2235967075 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5622aeb0c050 con 0x5622ae7921c0 2024-05-04T03:24:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.289+0000 7f7086ffd640 10 client.25555 handle_command_reply: tid=0 2024-05-04T03:24:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.289+0000 7f70b49c6740 1 client.25555 shutdown 2024-05-04T03:24:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.289+0000 7f70b49c6740 1 -- 172.21.15.1:0/2235967075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ae7dd4d0 msgr2=0x5622aebb0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.289+0000 7f70b49c6740 1 --2- 172.21.15.1:0/2235967075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ae7dd4d0 0x5622aebb0f20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7098000c00 tx=0x7f70980056f0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.289+0000 7f70b49c6740 1 -- 172.21.15.1:0/2235967075 shutdown_connections 2024-05-04T03:24:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.289+0000 7f70b49c6740 1 --2- 172.21.15.1:0/2235967075 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5622ae7921c0 0x5622aec259d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.289+0000 7f70b49c6740 1 --2- 172.21.15.1:0/2235967075 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622ae7fe1b0 0x5622aebb3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.289+0000 7f70b49c6740 1 --2- 172.21.15.1:0/2235967075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5622ae7dd4d0 0x5622aebb0f20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.289+0000 7f7086ffd640 0 client.25555 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:24:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.289+0000 7f70b49c6740 1 --2- 172.21.15.1:0/2235967075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5622ae7d52f0 0x5622aebb09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.289+0000 7f70b49c6740 1 -- 172.21.15.1:0/2235967075 >> 172.21.15.1:0/2235967075 conn(0x5622ae7f2300 msgr2=0x5622aeb9e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.289+0000 7f70b49c6740 1 -- 172.21.15.1:0/2235967075 shutdown_connections 2024-05-04T03:24:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.290+0000 7f70b49c6740 1 -- 172.21.15.1:0/2235967075 wait complete. 2024-05-04T03:24:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.290+0000 7f70b49c6740 20 client.25555 trim_cache size 0 max 16384 2024-05-04T03:24:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:24:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:24:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:24:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:24:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.290+0000 7f70ada3b640 1 -- 172.21.15.1:0/1047632966 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f70900b28a0 msgr2=0x7f70900b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.290+0000 7f70ada3b640 1 --2- 172.21.15.1:0/1047632966 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f70900b28a0 0x7f70900b4db0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f709c000c00 tx=0x7f709c0e80e0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.290+0000 7f70ada3b640 1 -- 172.21.15.1:0/1047632966 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f70a8103c60 msgr2=0x7f70a819a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.290+0000 7f70ada3b640 1 --2- 172.21.15.1:0/1047632966 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f70a8103c60 0x7f70a819a240 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f70a8103530 tx=0x7f709c00ee50 comp rx=0 tx=0).stop 2024-05-04T03:24:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.290+0000 7f70ada3b640 1 -- 172.21.15.1:0/1047632966 shutdown_connections 2024-05-04T03:24:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.290+0000 7f70ada3b640 1 --2- 172.21.15.1:0/1047632966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f70a8109ba0 0x7f70a81a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.290+0000 7f70ada3b640 1 --2- 172.21.15.1:0/1047632966 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70a8104ea0 0x7f70a819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.290+0000 7f70ada3b640 1 --2- 172.21.15.1:0/1047632966 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f70900b28a0 0x7f70900b4db0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.290+0000 7f70ada3b640 1 --2- 172.21.15.1:0/1047632966 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f70a8103c60 0x7f70a819a240 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.290+0000 7f70ada3b640 1 -- 172.21.15.1:0/1047632966 >> 172.21.15.1:0/1047632966 conn(0x7f70a80fee70 msgr2=0x7f70a8105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.291+0000 7f70ada3b640 1 -- 172.21.15.1:0/1047632966 shutdown_connections 2024-05-04T03:24:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:41.291+0000 7f70ada3b640 1 -- 172.21.15.1:0/1047632966 wait complete. 2024-05-04T03:24:42.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:42 smithi077 ceph-mon[33531]: pgmap v557: 97 pgs: 97 active+clean; 114 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.4 MiB/s wr, 876 op/s 2024-05-04T03:24:42.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:42 smithi072 ceph-mon[33332]: pgmap v557: 97 pgs: 97 active+clean; 114 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.4 MiB/s wr, 876 op/s 2024-05-04T03:24:42.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:42 smithi001 ceph-mon[29731]: pgmap v557: 97 pgs: 97 active+clean; 114 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.4 MiB/s wr, 876 op/s 2024-05-04T03:24:44.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:44 smithi077 ceph-mon[33531]: pgmap v558: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 879 op/s 2024-05-04T03:24:44.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:44 smithi072 ceph-mon[33332]: pgmap v558: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 879 op/s 2024-05-04T03:24:44.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:44 smithi001 ceph-mon[29731]: pgmap v558: 97 pgs: 97 active+clean; 111 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 879 op/s 2024-05-04T03:24:45.705 INFO:tasks.workunit.client.0.smithi001.stdout:copying to largest ever: 0x63ef9ee 2024-05-04T03:24:46.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:46 smithi077 ceph-mon[33531]: pgmap v559: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 3.0 MiB/s wr, 1.10k op/s 2024-05-04T03:24:46.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:46 smithi072 ceph-mon[33332]: pgmap v559: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 3.0 MiB/s wr, 1.10k op/s 2024-05-04T03:24:46.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:46 smithi001 ceph-mon[29731]: pgmap v559: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 3.0 MiB/s wr, 1.10k op/s 2024-05-04T03:24:48.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:48 smithi077 ceph-mon[33531]: pgmap v560: 97 pgs: 97 active+clean; 125 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.3 MiB/s wr, 858 op/s 2024-05-04T03:24:48.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:48 smithi072 ceph-mon[33332]: pgmap v560: 97 pgs: 97 active+clean; 125 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.3 MiB/s wr, 858 op/s 2024-05-04T03:24:48.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:48 smithi001 ceph-mon[29731]: pgmap v560: 97 pgs: 97 active+clean; 125 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.3 MiB/s wr, 858 op/s 2024-05-04T03:24:50.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:50 smithi077 ceph-mon[33531]: pgmap v561: 97 pgs: 97 active+clean; 125 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.7 MiB/s wr, 1.02k op/s 2024-05-04T03:24:50.681 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:50 smithi001 ceph-mon[29731]: pgmap v561: 97 pgs: 97 active+clean; 125 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.7 MiB/s wr, 1.02k op/s 2024-05-04T03:24:50.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:50 smithi072 ceph-mon[33332]: pgmap v561: 97 pgs: 97 active+clean; 125 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.7 MiB/s wr, 1.02k op/s 2024-05-04T03:24:52.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:52 smithi077 ceph-mon[33531]: pgmap v562: 97 pgs: 97 active+clean; 126 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.4 MiB/s wr, 931 op/s 2024-05-04T03:24:52.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:52 smithi072 ceph-mon[33332]: pgmap v562: 97 pgs: 97 active+clean; 126 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.4 MiB/s wr, 931 op/s 2024-05-04T03:24:52.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:52 smithi001 ceph-mon[29731]: pgmap v562: 97 pgs: 97 active+clean; 126 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.4 MiB/s wr, 931 op/s 2024-05-04T03:24:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.411+0000 7f4cedd38640 1 -- 172.21.15.1:0/2051141004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ce8103d90 msgr2=0x7f4ce81041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.411+0000 7f4cedd38640 1 --2- 172.21.15.1:0/2051141004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ce8103d90 0x7f4ce81041a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009900 tx=0x7f4cd002a410 comp rx=0 tx=0).stop 2024-05-04T03:24:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.411+0000 7f4cedd38640 1 -- 172.21.15.1:0/2051141004 shutdown_connections 2024-05-04T03:24:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.411+0000 7f4cedd38640 1 --2- 172.21.15.1:0/2051141004 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ce8108a90 0x7f4ce810aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.411+0000 7f4cedd38640 1 --2- 172.21.15.1:0/2051141004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ce8103d90 0x7f4ce81041a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.411+0000 7f4cedd38640 1 --2- 172.21.15.1:0/2051141004 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ce8102b50 0x7f4ce8103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.411+0000 7f4cedd38640 1 -- 172.21.15.1:0/2051141004 >> 172.21.15.1:0/2051141004 conn(0x7f4ce80fdd40 msgr2=0x7f4ce81001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.411+0000 7f4cedd38640 1 -- 172.21.15.1:0/2051141004 shutdown_connections 2024-05-04T03:24:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.411+0000 7f4cedd38640 1 -- 172.21.15.1:0/2051141004 wait complete. 2024-05-04T03:24:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.411+0000 7f4cedd38640 1 Processor -- start 2024-05-04T03:24:54.414 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.412+0000 7f4cedd38640 1 -- start start 2024-05-04T03:24:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.412+0000 7f4cedd38640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ce8102b50 0x7f4ce819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.412+0000 7f4cedd38640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ce8103d90 0x7f4ce819a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.412+0000 7f4cedd38640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ce8108a90 0x7f4ce81a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.412+0000 7f4cedd38640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce810e6c0 con 0x7f4ce8108a90 2024-05-04T03:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.412+0000 7f4cedd38640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce810e540 con 0x7f4ce8102b50 2024-05-04T03:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.412+0000 7f4cedd38640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce810e840 con 0x7f4ce8103d90 2024-05-04T03:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.412+0000 7f4ce77fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ce8102b50 0x7f4ce819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.412+0000 7f4ce77fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ce8102b50 0x7f4ce819a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:48514/0 (socket says 172.21.15.1:48514) 2024-05-04T03:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.412+0000 7f4ce77fe640 1 -- 172.21.15.1:0/4092302140 learned_addr learned my addr 172.21.15.1:0/4092302140 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.412+0000 7f4cdeffd640 1 --2- 172.21.15.1:0/4092302140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ce8103d90 0x7f4ce819a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.412+0000 7f4ce7fff640 1 --2- 172.21.15.1:0/4092302140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ce8108a90 0x7f4ce81a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.412+0000 7f4ce77fe640 1 -- 172.21.15.1:0/4092302140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ce8103d90 msgr2=0x7f4ce819a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.412+0000 7f4ce77fe640 1 --2- 172.21.15.1:0/4092302140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ce8103d90 0x7f4ce819a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.412+0000 7f4ce77fe640 1 -- 172.21.15.1:0/4092302140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ce8108a90 msgr2=0x7f4ce81a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.412+0000 7f4ce77fe640 1 --2- 172.21.15.1:0/4092302140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ce8108a90 0x7f4ce81a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.412+0000 7f4ce77fe640 1 -- 172.21.15.1:0/4092302140 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce81a1e10 con 0x7f4ce8102b50 2024-05-04T03:24:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.413+0000 7f4ce77fe640 1 --2- 172.21.15.1:0/4092302140 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ce8102b50 0x7f4ce819a2d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4cd800e9f0 tx=0x7f4cd800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.413+0000 7f4ce57fa640 1 -- 172.21.15.1:0/4092302140 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd800ccc0 con 0x7f4ce8102b50 2024-05-04T03:24:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.413+0000 7f4ce57fa640 1 -- 172.21.15.1:0/4092302140 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cd80035f0 con 0x7f4ce8102b50 2024-05-04T03:24:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.413+0000 7f4cedd38640 1 -- 172.21.15.1:0/4092302140 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ce81a2100 con 0x7f4ce8102b50 2024-05-04T03:24:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.413+0000 7f4ce57fa640 1 -- 172.21.15.1:0/4092302140 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd800ce60 con 0x7f4ce8102b50 2024-05-04T03:24:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.413+0000 7f4cedd38640 1 -- 172.21.15.1:0/4092302140 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ce81a2620 con 0x7f4ce8102b50 2024-05-04T03:24:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.415+0000 7f4ce57fa640 1 -- 172.21.15.1:0/4092302140 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f4cd8010690 con 0x7f4ce8102b50 2024-05-04T03:24:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.415+0000 7f4cf4bef740 1 Processor -- start 2024-05-04T03:24:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.416+0000 7f4cf4bef740 1 -- start start 2024-05-04T03:24:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.416+0000 7f4cf4bef740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559844a731b0 0x559844e14a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.416+0000 7f4cf4bef740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559844a4a2f0 0x559844e14fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.416+0000 7f4cf4bef740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559844a524d0 0x559844e1bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.416+0000 7f4cf4bef740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559844d795d0 con 0x559844a4a2f0 2024-05-04T03:24:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.416+0000 7f4cf4bef740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x559844d32890 con 0x559844a731b0 2024-05-04T03:24:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.416+0000 7f4cf4bef740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559844d75590 con 0x559844a524d0 2024-05-04T03:24:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.416+0000 7f4ce77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559844a4a2f0 0x559844e14fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.416+0000 7f4ce7fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559844a731b0 0x559844e14a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.416+0000 7f4ce77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559844a4a2f0 0x559844e14fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42142/0 (socket says 172.21.15.1:42142) 2024-05-04T03:24:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.416+0000 7f4ce77fe640 1 -- 172.21.15.1:0/3453708693 learned_addr learned my addr 172.21.15.1:0/3453708693 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.416+0000 7f4cdeffd640 1 --2- 172.21.15.1:0/3453708693 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559844a524d0 0x559844e1bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.416+0000 7f4ce77fe640 1 -- 172.21.15.1:0/3453708693 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559844a524d0 msgr2=0x559844e1bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.416+0000 7f4ce77fe640 1 --2- 172.21.15.1:0/3453708693 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559844a524d0 0x559844e1bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.416+0000 7f4ce77fe640 1 -- 172.21.15.1:0/3453708693 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559844a731b0 msgr2=0x559844e14a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.416+0000 7f4ce77fe640 1 --2- 172.21.15.1:0/3453708693 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559844a731b0 0x559844e14a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.416+0000 7f4ce57fa640 1 --2- 172.21.15.1:0/4092302140 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4cd40b2920 0x7f4cd40b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.416+0000 7f4ce77fe640 1 -- 172.21.15.1:0/3453708693 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559844a8b210 con 0x559844a4a2f0 2024-05-04T03:24:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.416+0000 7f4ce7fff640 1 --2- 172.21.15.1:0/3453708693 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559844a731b0 0x559844e14a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:24:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.416+0000 7f4cdeffd640 1 --2- 172.21.15.1:0/3453708693 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559844a524d0 0x559844e1bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:24:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.416+0000 7f4cdeffd640 1 --2- 172.21.15.1:0/4092302140 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4cd40b2920 0x7f4cd40b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.417+0000 7f4ce57fa640 1 -- 172.21.15.1:0/4092302140 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f4cd80e23e0 con 0x7f4ce8102b50 2024-05-04T03:24:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.417+0000 7f4ce77fe640 1 --2- 172.21.15.1:0/3453708693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559844a4a2f0 0x559844e14fe0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4cd80a5b40 tx=0x7f4cd80a61d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491aa1414238315f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.417+0000 7f4cddffb640 1 -- 172.21.15.1:0/3453708693 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd80a9890 con 0x559844a4a2f0 2024-05-04T03:24:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.417+0000 7f4cddffb640 1 -- 172.21.15.1:0/3453708693 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cd80a95d0 con 0x559844a4a2f0 2024-05-04T03:24:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.417+0000 7f4cddffb640 1 -- 172.21.15.1:0/3453708693 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd80a9230 con 0x559844a4a2f0 2024-05-04T03:24:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.418+0000 7f4cf4bef740 1 -- 172.21.15.1:0/3453708693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559844a4a2f0 msgr2=0x559844e14fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.418+0000 7f4cf4bef740 1 --2- 172.21.15.1:0/3453708693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559844a4a2f0 0x559844e14fe0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4cd80a5b40 tx=0x7f4cd80a61d0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.418+0000 7f4cdeffd640 1 --2- 172.21.15.1:0/4092302140 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4cd40b2920 0x7f4cd40b4e30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0004770 tx=0x7f4cd00314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.418+0000 7f4cf4bef740 1 -- 172.21.15.1:0/3453708693 shutdown_connections 2024-05-04T03:24:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.418+0000 7f4cf4bef740 1 --2- 172.21.15.1:0/3453708693 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559844a524d0 0x559844e1bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.418+0000 7f4cf4bef740 1 --2- 172.21.15.1:0/3453708693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559844a4a2f0 0x559844e14fe0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.418+0000 7f4cf4bef740 1 --2- 172.21.15.1:0/3453708693 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559844a731b0 0x559844e14a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.418+0000 7f4cf4bef740 1 -- 172.21.15.1:0/3453708693 >> 172.21.15.1:0/3453708693 conn(0x559844a67300 msgr2=0x559844e103a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.418+0000 7f4cf4bef740 1 -- 172.21.15.1:0/3453708693 shutdown_connections 2024-05-04T03:24:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.418+0000 7f4cf4bef740 1 -- 172.21.15.1:0/3453708693 wait complete. 2024-05-04T03:24:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.418+0000 7f4cf4bef740 1 -- start start 2024-05-04T03:24:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.418+0000 7f4cf4bef740 1 Processor -- start 2024-05-04T03:24:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.418+0000 7f4cf4bef740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559844a4a2f0 0x559844e1e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.419+0000 7f4cf4bef740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559844a524d0 0x559844e1ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.419+0000 7f4cf4bef740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559844a731b0 0x559844e25b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.419+0000 7f4cf4bef740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x559844e1c930 con 0x559844a731b0 2024-05-04T03:24:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.419+0000 7f4cf4bef740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x559844d36370 con 0x559844a4a2f0 2024-05-04T03:24:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.419+0000 7f4cf4bef740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559844e1cab0 con 0x559844a524d0 2024-05-04T03:24:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.419+0000 7f4cf4bef740 10 client.0 _ll_register_callbacks cb 0x559844a8aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:24:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.419+0000 7f4cdeffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559844a524d0 0x559844e1ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.419+0000 7f4ce77fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559844a4a2f0 0x559844e1e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.419+0000 7f4cdeffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559844a524d0 0x559844e1ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.1:33368/0 (socket says 172.21.15.1:33368) 2024-05-04T03:24:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.419+0000 7f4ce77fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559844a4a2f0 0x559844e1e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:48542/0 (socket says 172.21.15.1:48542) 2024-05-04T03:24:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.419+0000 7f4cdeffd640 1 -- 172.21.15.1:0/3168045470 learned_addr learned my addr 172.21.15.1:0/3168045470 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.419+0000 7f4ce7fff640 1 --2- 172.21.15.1:0/3168045470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559844a731b0 0x559844e25b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.419+0000 7f4cdeffd640 1 -- 172.21.15.1:0/3168045470 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559844a4a2f0 msgr2=0x559844e1e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:54.427 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.419+0000 7f4cdeffd640 1 --2- 172.21.15.1:0/3168045470 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559844a4a2f0 0x559844e1e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.419+0000 7f4cdeffd640 1 -- 172.21.15.1:0/3168045470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559844a731b0 msgr2=0x559844e25b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.419+0000 7f4cdeffd640 1 --2- 172.21.15.1:0/3168045470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559844a731b0 0x559844e25b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.419+0000 7f4cdeffd640 1 -- 172.21.15.1:0/3168045470 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559844b57620 con 0x559844a524d0 2024-05-04T03:24:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.419+0000 7f4ce7fff640 1 --2- 172.21.15.1:0/3168045470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559844a731b0 0x559844e25b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:24:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.419+0000 7f4cdeffd640 1 --2- 172.21.15.1:0/3168045470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559844a524d0 0x559844e1ec10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009900 tx=0x7f4cd0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1657d73a68877efb server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.419+0000 7f4cc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-04T03:24:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.419+0000 7f4cc6ffd640 1 -- 172.21.15.1:0/3168045470 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0035040 con 0x559844a524d0 2024-05-04T03:24:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.420+0000 7f4cf4bef740 1 -- 172.21.15.1:0/3168045470 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cd80a95d0 con 0x559844a524d0 2024-05-04T03:24:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.420+0000 7f4cc6ffd640 1 -- 172.21.15.1:0/3168045470 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f4cd00315a0 con 0x559844a524d0 2024-05-04T03:24:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.420+0000 7f4cc6ffd640 1 -- 172.21.15.1:0/3168045470 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4cd00317a0 con 0x559844a524d0 2024-05-04T03:24:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.420+0000 7f4cf4bef740 10 client.25567 fetch_fsmap learned FSMap version 32 2024-05-04T03:24:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.420+0000 7f4cf4bef740 1 -- 172.21.15.1:0/3168045470 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559844e28b60 con 0x559844a524d0 2024-05-04T03:24:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.420+0000 7f4cc6ffd640 1 -- 172.21.15.1:0/3168045470 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cd80a95d0 con 0x559844a524d0 2024-05-04T03:24:54.428 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.420+0000 7f4cc6ffd640 1 -- 172.21.15.1:0/3168045470 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f4cd0044030 con 0x559844a524d0 2024-05-04T03:24:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.420+0000 7f4cf4bef740 10 client.25567 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:24:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.420+0000 7f4cf4bef740 10 client.25567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:24:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.420+0000 7f4cf4bef740 20 client.25567 populate_metadata read hostname 'smithi001' 2024-05-04T03:24:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.420+0000 7f4cf4bef740 1 --2- 172.21.15.1:0/3168045470 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x559844a31d80 0x559844e2c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.420+0000 7f4cf4bef740 4 client.25567 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:24:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.420+0000 7f4cf4bef740 1 -- 172.21.15.1:0/3168045470 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559844d871e0 con 0x559844a31d80 2024-05-04T03:24:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.420+0000 7f4ce77fe640 1 --2- 172.21.15.1:0/3168045470 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x559844a31d80 0x559844e2c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.421+0000 7f4cc6ffd640 10 client.25567 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:24:54.429 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.421+0000 7f4ce77fe640 1 --2- 172.21.15.1:0/3168045470 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x559844a31d80 0x559844e2c230 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eca70f25650e5639 server_cookie=5bba322c4b80ab0d in_seq=0 out_seq=0 2024-05-04T03:24:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.422+0000 7f4cc6ffd640 1 -- 172.21.15.1:0/3168045470 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559844d871e0 con 0x559844a31d80 2024-05-04T03:24:54.430 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.422+0000 7f4cc6ffd640 10 client.25567 handle_command_reply: tid=0 2024-05-04T03:24:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.422+0000 7f4cf4bef740 1 client.25567 shutdown 2024-05-04T03:24:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.422+0000 7f4cf4bef740 1 -- 172.21.15.1:0/3168045470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559844a524d0 msgr2=0x559844e1ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.422+0000 7f4cf4bef740 1 --2- 172.21.15.1:0/3168045470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559844a524d0 0x559844e1ec10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009900 tx=0x7f4cd0009260 comp rx=0 tx=0).stop 2024-05-04T03:24:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.422+0000 7f4cf4bef740 1 -- 172.21.15.1:0/3168045470 shutdown_connections 2024-05-04T03:24:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.422+0000 7f4cf4bef740 1 --2- 172.21.15.1:0/3168045470 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x559844a31d80 0x559844e2c230 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.422+0000 7f4cf4bef740 1 --2- 172.21.15.1:0/3168045470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559844a731b0 0x559844e25b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.422+0000 7f4cc6ffd640 0 client.25567 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:24:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.422+0000 7f4cf4bef740 1 --2- 172.21.15.1:0/3168045470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559844a524d0 0x559844e1ec10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.422+0000 7f4cf4bef740 1 --2- 172.21.15.1:0/3168045470 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559844a4a2f0 0x559844e1e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.422+0000 7f4cf4bef740 1 -- 172.21.15.1:0/3168045470 >> 172.21.15.1:0/3168045470 conn(0x559844a67300 msgr2=0x559844e13300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.422+0000 7f4cf4bef740 1 -- 172.21.15.1:0/3168045470 shutdown_connections 2024-05-04T03:24:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.422+0000 7f4cf4bef740 1 -- 172.21.15.1:0/3168045470 wait complete. 2024-05-04T03:24:54.431 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.422+0000 7f4cf4bef740 20 client.25567 trim_cache size 0 max 16384 2024-05-04T03:24:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.443+0000 7f4cf4bef740 1 Processor -- start 2024-05-04T03:24:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.443+0000 7f4cf4bef740 1 -- start start 2024-05-04T03:24:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.443+0000 7f4cf4bef740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559844a31d80 0x559844e10310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.443+0000 7f4cf4bef740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559844a4a2f0 0x559844e25a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.443+0000 7f4cf4bef740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559844a524d0 0x559844e25fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.443+0000 7f4cf4bef740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81ad950 con 0x559844a4a2f0 2024-05-04T03:24:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.443+0000 7f4cf4bef740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81ab220 con 0x559844a31d80 2024-05-04T03:24:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.443+0000 7f4cf4bef740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81a9090 con 0x559844a524d0 2024-05-04T03:24:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.444+0000 7f4cdeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559844a4a2f0 0x559844e25a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.444+0000 7f4cdeffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559844a4a2f0 0x559844e25a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42170/0 (socket says 172.21.15.1:42170) 2024-05-04T03:24:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.444+0000 7f4cdeffd640 1 -- 172.21.15.1:0/2109908586 learned_addr learned my addr 172.21.15.1:0/2109908586 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.444+0000 7f4ce7fff640 1 --2- 172.21.15.1:0/2109908586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559844a524d0 0x559844e25fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.444+0000 7f4ce77fe640 1 --2- 172.21.15.1:0/2109908586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559844a31d80 0x559844e10310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.444+0000 7f4cdeffd640 1 -- 172.21.15.1:0/2109908586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559844a524d0 msgr2=0x559844e25fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.444+0000 7f4cdeffd640 1 --2- 172.21.15.1:0/2109908586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559844a524d0 0x559844e25fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.444+0000 7f4cdeffd640 1 -- 172.21.15.1:0/2109908586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559844a31d80 msgr2=0x559844e10310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.444+0000 7f4cdeffd640 1 --2- 172.21.15.1:0/2109908586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559844a31d80 0x559844e10310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.444+0000 7f4cdeffd640 1 -- 172.21.15.1:0/2109908586 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559844e0a4d0 con 0x559844a4a2f0 2024-05-04T03:24:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.444+0000 7f4ce77fe640 1 --2- 172.21.15.1:0/2109908586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559844a31d80 0x559844e10310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:24:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.444+0000 7f4cdeffd640 1 --2- 172.21.15.1:0/2109908586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559844a4a2f0 0x559844e25a90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0031440 tx=0x7f4cd0031730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462ecd793cad9e90 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.445+0000 7f4cdd7fa640 1 -- 172.21.15.1:0/2109908586 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0035040 con 0x559844a4a2f0 2024-05-04T03:24:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.445+0000 7f4cdd7fa640 1 -- 172.21.15.1:0/2109908586 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cd0061070 con 0x559844a4a2f0 2024-05-04T03:24:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.445+0000 7f4cdd7fa640 1 -- 172.21.15.1:0/2109908586 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0031ba0 con 0x559844a4a2f0 2024-05-04T03:24:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.446+0000 7f4cf4bef740 1 -- 172.21.15.1:0/2109908586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559844a4a2f0 msgr2=0x559844e25a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.446+0000 7f4cf4bef740 1 --2- 172.21.15.1:0/2109908586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559844a4a2f0 0x559844e25a90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0031440 tx=0x7f4cd0031730 comp rx=0 tx=0).stop 2024-05-04T03:24:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.446+0000 7f4cf4bef740 1 -- 172.21.15.1:0/2109908586 shutdown_connections 2024-05-04T03:24:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.446+0000 7f4cf4bef740 1 --2- 172.21.15.1:0/2109908586 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559844a524d0 0x559844e25fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.446+0000 7f4cf4bef740 1 --2- 172.21.15.1:0/2109908586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559844a4a2f0 0x559844e25a90 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.446+0000 7f4cf4bef740 1 --2- 172.21.15.1:0/2109908586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559844a31d80 0x559844e10310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.446+0000 7f4cf4bef740 1 -- 172.21.15.1:0/2109908586 >> 172.21.15.1:0/2109908586 conn(0x559844a67300 msgr2=0x559844e0fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.446+0000 7f4cf4bef740 1 -- 172.21.15.1:0/2109908586 shutdown_connections 2024-05-04T03:24:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.446+0000 7f4cf4bef740 1 -- 172.21.15.1:0/2109908586 wait complete. 2024-05-04T03:24:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.446+0000 7f4cf4bef740 1 -- start start 2024-05-04T03:24:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.447+0000 7f4cf4bef740 1 Processor -- start 2024-05-04T03:24:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.447+0000 7f4cf4bef740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559844a31d80 0x559844e23510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.447+0000 7f4cf4bef740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559844a4a2f0 0x559844e23a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.447+0000 7f4cf4bef740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559844a524d0 0x559844e24980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.447+0000 7f4cf4bef740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81a96e0 con 0x559844a31d80 2024-05-04T03:24:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.447+0000 7f4cf4bef740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81aab50 con 0x559844a4a2f0 2024-05-04T03:24:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.447+0000 7f4cf4bef740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81a9db0 con 0x559844a524d0 2024-05-04T03:24:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.447+0000 7f4cf4bef740 10 client.0 _ll_register_callbacks cb 0x7f4cd0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:24:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.447+0000 7f4ce77fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559844a31d80 0x559844e23510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.447+0000 7f4cdeffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559844a4a2f0 0x559844e23a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.447+0000 7f4ce7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559844a524d0 0x559844e24980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.447+0000 7f4cdeffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559844a4a2f0 0x559844e23a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:48566/0 (socket says 172.21.15.1:48566) 2024-05-04T03:24:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.447+0000 7f4cdeffd640 1 -- 172.21.15.1:0/1416868928 learned_addr learned my addr 172.21.15.1:0/1416868928 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:24:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.448+0000 7f4cdeffd640 1 -- 172.21.15.1:0/1416868928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559844a524d0 msgr2=0x559844e24980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.448+0000 7f4cdeffd640 1 --2- 172.21.15.1:0/1416868928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559844a524d0 0x559844e24980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.448+0000 7f4cdeffd640 1 -- 172.21.15.1:0/1416868928 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559844a31d80 msgr2=0x559844e23510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:24:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.448+0000 7f4cdeffd640 1 --2- 172.21.15.1:0/1416868928 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559844a31d80 0x559844e23510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.448+0000 7f4cdeffd640 1 -- 172.21.15.1:0/1416868928 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559844e24f40 con 0x559844a4a2f0 2024-05-04T03:24:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.448+0000 7f4ce7fff640 1 --2- 172.21.15.1:0/1416868928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559844a524d0 0x559844e24980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:24:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.448+0000 7f4cdeffd640 1 --2- 172.21.15.1:0/1416868928 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559844a4a2f0 0x559844e23a50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00517a0 tx=0x7f4cd00517d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58553a4c257960d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:24:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.448+0000 7f4cc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-04T03:24:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.448+0000 7f4cc6ffd640 1 -- 172.21.15.1:0/1416868928 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd006a060 con 0x559844a4a2f0 2024-05-04T03:24:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.448+0000 7f4cc6ffd640 1 -- 172.21.15.1:0/1416868928 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f4cd0031b50 con 0x559844a4a2f0 2024-05-04T03:24:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.448+0000 7f4cf4bef740 1 -- 172.21.15.1:0/1416868928 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cd0061070 con 0x559844a4a2f0 2024-05-04T03:24:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.448+0000 7f4cc6ffd640 1 -- 172.21.15.1:0/1416868928 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4cd0035070 con 0x559844a4a2f0 2024-05-04T03:24:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.448+0000 7f4cf4bef740 10 client.25607 fetch_fsmap learned FSMap version 32 2024-05-04T03:24:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.448+0000 7f4cf4bef740 1 -- 172.21.15.1:0/1416868928 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559844e26170 con 0x559844a4a2f0 2024-05-04T03:24:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.449+0000 7f4cc6ffd640 1 -- 172.21.15.1:0/1416868928 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cd0061070 con 0x559844a4a2f0 2024-05-04T03:24:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.449+0000 7f4cc6ffd640 1 -- 172.21.15.1:0/1416868928 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f4cd0044070 con 0x559844a4a2f0 2024-05-04T03:24:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.449+0000 7f4cf4bef740 10 client.25607 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:24:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.449+0000 7f4cf4bef740 10 client.25607 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:24:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.449+0000 7f4cf4bef740 20 client.25607 populate_metadata read hostname 'smithi001' 2024-05-04T03:24:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.449+0000 7f4cf4bef740 1 --2- 172.21.15.1:0/1416868928 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x559844a731b0 0x559844e968e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:24:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.449+0000 7f4cf4bef740 4 client.25607 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:24:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.449+0000 7f4cf4bef740 1 -- 172.21.15.1:0/1416868928 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559844d81050 con 0x559844a731b0 2024-05-04T03:24:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.450+0000 7f4ce77fe640 1 --2- 172.21.15.1:0/1416868928 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x559844a731b0 0x559844e968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:24:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.450+0000 7f4ce77fe640 1 --2- 172.21.15.1:0/1416868928 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x559844a731b0 0x559844e968e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e841af35bfa8c656 server_cookie=ef188d182d4fd853 in_seq=0 out_seq=0 2024-05-04T03:24:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.450+0000 7f4cc6ffd640 10 client.25607 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:24:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.458+0000 7f4cc6ffd640 1 -- 172.21.15.1:0/1416868928 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559844d81050 con 0x559844a731b0 2024-05-04T03:24:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.458+0000 7f4cc6ffd640 10 client.25607 handle_command_reply: tid=0 2024-05-04T03:24:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.458+0000 7f4cf4bef740 1 client.25607 shutdown 2024-05-04T03:24:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.458+0000 7f4cf4bef740 1 -- 172.21.15.1:0/1416868928 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559844a4a2f0 msgr2=0x559844e23a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.458+0000 7f4cf4bef740 1 --2- 172.21.15.1:0/1416868928 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559844a4a2f0 0x559844e23a50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00517a0 tx=0x7f4cd00517d0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.458+0000 7f4cf4bef740 1 -- 172.21.15.1:0/1416868928 shutdown_connections 2024-05-04T03:24:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.458+0000 7f4cf4bef740 1 --2- 172.21.15.1:0/1416868928 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x559844a731b0 0x559844e968e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.458+0000 7f4cf4bef740 1 --2- 172.21.15.1:0/1416868928 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559844a4a2f0 0x559844e23a50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.461 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.458+0000 7f4cc6ffd640 0 client.25607 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:24:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.458+0000 7f4cf4bef740 1 --2- 172.21.15.1:0/1416868928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559844a524d0 0x559844e24980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.458+0000 7f4cf4bef740 1 --2- 172.21.15.1:0/1416868928 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559844a31d80 0x559844e23510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.458+0000 7f4cf4bef740 1 -- 172.21.15.1:0/1416868928 >> 172.21.15.1:0/1416868928 conn(0x559844a67300 msgr2=0x559844e30a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.458+0000 7f4cf4bef740 1 -- 172.21.15.1:0/1416868928 shutdown_connections 2024-05-04T03:24:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.458+0000 7f4cf4bef740 1 -- 172.21.15.1:0/1416868928 wait complete. 2024-05-04T03:24:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.459+0000 7f4cf4bef740 20 client.25607 trim_cache size 0 max 16384 2024-05-04T03:24:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:24:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:24:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:24:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:24:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.459+0000 7f4cedd38640 1 -- 172.21.15.1:0/4092302140 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4cd40b2920 msgr2=0x7f4cd40b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.459+0000 7f4cedd38640 1 --2- 172.21.15.1:0/4092302140 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4cd40b2920 0x7f4cd40b4e30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0004770 tx=0x7f4cd00314a0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.459+0000 7f4cedd38640 1 -- 172.21.15.1:0/4092302140 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ce8102b50 msgr2=0x7f4ce819a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:24:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.459+0000 7f4cedd38640 1 --2- 172.21.15.1:0/4092302140 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ce8102b50 0x7f4ce819a2d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4cd800e9f0 tx=0x7f4cd800eeb0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.459+0000 7f4cedd38640 1 -- 172.21.15.1:0/4092302140 shutdown_connections 2024-05-04T03:24:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.459+0000 7f4cedd38640 1 --2- 172.21.15.1:0/4092302140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ce8108a90 0x7f4ce81a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.459+0000 7f4cedd38640 1 --2- 172.21.15.1:0/4092302140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ce8103d90 0x7f4ce819a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.459+0000 7f4cedd38640 1 --2- 172.21.15.1:0/4092302140 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f4cd40b2920 0x7f4cd40b4e30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.459+0000 7f4cedd38640 1 --2- 172.21.15.1:0/4092302140 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4ce8102b50 0x7f4ce819a2d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:24:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.459+0000 7f4cedd38640 1 -- 172.21.15.1:0/4092302140 >> 172.21.15.1:0/4092302140 conn(0x7f4ce80fdd40 msgr2=0x7f4ce8109e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:24:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.459+0000 7f4cedd38640 1 -- 172.21.15.1:0/4092302140 shutdown_connections 2024-05-04T03:24:54.464 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:24:54.459+0000 7f4cedd38640 1 -- 172.21.15.1:0/4092302140 wait complete. 2024-05-04T03:24:54.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:54 smithi077 ceph-mon[33531]: pgmap v563: 97 pgs: 97 active+clean; 120 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 956 op/s 2024-05-04T03:24:54.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:54 smithi072 ceph-mon[33332]: pgmap v563: 97 pgs: 97 active+clean; 120 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 956 op/s 2024-05-04T03:24:54.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:54 smithi001 ceph-mon[29731]: pgmap v563: 97 pgs: 97 active+clean; 120 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 956 op/s 2024-05-04T03:24:56.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:56 smithi077 ceph-mon[33531]: pgmap v564: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 3.0 MiB/s wr, 1.15k op/s 2024-05-04T03:24:56.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:56 smithi072 ceph-mon[33332]: pgmap v564: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 3.0 MiB/s wr, 1.15k op/s 2024-05-04T03:24:56.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:56 smithi001 ceph-mon[29731]: pgmap v564: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 3.0 MiB/s wr, 1.15k op/s 2024-05-04T03:24:58.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:24:58 smithi077 ceph-mon[33531]: pgmap v565: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.2 MiB/s wr, 878 op/s 2024-05-04T03:24:58.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:24:58 smithi072 ceph-mon[33332]: pgmap v565: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.2 MiB/s wr, 878 op/s 2024-05-04T03:24:58.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:24:58 smithi001 ceph-mon[29731]: pgmap v565: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.2 MiB/s wr, 878 op/s 2024-05-04T03:25:00.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:00 smithi077 ceph-mon[33531]: pgmap v566: 97 pgs: 97 active+clean; 126 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.6 MiB/s wr, 1.03k op/s 2024-05-04T03:25:00.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:00 smithi072 ceph-mon[33332]: pgmap v566: 97 pgs: 97 active+clean; 126 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.6 MiB/s wr, 1.03k op/s 2024-05-04T03:25:00.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:00 smithi001 ceph-mon[29731]: pgmap v566: 97 pgs: 97 active+clean; 126 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.6 MiB/s wr, 1.03k op/s 2024-05-04T03:25:02.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:02 smithi077 ceph-mon[33531]: pgmap v567: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.4 MiB/s wr, 925 op/s 2024-05-04T03:25:02.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:02 smithi072 ceph-mon[33332]: pgmap v567: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.4 MiB/s wr, 925 op/s 2024-05-04T03:25:02.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:02 smithi001 ceph-mon[29731]: pgmap v567: 97 pgs: 97 active+clean; 133 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.4 MiB/s wr, 925 op/s 2024-05-04T03:25:04.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:04 smithi077 ceph-mon[33531]: pgmap v568: 97 pgs: 97 active+clean; 129 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.4 MiB/s wr, 929 op/s 2024-05-04T03:25:04.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:04 smithi072 ceph-mon[33332]: pgmap v568: 97 pgs: 97 active+clean; 129 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.4 MiB/s wr, 929 op/s 2024-05-04T03:25:04.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:04 smithi001 ceph-mon[29731]: pgmap v568: 97 pgs: 97 active+clean; 129 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.4 MiB/s wr, 929 op/s 2024-05-04T03:25:06.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:06 smithi077 ceph-mon[33531]: pgmap v569: 97 pgs: 97 active+clean; 129 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.8 MiB/s wr, 1.05k op/s 2024-05-04T03:25:06.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:06 smithi072 ceph-mon[33332]: pgmap v569: 97 pgs: 97 active+clean; 129 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.8 MiB/s wr, 1.05k op/s 2024-05-04T03:25:06.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:06 smithi001 ceph-mon[29731]: pgmap v569: 97 pgs: 97 active+clean; 129 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.8 MiB/s wr, 1.05k op/s 2024-05-04T03:25:07.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.584+0000 7f0fda82b640 1 -- 172.21.15.1:0/2955968731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fd4102b50 msgr2=0x7f0fd4103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:07.587 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.584+0000 7f0fda82b640 1 --2- 172.21.15.1:0/2955968731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fd4102b50 0x7f0fd4103850 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009900 tx=0x7f0fc402a410 comp rx=0 tx=0).stop 2024-05-04T03:25:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.586+0000 7f0fda82b640 1 -- 172.21.15.1:0/2955968731 shutdown_connections 2024-05-04T03:25:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.586+0000 7f0fda82b640 1 --2- 172.21.15.1:0/2955968731 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0fd4108a90 0x7f0fd410aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.586+0000 7f0fda82b640 1 --2- 172.21.15.1:0/2955968731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fd4103d90 0x7f0fd41041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.586+0000 7f0fda82b640 1 --2- 172.21.15.1:0/2955968731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fd4102b50 0x7f0fd4103850 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.586+0000 7f0fda82b640 1 -- 172.21.15.1:0/2955968731 >> 172.21.15.1:0/2955968731 conn(0x7f0fd40fdd40 msgr2=0x7f0fd41001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.586+0000 7f0fda82b640 1 -- 172.21.15.1:0/2955968731 shutdown_connections 2024-05-04T03:25:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.587+0000 7f0fda82b640 1 -- 172.21.15.1:0/2955968731 wait complete. 2024-05-04T03:25:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.588+0000 7f0fda82b640 1 Processor -- start 2024-05-04T03:25:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.588+0000 7f0fda82b640 1 -- start start 2024-05-04T03:25:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.588+0000 7f0fda82b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0fd4102b50 0x7f0fd406b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.588+0000 7f0fda82b640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fd4103d90 0x7f0fd41a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.588+0000 7f0fda82b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fd4108a90 0x7f0fd41a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.588+0000 7f0fda82b640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd410e6a0 con 0x7f0fd4103d90 2024-05-04T03:25:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.588+0000 7f0fda82b640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd410e520 con 0x7f0fd4102b50 2024-05-04T03:25:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.588+0000 7f0fda82b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd410e820 con 0x7f0fd4108a90 2024-05-04T03:25:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.588+0000 7f0fd37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fd4103d90 0x7f0fd41a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.588+0000 7f0fd3fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0fd4102b50 0x7f0fd406b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.588+0000 7f0fd37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fd4103d90 0x7f0fd41a1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60032/0 (socket says 172.21.15.1:60032) 2024-05-04T03:25:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.589+0000 7f0fd37fe640 1 -- 172.21.15.1:0/1781649689 learned_addr learned my addr 172.21.15.1:0/1781649689 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:25:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.589+0000 7f0fd8dc8640 1 --2- 172.21.15.1:0/1781649689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fd4108a90 0x7f0fd41a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.589+0000 7f0fd37fe640 1 -- 172.21.15.1:0/1781649689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fd4108a90 msgr2=0x7f0fd41a39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.589+0000 7f0fd37fe640 1 --2- 172.21.15.1:0/1781649689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fd4108a90 0x7f0fd41a39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.589+0000 7f0fd37fe640 1 -- 172.21.15.1:0/1781649689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0fd4102b50 msgr2=0x7f0fd406b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.589+0000 7f0fd37fe640 1 --2- 172.21.15.1:0/1781649689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0fd4102b50 0x7f0fd406b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.589+0000 7f0fd37fe640 1 -- 172.21.15.1:0/1781649689 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd41a4000 con 0x7f0fd4103d90 2024-05-04T03:25:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.589+0000 7f0fd3fff640 1 --2- 172.21.15.1:0/1781649689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0fd4102b50 0x7f0fd406b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:25:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.589+0000 7f0fd37fe640 1 --2- 172.21.15.1:0/1781649689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fd4103d90 0x7f0fd41a1470 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0fc800e9f0 tx=0x7f0fc800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.589+0000 7f0fd17fa640 1 -- 172.21.15.1:0/1781649689 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc800cd80 con 0x7f0fd4103d90 2024-05-04T03:25:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.589+0000 7f0fd17fa640 1 -- 172.21.15.1:0/1781649689 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fc80054d0 con 0x7f0fd4103d90 2024-05-04T03:25:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.589+0000 7f0fda82b640 1 -- 172.21.15.1:0/1781649689 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fd41a42f0 con 0x7f0fd4103d90 2024-05-04T03:25:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.589+0000 7f0fd17fa640 1 -- 172.21.15.1:0/1781649689 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc8010690 con 0x7f0fd4103d90 2024-05-04T03:25:07.593 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.590+0000 7f0fda82b640 1 -- 172.21.15.1:0/1781649689 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fd41a4840 con 0x7f0fd4103d90 2024-05-04T03:25:07.594 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.592+0000 7f0fd17fa640 1 -- 172.21.15.1:0/1781649689 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f0fc8010830 con 0x7f0fd4103d90 2024-05-04T03:25:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.593+0000 7f0fd17fa640 1 --2- 172.21.15.1:0/1781649689 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0fbc0b2920 0x7f0fbc0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.593+0000 7f0fe17c4740 1 Processor -- start 2024-05-04T03:25:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.593+0000 7f0fd17fa640 1 -- 172.21.15.1:0/1781649689 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f0fc80e2500 con 0x7f0fd4103d90 2024-05-04T03:25:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.593+0000 7f0fe17c4740 1 -- start start 2024-05-04T03:25:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.593+0000 7f0fd3fff640 1 --2- 172.21.15.1:0/1781649689 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0fbc0b2920 0x7f0fbc0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.593+0000 7f0fe17c4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645e402b1b0 0x5645e43cca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.593+0000 7f0fe17c4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5645e40022f0 0x5645e43ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.593+0000 7f0fe17c4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645e400a4d0 0x5645e43d3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.593+0000 7f0fe17c4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5645e43315d0 con 0x5645e402b1b0 2024-05-04T03:25:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.594+0000 7f0fe17c4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5645e42ea890 con 0x5645e40022f0 2024-05-04T03:25:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.594+0000 7f0fe17c4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5645e432d590 con 0x5645e400a4d0 2024-05-04T03:25:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.594+0000 7f0fd37fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5645e40022f0 0x5645e43ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:07.597 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.594+0000 7f0fd8dc8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645e400a4d0 0x5645e43d3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.594+0000 7f0fd37fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5645e40022f0 0x5645e43ccfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:49244/0 (socket says 172.21.15.1:49244) 2024-05-04T03:25:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.594+0000 7f0fd37fe640 1 -- 172.21.15.1:0/3135600519 learned_addr learned my addr 172.21.15.1:0/3135600519 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:25:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.594+0000 7f0fd8dc8640 1 -- 172.21.15.1:0/3135600519 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5645e40022f0 msgr2=0x5645e43ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.594+0000 7f0fd8dc8640 1 --2- 172.21.15.1:0/3135600519 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5645e40022f0 0x5645e43ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.594+0000 7f0fd8dc8640 1 -- 172.21.15.1:0/3135600519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645e402b1b0 msgr2=0x5645e43cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.594+0000 7f0fd8dc8640 1 --2- 172.21.15.1:0/3135600519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645e402b1b0 0x5645e43cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.594+0000 7f0fd8dc8640 1 -- 172.21.15.1:0/3135600519 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645e4043210 con 0x5645e400a4d0 2024-05-04T03:25:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.594+0000 7f0fd37fe640 1 --2- 172.21.15.1:0/3135600519 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5645e40022f0 0x5645e43ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:25:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.594+0000 7f0fd3fff640 1 --2- 172.21.15.1:0/1781649689 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0fbc0b2920 0x7f0fbc0b4e30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0fc402aa00 tx=0x7f0fc40314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.594+0000 7f0fd8dc8640 1 --2- 172.21.15.1:0/3135600519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645e400a4d0 0x5645e43d3ee0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc009f00 tx=0x7f0fcc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c41b6f68b9f4075a server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.595+0000 7f0fba7fc640 1 -- 172.21.15.1:0/3135600519 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc004d20 con 0x5645e400a4d0 2024-05-04T03:25:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.595+0000 7f0fba7fc640 1 -- 172.21.15.1:0/3135600519 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fcc004030 con 0x5645e400a4d0 2024-05-04T03:25:07.598 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.595+0000 7f0fba7fc640 1 -- 172.21.15.1:0/3135600519 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc005280 con 0x5645e400a4d0 2024-05-04T03:25:07.603 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.595+0000 7f0fe17c4740 1 -- 172.21.15.1:0/3135600519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645e400a4d0 msgr2=0x5645e43d3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:07.603 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.595+0000 7f0fe17c4740 1 --2- 172.21.15.1:0/3135600519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645e400a4d0 0x5645e43d3ee0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc009f00 tx=0x7f0fcc011810 comp rx=0 tx=0).stop 2024-05-04T03:25:07.603 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.596+0000 7f0fe17c4740 1 -- 172.21.15.1:0/3135600519 shutdown_connections 2024-05-04T03:25:07.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.596+0000 7f0fe17c4740 1 --2- 172.21.15.1:0/3135600519 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5645e40022f0 0x5645e43ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.596+0000 7f0fe17c4740 1 --2- 172.21.15.1:0/3135600519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645e400a4d0 0x5645e43d3ee0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.596+0000 7f0fe17c4740 1 --2- 172.21.15.1:0/3135600519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645e402b1b0 0x5645e43cca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.596+0000 7f0fe17c4740 1 -- 172.21.15.1:0/3135600519 >> 172.21.15.1:0/3135600519 conn(0x5645e401f300 msgr2=0x5645e43c83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:07.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.596+0000 7f0fe17c4740 1 -- 172.21.15.1:0/3135600519 shutdown_connections 2024-05-04T03:25:07.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.596+0000 7f0fe17c4740 1 -- 172.21.15.1:0/3135600519 wait complete. 2024-05-04T03:25:07.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.596+0000 7f0fe17c4740 1 -- start start 2024-05-04T03:25:07.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.596+0000 7f0fe17c4740 1 Processor -- start 2024-05-04T03:25:07.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.596+0000 7f0fe17c4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645e40022f0 0x5645e43d66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:07.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.597+0000 7f0fe17c4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645e400a4d0 0x5645e43d6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:07.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.597+0000 7f0fe17c4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5645e402b1b0 0x5645e43ddb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:07.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.597+0000 7f0fe17c4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x5645e43d4930 con 0x5645e400a4d0 2024-05-04T03:25:07.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.597+0000 7f0fe17c4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5645e42ee370 con 0x5645e402b1b0 2024-05-04T03:25:07.604 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.597+0000 7f0fe17c4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5645e43d4ab0 con 0x5645e40022f0 2024-05-04T03:25:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.597+0000 7f0fd37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645e400a4d0 0x5645e43d6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.597+0000 7f0fd8dc8640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5645e402b1b0 0x5645e43ddb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.597+0000 7f0fd3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645e40022f0 0x5645e43d66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.597+0000 7f0fd8dc8640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5645e402b1b0 0x5645e43ddb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:49254/0 (socket says 172.21.15.1:49254) 2024-05-04T03:25:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.597+0000 7f0fd8dc8640 1 -- 172.21.15.1:0/309850675 learned_addr learned my addr 172.21.15.1:0/309850675 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:25:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.597+0000 7f0fe17c4740 10 client.0 _ll_register_callbacks cb 0x5645e4042a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:25:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.597+0000 7f0fd3fff640 1 -- 172.21.15.1:0/309850675 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5645e402b1b0 msgr2=0x5645e43ddb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.597+0000 7f0fd3fff640 1 --2- 172.21.15.1:0/309850675 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5645e402b1b0 0x5645e43ddb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.597+0000 7f0fd3fff640 1 -- 172.21.15.1:0/309850675 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645e400a4d0 msgr2=0x5645e43d6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.597+0000 7f0fd3fff640 1 --2- 172.21.15.1:0/309850675 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645e400a4d0 0x5645e43d6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.597+0000 7f0fd3fff640 1 -- 172.21.15.1:0/309850675 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645e410f620 con 0x5645e40022f0 2024-05-04T03:25:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.597+0000 7f0fd37fe640 1 --2- 172.21.15.1:0/309850675 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645e400a4d0 0x5645e43d6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:25:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.597+0000 7f0fd8dc8640 1 --2- 172.21.15.1:0/309850675 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5645e402b1b0 0x5645e43ddb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:25:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.597+0000 7f0fd3fff640 1 --2- 172.21.15.1:0/309850675 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645e40022f0 0x5645e43d66a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009230 tx=0x7f0fc4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a55ed59754dc1d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.597+0000 7f0fb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-04T03:25:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.598+0000 7f0fb37fe640 1 -- 172.21.15.1:0/309850675 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc404d070 con 0x5645e40022f0 2024-05-04T03:25:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.598+0000 7f0fb37fe640 1 -- 172.21.15.1:0/309850675 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f0fc40317a0 con 0x5645e40022f0 2024-05-04T03:25:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.598+0000 7f0fe17c4740 1 -- 172.21.15.1:0/309850675 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fcc004030 con 0x5645e40022f0 2024-05-04T03:25:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.598+0000 7f0fb37fe640 1 -- 172.21.15.1:0/309850675 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0fc4059050 con 0x5645e40022f0 2024-05-04T03:25:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.598+0000 7f0fe17c4740 10 client.25585 fetch_fsmap learned FSMap version 32 2024-05-04T03:25:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.598+0000 7f0fe17c4740 1 -- 172.21.15.1:0/309850675 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5645e43e0ae0 con 0x5645e40022f0 2024-05-04T03:25:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.598+0000 7f0fb37fe640 1 -- 172.21.15.1:0/309850675 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fcc004030 con 0x5645e40022f0 2024-05-04T03:25:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.599+0000 7f0fb37fe640 1 -- 172.21.15.1:0/309850675 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f0fc4049950 con 0x5645e40022f0 2024-05-04T03:25:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.599+0000 7f0fe17c4740 10 client.25585 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:25:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.599+0000 7f0fe17c4740 10 client.25585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:25:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.599+0000 7f0fe17c4740 20 client.25585 populate_metadata read hostname 'smithi001' 2024-05-04T03:25:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.599+0000 7f0fe17c4740 1 --2- 172.21.15.1:0/309850675 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5645e3fe9d80 0x5645e43e4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.599+0000 7f0fe17c4740 4 client.25585 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:25:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.599+0000 7f0fe17c4740 1 -- 172.21.15.1:0/309850675 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645e433f1e0 con 0x5645e3fe9d80 2024-05-04T03:25:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.599+0000 7f0fd37fe640 1 --2- 172.21.15.1:0/309850675 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5645e3fe9d80 0x5645e43e4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.599+0000 7f0fb37fe640 10 client.25585 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:25:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.599+0000 7f0fd37fe640 1 --2- 172.21.15.1:0/309850675 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5645e3fe9d80 0x5645e43e4220 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32e4bac7c1b30d51 server_cookie=4f5f71ce48d4e935 in_seq=0 out_seq=0 2024-05-04T03:25:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.600+0000 7f0fb37fe640 1 -- 172.21.15.1:0/309850675 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5645e433f1e0 con 0x5645e3fe9d80 2024-05-04T03:25:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.600+0000 7f0fb37fe640 10 client.25585 handle_command_reply: tid=0 2024-05-04T03:25:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.600+0000 7f0fe17c4740 1 client.25585 shutdown 2024-05-04T03:25:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.600+0000 7f0fe17c4740 1 -- 172.21.15.1:0/309850675 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645e40022f0 msgr2=0x5645e43d66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.600+0000 7f0fe17c4740 1 --2- 172.21.15.1:0/309850675 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645e40022f0 0x5645e43d66a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009230 tx=0x7f0fc4009260 comp rx=0 tx=0).stop 2024-05-04T03:25:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.600+0000 7f0fe17c4740 1 -- 172.21.15.1:0/309850675 shutdown_connections 2024-05-04T03:25:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.600+0000 7f0fe17c4740 1 --2- 172.21.15.1:0/309850675 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5645e3fe9d80 0x5645e43e4220 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.600+0000 7f0fe17c4740 1 --2- 172.21.15.1:0/309850675 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5645e402b1b0 0x5645e43ddb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.600+0000 7f0fe17c4740 1 --2- 172.21.15.1:0/309850675 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645e400a4d0 0x5645e43d6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.601+0000 7f0fe17c4740 1 --2- 172.21.15.1:0/309850675 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645e40022f0 0x5645e43d66a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.601+0000 7f0fe17c4740 1 -- 172.21.15.1:0/309850675 >> 172.21.15.1:0/309850675 conn(0x5645e401f300 msgr2=0x5645e43cb300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.601+0000 7f0fb37fe640 0 client.25585 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:25:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.601+0000 7f0fe17c4740 1 -- 172.21.15.1:0/309850675 shutdown_connections 2024-05-04T03:25:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.601+0000 7f0fe17c4740 1 -- 172.21.15.1:0/309850675 wait complete. 2024-05-04T03:25:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.601+0000 7f0fe17c4740 20 client.25585 trim_cache size 0 max 16384 2024-05-04T03:25:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.619+0000 7f0fe17c4740 1 Processor -- start 2024-05-04T03:25:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.619+0000 7f0fe17c4740 1 -- start start 2024-05-04T03:25:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.619+0000 7f0fe17c4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5645e40022f0 0x5645e43c8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.619+0000 7f0fe17c4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645e400a4d0 0x5645e43ddad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.619+0000 7f0fe17c4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645e402b1b0 0x5645e43d6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.620+0000 7f0fe17c4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd41afbf0 con 0x5645e400a4d0 2024-05-04T03:25:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.620+0000 7f0fe17c4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd41ad4c0 con 0x5645e40022f0 2024-05-04T03:25:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.620+0000 7f0fe17c4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd41ab330 con 0x5645e402b1b0 2024-05-04T03:25:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.620+0000 7f0fd37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645e400a4d0 0x5645e43ddad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.620+0000 7f0fd3fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5645e40022f0 0x5645e43c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.620+0000 7f0fd3fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5645e40022f0 0x5645e43c8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:49260/0 (socket says 172.21.15.1:49260) 2024-05-04T03:25:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.620+0000 7f0fd37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645e400a4d0 0x5645e43ddad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60062/0 (socket says 172.21.15.1:60062) 2024-05-04T03:25:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.620+0000 7f0fd3fff640 1 -- 172.21.15.1:0/773347971 learned_addr learned my addr 172.21.15.1:0/773347971 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:25:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.620+0000 7f0fd8dc8640 1 --2- 172.21.15.1:0/773347971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645e402b1b0 0x5645e43d6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.620+0000 7f0fd3fff640 1 -- 172.21.15.1:0/773347971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645e402b1b0 msgr2=0x5645e43d6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.620+0000 7f0fd3fff640 1 --2- 172.21.15.1:0/773347971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645e402b1b0 0x5645e43d6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.620+0000 7f0fd3fff640 1 -- 172.21.15.1:0/773347971 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645e400a4d0 msgr2=0x5645e43ddad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.620+0000 7f0fd3fff640 1 --2- 172.21.15.1:0/773347971 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645e400a4d0 0x5645e43ddad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.620+0000 7f0fd3fff640 1 -- 172.21.15.1:0/773347971 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645e43c8d00 con 0x5645e40022f0 2024-05-04T03:25:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.620+0000 7f0fd3fff640 1 --2- 172.21.15.1:0/773347971 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5645e40022f0 0x5645e43c8470 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4000c00 tx=0x7f0fc40098a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59bad70b8c0e6e76 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.621+0000 7f0fb9ffb640 1 -- 172.21.15.1:0/773347971 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc404d040 con 0x5645e40022f0 2024-05-04T03:25:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.621+0000 7f0fb9ffb640 1 -- 172.21.15.1:0/773347971 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fc4062070 con 0x5645e40022f0 2024-05-04T03:25:07.624 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.621+0000 7f0fb9ffb640 1 -- 172.21.15.1:0/773347971 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4035050 con 0x5645e40022f0 2024-05-04T03:25:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.622+0000 7f0fe17c4740 1 -- 172.21.15.1:0/773347971 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5645e40022f0 msgr2=0x5645e43c8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.622+0000 7f0fe17c4740 1 --2- 172.21.15.1:0/773347971 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5645e40022f0 0x5645e43c8470 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4000c00 tx=0x7f0fc40098a0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.622+0000 7f0fe17c4740 1 -- 172.21.15.1:0/773347971 shutdown_connections 2024-05-04T03:25:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.622+0000 7f0fe17c4740 1 --2- 172.21.15.1:0/773347971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645e402b1b0 0x5645e43d6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.622+0000 7f0fe17c4740 1 --2- 172.21.15.1:0/773347971 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645e400a4d0 0x5645e43ddad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.622+0000 7f0fe17c4740 1 --2- 172.21.15.1:0/773347971 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5645e40022f0 0x5645e43c8470 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.622+0000 7f0fe17c4740 1 -- 172.21.15.1:0/773347971 >> 172.21.15.1:0/773347971 conn(0x5645e401f300 msgr2=0x5645e43c7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.622+0000 7f0fe17c4740 1 -- 172.21.15.1:0/773347971 shutdown_connections 2024-05-04T03:25:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.622+0000 7f0fe17c4740 1 -- 172.21.15.1:0/773347971 wait complete. 2024-05-04T03:25:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.622+0000 7f0fe17c4740 1 -- start start 2024-05-04T03:25:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.622+0000 7f0fe17c4740 1 Processor -- start 2024-05-04T03:25:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.623+0000 7f0fe17c4740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5645e40022f0 0x5645e43dd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.623+0000 7f0fe17c4740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645e400a4d0 0x5645e43da470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.623+0000 7f0fe17c4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645e402b1b0 0x5645e43da9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.623+0000 7f0fe17c4740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd41ab980 con 0x5645e400a4d0 2024-05-04T03:25:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.623+0000 7f0fe17c4740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd41acdf0 con 0x5645e40022f0 2024-05-04T03:25:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.623+0000 7f0fe17c4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd41ac050 con 0x5645e402b1b0 2024-05-04T03:25:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.623+0000 7f0fe17c4740 10 client.0 _ll_register_callbacks cb 0x7f0fc8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:25:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.623+0000 7f0fd37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645e400a4d0 0x5645e43da470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.623+0000 7f0fd3fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5645e40022f0 0x5645e43dd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.623+0000 7f0fd37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645e400a4d0 0x5645e43da470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60064/0 (socket says 172.21.15.1:60064) 2024-05-04T03:25:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.623+0000 7f0fd3fff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5645e40022f0 0x5645e43dd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:49270/0 (socket says 172.21.15.1:49270) 2024-05-04T03:25:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.623+0000 7f0fd37fe640 1 -- 172.21.15.1:0/992675144 learned_addr learned my addr 172.21.15.1:0/992675144 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:25:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.623+0000 7f0fd8dc8640 1 --2- 172.21.15.1:0/992675144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645e402b1b0 0x5645e43da9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.623+0000 7f0fd3fff640 1 -- 172.21.15.1:0/992675144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645e402b1b0 msgr2=0x5645e43da9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.623+0000 7f0fd3fff640 1 --2- 172.21.15.1:0/992675144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645e402b1b0 0x5645e43da9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.623+0000 7f0fd3fff640 1 -- 172.21.15.1:0/992675144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645e400a4d0 msgr2=0x5645e43da470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.623+0000 7f0fd3fff640 1 --2- 172.21.15.1:0/992675144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645e400a4d0 0x5645e43da470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.623+0000 7f0fd3fff640 1 -- 172.21.15.1:0/992675144 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645e444a590 con 0x5645e40022f0 2024-05-04T03:25:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.623+0000 7f0fd37fe640 1 --2- 172.21.15.1:0/992675144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645e400a4d0 0x5645e43da470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:25:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.623+0000 7f0fd3fff640 1 --2- 172.21.15.1:0/992675144 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5645e40022f0 0x5645e43dd8f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x5645e43c6810 tx=0x7f0fc4049680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c793935fb515d54f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.623+0000 7f0fb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-04T03:25:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.624+0000 7f0fb37fe640 1 -- 172.21.15.1:0/992675144 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc406b060 con 0x5645e40022f0 2024-05-04T03:25:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.624+0000 7f0fb37fe640 1 -- 172.21.15.1:0/992675144 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f0fc4005050 con 0x5645e40022f0 2024-05-04T03:25:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.624+0000 7f0fe17c4740 1 -- 172.21.15.1:0/992675144 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fc4062070 con 0x5645e40022f0 2024-05-04T03:25:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.624+0000 7f0fb37fe640 1 -- 172.21.15.1:0/992675144 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0fc4035070 con 0x5645e40022f0 2024-05-04T03:25:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.624+0000 7f0fe17c4740 10 client.25625 fetch_fsmap learned FSMap version 32 2024-05-04T03:25:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.624+0000 7f0fe17c4740 1 -- 172.21.15.1:0/992675144 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5645e444b6d0 con 0x5645e40022f0 2024-05-04T03:25:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.624+0000 7f0fb37fe640 1 -- 172.21.15.1:0/992675144 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fc4062070 con 0x5645e40022f0 2024-05-04T03:25:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.624+0000 7f0fb37fe640 1 -- 172.21.15.1:0/992675144 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f0fc4004c80 con 0x5645e40022f0 2024-05-04T03:25:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.624+0000 7f0fe17c4740 10 client.25625 fetch_fsmap finished waiting for FSMap version 32 2024-05-04T03:25:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.624+0000 7f0fe17c4740 10 client.25625 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:25:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.624+0000 7f0fe17c4740 20 client.25625 populate_metadata read hostname 'smithi001' 2024-05-04T03:25:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.624+0000 7f0fe17c4740 1 --2- 172.21.15.1:0/992675144 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5645e3fbf1c0 0x5645e444e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.624+0000 7f0fe17c4740 4 client.25625 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:25:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.624+0000 7f0fe17c4740 1 -- 172.21.15.1:0/992675144 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5645e4339050 con 0x5645e3fbf1c0 2024-05-04T03:25:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.625+0000 7f0fd37fe640 1 --2- 172.21.15.1:0/992675144 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5645e3fbf1c0 0x5645e444e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.625+0000 7f0fb37fe640 10 client.25625 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:25:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.625+0000 7f0fd37fe640 1 --2- 172.21.15.1:0/992675144 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5645e3fbf1c0 0x5645e444e010 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e080cace4d05c1f7 server_cookie=70648ab9700ec1a7 in_seq=0 out_seq=0 2024-05-04T03:25:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.633+0000 7f0fb37fe640 1 -- 172.21.15.1:0/992675144 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5645e4339050 con 0x5645e3fbf1c0 2024-05-04T03:25:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.633+0000 7f0fb37fe640 10 client.25625 handle_command_reply: tid=0 2024-05-04T03:25:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.633+0000 7f0fe17c4740 1 client.25625 shutdown 2024-05-04T03:25:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.633+0000 7f0fe17c4740 1 -- 172.21.15.1:0/992675144 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5645e40022f0 msgr2=0x5645e43dd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.633+0000 7f0fe17c4740 1 --2- 172.21.15.1:0/992675144 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5645e40022f0 0x5645e43dd8f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x5645e43c6810 tx=0x7f0fc4049680 comp rx=0 tx=0).stop 2024-05-04T03:25:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.633+0000 7f0fe17c4740 1 -- 172.21.15.1:0/992675144 shutdown_connections 2024-05-04T03:25:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.633+0000 7f0fe17c4740 1 --2- 172.21.15.1:0/992675144 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x5645e3fbf1c0 0x5645e444e010 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.633+0000 7f0fe17c4740 1 --2- 172.21.15.1:0/992675144 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5645e402b1b0 0x5645e43da9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.633+0000 7f0fe17c4740 1 --2- 172.21.15.1:0/992675144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5645e400a4d0 0x5645e43da470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.633+0000 7f0fe17c4740 1 --2- 172.21.15.1:0/992675144 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5645e40022f0 0x5645e43dd8f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.633+0000 7f0fe17c4740 1 -- 172.21.15.1:0/992675144 >> 172.21.15.1:0/992675144 conn(0x5645e401f300 msgr2=0x5645e43d6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:07.636 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.633+0000 7f0fb37fe640 0 client.25625 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:25:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.633+0000 7f0fe17c4740 1 -- 172.21.15.1:0/992675144 shutdown_connections 2024-05-04T03:25:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.633+0000 7f0fe17c4740 1 -- 172.21.15.1:0/992675144 wait complete. 2024-05-04T03:25:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.634+0000 7f0fe17c4740 20 client.25625 trim_cache size 0 max 16384 2024-05-04T03:25:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:25:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:25:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:25:07.637 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:25:07.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.634+0000 7f0fda82b640 1 -- 172.21.15.1:0/1781649689 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0fbc0b2920 msgr2=0x7f0fbc0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:07.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.634+0000 7f0fda82b640 1 --2- 172.21.15.1:0/1781649689 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0fbc0b2920 0x7f0fbc0b4e30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0fc402aa00 tx=0x7f0fc40314a0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.634+0000 7f0fda82b640 1 -- 172.21.15.1:0/1781649689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fd4103d90 msgr2=0x7f0fd41a1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:07.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.634+0000 7f0fda82b640 1 --2- 172.21.15.1:0/1781649689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fd4103d90 0x7f0fd41a1470 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0fc800e9f0 tx=0x7f0fc800edf0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.634+0000 7f0fda82b640 1 -- 172.21.15.1:0/1781649689 shutdown_connections 2024-05-04T03:25:07.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.634+0000 7f0fda82b640 1 --2- 172.21.15.1:0/1781649689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fd4108a90 0x7f0fd41a39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.634+0000 7f0fda82b640 1 --2- 172.21.15.1:0/1781649689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fd4103d90 0x7f0fd41a1470 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.638 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.634+0000 7f0fda82b640 1 --2- 172.21.15.1:0/1781649689 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0fbc0b2920 0x7f0fbc0b4e30 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.634+0000 7f0fda82b640 1 --2- 172.21.15.1:0/1781649689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0fd4102b50 0x7f0fd406b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:07.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.634+0000 7f0fda82b640 1 -- 172.21.15.1:0/1781649689 >> 172.21.15.1:0/1781649689 conn(0x7f0fd40fdd40 msgr2=0x7f0fd4104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:07.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.634+0000 7f0fda82b640 1 -- 172.21.15.1:0/1781649689 shutdown_connections 2024-05-04T03:25:07.639 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:07.635+0000 7f0fda82b640 1 -- 172.21.15.1:0/1781649689 wait complete. 2024-05-04T03:25:08.516 INFO:tasks.workunit.client.0.smithi001.stdout:All 50000 operations completed A-OK! 2024-05-04T03:25:08.521 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-04T03:25:08.521 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-04T03:25:08.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:08 smithi077 ceph-mon[33531]: pgmap v570: 97 pgs: 97 active+clean; 129 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.1 MiB/s wr, 809 op/s 2024-05-04T03:25:08.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:08 smithi072 ceph-mon[33332]: pgmap v570: 97 pgs: 97 active+clean; 129 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.1 MiB/s wr, 809 op/s 2024-05-04T03:25:08.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:08 smithi001 ceph-mon[29731]: pgmap v570: 97 pgs: 97 active+clean; 129 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.1 MiB/s wr, 809 op/s 2024-05-04T03:25:10.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:10 smithi077 ceph-mon[33531]: pgmap v571: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.6 MiB/s wr, 1.00k op/s 2024-05-04T03:25:10.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:10 smithi072 ceph-mon[33332]: pgmap v571: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.6 MiB/s wr, 1.00k op/s 2024-05-04T03:25:10.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:10 smithi001 ceph-mon[29731]: pgmap v571: 97 pgs: 97 active+clean; 121 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.6 MiB/s wr, 1.00k op/s 2024-05-04T03:25:12.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:12 smithi077 ceph-mon[33531]: pgmap v572: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.3 MiB/s wr, 859 op/s 2024-05-04T03:25:12.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:12 smithi072 ceph-mon[33332]: pgmap v572: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.3 MiB/s wr, 859 op/s 2024-05-04T03:25:12.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:12 smithi001 ceph-mon[29731]: pgmap v572: 97 pgs: 97 active+clean; 118 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.3 MiB/s wr, 859 op/s 2024-05-04T03:25:13.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:13 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:25:13.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:13 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:25:13.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:13 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:25:14.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:14 smithi072 ceph-mon[33332]: pgmap v573: 97 pgs: 97 active+clean; 116 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.1 MiB/s wr, 814 op/s 2024-05-04T03:25:14.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:14 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:25:14.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:14 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:25:14.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:14 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:25:14.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:14 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:25:14.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:14 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:25:14.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:14 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:25:14.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:14 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:25:14.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:14 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:25:14.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:14 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:25:14.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:14 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:25:14.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:14 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:25:14.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:14 smithi001 ceph-mon[29731]: pgmap v573: 97 pgs: 97 active+clean; 116 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.1 MiB/s wr, 814 op/s 2024-05-04T03:25:14.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:25:14.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:25:14.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:25:14.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:25:14.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:25:14.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:25:14.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:25:14.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:25:14.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:25:14.881 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:25:14.881 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:25:14.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:14 smithi077 ceph-mon[33531]: pgmap v573: 97 pgs: 97 active+clean; 116 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.1 MiB/s wr, 814 op/s 2024-05-04T03:25:14.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:14 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:25:14.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:14 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:25:14.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:14 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:25:14.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:14 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:25:14.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:14 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:25:14.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:14 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:25:14.930 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:14 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:25:14.930 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:14 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-04T03:25:14.930 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:14 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:25:14.930 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:14 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:25:14.930 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:14 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:25:15.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:15 smithi072 ceph-mon[33332]: Adjusting osd_memory_target on smithi072 to 176.1M 2024-05-04T03:25:15.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:15 smithi072 ceph-mon[33332]: Adjusting osd_memory_target on smithi077 to 176.2M 2024-05-04T03:25:15.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:15 smithi072 ceph-mon[33332]: Unable to set osd_memory_target on smithi072 to 184725811: error parsing value: Value '184725811' is below minimum 939524096 2024-05-04T03:25:15.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:15 smithi072 ceph-mon[33332]: Unable to set osd_memory_target on smithi077 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-05-04T03:25:15.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:15 smithi001 ceph-mon[29731]: Adjusting osd_memory_target on smithi072 to 176.1M 2024-05-04T03:25:15.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:15 smithi001 ceph-mon[29731]: Adjusting osd_memory_target on smithi077 to 176.2M 2024-05-04T03:25:15.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:15 smithi001 ceph-mon[29731]: Unable to set osd_memory_target on smithi072 to 184725811: error parsing value: Value '184725811' is below minimum 939524096 2024-05-04T03:25:15.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:15 smithi001 ceph-mon[29731]: Unable to set osd_memory_target on smithi077 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-05-04T03:25:15.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:15 smithi077 ceph-mon[33531]: Adjusting osd_memory_target on smithi072 to 176.1M 2024-05-04T03:25:15.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:15 smithi077 ceph-mon[33531]: Adjusting osd_memory_target on smithi077 to 176.2M 2024-05-04T03:25:15.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:15 smithi077 ceph-mon[33531]: Unable to set osd_memory_target on smithi072 to 184725811: error parsing value: Value '184725811' is below minimum 939524096 2024-05-04T03:25:15.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:15 smithi077 ceph-mon[33531]: Unable to set osd_memory_target on smithi077 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-05-04T03:25:16.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:16 smithi072 ceph-mon[33332]: pgmap v574: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.4 MiB/s wr, 745 op/s 2024-05-04T03:25:16.829 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:16 smithi077 ceph-mon[33531]: pgmap v574: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.4 MiB/s wr, 745 op/s 2024-05-04T03:25:16.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:16 smithi001 ceph-mon[29731]: pgmap v574: 97 pgs: 97 active+clean; 122 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.4 MiB/s wr, 745 op/s 2024-05-04T03:25:18.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:18 smithi072 ceph-mon[33332]: pgmap v575: 97 pgs: 97 active+clean; 121 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.7 MiB/s wr, 550 op/s 2024-05-04T03:25:18.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:18 smithi001 ceph-mon[29731]: pgmap v575: 97 pgs: 97 active+clean; 121 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.7 MiB/s wr, 550 op/s 2024-05-04T03:25:18.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:18 smithi077 ceph-mon[33531]: pgmap v575: 97 pgs: 97 active+clean; 121 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.7 MiB/s wr, 550 op/s 2024-05-04T03:25:20.243 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-05-04T03:25:20.243 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-04T03:25:20.543 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:20 smithi001 ceph-mon[29731]: pgmap v576: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.7 MiB/s wr, 611 op/s 2024-05-04T03:25:20.544 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:20 smithi001 ceph-mon[29731]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:25:20.545 DEBUG:teuthology.parallel:result is None 2024-05-04T03:25:20.545 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-04T03:25:20.586 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-04T03:25:20.587 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-05-04T03:25:20.598 INFO:tasks.quiescer:joining Quiescers 2024-05-04T03:25:20.599 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-05-04T03:25:20.599 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:25:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.737+0000 7efdc9c15640 1 -- 172.21.15.1:0/2051906593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efdc4102b50 msgr2=0x7efdc4103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.737+0000 7efdc9c15640 1 --2- 172.21.15.1:0/2051906593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efdc4102b50 0x7efdc4103850 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7efdb4009900 tx=0x7efdb402a410 comp rx=0 tx=0).stop 2024-05-04T03:25:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.737+0000 7efdc9c15640 1 -- 172.21.15.1:0/2051906593 shutdown_connections 2024-05-04T03:25:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.737+0000 7efdc9c15640 1 --2- 172.21.15.1:0/2051906593 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efdc4108a90 0x7efdc410aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.737+0000 7efdc9c15640 1 --2- 172.21.15.1:0/2051906593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdc4103d90 0x7efdc41041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.737+0000 7efdc9c15640 1 --2- 172.21.15.1:0/2051906593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efdc4102b50 0x7efdc4103850 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.737+0000 7efdc9c15640 1 -- 172.21.15.1:0/2051906593 >> 172.21.15.1:0/2051906593 conn(0x7efdc40fdd40 msgr2=0x7efdc41001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.737+0000 7efdc9c15640 1 -- 172.21.15.1:0/2051906593 shutdown_connections 2024-05-04T03:25:20.740 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.737+0000 7efdc9c15640 1 -- 172.21.15.1:0/2051906593 wait complete. 2024-05-04T03:25:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.738+0000 7efdc9c15640 1 Processor -- start 2024-05-04T03:25:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.738+0000 7efdc9c15640 1 -- start start 2024-05-04T03:25:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.739+0000 7efdc9c15640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdc4102b50 0x7efdc406b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.739+0000 7efdc9c15640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efdc4103d90 0x7efdc419f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.739+0000 7efdc9c15640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efdc4108a90 0x7efdc41a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.739+0000 7efdc9c15640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efdc410e650 con 0x7efdc4103d90 2024-05-04T03:25:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.739+0000 7efdc9c15640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7efdc410e4d0 con 0x7efdc4108a90 2024-05-04T03:25:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.739+0000 7efdc9c15640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efdc410e7d0 con 0x7efdc4102b50 2024-05-04T03:25:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.739+0000 7efdc2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efdc4103d90 0x7efdc419f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.739+0000 7efdc2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efdc4103d90 0x7efdc419f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38502/0 (socket says 172.21.15.1:38502) 2024-05-04T03:25:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.739+0000 7efdc2ffd640 1 -- 172.21.15.1:0/1732304776 learned_addr learned my addr 172.21.15.1:0/1732304776 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:25:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.739+0000 7efdc3fff640 1 --2- 172.21.15.1:0/1732304776 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efdc4108a90 0x7efdc41a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.739+0000 7efdc37fe640 1 --2- 172.21.15.1:0/1732304776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdc4102b50 0x7efdc406b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.739+0000 7efdc3fff640 1 -- 172.21.15.1:0/1732304776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdc4102b50 msgr2=0x7efdc406b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.739+0000 7efdc3fff640 1 --2- 172.21.15.1:0/1732304776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdc4102b50 0x7efdc406b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.739+0000 7efdc3fff640 1 -- 172.21.15.1:0/1732304776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efdc4103d90 msgr2=0x7efdc419f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.739+0000 7efdc3fff640 1 --2- 172.21.15.1:0/1732304776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efdc4103d90 0x7efdc419f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.739+0000 7efdc3fff640 1 -- 172.21.15.1:0/1732304776 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc41a1e20 con 0x7efdc4108a90 2024-05-04T03:25:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.740+0000 7efdc3fff640 1 --2- 172.21.15.1:0/1732304776 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efdc4108a90 0x7efdc41a1600 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efdbc00ec50 tx=0x7efdbc00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.740+0000 7efdc0ff9640 1 -- 172.21.15.1:0/1732304776 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc003b70 con 0x7efdc4108a90 2024-05-04T03:25:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.740+0000 7efdc0ff9640 1 -- 172.21.15.1:0/1732304776 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdbc003d10 con 0x7efdc4108a90 2024-05-04T03:25:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.740+0000 7efdc9c15640 1 -- 172.21.15.1:0/1732304776 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc41a2110 con 0x7efdc4108a90 2024-05-04T03:25:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.740+0000 7efdc0ff9640 1 -- 172.21.15.1:0/1732304776 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc010830 con 0x7efdc4108a90 2024-05-04T03:25:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.740+0000 7efdc9c15640 1 -- 172.21.15.1:0/1732304776 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc41a2610 con 0x7efdc4108a90 2024-05-04T03:25:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.742+0000 7efdd0ba3740 1 Processor -- start 2024-05-04T03:25:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.743+0000 7efdc0ff9640 1 -- 172.21.15.1:0/1732304776 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7efdbc0109d0 con 0x7efdc4108a90 2024-05-04T03:25:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.744+0000 7efdd0ba3740 1 -- start start 2024-05-04T03:25:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.744+0000 7efdd0ba3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56253f7a61b0 0x56253fb4be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.744+0000 7efdd0ba3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56253f77d2f0 0x56253fb4c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.744+0000 7efdd0ba3740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56253f7854d0 0x56253fb532f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:20.748 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.744+0000 7efdd0ba3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56253faac5d0 con 0x56253f77d2f0 2024-05-04T03:25:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.744+0000 7efdd0ba3740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x56253fa65890 con 0x56253f7854d0 2024-05-04T03:25:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.744+0000 7efdd0ba3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56253faa8590 con 0x56253f7a61b0 2024-05-04T03:25:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.744+0000 7efdc37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56253f77d2f0 0x56253fb4c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.744+0000 7efdc37fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56253f77d2f0 0x56253fb4c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38508/0 (socket says 172.21.15.1:38508) 2024-05-04T03:25:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.744+0000 7efdc37fe640 1 -- 172.21.15.1:0/4200925897 learned_addr learned my addr 172.21.15.1:0/4200925897 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:25:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.744+0000 7efdc2ffd640 1 --2- 172.21.15.1:0/4200925897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56253f7a61b0 0x56253fb4be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.744+0000 7efdc2ffd640 1 --2- 172.21.15.1:0/4200925897 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56253f7854d0 0x56253fb532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.744+0000 7efdc0ff9640 1 --2- 172.21.15.1:0/1732304776 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7efdac0b28a0 0x7efdac0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.744+0000 7efdc37fe640 1 -- 172.21.15.1:0/4200925897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56253f7a61b0 msgr2=0x56253fb4be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.744+0000 7efdc37fe640 1 --2- 172.21.15.1:0/4200925897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56253f7a61b0 0x56253fb4be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.744+0000 7efdc37fe640 1 -- 172.21.15.1:0/4200925897 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56253f7854d0 msgr2=0x56253fb532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.744+0000 7efdc37fe640 1 --2- 172.21.15.1:0/4200925897 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56253f7854d0 0x56253fb532f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.744+0000 7efdc37fe640 1 -- 172.21.15.1:0/4200925897 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56253f7be210 con 0x56253f77d2f0 2024-05-04T03:25:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.744+0000 7efdc2ffd640 1 --2- 172.21.15.1:0/4200925897 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56253f7854d0 0x56253fb532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:25:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.745+0000 7efdc3fff640 1 --2- 172.21.15.1:0/1732304776 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7efdac0b28a0 0x7efdac0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.745+0000 7efdc0ff9640 1 -- 172.21.15.1:0/1732304776 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7efdbc0e2690 con 0x7efdc4108a90 2024-05-04T03:25:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.745+0000 7efdc37fe640 1 --2- 172.21.15.1:0/4200925897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56253f77d2f0 0x56253fb4c3f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7efdb4009230 tx=0x7efdb4009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246a1a2dd211824d server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.745+0000 7efda9ffb640 1 -- 172.21.15.1:0/4200925897 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb4035030 con 0x56253f77d2f0 2024-05-04T03:25:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.745+0000 7efda9ffb640 1 -- 172.21.15.1:0/4200925897 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdb4040070 con 0x56253f77d2f0 2024-05-04T03:25:20.750 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.745+0000 7efda9ffb640 1 -- 172.21.15.1:0/4200925897 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb4003970 con 0x56253f77d2f0 2024-05-04T03:25:20.751 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:20 smithi072 ceph-mon[33332]: pgmap v576: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.7 MiB/s wr, 611 op/s 2024-05-04T03:25:20.751 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:20 smithi072 ceph-mon[33332]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:25:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.746+0000 7efdd0ba3740 1 -- 172.21.15.1:0/4200925897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56253f77d2f0 msgr2=0x56253fb4c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.746+0000 7efdd0ba3740 1 --2- 172.21.15.1:0/4200925897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56253f77d2f0 0x56253fb4c3f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7efdb4009230 tx=0x7efdb4009620 comp rx=0 tx=0).stop 2024-05-04T03:25:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.746+0000 7efdc3fff640 1 --2- 172.21.15.1:0/1732304776 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7efdac0b28a0 0x7efdac0b4db0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0a5df0 tx=0x7efdbc097f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.747+0000 7efdd0ba3740 1 -- 172.21.15.1:0/4200925897 shutdown_connections 2024-05-04T03:25:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.747+0000 7efdd0ba3740 1 --2- 172.21.15.1:0/4200925897 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56253f7854d0 0x56253fb532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.747+0000 7efdd0ba3740 1 --2- 172.21.15.1:0/4200925897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56253f77d2f0 0x56253fb4c3f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.747+0000 7efdd0ba3740 1 --2- 172.21.15.1:0/4200925897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56253f7a61b0 0x56253fb4be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.747+0000 7efdd0ba3740 1 -- 172.21.15.1:0/4200925897 >> 172.21.15.1:0/4200925897 conn(0x56253f79a300 msgr2=0x56253fb433a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.747+0000 7efdd0ba3740 1 -- 172.21.15.1:0/4200925897 shutdown_connections 2024-05-04T03:25:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.747+0000 7efdd0ba3740 1 -- 172.21.15.1:0/4200925897 wait complete. 2024-05-04T03:25:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.747+0000 7efdd0ba3740 1 -- start start 2024-05-04T03:25:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.747+0000 7efdd0ba3740 1 Processor -- start 2024-05-04T03:25:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.748+0000 7efdd0ba3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56253f77d2f0 0x56253fb55af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.748+0000 7efdd0ba3740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56253f7854d0 0x56253fb56030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.748+0000 7efdd0ba3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56253f7a61b0 0x56253fb5cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.748+0000 7efdd0ba3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x56253fb53ce0 con 0x56253f77d2f0 2024-05-04T03:25:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.748+0000 7efdd0ba3740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x56253fa69370 con 0x56253f7854d0 2024-05-04T03:25:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.748+0000 7efdd0ba3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56253fb53e60 con 0x56253f7a61b0 2024-05-04T03:25:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.748+0000 7efdd0ba3740 10 client.0 _ll_register_callbacks cb 0x56253f7bda00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:25:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.748+0000 7efdc2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56253f77d2f0 0x56253fb55af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.748+0000 7efdc2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56253f77d2f0 0x56253fb55af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38510/0 (socket says 172.21.15.1:38510) 2024-05-04T03:25:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.748+0000 7efdc2ffd640 1 -- 172.21.15.1:0/3930880381 learned_addr learned my addr 172.21.15.1:0/3930880381 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:25:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.748+0000 7efdc37fe640 1 --2- 172.21.15.1:0/3930880381 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56253f7854d0 0x56253fb56030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.748+0000 7efdc2ffd640 1 --2- 172.21.15.1:0/3930880381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56253f7a61b0 0x56253fb5cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.748+0000 7efdc2ffd640 1 -- 172.21.15.1:0/3930880381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56253f7a61b0 msgr2=0x56253fb5cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.748+0000 7efdc2ffd640 1 --2- 172.21.15.1:0/3930880381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56253f7a61b0 0x56253fb5cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.748+0000 7efdc2ffd640 1 -- 172.21.15.1:0/3930880381 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56253f7854d0 msgr2=0x56253fb56030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.748+0000 7efdc2ffd640 1 --2- 172.21.15.1:0/3930880381 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56253f7854d0 0x56253fb56030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.748+0000 7efdc2ffd640 1 -- 172.21.15.1:0/3930880381 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56253f88a620 con 0x56253f77d2f0 2024-05-04T03:25:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.749+0000 7efdc2ffd640 1 --2- 172.21.15.1:0/3930880381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56253f77d2f0 0x56253fb55af0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009e80 tx=0x7efdb8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdcb5488690c1f6c server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.749+0000 7efda2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:25:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.749+0000 7efda2ffd640 1 -- 172.21.15.1:0/3930880381 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb800fc10 con 0x56253f77d2f0 2024-05-04T03:25:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.749+0000 7efda2ffd640 1 -- 172.21.15.1:0/3930880381 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7efdb800fdb0 con 0x56253f77d2f0 2024-05-04T03:25:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.749+0000 7efdd0ba3740 1 -- 172.21.15.1:0/3930880381 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdb4040070 con 0x56253f77d2f0 2024-05-04T03:25:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.749+0000 7efda2ffd640 1 -- 172.21.15.1:0/3930880381 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7efdb8005020 con 0x56253f77d2f0 2024-05-04T03:25:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.749+0000 7efdd0ba3740 10 client.15927 fetch_fsmap learned FSMap version 33 2024-05-04T03:25:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.749+0000 7efdd0ba3740 1 -- 172.21.15.1:0/3930880381 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56253fb5ff90 con 0x56253f77d2f0 2024-05-04T03:25:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.749+0000 7efda2ffd640 1 -- 172.21.15.1:0/3930880381 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdb4040070 con 0x56253f77d2f0 2024-05-04T03:25:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.749+0000 7efda2ffd640 1 -- 172.21.15.1:0/3930880381 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 33) ==== 8342+0+0 (secure 0 0 0) 0x7efdb801d030 con 0x56253f77d2f0 2024-05-04T03:25:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.749+0000 7efdd0ba3740 10 client.15927 fetch_fsmap finished waiting for FSMap version 33 2024-05-04T03:25:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.749+0000 7efdd0ba3740 10 client.15927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:25:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.749+0000 7efdd0ba3740 20 client.15927 populate_metadata read hostname 'smithi001' 2024-05-04T03:25:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.750+0000 7efdd0ba3740 1 --2- 172.21.15.1:0/3930880381 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x56253f764d80 0x56253fb63690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.750+0000 7efdd0ba3740 4 client.15927 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:25:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.750+0000 7efdd0ba3740 1 -- 172.21.15.1:0/3930880381 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56253faba1e0 con 0x56253f764d80 2024-05-04T03:25:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.750+0000 7efdc37fe640 1 --2- 172.21.15.1:0/3930880381 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x56253f764d80 0x56253fb63690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.750+0000 7efda2ffd640 10 client.15927 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:25:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.750+0000 7efdc37fe640 1 --2- 172.21.15.1:0/3930880381 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x56253f764d80 0x56253fb63690 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8403644be55076a server_cookie=93c098e8c3edd174 in_seq=0 out_seq=0 2024-05-04T03:25:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.751+0000 7efda2ffd640 1 -- 172.21.15.1:0/3930880381 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56253faba1e0 con 0x56253f764d80 2024-05-04T03:25:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.751+0000 7efda2ffd640 10 client.15927 handle_command_reply: tid=0 2024-05-04T03:25:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.751+0000 7efdd0ba3740 1 client.15927 shutdown 2024-05-04T03:25:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.751+0000 7efdd0ba3740 1 -- 172.21.15.1:0/3930880381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56253f77d2f0 msgr2=0x56253fb55af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.751+0000 7efdd0ba3740 1 --2- 172.21.15.1:0/3930880381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56253f77d2f0 0x56253fb55af0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009e80 tx=0x7efdb8005f90 comp rx=0 tx=0).stop 2024-05-04T03:25:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.751+0000 7efdd0ba3740 1 -- 172.21.15.1:0/3930880381 shutdown_connections 2024-05-04T03:25:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.751+0000 7efdd0ba3740 1 --2- 172.21.15.1:0/3930880381 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x56253f764d80 0x56253fb63690 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.751+0000 7efdd0ba3740 1 --2- 172.21.15.1:0/3930880381 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56253f7854d0 0x56253fb56030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.751+0000 7efdd0ba3740 1 --2- 172.21.15.1:0/3930880381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56253f7a61b0 0x56253fb5cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.751+0000 7efda2ffd640 0 client.15927 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:25:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.751+0000 7efdd0ba3740 1 --2- 172.21.15.1:0/3930880381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56253f77d2f0 0x56253fb55af0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.751+0000 7efdd0ba3740 1 -- 172.21.15.1:0/3930880381 >> 172.21.15.1:0/3930880381 conn(0x56253f79a300 msgr2=0x56253fb46300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.751+0000 7efdd0ba3740 1 -- 172.21.15.1:0/3930880381 shutdown_connections 2024-05-04T03:25:20.765 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.751+0000 7efdd0ba3740 1 -- 172.21.15.1:0/3930880381 wait complete. 2024-05-04T03:25:20.766 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.751+0000 7efdd0ba3740 20 client.15927 trim_cache size 0 max 16384 2024-05-04T03:25:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.767+0000 7efdd0ba3740 1 Processor -- start 2024-05-04T03:25:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.767+0000 7efdd0ba3740 1 -- start start 2024-05-04T03:25:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.767+0000 7efdd0ba3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56253f77d2f0 0x56253fb505f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.767+0000 7efdd0ba3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56253f7854d0 0x56253fb433d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.767+0000 7efdd0ba3740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56253f7a61b0 0x56253fb5cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.767+0000 7efdd0ba3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efdc41ab880 con 0x56253f77d2f0 2024-05-04T03:25:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.767+0000 7efdd0ba3740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7efdc41a9120 con 0x56253f7a61b0 2024-05-04T03:25:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.767+0000 7efdd0ba3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efdc41a6f70 con 0x56253f7854d0 2024-05-04T03:25:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.768+0000 7efdc2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56253f77d2f0 0x56253fb505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.768+0000 7efdc2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56253f77d2f0 0x56253fb505f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38514/0 (socket says 172.21.15.1:38514) 2024-05-04T03:25:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.768+0000 7efdc2ffd640 1 -- 172.21.15.1:0/1564175783 learned_addr learned my addr 172.21.15.1:0/1564175783 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:25:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.768+0000 7efdc37fe640 1 --2- 172.21.15.1:0/1564175783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56253f7854d0 0x56253fb433d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.768+0000 7efdc2ffd640 1 --2- 172.21.15.1:0/1564175783 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56253f7a61b0 0x56253fb5cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.768+0000 7efdc37fe640 1 -- 172.21.15.1:0/1564175783 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56253f7a61b0 msgr2=0x56253fb5cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.768+0000 7efdc37fe640 1 --2- 172.21.15.1:0/1564175783 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56253f7a61b0 0x56253fb5cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.768+0000 7efdc37fe640 1 -- 172.21.15.1:0/1564175783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56253f77d2f0 msgr2=0x56253fb505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.768+0000 7efdc37fe640 1 --2- 172.21.15.1:0/1564175783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56253f77d2f0 0x56253fb505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.768+0000 7efdc37fe640 1 -- 172.21.15.1:0/1564175783 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56253fb43910 con 0x56253f7854d0 2024-05-04T03:25:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.768+0000 7efdc37fe640 1 --2- 172.21.15.1:0/1564175783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56253f7854d0 0x56253fb433d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efdb4031500 tx=0x7efdb4047040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30f12d710ff8ab2a server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.769+0000 7efda97fa640 1 -- 172.21.15.1:0/1564175783 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb4035030 con 0x56253f7854d0 2024-05-04T03:25:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.769+0000 7efda97fa640 1 -- 172.21.15.1:0/1564175783 <== mon.1 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdb4054070 con 0x56253f7854d0 2024-05-04T03:25:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.769+0000 7efda97fa640 1 -- 172.21.15.1:0/1564175783 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb4045b90 con 0x56253f7854d0 2024-05-04T03:25:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.770+0000 7efdd0ba3740 1 -- 172.21.15.1:0/1564175783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56253f7854d0 msgr2=0x56253fb433d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.770+0000 7efdd0ba3740 1 --2- 172.21.15.1:0/1564175783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56253f7854d0 0x56253fb433d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efdb4031500 tx=0x7efdb4047040 comp rx=0 tx=0).stop 2024-05-04T03:25:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.770+0000 7efdd0ba3740 1 -- 172.21.15.1:0/1564175783 shutdown_connections 2024-05-04T03:25:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.770+0000 7efdd0ba3740 1 --2- 172.21.15.1:0/1564175783 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56253f7a61b0 0x56253fb5cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.770+0000 7efdd0ba3740 1 --2- 172.21.15.1:0/1564175783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56253f7854d0 0x56253fb433d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.770+0000 7efdd0ba3740 1 --2- 172.21.15.1:0/1564175783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56253f77d2f0 0x56253fb505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.770+0000 7efdd0ba3740 1 -- 172.21.15.1:0/1564175783 >> 172.21.15.1:0/1564175783 conn(0x56253f79a300 msgr2=0x56253fb42cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.770+0000 7efdd0ba3740 1 -- 172.21.15.1:0/1564175783 shutdown_connections 2024-05-04T03:25:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.770+0000 7efdd0ba3740 1 -- 172.21.15.1:0/1564175783 wait complete. 2024-05-04T03:25:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.770+0000 7efdd0ba3740 1 -- start start 2024-05-04T03:25:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.771+0000 7efdd0ba3740 1 Processor -- start 2024-05-04T03:25:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.771+0000 7efdd0ba3740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56253f77d2f0 0x56253fb5a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.771+0000 7efdd0ba3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56253f7854d0 0x56253fb5ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.771+0000 7efdd0ba3740 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56253f7a61b0 0x56253fb5bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.771+0000 7efdd0ba3740 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7efdc41a75e0 con 0x56253f7a61b0 2024-05-04T03:25:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.771+0000 7efdd0ba3740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7efdc41a8a50 con 0x56253f77d2f0 2024-05-04T03:25:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.771+0000 7efdd0ba3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efdc41a7cb0 con 0x56253f7854d0 2024-05-04T03:25:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.771+0000 7efdd0ba3740 10 client.0 _ll_register_callbacks cb 0x7efdb8009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:25:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.771+0000 7efdc2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56253f7a61b0 0x56253fb5bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.771+0000 7efdc2ffd640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56253f7a61b0 0x56253fb5bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38528/0 (socket says 172.21.15.1:38528) 2024-05-04T03:25:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.771+0000 7efdc2ffd640 1 -- 172.21.15.1:0/1598625751 learned_addr learned my addr 172.21.15.1:0/1598625751 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:25:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.771+0000 7efdc37fe640 1 --2- 172.21.15.1:0/1598625751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56253f7854d0 0x56253fb5ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.771+0000 7efdc2ffd640 1 --2- 172.21.15.1:0/1598625751 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56253f77d2f0 0x56253fb5a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.772+0000 7efdc2ffd640 1 -- 172.21.15.1:0/1598625751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56253f7854d0 msgr2=0x56253fb5ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.772+0000 7efdc2ffd640 1 --2- 172.21.15.1:0/1598625751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56253f7854d0 0x56253fb5ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.772+0000 7efdc2ffd640 1 -- 172.21.15.1:0/1598625751 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56253f77d2f0 msgr2=0x56253fb5a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.772+0000 7efdc2ffd640 1 --2- 172.21.15.1:0/1598625751 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56253f77d2f0 0x56253fb5a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.772+0000 7efdc2ffd640 1 -- 172.21.15.1:0/1598625751 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56253fb5c310 con 0x56253f7a61b0 2024-05-04T03:25:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.772+0000 7efdc2ffd640 1 --2- 172.21.15.1:0/1598625751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56253f7a61b0 0x56253fb5bd50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009e80 tx=0x7efdb80097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9aa88a66f75f72 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.772+0000 7efda2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:25:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.772+0000 7efda2ffd640 1 -- 172.21.15.1:0/1598625751 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb802d020 con 0x56253f7a61b0 2024-05-04T03:25:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.772+0000 7efda2ffd640 1 -- 172.21.15.1:0/1598625751 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7efdb8025020 con 0x56253f7a61b0 2024-05-04T03:25:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.772+0000 7efdd0ba3740 1 -- 172.21.15.1:0/1598625751 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdb4054070 con 0x56253f7a61b0 2024-05-04T03:25:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.772+0000 7efda2ffd640 1 -- 172.21.15.1:0/1598625751 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7efdb803e030 con 0x56253f7a61b0 2024-05-04T03:25:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.772+0000 7efdd0ba3740 10 client.15933 fetch_fsmap learned FSMap version 33 2024-05-04T03:25:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.772+0000 7efdd0ba3740 1 -- 172.21.15.1:0/1598625751 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56253fb5d5a0 con 0x56253f7a61b0 2024-05-04T03:25:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.773+0000 7efda2ffd640 1 -- 172.21.15.1:0/1598625751 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdb4054070 con 0x56253f7a61b0 2024-05-04T03:25:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.773+0000 7efda2ffd640 1 -- 172.21.15.1:0/1598625751 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 33) ==== 8342+0+0 (secure 0 0 0) 0x7efdb801d040 con 0x56253f7a61b0 2024-05-04T03:25:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.773+0000 7efdd0ba3740 10 client.15933 fetch_fsmap finished waiting for FSMap version 33 2024-05-04T03:25:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.773+0000 7efdd0ba3740 10 client.15933 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-04T03:25:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.773+0000 7efdd0ba3740 20 client.15933 populate_metadata read hostname 'smithi001' 2024-05-04T03:25:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.773+0000 7efdd0ba3740 1 --2- 172.21.15.1:0/1598625751 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x56253f73a1c0 0x56253fbcdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.773+0000 7efdd0ba3740 4 client.15933 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-04T03:25:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.773+0000 7efdd0ba3740 1 -- 172.21.15.1:0/1598625751 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56253fab4050 con 0x56253f73a1c0 2024-05-04T03:25:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.773+0000 7efdc37fe640 1 --2- 172.21.15.1:0/1598625751 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x56253f73a1c0 0x56253fbcdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.773+0000 7efda2ffd640 10 client.15933 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:25:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.774+0000 7efdc37fe640 1 --2- 172.21.15.1:0/1598625751 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x56253f73a1c0 0x56253fbcdcd0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7081f5f3066e2ce4 server_cookie=d5b47e0422abe4da in_seq=0 out_seq=0 2024-05-04T03:25:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.819+0000 7efda2ffd640 1 -- 172.21.15.1:0/1598625751 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56253fab4050 con 0x56253f73a1c0 2024-05-04T03:25:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.819+0000 7efda2ffd640 10 client.15933 handle_command_reply: tid=0 2024-05-04T03:25:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.819+0000 7efdd0ba3740 1 client.15933 shutdown 2024-05-04T03:25:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.820+0000 7efdd0ba3740 1 -- 172.21.15.1:0/1598625751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56253f7a61b0 msgr2=0x56253fb5bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.820+0000 7efdd0ba3740 1 --2- 172.21.15.1:0/1598625751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56253f7a61b0 0x56253fb5bd50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009e80 tx=0x7efdb80097d0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.820+0000 7efdd0ba3740 1 -- 172.21.15.1:0/1598625751 shutdown_connections 2024-05-04T03:25:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.820+0000 7efdd0ba3740 1 --2- 172.21.15.1:0/1598625751 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x56253f73a1c0 0x56253fbcdcd0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.820+0000 7efdd0ba3740 1 --2- 172.21.15.1:0/1598625751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56253f7a61b0 0x56253fb5bd50 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.820+0000 7efdd0ba3740 1 --2- 172.21.15.1:0/1598625751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56253f7854d0 0x56253fb5ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.820+0000 7efda2ffd640 0 client.15933 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:25:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.820+0000 7efdd0ba3740 1 --2- 172.21.15.1:0/1598625751 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56253f77d2f0 0x56253fb5a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.820+0000 7efdd0ba3740 1 -- 172.21.15.1:0/1598625751 >> 172.21.15.1:0/1598625751 conn(0x56253f79a300 msgr2=0x56253fb67e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.820+0000 7efdd0ba3740 1 -- 172.21.15.1:0/1598625751 shutdown_connections 2024-05-04T03:25:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.820+0000 7efdd0ba3740 1 -- 172.21.15.1:0/1598625751 wait complete. 2024-05-04T03:25:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.820+0000 7efdd0ba3740 20 client.15933 trim_cache size 0 max 16384 2024-05-04T03:25:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:{ 2024-05-04T03:25:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "message": "", 2024-05-04T03:25:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout: "return_code": 0 2024-05-04T03:25:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stdout:} 2024-05-04T03:25:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.820+0000 7efdc9c15640 1 -- 172.21.15.1:0/1732304776 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7efdac0b28a0 msgr2=0x7efdac0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.820+0000 7efdc9c15640 1 --2- 172.21.15.1:0/1732304776 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7efdac0b28a0 0x7efdac0b4db0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0a5df0 tx=0x7efdbc097f90 comp rx=0 tx=0).stop 2024-05-04T03:25:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.820+0000 7efdc9c15640 1 -- 172.21.15.1:0/1732304776 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efdc4108a90 msgr2=0x7efdc41a1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:20.825 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.820+0000 7efdc9c15640 1 --2- 172.21.15.1:0/1732304776 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efdc4108a90 0x7efdc41a1600 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efdbc00ec50 tx=0x7efdbc00c6a0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.820+0000 7efdc9c15640 1 -- 172.21.15.1:0/1732304776 shutdown_connections 2024-05-04T03:25:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.820+0000 7efdc9c15640 1 --2- 172.21.15.1:0/1732304776 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7efdac0b28a0 0x7efdac0b4db0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.820+0000 7efdc9c15640 1 --2- 172.21.15.1:0/1732304776 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efdc4108a90 0x7efdc41a1600 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.820+0000 7efdc9c15640 1 --2- 172.21.15.1:0/1732304776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efdc4103d90 0x7efdc419f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.821+0000 7efdc9c15640 1 --2- 172.21.15.1:0/1732304776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efdc4102b50 0x7efdc406b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.821+0000 7efdc9c15640 1 -- 172.21.15.1:0/1732304776 >> 172.21.15.1:0/1732304776 conn(0x7efdc40fdd40 msgr2=0x7efdc4104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.821+0000 7efdc9c15640 1 -- 172.21.15.1:0/1732304776 shutdown_connections 2024-05-04T03:25:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi001.front.sepia.ceph.com.smithi001.stderr:2024-05-04T03:25:20.821+0000 7efdc9c15640 1 -- 172.21.15.1:0/1732304776 wait complete. 2024-05-04T03:25:20.913 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:25:20.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:20 smithi077 ceph-mon[33531]: pgmap v576: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.7 MiB/s wr, 611 op/s 2024-05-04T03:25:20.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:20 smithi077 ceph-mon[33531]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:25:22.607 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:22 smithi001 ceph-mon[29731]: pgmap v577: 97 pgs: 97 active+clean; 122 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.4 MiB/s wr, 438 op/s 2024-05-04T03:25:22.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:22 smithi072 ceph-mon[33332]: pgmap v577: 97 pgs: 97 active+clean; 122 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.4 MiB/s wr, 438 op/s 2024-05-04T03:25:22.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.843+0000 7feb02114700 1 -- 172.21.15.1:0/3954541079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feafc1034c0 msgr2=0x7feafc1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:22.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.843+0000 7feb02114700 1 --2- 172.21.15.1:0/3954541079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feafc1034c0 0x7feafc1038d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7feaf00099c0 tx=0x7feaf0009cd0 comp rx=0 tx=0).stop 2024-05-04T03:25:22.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.844+0000 7feb02114700 1 -- 172.21.15.1:0/3954541079 shutdown_connections 2024-05-04T03:25:22.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.844+0000 7feb02114700 1 --2- 172.21.15.1:0/3954541079 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feafc108740 0x7feafc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:22.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.844+0000 7feb02114700 1 --2- 172.21.15.1:0/3954541079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feafc1034c0 0x7feafc1038d0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:22.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.844+0000 7feb02114700 1 --2- 172.21.15.1:0/3954541079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feafc107df0 0x7feafc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:22.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.844+0000 7feb02114700 1 -- 172.21.15.1:0/3954541079 >> 172.21.15.1:0/3954541079 conn(0x7feafc0fedc0 msgr2=0x7feafc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:22.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.844+0000 7feb02114700 1 -- 172.21.15.1:0/3954541079 shutdown_connections 2024-05-04T03:25:22.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.844+0000 7feb02114700 1 -- 172.21.15.1:0/3954541079 wait complete. 2024-05-04T03:25:22.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.844+0000 7feb02114700 1 Processor -- start 2024-05-04T03:25:22.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.845+0000 7feb02114700 1 -- start start 2024-05-04T03:25:22.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.845+0000 7feb02114700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feafc1034c0 0x7feafc19bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:22.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.845+0000 7feb02114700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feafc107df0 0x7feafc19c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:22.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.845+0000 7feb02114700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feafc108740 0x7feafc19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:22.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.845+0000 7feb02114700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7feafc113370 con 0x7feafc108740 2024-05-04T03:25:22.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.845+0000 7feb02114700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7feafc1131f0 con 0x7feafc107df0 2024-05-04T03:25:22.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.845+0000 7feb02114700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feafc1134f0 con 0x7feafc1034c0 2024-05-04T03:25:22.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.845+0000 7feafbfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feafc108740 0x7feafc19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:22.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.845+0000 7feafbfff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feafc108740 0x7feafc19c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40154/0 (socket says 172.21.15.1:40154) 2024-05-04T03:25:22.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.845+0000 7feafbfff700 1 -- 172.21.15.1:0/3781074218 learned_addr learned my addr 172.21.15.1:0/3781074218 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:25:22.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.845+0000 7feafb7fe700 1 --2- 172.21.15.1:0/3781074218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feafc1034c0 0x7feafc19bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:22.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.845+0000 7feafaffd700 1 --2- 172.21.15.1:0/3781074218 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feafc107df0 0x7feafc19c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:22.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.845+0000 7feafbfff700 1 -- 172.21.15.1:0/3781074218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feafc1034c0 msgr2=0x7feafc19bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:22.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.845+0000 7feafbfff700 1 --2- 172.21.15.1:0/3781074218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feafc1034c0 0x7feafc19bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:22.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.845+0000 7feafbfff700 1 -- 172.21.15.1:0/3781074218 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feafc107df0 msgr2=0x7feafc19c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:22.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.845+0000 7feafbfff700 1 --2- 172.21.15.1:0/3781074218 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feafc107df0 0x7feafc19c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:22.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.845+0000 7feafbfff700 1 -- 172.21.15.1:0/3781074218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feafc1a6f80 con 0x7feafc108740 2024-05-04T03:25:22.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.846+0000 7feafbfff700 1 --2- 172.21.15.1:0/3781074218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feafc108740 0x7feafc19c6d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7feaf400ef30 tx=0x7feaf400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:22.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.846+0000 7feaf8ff9700 1 -- 172.21.15.1:0/3781074218 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf40092e0 con 0x7feafc108740 2024-05-04T03:25:22.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.846+0000 7feaf8ff9700 1 -- 172.21.15.1:0/3781074218 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaf4004bb0 con 0x7feafc108740 2024-05-04T03:25:22.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.846+0000 7feaf8ff9700 1 -- 172.21.15.1:0/3781074218 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf4004020 con 0x7feafc108740 2024-05-04T03:25:22.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.846+0000 7feb02114700 1 -- 172.21.15.1:0/3781074218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feafc1a7210 con 0x7feafc108740 2024-05-04T03:25:22.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.846+0000 7feb02114700 1 -- 172.21.15.1:0/3781074218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feafc1a7750 con 0x7feafc108740 2024-05-04T03:25:22.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.847+0000 7feb02114700 1 -- 172.21.15.1:0/3781074218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feafc0512c0 con 0x7feafc108740 2024-05-04T03:25:22.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.848+0000 7feaf8ff9700 1 -- 172.21.15.1:0/3781074218 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7feaf401b030 con 0x7feafc108740 2024-05-04T03:25:22.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.849+0000 7feaf8ff9700 1 --2- 172.21.15.1:0/3781074218 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7feae40b2900 0x7feae40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:22.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.849+0000 7feafb7fe700 1 --2- 172.21.15.1:0/3781074218 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7feae40b2900 0x7feae40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:22.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.849+0000 7feaf8ff9700 1 -- 172.21.15.1:0/3781074218 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7feaf40e1ba0 con 0x7feafc108740 2024-05-04T03:25:22.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.850+0000 7feafb7fe700 1 --2- 172.21.15.1:0/3781074218 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7feae40b2900 0x7feae40b4e10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7feaf0000c00 tx=0x7feaf001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:22.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:22.854+0000 7feaf8ff9700 1 -- 172.21.15.1:0/3781074218 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feaf4022020 con 0x7feafc108740 2024-05-04T03:25:22.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:22 smithi077 ceph-mon[33531]: pgmap v577: 97 pgs: 97 active+clean; 122 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.4 MiB/s wr, 438 op/s 2024-05-04T03:25:23.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:23.118+0000 7feb02114700 1 -- 172.21.15.1:0/3781074218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feafc02e5f0 con 0x7feafc108740 2024-05-04T03:25:23.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:23.119+0000 7feaf8ff9700 1 -- 172.21.15.1:0/3781074218 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11846 (secure 0 0 0) 0x7feaf4010040 con 0x7feafc108740 2024-05-04T03:25:23.121 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:25:23.121 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2186681910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2186681910},{"type":"v1","addr":"172.21.15.1:6835","nonce":2186681910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2747191047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2747191047},{"type":"v1","addr":"172.21.15.1:6837","nonce":2747191047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4115120540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4115120540},{"type":"v1","addr":"172.21.15.1:6839","nonce":4115120540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3405709780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3405709780},{"type":"v1","addr":"172.21.15.1:6841","nonce":3405709780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/3816098761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":3816098761},{"type":"v1","addr":"172.21.15.72:6833","nonce":3816098761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/379026128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":379026128},{"type":"v1","addr":"172.21.15.77:6833","nonce":379026128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:15:49.823471+0000","modified":"2024-05-04T03:25:21.576332+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24472,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/3930984976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3930984976},{"type":"v1","addr":"172.21.15.72:6835","nonce":3930984976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/4279951305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4279951305},{"type":"v1","addr":"172.21.15.77:6835","nonce":4279951305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2003521417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2003521417},{"type":"v1","addr":"172.21.15.77:6837","nonce":2003521417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/1417614203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1417614203},{"type":"v1","addr":"172.21.15.72:6837","nonce":1417614203}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.77:6839/922723189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":922723189},{"type":"v1","addr":"172.21.15.77:6839","nonce":922723189}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2976050650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2976050650},{"type":"v1","addr":"172.21.15.72:6839","nonce":2976050650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24472,24494,24467]},"id":1}]} 2024-05-04T03:25:23.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:23.121+0000 7feb02114700 1 -- 172.21.15.1:0/3781074218 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7feae40b2900 msgr2=0x7feae40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:23.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:23.121+0000 7feb02114700 1 --2- 172.21.15.1:0/3781074218 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7feae40b2900 0x7feae40b4e10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7feaf0000c00 tx=0x7feaf001a040 comp rx=0 tx=0).stop 2024-05-04T03:25:23.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:23.121+0000 7feb02114700 1 -- 172.21.15.1:0/3781074218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feafc108740 msgr2=0x7feafc19c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:23.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:23.121+0000 7feb02114700 1 --2- 172.21.15.1:0/3781074218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feafc108740 0x7feafc19c6d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7feaf400ef30 tx=0x7feaf400c3a0 comp rx=0 tx=0).stop 2024-05-04T03:25:23.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:23.121+0000 7feb02114700 1 -- 172.21.15.1:0/3781074218 shutdown_connections 2024-05-04T03:25:23.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:23.121+0000 7feb02114700 1 --2- 172.21.15.1:0/3781074218 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7feae40b2900 0x7feae40b4e10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:23.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:23.121+0000 7feb02114700 1 --2- 172.21.15.1:0/3781074218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feafc108740 0x7feafc19c6d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:23.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:23.121+0000 7feb02114700 1 --2- 172.21.15.1:0/3781074218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feafc1034c0 0x7feafc19bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:23.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:23.121+0000 7feb02114700 1 --2- 172.21.15.1:0/3781074218 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feafc107df0 0x7feafc19c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:23.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:23.121+0000 7feb02114700 1 -- 172.21.15.1:0/3781074218 >> 172.21.15.1:0/3781074218 conn(0x7feafc0fedc0 msgr2=0x7feafc101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:23.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:23.122+0000 7feb02114700 1 -- 172.21.15.1:0/3781074218 shutdown_connections 2024-05-04T03:25:23.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:23.122+0000 7feb02114700 1 -- 172.21.15.1:0/3781074218 wait complete. 2024-05-04T03:25:23.125 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 34 2024-05-04T03:25:23.674 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --cancel --all' 2024-05-04T03:25:23.674 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24494 quiesce db --cancel --all 2024-05-04T03:25:23.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:23 smithi072 ceph-mon[33332]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:25:23.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:23 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/3781074218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:25:23.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:23 smithi001 ceph-mon[29731]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:25:23.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:23 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3781074218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:25:23.925 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:25:23.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:23 smithi077 ceph-mon[33531]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:25:23.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:23 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/3781074218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:25:24.634 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:24 smithi001 ceph-mon[29731]: pgmap v578: 97 pgs: 97 active+clean; 113 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 430 op/s 2024-05-04T03:25:24.748 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:24 smithi072 ceph-mon[33332]: pgmap v578: 97 pgs: 97 active+clean; 113 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 430 op/s 2024-05-04T03:25:24.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:24 smithi077 ceph-mon[33531]: pgmap v578: 97 pgs: 97 active+clean; 113 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 430 op/s 2024-05-04T03:25:25.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.548+0000 7f1bde26a700 1 -- 172.21.15.1:0/1647228263 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bd8107df0 msgr2=0x7f1bd8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:25.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.548+0000 7f1bde26a700 1 --2- 172.21.15.1:0/1647228263 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bd8107df0 0x7f1bd8108200 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0099c0 tx=0x7f1bcc009cd0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.548+0000 7f1bde26a700 1 -- 172.21.15.1:0/1647228263 shutdown_connections 2024-05-04T03:25:25.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.548+0000 7f1bde26a700 1 --2- 172.21.15.1:0/1647228263 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1bd81034c0 0x7f1bd81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.548+0000 7f1bde26a700 1 --2- 172.21.15.1:0/1647228263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bd8108740 0x7f1bd81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.548+0000 7f1bde26a700 1 --2- 172.21.15.1:0/1647228263 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bd8107df0 0x7f1bd8108200 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.548+0000 7f1bde26a700 1 -- 172.21.15.1:0/1647228263 >> 172.21.15.1:0/1647228263 conn(0x7f1bd80fedf0 msgr2=0x7f1bd8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:25.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.549+0000 7f1bde26a700 1 -- 172.21.15.1:0/1647228263 shutdown_connections 2024-05-04T03:25:25.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.549+0000 7f1bde26a700 1 -- 172.21.15.1:0/1647228263 wait complete. 2024-05-04T03:25:25.552 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:25 smithi001 ceph-mon[29731]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:25:25.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.549+0000 7f1bde26a700 1 Processor -- start 2024-05-04T03:25:25.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.549+0000 7f1bde26a700 1 -- start start 2024-05-04T03:25:25.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.550+0000 7f1bde26a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bd81034c0 0x7f1bd819bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:25.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.550+0000 7f1bde26a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bd8107df0 0x7f1bd819c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:25.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.550+0000 7f1bde26a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1bd8108740 0x7f1bd819c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:25.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.550+0000 7f1bde26a700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd8113350 con 0x7f1bd81034c0 2024-05-04T03:25:25.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.550+0000 7f1bde26a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd81131d0 con 0x7f1bd8108740 2024-05-04T03:25:25.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.550+0000 7f1bde26a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd81134d0 con 0x7f1bd8107df0 2024-05-04T03:25:25.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.550+0000 7f1bd7fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bd81034c0 0x7f1bd819bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:25.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.550+0000 7f1bd7fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bd81034c0 0x7f1bd819bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40172/0 (socket says 172.21.15.1:40172) 2024-05-04T03:25:25.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.550+0000 7f1bd7fff700 1 -- 172.21.15.1:0/1893370380 learned_addr learned my addr 172.21.15.1:0/1893370380 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:25:25.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.550+0000 7f1bdc807700 1 --2- 172.21.15.1:0/1893370380 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1bd8108740 0x7f1bd819c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:25.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.550+0000 7f1bd77fe700 1 --2- 172.21.15.1:0/1893370380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bd8107df0 0x7f1bd819c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:25.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.550+0000 7f1bd7fff700 1 -- 172.21.15.1:0/1893370380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bd8107df0 msgr2=0x7f1bd819c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:25.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.550+0000 7f1bd7fff700 1 --2- 172.21.15.1:0/1893370380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bd8107df0 0x7f1bd819c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.550+0000 7f1bd7fff700 1 -- 172.21.15.1:0/1893370380 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1bd8108740 msgr2=0x7f1bd819c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:25.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.550+0000 7f1bd7fff700 1 --2- 172.21.15.1:0/1893370380 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1bd8108740 0x7f1bd819c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.550+0000 7f1bd7fff700 1 -- 172.21.15.1:0/1893370380 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd81a6fe0 con 0x7f1bd81034c0 2024-05-04T03:25:25.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.550+0000 7f1bd7fff700 1 --2- 172.21.15.1:0/1893370380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bd81034c0 0x7f1bd819bc90 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1bc800e9a0 tx=0x7f1bc800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:25.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.551+0000 7f1bd57fa700 1 -- 172.21.15.1:0/1893370380 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc800cbd0 con 0x7f1bd81034c0 2024-05-04T03:25:25.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.551+0000 7f1bd57fa700 1 -- 172.21.15.1:0/1893370380 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bc800cd70 con 0x7f1bd81034c0 2024-05-04T03:25:25.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.551+0000 7f1bd57fa700 1 -- 172.21.15.1:0/1893370380 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8018680 con 0x7f1bd81034c0 2024-05-04T03:25:25.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.551+0000 7f1bde26a700 1 -- 172.21.15.1:0/1893370380 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd81a72d0 con 0x7f1bd81034c0 2024-05-04T03:25:25.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.551+0000 7f1bde26a700 1 -- 172.21.15.1:0/1893370380 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd81a7830 con 0x7f1bd81034c0 2024-05-04T03:25:25.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.553+0000 7f1bd57fa700 1 -- 172.21.15.1:0/1893370380 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f1bc8010430 con 0x7f1bd81034c0 2024-05-04T03:25:25.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.554+0000 7f1bd57fa700 1 --2- 172.21.15.1:0/1893370380 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1bc00b2900 0x7f1bc00b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:25.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.554+0000 7f1bd77fe700 1 --2- 172.21.15.1:0/1893370380 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1bc00b2900 0x7f1bc00b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:25.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.554+0000 7f1bd57fa700 1 -- 172.21.15.1:0/1893370380 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f1bc80e2360 con 0x7f1bd81034c0 2024-05-04T03:25:25.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.555+0000 7f1becc7eb80 1 Processor -- start 2024-05-04T03:25:25.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.555+0000 7f1becc7eb80 1 -- start start 2024-05-04T03:25:25.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.555+0000 7f1bd77fe700 1 --2- 172.21.15.1:0/1893370380 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1bc00b2900 0x7f1bc00b4e10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc000c00 tx=0x7f1bcc01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:25.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.555+0000 7f1becc7eb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dffce97a40 0x55dffce97e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:25.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.555+0000 7f1becc7eb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dffce98390 0x55dffcea2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:25.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.555+0000 7f1becc7eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dffcea2bb0 0x55dffcea4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:25.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.555+0000 7f1becc7eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55dffcdba9b0 con 0x55dffcea2bb0 2024-05-04T03:25:25.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.555+0000 7f1becc7eb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55dffcd1fe10 con 0x55dffce98390 2024-05-04T03:25:25.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.555+0000 7f1becc7eb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55dffcdb6970 con 0x55dffce97a40 2024-05-04T03:25:25.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.555+0000 7f1bdc807700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dffcea2bb0 0x55dffcea4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:25.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.555+0000 7f1bdc807700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dffcea2bb0 0x55dffcea4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40182/0 (socket says 172.21.15.1:40182) 2024-05-04T03:25:25.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.555+0000 7f1bdc807700 1 -- 172.21.15.1:0/3548144729 learned_addr learned my addr 172.21.15.1:0/3548144729 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:25:25.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.555+0000 7f1bd7fff700 1 --2- 172.21.15.1:0/3548144729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dffce97a40 0x55dffce97e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:25.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.555+0000 7f1bd77fe700 1 --2- 172.21.15.1:0/3548144729 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dffce98390 0x55dffcea2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:25.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.555+0000 7f1bdc807700 1 -- 172.21.15.1:0/3548144729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dffce97a40 msgr2=0x55dffce97e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:25.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.555+0000 7f1bdc807700 1 --2- 172.21.15.1:0/3548144729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dffce97a40 0x55dffce97e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.555+0000 7f1bdc807700 1 -- 172.21.15.1:0/3548144729 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dffce98390 msgr2=0x55dffcea2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:25.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.555+0000 7f1bdc807700 1 --2- 172.21.15.1:0/3548144729 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dffce98390 0x55dffcea2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.556+0000 7f1bdc807700 1 -- 172.21.15.1:0/3548144729 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dffcd07940 con 0x55dffcea2bb0 2024-05-04T03:25:25.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.556+0000 7f1bdc807700 1 --2- 172.21.15.1:0/3548144729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dffcea2bb0 0x55dffcea4fd0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0003f10 tx=0x7f1bd0009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b767369ad4c9e31f server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:25.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.556+0000 7f1bbe7fc700 1 -- 172.21.15.1:0/3548144729 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0004bb0 con 0x55dffcea2bb0 2024-05-04T03:25:25.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.556+0000 7f1bbe7fc700 1 -- 172.21.15.1:0/3548144729 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bd0004d50 con 0x55dffcea2bb0 2024-05-04T03:25:25.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.556+0000 7f1bbe7fc700 1 -- 172.21.15.1:0/3548144729 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0007d60 con 0x55dffcea2bb0 2024-05-04T03:25:25.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.557+0000 7f1becc7eb80 1 -- 172.21.15.1:0/3548144729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dffcea2bb0 msgr2=0x55dffcea4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:25.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.557+0000 7f1becc7eb80 1 --2- 172.21.15.1:0/3548144729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dffcea2bb0 0x55dffcea4fd0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0003f10 tx=0x7f1bd0009ba0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.557+0000 7f1becc7eb80 1 -- 172.21.15.1:0/3548144729 shutdown_connections 2024-05-04T03:25:25.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.557+0000 7f1becc7eb80 1 --2- 172.21.15.1:0/3548144729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dffcea2bb0 0x55dffcea4fd0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.557+0000 7f1becc7eb80 1 --2- 172.21.15.1:0/3548144729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dffce97a40 0x55dffce97e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.557+0000 7f1becc7eb80 1 --2- 172.21.15.1:0/3548144729 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dffce98390 0x55dffcea2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.557+0000 7f1becc7eb80 1 -- 172.21.15.1:0/3548144729 >> 172.21.15.1:0/3548144729 conn(0x55dffcd1aeb0 msgr2=0x55dffce93940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:25.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.557+0000 7f1becc7eb80 1 -- 172.21.15.1:0/3548144729 shutdown_connections 2024-05-04T03:25:25.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.557+0000 7f1becc7eb80 1 -- 172.21.15.1:0/3548144729 wait complete. 2024-05-04T03:25:25.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.557+0000 7f1becc7eb80 1 -- start start 2024-05-04T03:25:25.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.558+0000 7f1becc7eb80 1 Processor -- start 2024-05-04T03:25:25.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.558+0000 7f1becc7eb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dffce97a40 0x55dffce9fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:25.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.558+0000 7f1becc7eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dffce98390 0x55dffcea0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:25.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.558+0000 7f1becc7eb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dffcea2bb0 0x55dffcea06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:25.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.558+0000 7f1becc7eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x55dffcea5980 con 0x55dffce98390 2024-05-04T03:25:25.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.558+0000 7f1becc7eb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55dffcd73f40 con 0x55dffcea2bb0 2024-05-04T03:25:25.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.558+0000 7f1becc7eb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55dffcea5b00 con 0x55dffce97a40 2024-05-04T03:25:25.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.558+0000 7f1becc7eb80 10 client.0 _ll_register_callbacks cb 0x55dffcd529c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:25:25.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.558+0000 7f1bd77fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dffce98390 0x55dffcea0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:25.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.558+0000 7f1bd7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dffce97a40 0x55dffce9fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:25.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.558+0000 7f1bd7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dffce97a40 0x55dffce9fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.1:41000/0 (socket says 172.21.15.1:41000) 2024-05-04T03:25:25.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.558+0000 7f1bd7fff700 1 -- 172.21.15.1:0/746122181 learned_addr learned my addr 172.21.15.1:0/746122181 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:25:25.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.558+0000 7f1bd77fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dffce98390 0x55dffcea0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40190/0 (socket says 172.21.15.1:40190) 2024-05-04T03:25:25.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.558+0000 7f1bdc807700 1 --2- 172.21.15.1:0/746122181 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dffcea2bb0 0x55dffcea06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:25.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.559+0000 7f1bd7fff700 1 -- 172.21.15.1:0/746122181 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dffcea2bb0 msgr2=0x55dffcea06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:25.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.559+0000 7f1bd7fff700 1 --2- 172.21.15.1:0/746122181 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dffcea2bb0 0x55dffcea06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.559+0000 7f1bd7fff700 1 -- 172.21.15.1:0/746122181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dffce98390 msgr2=0x55dffcea0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.559+0000 7f1bd7fff700 1 --2- 172.21.15.1:0/746122181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dffce98390 0x55dffcea0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.559+0000 7f1bd7fff700 1 -- 172.21.15.1:0/746122181 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dffcd717d0 con 0x55dffce97a40 2024-05-04T03:25:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.559+0000 7f1bd77fe700 1 --2- 172.21.15.1:0/746122181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dffce98390 0x55dffcea0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-04T03:25:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.559+0000 7f1bd7fff700 1 --2- 172.21.15.1:0/746122181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dffce97a40 0x55dffce9fc30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80a9740 tx=0x7f1bc80a9f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1969b45a6fd8cd61 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.559+0000 7f1bb77fe700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-04T03:25:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.559+0000 7f1bb77fe700 1 -- 172.21.15.1:0/746122181 <== mon.1 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc80b2040 con 0x55dffce97a40 2024-05-04T03:25:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.559+0000 7f1bb77fe700 1 -- 172.21.15.1:0/746122181 <== mon.1 v2:172.21.15.77:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f1bc8010930 con 0x55dffce97a40 2024-05-04T03:25:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.559+0000 7f1becc7eb80 1 -- 172.21.15.1:0/746122181 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bd0004d50 con 0x55dffce97a40 2024-05-04T03:25:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.559+0000 7f1bb77fe700 1 -- 172.21.15.1:0/746122181 <== mon.1 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1bc80a51d0 con 0x55dffce97a40 2024-05-04T03:25:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.559+0000 7f1becc7eb80 10 client.25624 fetch_fsmap learned FSMap version 35 2024-05-04T03:25:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.559+0000 7f1becc7eb80 1 -- 172.21.15.1:0/746122181 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55dffceb0660 con 0x55dffce97a40 2024-05-04T03:25:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.559+0000 7f1bb77fe700 1 -- 172.21.15.1:0/746122181 <== mon.1 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bd0004d50 con 0x55dffce97a40 2024-05-04T03:25:25.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.560+0000 7f1bb77fe700 1 -- 172.21.15.1:0/746122181 <== mon.1 v2:172.21.15.77:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7f1bc80cc0c0 con 0x55dffce97a40 2024-05-04T03:25:25.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.560+0000 7f1becc7eb80 10 client.25624 fetch_fsmap finished waiting for FSMap version 35 2024-05-04T03:25:25.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.560+0000 7f1becc7eb80 10 client.25624 resolve_mds: validated gid 24494 aka daemon mds.b 2024-05-04T03:25:25.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.560+0000 7f1becc7eb80 20 client.25624 populate_metadata read hostname 'smithi001' 2024-05-04T03:25:25.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.560+0000 7f1becc7eb80 1 --2- 172.21.15.1:0/746122181 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55dffceb2ca0 0x55dffceb50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:25.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.560+0000 7f1becc7eb80 4 client.25624 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-04T03:25:25.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.560+0000 7f1becc7eb80 1 -- 172.21.15.1:0/746122181 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dffcdc8390 con 0x55dffceb2ca0 2024-05-04T03:25:25.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.560+0000 7f1bd77fe700 1 --2- 172.21.15.1:0/746122181 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55dffceb2ca0 0x55dffceb50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:25.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.560+0000 7f1bd77fe700 1 --2- 172.21.15.1:0/746122181 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55dffceb2ca0 0x55dffceb50e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0d747a387e6c678 server_cookie=8495e8c58974ad22 in_seq=0 out_seq=0 2024-05-04T03:25:25.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.560+0000 7f1bb77fe700 10 client.25624 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:25:25.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.561+0000 7f1bb77fe700 1 -- 172.21.15.1:0/746122181 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dffcdc8390 con 0x55dffceb2ca0 2024-05-04T03:25:25.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.561+0000 7f1bb77fe700 10 client.25624 handle_command_reply: tid=0 2024-05-04T03:25:25.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.561+0000 7f1becc7eb80 1 client.25624 shutdown 2024-05-04T03:25:25.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.561+0000 7f1becc7eb80 1 -- 172.21.15.1:0/746122181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dffce97a40 msgr2=0x55dffce9fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:25.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.561+0000 7f1becc7eb80 1 --2- 172.21.15.1:0/746122181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dffce97a40 0x55dffce9fc30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80a9740 tx=0x7f1bc80a9f80 comp rx=0 tx=0).stop 2024-05-04T03:25:25.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.561+0000 7f1becc7eb80 1 -- 172.21.15.1:0/746122181 shutdown_connections 2024-05-04T03:25:25.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.561+0000 7f1becc7eb80 1 --2- 172.21.15.1:0/746122181 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55dffceb2ca0 0x55dffceb50e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.561+0000 7f1becc7eb80 1 --2- 172.21.15.1:0/746122181 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dffcea2bb0 0x55dffcea06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.561+0000 7f1becc7eb80 1 --2- 172.21.15.1:0/746122181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dffce97a40 0x55dffce9fc30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.561+0000 7f1bb77fe700 0 client.25624 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:25:25.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.561+0000 7f1becc7eb80 1 --2- 172.21.15.1:0/746122181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dffce98390 0x55dffcea0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.561+0000 7f1becc7eb80 1 -- 172.21.15.1:0/746122181 >> 172.21.15.1:0/746122181 conn(0x55dffcd1aeb0 msgr2=0x55dffce994e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:25.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.562+0000 7f1becc7eb80 1 -- 172.21.15.1:0/746122181 shutdown_connections 2024-05-04T03:25:25.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.562+0000 7f1becc7eb80 1 -- 172.21.15.1:0/746122181 wait complete. 2024-05-04T03:25:25.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.562+0000 7f1becc7eb80 20 client.25624 trim_cache size 0 max 16384 2024-05-04T03:25:25.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.582+0000 7f1becc7eb80 1 Processor -- start 2024-05-04T03:25:25.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.582+0000 7f1becc7eb80 1 -- start start 2024-05-04T03:25:25.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.582+0000 7f1becc7eb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dffce97a40 0x55dffcea1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:25.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.582+0000 7f1becc7eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dffce98390 0x55dffcea04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:25.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.582+0000 7f1becc7eb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dffcea2bb0 0x55dffce93490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:25.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.582+0000 7f1becc7eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd81af1d0 con 0x55dffce98390 2024-05-04T03:25:25.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.582+0000 7f1becc7eb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd81aca00 con 0x55dffce97a40 2024-05-04T03:25:25.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.582+0000 7f1becc7eb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd81aa8d0 con 0x55dffcea2bb0 2024-05-04T03:25:25.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.582+0000 7f1bd77fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dffce98390 0x55dffcea04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:25.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.582+0000 7f1bd77fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dffce98390 0x55dffcea04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40192/0 (socket says 172.21.15.1:40192) 2024-05-04T03:25:25.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.582+0000 7f1bd77fe700 1 -- 172.21.15.1:0/843727846 learned_addr learned my addr 172.21.15.1:0/843727846 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:25:25.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.583+0000 7f1bdc807700 1 --2- 172.21.15.1:0/843727846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dffcea2bb0 0x55dffce93490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:25.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.583+0000 7f1bd7fff700 1 --2- 172.21.15.1:0/843727846 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dffce97a40 0x55dffcea1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:25.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.583+0000 7f1bd77fe700 1 -- 172.21.15.1:0/843727846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dffcea2bb0 msgr2=0x55dffce93490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:25.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.583+0000 7f1bd77fe700 1 --2- 172.21.15.1:0/843727846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dffcea2bb0 0x55dffce93490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.583+0000 7f1bd77fe700 1 -- 172.21.15.1:0/843727846 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dffce97a40 msgr2=0x55dffcea1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:25.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.583+0000 7f1bd77fe700 1 --2- 172.21.15.1:0/843727846 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dffce97a40 0x55dffcea1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.583+0000 7f1bd77fe700 1 -- 172.21.15.1:0/843727846 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dffce94480 con 0x55dffce98390 2024-05-04T03:25:25.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.583+0000 7f1bd77fe700 1 --2- 172.21.15.1:0/843727846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dffce98390 0x55dffcea04f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0099c0 tx=0x7f1bcc010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e2bf58f1782e94 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:25.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.583+0000 7f1bbdffb700 1 -- 172.21.15.1:0/843727846 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc03c070 con 0x55dffce98390 2024-05-04T03:25:25.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.583+0000 7f1bbdffb700 1 -- 172.21.15.1:0/843727846 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bcc037e50 con 0x55dffce98390 2024-05-04T03:25:25.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.583+0000 7f1bbdffb700 1 -- 172.21.15.1:0/843727846 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc02a740 con 0x55dffce98390 2024-05-04T03:25:25.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.584+0000 7f1becc7eb80 1 -- 172.21.15.1:0/843727846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dffce98390 msgr2=0x55dffcea04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.584+0000 7f1becc7eb80 1 --2- 172.21.15.1:0/843727846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dffce98390 0x55dffcea04f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0099c0 tx=0x7f1bcc010000 comp rx=0 tx=0).stop 2024-05-04T03:25:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.584+0000 7f1becc7eb80 1 -- 172.21.15.1:0/843727846 shutdown_connections 2024-05-04T03:25:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.584+0000 7f1becc7eb80 1 --2- 172.21.15.1:0/843727846 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dffce97a40 0x55dffcea1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.584+0000 7f1becc7eb80 1 --2- 172.21.15.1:0/843727846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dffcea2bb0 0x55dffce93490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.584+0000 7f1becc7eb80 1 --2- 172.21.15.1:0/843727846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dffce98390 0x55dffcea04f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.584+0000 7f1becc7eb80 1 -- 172.21.15.1:0/843727846 >> 172.21.15.1:0/843727846 conn(0x55dffcd1aeb0 msgr2=0x55dffce92d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.584+0000 7f1becc7eb80 1 -- 172.21.15.1:0/843727846 shutdown_connections 2024-05-04T03:25:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.584+0000 7f1becc7eb80 1 -- 172.21.15.1:0/843727846 wait complete. 2024-05-04T03:25:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.584+0000 7f1becc7eb80 1 -- start start 2024-05-04T03:25:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.585+0000 7f1becc7eb80 1 Processor -- start 2024-05-04T03:25:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.585+0000 7f1becc7eb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dffce97a40 0x55dffceaad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.585+0000 7f1becc7eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dffce98390 0x55dffceab2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.585+0000 7f1becc7eb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dffcea2bb0 0x55dffcea7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:25.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.585+0000 7f1becc7eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd81aaf20 con 0x55dffce98390 2024-05-04T03:25:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.585+0000 7f1becc7eb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd81ac330 con 0x55dffcea2bb0 2024-05-04T03:25:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.585+0000 7f1becc7eb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd81ab590 con 0x55dffce97a40 2024-05-04T03:25:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.585+0000 7f1becc7eb80 10 client.0 _ll_register_callbacks cb 0x55dffcd529c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-04T03:25:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.585+0000 7f1bd77fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dffce98390 0x55dffceab2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.585+0000 7f1bd77fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dffce98390 0x55dffceab2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40200/0 (socket says 172.21.15.1:40200) 2024-05-04T03:25:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.585+0000 7f1bd77fe700 1 -- 172.21.15.1:0/1262015033 learned_addr learned my addr 172.21.15.1:0/1262015033 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:25:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.585+0000 7f1bd7fff700 1 --2- 172.21.15.1:0/1262015033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dffce97a40 0x55dffceaad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.585+0000 7f1bdc807700 1 --2- 172.21.15.1:0/1262015033 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dffcea2bb0 0x55dffcea7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.586+0000 7f1bd77fe700 1 -- 172.21.15.1:0/1262015033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dffce97a40 msgr2=0x55dffceaad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.586+0000 7f1bd77fe700 1 --2- 172.21.15.1:0/1262015033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dffce97a40 0x55dffceaad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.586+0000 7f1bd77fe700 1 -- 172.21.15.1:0/1262015033 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dffcea2bb0 msgr2=0x55dffcea7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-04T03:25:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.586+0000 7f1bd77fe700 1 --2- 172.21.15.1:0/1262015033 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dffcea2bb0 0x55dffcea7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.586+0000 7f1bd77fe700 1 -- 172.21.15.1:0/1262015033 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dffcea82a0 con 0x55dffce98390 2024-05-04T03:25:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.586+0000 7f1bdc807700 1 --2- 172.21.15.1:0/1262015033 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dffcea2bb0 0x55dffcea7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:25:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.586+0000 7f1bd77fe700 1 --2- 172.21.15.1:0/1262015033 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dffce98390 0x55dffceab2d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009d10 tx=0x7f1bcc004ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5dfce2299362cd server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.586+0000 7f1bb77fe700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-05-04T03:25:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.586+0000 7f1bb77fe700 1 -- 172.21.15.1:0/1262015033 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc044020 con 0x55dffce98390 2024-05-04T03:25:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.586+0000 7f1bb77fe700 1 -- 172.21.15.1:0/1262015033 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f1bcc037480 con 0x55dffce98390 2024-05-04T03:25:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.586+0000 7f1becc7eb80 1 -- 172.21.15.1:0/1262015033 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bcc037e50 con 0x55dffce98390 2024-05-04T03:25:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.586+0000 7f1bb77fe700 1 -- 172.21.15.1:0/1262015033 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1bcc03c040 con 0x55dffce98390 2024-05-04T03:25:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.586+0000 7f1becc7eb80 10 client.15957 fetch_fsmap learned FSMap version 35 2024-05-04T03:25:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.586+0000 7f1becc7eb80 1 -- 172.21.15.1:0/1262015033 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55dffcf1da30 con 0x55dffce98390 2024-05-04T03:25:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.586+0000 7f1bb77fe700 1 -- 172.21.15.1:0/1262015033 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bcc037e50 con 0x55dffce98390 2024-05-04T03:25:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.586+0000 7f1bb77fe700 1 -- 172.21.15.1:0/1262015033 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 35) ==== 8350+0+0 (secure 0 0 0) 0x7f1bcc02ab00 con 0x55dffce98390 2024-05-04T03:25:25.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.587+0000 7f1becc7eb80 10 client.15957 fetch_fsmap finished waiting for FSMap version 35 2024-05-04T03:25:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.587+0000 7f1becc7eb80 10 client.15957 resolve_mds: validated gid 24494 aka daemon mds.b 2024-05-04T03:25:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.587+0000 7f1becc7eb80 20 client.15957 populate_metadata read hostname 'smithi001' 2024-05-04T03:25:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.587+0000 7f1becc7eb80 1 --2- 172.21.15.1:0/1262015033 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55dffcf1e0c0 0x55dffcf204e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.587+0000 7f1becc7eb80 4 client.15957 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-05-04T03:25:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.587+0000 7f1becc7eb80 1 -- 172.21.15.1:0/1262015033 --> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55dffcdc2430 con 0x55dffcf1e0c0 2024-05-04T03:25:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.587+0000 7f1bd7fff700 1 --2- 172.21.15.1:0/1262015033 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55dffcf1e0c0 0x55dffcf204e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.587+0000 7f1bb77fe700 10 client.15957 ms_handle_connect on v2:172.21.15.72:6838/2976050650 2024-05-04T03:25:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.587+0000 7f1bd7fff700 1 --2- 172.21.15.1:0/1262015033 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55dffcf1e0c0 0x55dffcf204e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5caa0b984d17c93b server_cookie=7570e552b7fecc0 in_seq=0 out_seq=0 2024-05-04T03:25:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.588+0000 7f1bb77fe700 1 -- 172.21.15.1:0/1262015033 <== mds.0 v2:172.21.15.72:6838/2976050650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55dffcdc2430 con 0x55dffcf1e0c0 2024-05-04T03:25:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.588+0000 7f1bb77fe700 10 client.15957 handle_command_reply: tid=0 2024-05-04T03:25:25.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.588+0000 7f1becc7eb80 1 client.15957 shutdown 2024-05-04T03:25:25.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.588+0000 7f1becc7eb80 1 -- 172.21.15.1:0/1262015033 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dffce98390 msgr2=0x55dffceab2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:25.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.588+0000 7f1becc7eb80 1 --2- 172.21.15.1:0/1262015033 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dffce98390 0x55dffceab2d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009d10 tx=0x7f1bcc004ec0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.588+0000 7f1becc7eb80 1 -- 172.21.15.1:0/1262015033 shutdown_connections 2024-05-04T03:25:25.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.588+0000 7f1becc7eb80 1 --2- 172.21.15.1:0/1262015033 >> [v2:172.21.15.72:6838/2976050650,v1:172.21.15.72:6839/2976050650] conn(0x55dffcf1e0c0 0x55dffcf204e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.588+0000 7f1becc7eb80 1 --2- 172.21.15.1:0/1262015033 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dffcea2bb0 0x55dffcea7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.588+0000 7f1becc7eb80 1 --2- 172.21.15.1:0/1262015033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55dffce97a40 0x55dffceaad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.588+0000 7f1bb77fe700 0 client.15957 ms_handle_reset on v2:172.21.15.72:6838/2976050650 2024-05-04T03:25:25.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.588+0000 7f1becc7eb80 1 --2- 172.21.15.1:0/1262015033 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dffce98390 0x55dffceab2d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.588+0000 7f1becc7eb80 1 -- 172.21.15.1:0/1262015033 >> 172.21.15.1:0/1262015033 conn(0x55dffcd1aeb0 msgr2=0x55dffce937b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:25.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.588+0000 7f1becc7eb80 1 -- 172.21.15.1:0/1262015033 shutdown_connections 2024-05-04T03:25:25.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.588+0000 7f1becc7eb80 1 -- 172.21.15.1:0/1262015033 wait complete. 2024-05-04T03:25:25.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.588+0000 7f1becc7eb80 20 client.15957 trim_cache size 0 max 16384 2024-05-04T03:25:25.596 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-04T03:25:25.596 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 35, 2024-05-04T03:25:25.596 INFO:teuthology.orchestra.run.smithi001.stdout: "leader": 24494, 2024-05-04T03:25:25.596 INFO:teuthology.orchestra.run.smithi001.stdout: "set_version": 16, 2024-05-04T03:25:25.596 INFO:teuthology.orchestra.run.smithi001.stdout: "sets": {} 2024-05-04T03:25:25.596 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-04T03:25:25.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.589+0000 7f1bde26a700 1 -- 172.21.15.1:0/1893370380 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1bc00b2900 msgr2=0x7f1bc00b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:25.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.589+0000 7f1bde26a700 1 --2- 172.21.15.1:0/1893370380 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1bc00b2900 0x7f1bc00b4e10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc000c00 tx=0x7f1bcc01a040 comp rx=0 tx=0).stop 2024-05-04T03:25:25.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.589+0000 7f1bde26a700 1 -- 172.21.15.1:0/1893370380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bd81034c0 msgr2=0x7f1bd819bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:25.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.589+0000 7f1bde26a700 1 --2- 172.21.15.1:0/1893370380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bd81034c0 0x7f1bd819bc90 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1bc800e9a0 tx=0x7f1bc800ecb0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.589+0000 7f1bde26a700 1 -- 172.21.15.1:0/1893370380 shutdown_connections 2024-05-04T03:25:25.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.589+0000 7f1bde26a700 1 --2- 172.21.15.1:0/1893370380 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1bd8108740 0x7f1bd819c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.589+0000 7f1bde26a700 1 --2- 172.21.15.1:0/1893370380 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f1bc00b2900 0x7f1bc00b4e10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.589+0000 7f1bde26a700 1 --2- 172.21.15.1:0/1893370380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1bd81034c0 0x7f1bd819bc90 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.589+0000 7f1bde26a700 1 --2- 172.21.15.1:0/1893370380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1bd8107df0 0x7f1bd819c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:25.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.589+0000 7f1bde26a700 1 -- 172.21.15.1:0/1893370380 >> 172.21.15.1:0/1893370380 conn(0x7f1bd80fedf0 msgr2=0x7f1bd81096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:25.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.589+0000 7f1bde26a700 1 -- 172.21.15.1:0/1893370380 shutdown_connections 2024-05-04T03:25:25.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:25.589+0000 7f1bde26a700 1 -- 172.21.15.1:0/1893370380 wait complete. 2024-05-04T03:25:25.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:25 smithi072 ceph-mon[33332]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:25:25.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:25 smithi077 ceph-mon[33531]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-05-04T03:25:26.565 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:26 smithi001 ceph-mon[29731]: pgmap v579: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 425 op/s 2024-05-04T03:25:26.603 INFO:tasks.quiescer:done joining Quiescers 2024-05-04T03:25:26.603 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-04T03:25:26.615 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi001.front.sepia.ceph.com 2024-05-04T03:25:26.625 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-04T03:25:26.659 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-04T03:25:26.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:26 smithi072 ceph-mon[33332]: pgmap v579: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 425 op/s 2024-05-04T03:25:26.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.788+0000 7ffb99d86640 1 -- 172.21.15.1:0/2477840245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb94103dd0 msgr2=0x7ffb941041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:26.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.788+0000 7ffb99d86640 1 --2- 172.21.15.1:0/2477840245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb94103dd0 0x7ffb941041e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ffb88009900 tx=0x7ffb8802a410 comp rx=0 tx=0).stop 2024-05-04T03:25:26.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.789+0000 7ffb99d86640 1 -- 172.21.15.1:0/2477840245 shutdown_connections 2024-05-04T03:25:26.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.789+0000 7ffb99d86640 1 --2- 172.21.15.1:0/2477840245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb94108ad0 0x7ffb9410aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:26.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.789+0000 7ffb99d86640 1 --2- 172.21.15.1:0/2477840245 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb94103dd0 0x7ffb941041e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:26.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.789+0000 7ffb99d86640 1 --2- 172.21.15.1:0/2477840245 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb94102b90 0x7ffb94103890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:26.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.789+0000 7ffb99d86640 1 -- 172.21.15.1:0/2477840245 >> 172.21.15.1:0/2477840245 conn(0x7ffb940fdda0 msgr2=0x7ffb94100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:26.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.789+0000 7ffb99d86640 1 -- 172.21.15.1:0/2477840245 shutdown_connections 2024-05-04T03:25:26.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.789+0000 7ffb99d86640 1 -- 172.21.15.1:0/2477840245 wait complete. 2024-05-04T03:25:26.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.789+0000 7ffb99d86640 1 Processor -- start 2024-05-04T03:25:26.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.790+0000 7ffb99d86640 1 -- start start 2024-05-04T03:25:26.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.790+0000 7ffb99d86640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb94102b90 0x7ffb9419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:26.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.790+0000 7ffb99d86640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb94103dd0 0x7ffb9419a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:26.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.790+0000 7ffb99d86640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb94108ad0 0x7ffb941a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:26.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.790+0000 7ffb99d86640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9410f740 con 0x7ffb94102b90 2024-05-04T03:25:26.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.790+0000 7ffb99d86640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9410f5c0 con 0x7ffb94103dd0 2024-05-04T03:25:26.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.790+0000 7ffb99d86640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9410f8c0 con 0x7ffb94108ad0 2024-05-04T03:25:26.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.790+0000 7ffb937fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb94102b90 0x7ffb9419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:26.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.790+0000 7ffb937fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb94102b90 0x7ffb9419a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40222/0 (socket says 172.21.15.1:40222) 2024-05-04T03:25:26.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.790+0000 7ffb937fe640 1 -- 172.21.15.1:0/198793766 learned_addr learned my addr 172.21.15.1:0/198793766 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:25:26.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.790+0000 7ffb92ffd640 1 --2- 172.21.15.1:0/198793766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb94103dd0 0x7ffb9419a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:26.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.790+0000 7ffb93fff640 1 --2- 172.21.15.1:0/198793766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb94108ad0 0x7ffb941a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:26.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.790+0000 7ffb937fe640 1 -- 172.21.15.1:0/198793766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb94108ad0 msgr2=0x7ffb941a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:26.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.790+0000 7ffb937fe640 1 --2- 172.21.15.1:0/198793766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb94108ad0 0x7ffb941a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:26.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.790+0000 7ffb937fe640 1 -- 172.21.15.1:0/198793766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb94103dd0 msgr2=0x7ffb9419a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:26.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.790+0000 7ffb937fe640 1 --2- 172.21.15.1:0/198793766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb94103dd0 0x7ffb9419a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:26.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.790+0000 7ffb937fe640 1 -- 172.21.15.1:0/198793766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb941a1da0 con 0x7ffb94102b90 2024-05-04T03:25:26.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.790+0000 7ffb937fe640 1 --2- 172.21.15.1:0/198793766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb94102b90 0x7ffb9419a2b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400d740 tx=0x7ffb8400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:26.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.791+0000 7ffb90ff9640 1 -- 172.21.15.1:0/198793766 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb84017070 con 0x7ffb94102b90 2024-05-04T03:25:26.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.791+0000 7ffb90ff9640 1 -- 172.21.15.1:0/198793766 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb8400de50 con 0x7ffb94102b90 2024-05-04T03:25:26.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.791+0000 7ffb99d86640 1 -- 172.21.15.1:0/198793766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb941a2090 con 0x7ffb94102b90 2024-05-04T03:25:26.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.791+0000 7ffb90ff9640 1 -- 172.21.15.1:0/198793766 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb84003c20 con 0x7ffb94102b90 2024-05-04T03:25:26.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.791+0000 7ffb99d86640 1 -- 172.21.15.1:0/198793766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb941a2630 con 0x7ffb94102b90 2024-05-04T03:25:26.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.792+0000 7ffb99d86640 1 -- 172.21.15.1:0/198793766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb941078e0 con 0x7ffb94102b90 2024-05-04T03:25:26.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.793+0000 7ffb90ff9640 1 -- 172.21.15.1:0/198793766 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7ffb84003dc0 con 0x7ffb94102b90 2024-05-04T03:25:26.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.794+0000 7ffb90ff9640 1 --2- 172.21.15.1:0/198793766 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ffb7c0b2920 0x7ffb7c0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:26.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.794+0000 7ffb92ffd640 1 --2- 172.21.15.1:0/198793766 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ffb7c0b2920 0x7ffb7c0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:26.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.794+0000 7ffb90ff9640 1 -- 172.21.15.1:0/198793766 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7ffb840e1e70 con 0x7ffb94102b90 2024-05-04T03:25:26.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.795+0000 7ffb92ffd640 1 --2- 172.21.15.1:0/198793766 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ffb7c0b2920 0x7ffb7c0b4e30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ffb88009230 tx=0x7ffb88031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:26.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.798+0000 7ffb90ff9640 1 -- 172.21.15.1:0/198793766 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffb840a9680 con 0x7ffb94102b90 2024-05-04T03:25:26.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:26 smithi077 ceph-mon[33531]: pgmap v579: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 425 op/s 2024-05-04T03:25:26.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.992+0000 7ffb99d86640 1 -- 172.21.15.1:0/198793766 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffb940638d0 con 0x7ffb94102b90 2024-05-04T03:25:26.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.994+0000 7ffb90ff9640 1 -- 172.21.15.1:0/198793766 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22380 (secure 0 0 0) 0x7ffb840ad980 con 0x7ffb94102b90 2024-05-04T03:25:26.996 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:25:26.996 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":83,"fsid":"4a572aa8-09c3-11ef-bc97-c7b262605968","created":"2024-05-04T03:07:32.845547+0000","modified":"2024-05-04T03:16:34.321580+0000","last_up_change":"2024-05-04T03:14:44.192571+0000","last_in_change":"2024-05-04T03:14:23.353342+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T03:11:01.139189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T03:15:43.151696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T03:15:46.753539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"100fe035-2454-4c95-90c7-759956ec36a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6803","nonce":3479011484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6805","nonce":3479011484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6809","nonce":3479011484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6807","nonce":3479011484}]},"public_addr":"172.21.15.1:6803/3479011484","cluster_addr":"172.21.15.1:6805/3479011484","heartbeat_back_addr":"172.21.15.1:6809/3479011484","heartbeat_front_addr":"172.21.15.1:6807/3479011484","state":["exists","up"]},{"osd":1,"uuid":"5826ac76-ef6b-4040-8ad1-305db355af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6801","nonce":3725706149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6803","nonce":3725706149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6807","nonce":3725706149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6805","nonce":3725706149}]},"public_addr":"172.21.15.72:6801/3725706149","cluster_addr":"172.21.15.72:6803/3725706149","heartbeat_back_addr":"172.21.15.72:6807/3725706149","heartbeat_front_addr":"172.21.15.72:6805/3725706149","state":["exists","up"]},{"osd":2,"uuid":"923d76e5-b032-4190-9e35-546d27722a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6801","nonce":515038343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6803","nonce":515038343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6807","nonce":515038343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6805","nonce":515038343}]},"public_addr":"172.21.15.77:6801/515038343","cluster_addr":"172.21.15.77:6803/515038343","heartbeat_back_addr":"172.21.15.77:6807/515038343","heartbeat_front_addr":"172.21.15.77:6805/515038343","state":["exists","up"]},{"osd":3,"uuid":"495cc0c5-6a99-481f-ad9e-38f05fb84356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6811","nonce":4050439365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6813","nonce":4050439365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6817","nonce":4050439365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6815","nonce":4050439365}]},"public_addr":"172.21.15.1:6811/4050439365","cluster_addr":"172.21.15.1:6813/4050439365","heartbeat_back_addr":"172.21.15.1:6817/4050439365","heartbeat_front_addr":"172.21.15.1:6815/4050439365","state":["exists","up"]},{"osd":4,"uuid":"b86b8026-79c0-4056-9e67-a61f54b6ab5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6809","nonce":2501732391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6811","nonce":2501732391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6815","nonce":2501732391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6813","nonce":2501732391}]},"public_addr":"172.21.15.72:6809/2501732391","cluster_addr":"172.21.15.72:6811/2501732391","heartbeat_back_addr":"172.21.15.72:6815/2501732391","heartbeat_front_addr":"172.21.15.72:6813/2501732391","state":["exists","up"]},{"osd":5,"uuid":"80975cb9-39e0-46b8-a343-9ff69194a14e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6809","nonce":3582852165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6811","nonce":3582852165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6815","nonce":3582852165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6813","nonce":3582852165}]},"public_addr":"172.21.15.77:6809/3582852165","cluster_addr":"172.21.15.77:6811/3582852165","heartbeat_back_addr":"172.21.15.77:6815/3582852165","heartbeat_front_addr":"172.21.15.77:6813/3582852165","state":["exists","up"]},{"osd":6,"uuid":"aca3fac5-fbb5-42de-bf0e-e4a0db7381e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6819","nonce":1799033649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6821","nonce":1799033649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6825","nonce":1799033649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6823","nonce":1799033649}]},"public_addr":"172.21.15.1:6819/1799033649","cluster_addr":"172.21.15.1:6821/1799033649","heartbeat_back_addr":"172.21.15.1:6825/1799033649","heartbeat_front_addr":"172.21.15.1:6823/1799033649","state":["exists","up"]},{"osd":7,"uuid":"cbff6b8a-c463-4249-ae45-fdc6f99a5489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6817","nonce":3460835447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6819","nonce":3460835447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6823","nonce":3460835447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6821","nonce":3460835447}]},"public_addr":"172.21.15.72:6817/3460835447","cluster_addr":"172.21.15.72:6819/3460835447","heartbeat_back_addr":"172.21.15.72:6823/3460835447","heartbeat_front_addr":"172.21.15.72:6821/3460835447","state":["exists","up"]},{"osd":8,"uuid":"e140bf13-4887-4d68-9373-3d4fc1c4584a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6817","nonce":1124479323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6819","nonce":1124479323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6823","nonce":1124479323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6821","nonce":1124479323}]},"public_addr":"172.21.15.77:6817/1124479323","cluster_addr":"172.21.15.77:6819/1124479323","heartbeat_back_addr":"172.21.15.77:6823/1124479323","heartbeat_front_addr":"172.21.15.77:6821/1124479323","state":["exists","up"]},{"osd":9,"uuid":"f1e8a232-cb11-4b9d-8789-e1637fa5aaa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6827","nonce":2681322272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6829","nonce":2681322272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6833","nonce":2681322272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6831","nonce":2681322272}]},"public_addr":"172.21.15.1:6827/2681322272","cluster_addr":"172.21.15.1:6829/2681322272","heartbeat_back_addr":"172.21.15.1:6833/2681322272","heartbeat_front_addr":"172.21.15.1:6831/2681322272","state":["exists","up"]},{"osd":10,"uuid":"79b570af-3d1e-439d-a328-9e6bc7d67839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6825","nonce":3293703569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6827","nonce":3293703569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6831","nonce":3293703569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6829","nonce":3293703569}]},"public_addr":"172.21.15.72:6825/3293703569","cluster_addr":"172.21.15.72:6827/3293703569","heartbeat_back_addr":"172.21.15.72:6831/3293703569","heartbeat_front_addr":"172.21.15.72:6829/3293703569","state":["exists","up"]},{"osd":11,"uuid":"e35bf83d-9ee4-4745-96c2-1c35cc8eefba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6825","nonce":4155947548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6827","nonce":4155947548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6831","nonce":4155947548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6829","nonce":4155947548}]},"public_addr":"172.21.15.77:6825/4155947548","cluster_addr":"172.21.15.77:6827/4155947548","heartbeat_back_addr":"172.21.15.77:6831/4155947548","heartbeat_front_addr":"172.21.15.77:6829/4155947548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:09.467126+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:33.861139+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:57.741962+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:11:21.746021+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:11:46.323252+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:12:10.640180+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:12:35.750591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:00.306549+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:25.012495+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:51.185229+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:14:16.403850+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:14:41.653316+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/2029897106":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/402488450":"2024-05-05T03:08:33.045884+0000","172.21.15.1:6801/128793132":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/2233526606":"2024-05-05T03:08:06.135783+0000","172.21.15.1:0/2715541855":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/2415697325":"2024-05-05T03:08:06.135783+0000","172.21.15.1:0/2305298106":"2024-05-05T03:08:06.135783+0000","172.21.15.1:6800/128793132":"2024-05-05T03:08:33.045884+0000","172.21.15.1:6800/4200529863":"2024-05-05T03:08:06.135783+0000","172.21.15.1:6801/4200529863":"2024-05-05T03:08:06.135783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T03:25:26.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.996+0000 7ffb99d86640 1 -- 172.21.15.1:0/198793766 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ffb7c0b2920 msgr2=0x7ffb7c0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:26.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.996+0000 7ffb99d86640 1 --2- 172.21.15.1:0/198793766 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ffb7c0b2920 0x7ffb7c0b4e30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ffb88009230 tx=0x7ffb88031400 comp rx=0 tx=0).stop 2024-05-04T03:25:26.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.996+0000 7ffb99d86640 1 -- 172.21.15.1:0/198793766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb94102b90 msgr2=0x7ffb9419a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:26.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.996+0000 7ffb99d86640 1 --2- 172.21.15.1:0/198793766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb94102b90 0x7ffb9419a2b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400d740 tx=0x7ffb8400dc00 comp rx=0 tx=0).stop 2024-05-04T03:25:26.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.996+0000 7ffb99d86640 1 -- 172.21.15.1:0/198793766 shutdown_connections 2024-05-04T03:25:26.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.996+0000 7ffb99d86640 1 --2- 172.21.15.1:0/198793766 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7ffb7c0b2920 0x7ffb7c0b4e30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:26.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.996+0000 7ffb99d86640 1 --2- 172.21.15.1:0/198793766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ffb94103dd0 0x7ffb9419a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:26.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.996+0000 7ffb99d86640 1 --2- 172.21.15.1:0/198793766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb94108ad0 0x7ffb941a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:26.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.996+0000 7ffb99d86640 1 --2- 172.21.15.1:0/198793766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb94102b90 0x7ffb9419a2b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:26.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.996+0000 7ffb99d86640 1 -- 172.21.15.1:0/198793766 >> 172.21.15.1:0/198793766 conn(0x7ffb940fdda0 msgr2=0x7ffb94104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:26.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.996+0000 7ffb99d86640 1 -- 172.21.15.1:0/198793766 shutdown_connections 2024-05-04T03:25:26.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:26.996+0000 7ffb99d86640 1 -- 172.21.15.1:0/198793766 wait complete. 2024-05-04T03:25:27.011 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-04T03:11:01.139189+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-04T03:15:43.151696+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-04T03:15:46.753539+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-04T03:25:27.011 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-04T03:25:27.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.124+0000 7fad1a6f9640 1 -- 172.21.15.1:0/926451393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad14107200 msgr2=0x7fad14109620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:27.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.124+0000 7fad1a6f9640 1 --2- 172.21.15.1:0/926451393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad14107200 0x7fad14109620 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fad08009900 tx=0x7fad0802a410 comp rx=0 tx=0).stop 2024-05-04T03:25:27.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.124+0000 7fad1a6f9640 1 -- 172.21.15.1:0/926451393 shutdown_connections 2024-05-04T03:25:27.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.124+0000 7fad1a6f9640 1 --2- 172.21.15.1:0/926451393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad14109b60 0x7fad1410bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:27.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.124+0000 7fad1a6f9640 1 --2- 172.21.15.1:0/926451393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad14107200 0x7fad14109620 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:27.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.124+0000 7fad1a6f9640 1 --2- 172.21.15.1:0/926451393 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad14102050 0x7fad14106cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:27.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.124+0000 7fad1a6f9640 1 -- 172.21.15.1:0/926451393 >> 172.21.15.1:0/926451393 conn(0x7fad140fd4b0 msgr2=0x7fad140ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:27.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.124+0000 7fad1a6f9640 1 -- 172.21.15.1:0/926451393 shutdown_connections 2024-05-04T03:25:27.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.124+0000 7fad1a6f9640 1 -- 172.21.15.1:0/926451393 wait complete. 2024-05-04T03:25:27.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.125+0000 7fad1a6f9640 1 Processor -- start 2024-05-04T03:25:27.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.125+0000 7fad1a6f9640 1 -- start start 2024-05-04T03:25:27.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.125+0000 7fad1a6f9640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad14102050 0x7fad1419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:27.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.125+0000 7fad1a6f9640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad14107200 0x7fad1419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:27.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.125+0000 7fad1a6f9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad14109b60 0x7fad141a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:27.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.125+0000 7fad1a6f9640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fad1410f860 con 0x7fad14107200 2024-05-04T03:25:27.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.125+0000 7fad1a6f9640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fad1410f6e0 con 0x7fad14102050 2024-05-04T03:25:27.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.125+0000 7fad1a6f9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fad1410f9e0 con 0x7fad14109b60 2024-05-04T03:25:27.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.125+0000 7fad137fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad14107200 0x7fad1419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:27.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.125+0000 7fad137fe640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad14107200 0x7fad1419a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40238/0 (socket says 172.21.15.1:40238) 2024-05-04T03:25:27.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.125+0000 7fad137fe640 1 -- 172.21.15.1:0/2863787629 learned_addr learned my addr 172.21.15.1:0/2863787629 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:25:27.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.126+0000 7fad13fff640 1 --2- 172.21.15.1:0/2863787629 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad14102050 0x7fad1419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:27.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.126+0000 7fad18c96640 1 --2- 172.21.15.1:0/2863787629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad14109b60 0x7fad141a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:27.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.126+0000 7fad137fe640 1 -- 172.21.15.1:0/2863787629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad14109b60 msgr2=0x7fad141a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:27.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.126+0000 7fad137fe640 1 --2- 172.21.15.1:0/2863787629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad14109b60 0x7fad141a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:27.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.126+0000 7fad137fe640 1 -- 172.21.15.1:0/2863787629 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad14102050 msgr2=0x7fad1419a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:27.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.126+0000 7fad137fe640 1 --2- 172.21.15.1:0/2863787629 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad14102050 0x7fad1419a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:27.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.126+0000 7fad137fe640 1 -- 172.21.15.1:0/2863787629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad141a1f20 con 0x7fad14107200 2024-05-04T03:25:27.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.126+0000 7fad13fff640 1 --2- 172.21.15.1:0/2863787629 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad14102050 0x7fad1419a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:25:27.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.126+0000 7fad137fe640 1 --2- 172.21.15.1:0/2863787629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad14107200 0x7fad1419a920 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fad08006010 tx=0x7fad08004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:27.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.126+0000 7fad117fa640 1 -- 172.21.15.1:0/2863787629 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0803f030 con 0x7fad14107200 2024-05-04T03:25:27.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.126+0000 7fad117fa640 1 -- 172.21.15.1:0/2863787629 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad08003cb0 con 0x7fad14107200 2024-05-04T03:25:27.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.126+0000 7fad1a6f9640 1 -- 172.21.15.1:0/2863787629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad141a21b0 con 0x7fad14107200 2024-05-04T03:25:27.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.126+0000 7fad117fa640 1 -- 172.21.15.1:0/2863787629 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad08035070 con 0x7fad14107200 2024-05-04T03:25:27.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.126+0000 7fad1a6f9640 1 -- 172.21.15.1:0/2863787629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad141a26f0 con 0x7fad14107200 2024-05-04T03:25:27.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.127+0000 7fad1a6f9640 1 -- 172.21.15.1:0/2863787629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad140fefd0 con 0x7fad14107200 2024-05-04T03:25:27.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.129+0000 7fad117fa640 1 -- 172.21.15.1:0/2863787629 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7fad08042030 con 0x7fad14107200 2024-05-04T03:25:27.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.129+0000 7fad117fa640 1 --2- 172.21.15.1:0/2863787629 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7facfc0b2870 0x7facfc0b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:27.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.130+0000 7fad117fa640 1 -- 172.21.15.1:0/2863787629 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fad08102ed0 con 0x7fad14107200 2024-05-04T03:25:27.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.130+0000 7fad13fff640 1 --2- 172.21.15.1:0/2863787629 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7facfc0b2870 0x7facfc0b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:27.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.130+0000 7fad13fff640 1 --2- 172.21.15.1:0/2863787629 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7facfc0b2870 0x7facfc0b4d80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fad04005d00 tx=0x7fad04005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:27.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.133+0000 7fad117fa640 1 -- 172.21.15.1:0/2863787629 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fad080cacb0 con 0x7fad14107200 2024-05-04T03:25:27.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.324+0000 7fad1a6f9640 1 -- 172.21.15.1:0/2863787629 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fad1402ddc0 con 0x7fad14107200 2024-05-04T03:25:27.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.325+0000 7fad117fa640 1 -- 172.21.15.1:0/2863787629 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7fad1402ddc0 con 0x7fad14107200 2024-05-04T03:25:27.327 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-05-04T03:25:27.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.327+0000 7fad1a6f9640 1 -- 172.21.15.1:0/2863787629 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7facfc0b2870 msgr2=0x7facfc0b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:27.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.327+0000 7fad1a6f9640 1 --2- 172.21.15.1:0/2863787629 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7facfc0b2870 0x7facfc0b4d80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fad04005d00 tx=0x7fad04005c70 comp rx=0 tx=0).stop 2024-05-04T03:25:27.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.327+0000 7fad1a6f9640 1 -- 172.21.15.1:0/2863787629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad14107200 msgr2=0x7fad1419a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:27.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.327+0000 7fad1a6f9640 1 --2- 172.21.15.1:0/2863787629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad14107200 0x7fad1419a920 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fad08006010 tx=0x7fad08004340 comp rx=0 tx=0).stop 2024-05-04T03:25:27.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.327+0000 7fad1a6f9640 1 -- 172.21.15.1:0/2863787629 shutdown_connections 2024-05-04T03:25:27.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.327+0000 7fad1a6f9640 1 --2- 172.21.15.1:0/2863787629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fad14109b60 0x7fad141a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:27.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.327+0000 7fad1a6f9640 1 --2- 172.21.15.1:0/2863787629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad14107200 0x7fad1419a920 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:27.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.327+0000 7fad1a6f9640 1 --2- 172.21.15.1:0/2863787629 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7facfc0b2870 0x7facfc0b4d80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:27.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.327+0000 7fad1a6f9640 1 --2- 172.21.15.1:0/2863787629 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad14102050 0x7fad1419a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:27.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.327+0000 7fad1a6f9640 1 -- 172.21.15.1:0/2863787629 >> 172.21.15.1:0/2863787629 conn(0x7fad140fd4b0 msgr2=0x7fad141056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:27.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.327+0000 7fad1a6f9640 1 -- 172.21.15.1:0/2863787629 shutdown_connections 2024-05-04T03:25:27.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.327+0000 7fad1a6f9640 1 -- 172.21.15.1:0/2863787629 wait complete. 2024-05-04T03:25:27.341 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-04T03:25:27.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.453+0000 7f3f46fba640 1 -- 172.21.15.1:0/2400016302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f40104ea0 msgr2=0x7f3f401052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:27.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.453+0000 7f3f46fba640 1 --2- 172.21.15.1:0/2400016302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f40104ea0 0x7f3f401052b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3f30009900 tx=0x7f3f3002a410 comp rx=0 tx=0).stop 2024-05-04T03:25:27.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.453+0000 7f3f46fba640 1 -- 172.21.15.1:0/2400016302 shutdown_connections 2024-05-04T03:25:27.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.453+0000 7f3f46fba640 1 --2- 172.21.15.1:0/2400016302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f40109ba0 0x7f3f4010bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:27.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.453+0000 7f3f46fba640 1 --2- 172.21.15.1:0/2400016302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f40104ea0 0x7f3f401052b0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:27.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.453+0000 7f3f46fba640 1 --2- 172.21.15.1:0/2400016302 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f40103c60 0x7f3f40104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:27.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.453+0000 7f3f46fba640 1 -- 172.21.15.1:0/2400016302 >> 172.21.15.1:0/2400016302 conn(0x7f3f400fee70 msgr2=0x7f3f40101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:27.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.453+0000 7f3f46fba640 1 -- 172.21.15.1:0/2400016302 shutdown_connections 2024-05-04T03:25:27.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.453+0000 7f3f46fba640 1 -- 172.21.15.1:0/2400016302 wait complete. 2024-05-04T03:25:27.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.454+0000 7f3f46fba640 1 Processor -- start 2024-05-04T03:25:27.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.454+0000 7f3f46fba640 1 -- start start 2024-05-04T03:25:27.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.454+0000 7f3f46fba640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f40103c60 0x7f3f4019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:27.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.454+0000 7f3f46fba640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f40104ea0 0x7f3f4019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:27.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.454+0000 7f3f46fba640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f40109ba0 0x7f3f401a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:27.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.454+0000 7f3f46fba640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4010f7e0 con 0x7f3f40103c60 2024-05-04T03:25:27.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.454+0000 7f3f46fba640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4010f660 con 0x7f3f40104ea0 2024-05-04T03:25:27.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.454+0000 7f3f46fba640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4010f960 con 0x7f3f40109ba0 2024-05-04T03:25:27.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.454+0000 7f3f44d56640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f40103c60 0x7f3f4019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:27.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.454+0000 7f3f44d56640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f40103c60 0x7f3f4019a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40258/0 (socket says 172.21.15.1:40258) 2024-05-04T03:25:27.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.454+0000 7f3f44d56640 1 -- 172.21.15.1:0/4228828802 learned_addr learned my addr 172.21.15.1:0/4228828802 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:25:27.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.454+0000 7f3f37fff640 1 --2- 172.21.15.1:0/4228828802 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f40104ea0 0x7f3f4019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:27.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.455+0000 7f3f45557640 1 --2- 172.21.15.1:0/4228828802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f40109ba0 0x7f3f401a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:27.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.455+0000 7f3f44d56640 1 -- 172.21.15.1:0/4228828802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f40109ba0 msgr2=0x7f3f401a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:27.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.455+0000 7f3f44d56640 1 --2- 172.21.15.1:0/4228828802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f40109ba0 0x7f3f401a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:27.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.455+0000 7f3f44d56640 1 -- 172.21.15.1:0/4228828802 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f40104ea0 msgr2=0x7f3f4019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:27.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.455+0000 7f3f44d56640 1 --2- 172.21.15.1:0/4228828802 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f40104ea0 0x7f3f4019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:27.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.455+0000 7f3f44d56640 1 -- 172.21.15.1:0/4228828802 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f401a1e40 con 0x7f3f40103c60 2024-05-04T03:25:27.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.455+0000 7f3f44d56640 1 --2- 172.21.15.1:0/4228828802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f40103c60 0x7f3f4019a3b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00c970 tx=0x7f3f3c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:27.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.455+0000 7f3f35ffb640 1 -- 172.21.15.1:0/4228828802 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c007c70 con 0x7f3f40103c60 2024-05-04T03:25:27.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.455+0000 7f3f35ffb640 1 -- 172.21.15.1:0/4228828802 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f3c007e10 con 0x7f3f40103c60 2024-05-04T03:25:27.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.455+0000 7f3f46fba640 1 -- 172.21.15.1:0/4228828802 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f401a2130 con 0x7f3f40103c60 2024-05-04T03:25:27.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.455+0000 7f3f35ffb640 1 -- 172.21.15.1:0/4228828802 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c0187b0 con 0x7f3f40103c60 2024-05-04T03:25:27.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.455+0000 7f3f46fba640 1 -- 172.21.15.1:0/4228828802 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f401a2700 con 0x7f3f40103c60 2024-05-04T03:25:27.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.456+0000 7f3f46fba640 1 -- 172.21.15.1:0/4228828802 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f401089b0 con 0x7f3f40103c60 2024-05-04T03:25:27.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.458+0000 7f3f35ffb640 1 -- 172.21.15.1:0/4228828802 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f3f3c00fab0 con 0x7f3f40103c60 2024-05-04T03:25:27.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.458+0000 7f3f35ffb640 1 --2- 172.21.15.1:0/4228828802 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f3f280b27f0 0x7f3f280b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:27.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.458+0000 7f3f37fff640 1 --2- 172.21.15.1:0/4228828802 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f3f280b27f0 0x7f3f280b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:27.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.458+0000 7f3f35ffb640 1 -- 172.21.15.1:0/4228828802 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f3f3c0e2290 con 0x7f3f40103c60 2024-05-04T03:25:27.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.459+0000 7f3f37fff640 1 --2- 172.21.15.1:0/4228828802 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f3f280b27f0 0x7f3f280b4d00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3f30000c00 tx=0x7f3f30031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:27.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.463+0000 7f3f35ffb640 1 -- 172.21.15.1:0/4228828802 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3f3c014030 con 0x7f3f40103c60 2024-05-04T03:25:27.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:27 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/198793766' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:25:27.629 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:27 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2863787629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-04T03:25:27.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.653+0000 7f3f46fba640 1 -- 172.21.15.1:0/4228828802 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3f4002de50 con 0x7f3f40103c60 2024-05-04T03:25:27.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.654+0000 7f3f35ffb640 1 -- 172.21.15.1:0/4228828802 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) ==== 104+0+11 (secure 0 0 0) 0x7f3f3c0aa070 con 0x7f3f40103c60 2024-05-04T03:25:27.656 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-05-04T03:25:27.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.656+0000 7f3f46fba640 1 -- 172.21.15.1:0/4228828802 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f3f280b27f0 msgr2=0x7f3f280b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:27.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.656+0000 7f3f46fba640 1 --2- 172.21.15.1:0/4228828802 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f3f280b27f0 0x7f3f280b4d00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3f30000c00 tx=0x7f3f30031400 comp rx=0 tx=0).stop 2024-05-04T03:25:27.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.656+0000 7f3f46fba640 1 -- 172.21.15.1:0/4228828802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f40103c60 msgr2=0x7f3f4019a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:27.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.656+0000 7f3f46fba640 1 --2- 172.21.15.1:0/4228828802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f40103c60 0x7f3f4019a3b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00c970 tx=0x7f3f3c00ce30 comp rx=0 tx=0).stop 2024-05-04T03:25:27.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.656+0000 7f3f46fba640 1 -- 172.21.15.1:0/4228828802 shutdown_connections 2024-05-04T03:25:27.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.656+0000 7f3f46fba640 1 --2- 172.21.15.1:0/4228828802 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f3f280b27f0 0x7f3f280b4d00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:27.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.656+0000 7f3f46fba640 1 --2- 172.21.15.1:0/4228828802 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3f40104ea0 0x7f3f4019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:27.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.656+0000 7f3f46fba640 1 --2- 172.21.15.1:0/4228828802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f40109ba0 0x7f3f401a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:27.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.656+0000 7f3f46fba640 1 --2- 172.21.15.1:0/4228828802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f40103c60 0x7f3f4019a3b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:27.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.656+0000 7f3f46fba640 1 -- 172.21.15.1:0/4228828802 >> 172.21.15.1:0/4228828802 conn(0x7f3f400fee70 msgr2=0x7f3f40105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:27.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.656+0000 7f3f46fba640 1 -- 172.21.15.1:0/4228828802 shutdown_connections 2024-05-04T03:25:27.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.656+0000 7f3f46fba640 1 -- 172.21.15.1:0/4228828802 wait complete. 2024-05-04T03:25:27.670 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-04T03:25:27.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:27 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/198793766' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:25:27.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:27 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/2863787629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-04T03:25:27.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.783+0000 7f17e3043640 1 -- 172.21.15.1:0/3757690999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17dc103960 msgr2=0x7f17dc103d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:27.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.783+0000 7f17e3043640 1 --2- 172.21.15.1:0/3757690999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17dc103960 0x7f17dc103d70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f17d0009900 tx=0x7f17d002a410 comp rx=0 tx=0).stop 2024-05-04T03:25:27.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.783+0000 7f17e3043640 1 -- 172.21.15.1:0/3757690999 shutdown_connections 2024-05-04T03:25:27.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.783+0000 7f17e3043640 1 --2- 172.21.15.1:0/3757690999 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f17dc109ba0 0x7f17dc10bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:27.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.783+0000 7f17e3043640 1 --2- 172.21.15.1:0/3757690999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17dc103960 0x7f17dc103d70 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:27.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.783+0000 7f17e3043640 1 --2- 172.21.15.1:0/3757690999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17dc1052a0 0x7f17dc103420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:27.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.783+0000 7f17e3043640 1 -- 172.21.15.1:0/3757690999 >> 172.21.15.1:0/3757690999 conn(0x7f17dc0fee70 msgr2=0x7f17dc101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:27.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.783+0000 7f17e3043640 1 -- 172.21.15.1:0/3757690999 shutdown_connections 2024-05-04T03:25:27.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.783+0000 7f17e3043640 1 -- 172.21.15.1:0/3757690999 wait complete. 2024-05-04T03:25:27.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.784+0000 7f17e3043640 1 Processor -- start 2024-05-04T03:25:27.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.784+0000 7f17e3043640 1 -- start start 2024-05-04T03:25:27.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.784+0000 7f17e3043640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f17dc103960 0x7f17dc077a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:27.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.784+0000 7f17e3043640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17dc1052a0 0x7f17dc076130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:27.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.784+0000 7f17e3043640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17dc109ba0 0x7f17dc076670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:27.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.784+0000 7f17e3043640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc10f8a0 con 0x7f17dc1052a0 2024-05-04T03:25:27.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.784+0000 7f17e3043640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc10f720 con 0x7f17dc103960 2024-05-04T03:25:27.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.784+0000 7f17e3043640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc10fa20 con 0x7f17dc109ba0 2024-05-04T03:25:27.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.784+0000 7f17dbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17dc1052a0 0x7f17dc076130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:27.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.784+0000 7f17dbfff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17dc1052a0 0x7f17dc076130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40278/0 (socket says 172.21.15.1:40278) 2024-05-04T03:25:27.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.784+0000 7f17dbfff640 1 -- 172.21.15.1:0/1593336909 learned_addr learned my addr 172.21.15.1:0/1593336909 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:25:27.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.784+0000 7f17e0ddf640 1 --2- 172.21.15.1:0/1593336909 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f17dc103960 0x7f17dc077a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:27.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.784+0000 7f17e15e0640 1 --2- 172.21.15.1:0/1593336909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17dc109ba0 0x7f17dc076670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:27.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.785+0000 7f17dbfff640 1 -- 172.21.15.1:0/1593336909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17dc109ba0 msgr2=0x7f17dc076670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:27.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.785+0000 7f17dbfff640 1 --2- 172.21.15.1:0/1593336909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17dc109ba0 0x7f17dc076670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:27.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.785+0000 7f17dbfff640 1 -- 172.21.15.1:0/1593336909 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f17dc103960 msgr2=0x7f17dc077a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:27.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.785+0000 7f17dbfff640 1 --2- 172.21.15.1:0/1593336909 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f17dc103960 0x7f17dc077a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:27.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.785+0000 7f17dbfff640 1 -- 172.21.15.1:0/1593336909 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17dc076da0 con 0x7f17dc1052a0 2024-05-04T03:25:27.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.785+0000 7f17dbfff640 1 --2- 172.21.15.1:0/1593336909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17dc1052a0 0x7f17dc076130 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f17d0006010 tx=0x7f17d0004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:27.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.785+0000 7f17d9ffb640 1 -- 172.21.15.1:0/1593336909 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d003f030 con 0x7f17dc1052a0 2024-05-04T03:25:27.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.785+0000 7f17d9ffb640 1 -- 172.21.15.1:0/1593336909 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17d0003cb0 con 0x7f17dc1052a0 2024-05-04T03:25:27.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.785+0000 7f17e3043640 1 -- 172.21.15.1:0/1593336909 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17dc1a4510 con 0x7f17dc1052a0 2024-05-04T03:25:27.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.785+0000 7f17d9ffb640 1 -- 172.21.15.1:0/1593336909 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d0035070 con 0x7f17dc1052a0 2024-05-04T03:25:27.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.785+0000 7f17e3043640 1 -- 172.21.15.1:0/1593336909 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17dc1a4970 con 0x7f17dc1052a0 2024-05-04T03:25:27.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.786+0000 7f17e3043640 1 -- 172.21.15.1:0/1593336909 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17dc06a340 con 0x7f17dc1052a0 2024-05-04T03:25:27.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.788+0000 7f17d9ffb640 1 -- 172.21.15.1:0/1593336909 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f17d0042030 con 0x7f17dc1052a0 2024-05-04T03:25:27.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.789+0000 7f17d9ffb640 1 --2- 172.21.15.1:0/1593336909 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f17c40b2870 0x7f17c40b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:27.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.789+0000 7f17d9ffb640 1 -- 172.21.15.1:0/1593336909 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f17d0102d60 con 0x7f17dc1052a0 2024-05-04T03:25:27.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.789+0000 7f17e0ddf640 1 --2- 172.21.15.1:0/1593336909 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f17c40b2870 0x7f17c40b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:27.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.789+0000 7f17e0ddf640 1 --2- 172.21.15.1:0/1593336909 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f17c40b2870 0x7f17c40b4d80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f17d4005c40 tx=0x7f17d400a590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:27.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.792+0000 7f17d9ffb640 1 -- 172.21.15.1:0/1593336909 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f17d0003e60 con 0x7f17dc1052a0 2024-05-04T03:25:27.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:27 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/198793766' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:25:27.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:27 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/2863787629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-04T03:25:27.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.982+0000 7f17e3043640 1 -- 172.21.15.1:0/1593336909 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f17dc0638d0 con 0x7f17dc1052a0 2024-05-04T03:25:27.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.982+0000 7f17d9ffb640 1 -- 172.21.15.1:0/1593336909 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) ==== 100+0+11 (secure 0 0 0) 0x7f17d00ca450 con 0x7f17dc1052a0 2024-05-04T03:25:27.985 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-05-04T03:25:27.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.985+0000 7f17e3043640 1 -- 172.21.15.1:0/1593336909 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f17c40b2870 msgr2=0x7f17c40b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:27.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.985+0000 7f17e3043640 1 --2- 172.21.15.1:0/1593336909 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f17c40b2870 0x7f17c40b4d80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f17d4005c40 tx=0x7f17d400a590 comp rx=0 tx=0).stop 2024-05-04T03:25:27.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.985+0000 7f17e3043640 1 -- 172.21.15.1:0/1593336909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17dc1052a0 msgr2=0x7f17dc076130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:27.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.985+0000 7f17e3043640 1 --2- 172.21.15.1:0/1593336909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17dc1052a0 0x7f17dc076130 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f17d0006010 tx=0x7f17d0004340 comp rx=0 tx=0).stop 2024-05-04T03:25:27.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.985+0000 7f17e3043640 1 -- 172.21.15.1:0/1593336909 shutdown_connections 2024-05-04T03:25:27.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.985+0000 7f17e3043640 1 --2- 172.21.15.1:0/1593336909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17dc109ba0 0x7f17dc076670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:27.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.985+0000 7f17e3043640 1 --2- 172.21.15.1:0/1593336909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f17dc1052a0 0x7f17dc076130 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:27.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.985+0000 7f17e3043640 1 --2- 172.21.15.1:0/1593336909 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f17c40b2870 0x7f17c40b4d80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:27.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.985+0000 7f17e3043640 1 --2- 172.21.15.1:0/1593336909 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f17dc103960 0x7f17dc077a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:27.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.985+0000 7f17e3043640 1 -- 172.21.15.1:0/1593336909 >> 172.21.15.1:0/1593336909 conn(0x7f17dc0fee70 msgr2=0x7f17dc105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:27.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.985+0000 7f17e3043640 1 -- 172.21.15.1:0/1593336909 shutdown_connections 2024-05-04T03:25:27.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:27.985+0000 7f17e3043640 1 -- 172.21.15.1:0/1593336909 wait complete. 2024-05-04T03:25:27.999 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-04T03:25:28.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.109+0000 7f32753ce640 1 -- 172.21.15.1:0/3014610643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3270103d70 msgr2=0x7f3270104180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:28.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.109+0000 7f32753ce640 1 --2- 172.21.15.1:0/3014610643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3270103d70 0x7f3270104180 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f3258009900 tx=0x7f325802a410 comp rx=0 tx=0).stop 2024-05-04T03:25:28.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.109+0000 7f32753ce640 1 -- 172.21.15.1:0/3014610643 shutdown_connections 2024-05-04T03:25:28.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.109+0000 7f32753ce640 1 --2- 172.21.15.1:0/3014610643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3270108a70 0x7f327010ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:28.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.109+0000 7f32753ce640 1 --2- 172.21.15.1:0/3014610643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3270103d70 0x7f3270104180 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:28.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.109+0000 7f32753ce640 1 --2- 172.21.15.1:0/3014610643 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3270102b30 0x7f3270103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:28.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.109+0000 7f32753ce640 1 -- 172.21.15.1:0/3014610643 >> 172.21.15.1:0/3014610643 conn(0x7f32700fdd20 msgr2=0x7f32701001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.109+0000 7f32753ce640 1 -- 172.21.15.1:0/3014610643 shutdown_connections 2024-05-04T03:25:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.109+0000 7f32753ce640 1 -- 172.21.15.1:0/3014610643 wait complete. 2024-05-04T03:25:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.110+0000 7f32753ce640 1 Processor -- start 2024-05-04T03:25:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.110+0000 7f32753ce640 1 -- start start 2024-05-04T03:25:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.110+0000 7f32753ce640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3270102b30 0x7f327019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.110+0000 7f32753ce640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3270103d70 0x7f327019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.110+0000 7f32753ce640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3270108a70 0x7f32701a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.110+0000 7f32753ce640 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f327010e630 con 0x7f3270103d70 2024-05-04T03:25:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.110+0000 7f32753ce640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f327010e4b0 con 0x7f3270102b30 2024-05-04T03:25:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.110+0000 7f32753ce640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f327010e7b0 con 0x7f3270108a70 2024-05-04T03:25:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.110+0000 7f3267fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3270103d70 0x7f327019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.110+0000 7f3267fff640 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3270103d70 0x7f327019a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40294/0 (socket says 172.21.15.1:40294) 2024-05-04T03:25:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.110+0000 7f3267fff640 1 -- 172.21.15.1:0/1996677302 learned_addr learned my addr 172.21.15.1:0/1996677302 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:25:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.111+0000 7f326f7fe640 1 --2- 172.21.15.1:0/1996677302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3270108a70 0x7f32701a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:28.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.111+0000 7f326effd640 1 --2- 172.21.15.1:0/1996677302 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3270102b30 0x7f327019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:28.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.111+0000 7f3267fff640 1 -- 172.21.15.1:0/1996677302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3270108a70 msgr2=0x7f32701a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:28.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.111+0000 7f3267fff640 1 --2- 172.21.15.1:0/1996677302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3270108a70 0x7f32701a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:28.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.111+0000 7f3267fff640 1 -- 172.21.15.1:0/1996677302 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3270102b30 msgr2=0x7f327019a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:28.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.111+0000 7f3267fff640 1 --2- 172.21.15.1:0/1996677302 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3270102b30 0x7f327019a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:28.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.111+0000 7f3267fff640 1 -- 172.21.15.1:0/1996677302 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32701a1da0 con 0x7f3270103d70 2024-05-04T03:25:28.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.111+0000 7f326f7fe640 1 --2- 172.21.15.1:0/1996677302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3270108a70 0x7f32701a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:25:28.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.111+0000 7f326effd640 1 --2- 172.21.15.1:0/1996677302 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3270102b30 0x7f327019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:25:28.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.111+0000 7f3267fff640 1 --2- 172.21.15.1:0/1996677302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3270103d70 0x7f327019a7a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3258006010 tx=0x7f3258031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:28.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.111+0000 7f326cff9640 1 -- 172.21.15.1:0/1996677302 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325803f030 con 0x7f3270103d70 2024-05-04T03:25:28.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.111+0000 7f326cff9640 1 -- 172.21.15.1:0/1996677302 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3258003aa0 con 0x7f3270103d70 2024-05-04T03:25:28.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.111+0000 7f32753ce640 1 -- 172.21.15.1:0/1996677302 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32701a2030 con 0x7f3270103d70 2024-05-04T03:25:28.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.111+0000 7f326cff9640 1 -- 172.21.15.1:0/1996677302 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3258035070 con 0x7f3270103d70 2024-05-04T03:25:28.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.111+0000 7f32753ce640 1 -- 172.21.15.1:0/1996677302 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32701a2570 con 0x7f3270103d70 2024-05-04T03:25:28.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.112+0000 7f32753ce640 1 -- 172.21.15.1:0/1996677302 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32701a2890 con 0x7f3270103d70 2024-05-04T03:25:28.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.114+0000 7f326cff9640 1 -- 172.21.15.1:0/1996677302 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f3258042030 con 0x7f3270103d70 2024-05-04T03:25:28.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.115+0000 7f326cff9640 1 --2- 172.21.15.1:0/1996677302 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f325c0b2920 0x7f325c0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:25:28.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.115+0000 7f326effd640 1 --2- 172.21.15.1:0/1996677302 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f325c0b2920 0x7f325c0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:25:28.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.115+0000 7f326cff9640 1 -- 172.21.15.1:0/1996677302 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f3258102eb0 con 0x7f3270103d70 2024-05-04T03:25:28.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.115+0000 7f326effd640 1 --2- 172.21.15.1:0/1996677302 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f325c0b2920 0x7f325c0b4e30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3270103730 tx=0x7f3260005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:25:28.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.119+0000 7f326cff9640 1 -- 172.21.15.1:0/1996677302 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f32580cabf0 con 0x7f3270103d70 2024-05-04T03:25:28.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.360+0000 7f32753ce640 1 -- 172.21.15.1:0/1996677302 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f32701a2a60 con 0x7f3270103d70 2024-05-04T03:25:28.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.370+0000 7f326cff9640 1 -- 172.21.15.1:0/1996677302 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+815629 (secure 0 0 0) 0x7f32580ca4f0 con 0x7f3270103d70 2024-05-04T03:25:28.375 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:25:28.375 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-04T03:25:28.375 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 20, 2024-05-04T03:25:28.375 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.375 INFO:teuthology.orchestra.run.smithi001.stdout: "active_gid": 14150, 2024-05-04T03:25:28.375 INFO:teuthology.orchestra.run.smithi001.stdout: "active_name": "x", 2024-05-04T03:25:28.375 INFO:teuthology.orchestra.run.smithi001.stdout: "active_addrs": { 2024-05-04T03:25:28.375 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-04T03:25:28.375 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.375 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-04T03:25:28.375 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6800", 2024-05-04T03:25:28.375 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 402647351 2024-05-04T03:25:28.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.376 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-05-04T03:25:28.376 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6801", 2024-05-04T03:25:28.376 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 402647351 2024-05-04T03:25:28.376 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.376 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-04T03:25:28.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.376 INFO:teuthology.orchestra.run.smithi001.stdout: "active_addr": "172.21.15.1:6801/402647351", 2024-05-04T03:25:28.376 INFO:teuthology.orchestra.run.smithi001.stdout: "active_change": "2024-05-04T03:08:33.046021+0000", 2024-05-04T03:25:28.376 INFO:teuthology.orchestra.run.smithi001.stdout: "active_mgr_features": 4540701547738038271, 2024-05-04T03:25:28.376 INFO:teuthology.orchestra.run.smithi001.stdout: "available": true, 2024-05-04T03:25:28.376 INFO:teuthology.orchestra.run.smithi001.stdout: "standbys": [ 2024-05-04T03:25:28.376 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.377 INFO:teuthology.orchestra.run.smithi001.stdout: "gid": 24106, 2024-05-04T03:25:28.377 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "z", 2024-05-04T03:25:28.377 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_features": 4540701547738038271, 2024-05-04T03:25:28.377 INFO:teuthology.orchestra.run.smithi001.stdout: "available_modules": [ 2024-05-04T03:25:28.377 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.377 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "alerts", 2024-05-04T03:25:28.377 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.377 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.377 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.377 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-05-04T03:25:28.377 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-05-04T03:25:28.377 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.377 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.377 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.377 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-05-04T03:25:28.377 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.378 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.378 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.378 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How frequently to reexamine health status", 2024-05-04T03:25:28.378 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.378 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.378 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.378 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.378 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.378 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.378 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.378 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.378 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.378 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.379 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.379 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.379 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.379 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.379 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.379 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.379 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.379 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.379 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.379 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.379 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.379 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.379 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.379 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.380 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.380 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.380 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.380 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.380 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.380 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.380 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.380 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.380 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.380 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.380 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.380 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.380 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.380 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.381 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.381 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.381 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.381 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.381 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.381 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.381 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.381 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.381 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.381 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.381 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.381 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.381 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.381 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.381 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.381 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.381 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.382 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.382 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.382 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.382 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.382 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.382 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.382 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.382 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.382 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.382 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.382 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.382 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.382 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.383 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.383 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_destination": { 2024-05-04T03:25:28.383 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_destination", 2024-05-04T03:25:28.383 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.383 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.383 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.383 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.383 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.383 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.383 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.383 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email address to send alerts to", 2024-05-04T03:25:28.383 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.383 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.383 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.384 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_from_name": { 2024-05-04T03:25:28.384 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_from_name", 2024-05-04T03:25:28.384 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.384 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.384 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.384 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "Ceph", 2024-05-04T03:25:28.384 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.384 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.384 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.384 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email From: name", 2024-05-04T03:25:28.384 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.384 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.384 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.384 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_host": { 2024-05-04T03:25:28.385 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_host", 2024-05-04T03:25:28.385 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.385 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.385 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.385 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.385 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.385 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.385 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.385 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP server", 2024-05-04T03:25:28.385 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.385 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.385 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.385 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_password": { 2024-05-04T03:25:28.385 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_password", 2024-05-04T03:25:28.385 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.386 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.386 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.386 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.386 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.386 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.386 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.386 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Password to authenticate with", 2024-05-04T03:25:28.386 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.386 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.386 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.386 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_port": { 2024-05-04T03:25:28.386 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_port", 2024-05-04T03:25:28.386 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.386 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.387 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.387 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "465", 2024-05-04T03:25:28.387 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.387 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.387 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.387 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP port", 2024-05-04T03:25:28.387 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.387 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.387 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.387 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_sender": { 2024-05-04T03:25:28.387 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_sender", 2024-05-04T03:25:28.387 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.387 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.387 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.387 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.387 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.388 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.388 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.388 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP envelope sender", 2024-05-04T03:25:28.388 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.388 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.388 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.388 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_ssl": { 2024-05-04T03:25:28.388 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_ssl", 2024-05-04T03:25:28.388 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.388 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.388 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.388 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.388 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.388 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.388 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.389 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-04T03:25:28.389 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.389 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.389 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.389 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_user": { 2024-05-04T03:25:28.389 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_user", 2024-05-04T03:25:28.389 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.389 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.389 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.389 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.389 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.389 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.389 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.389 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "User to authenticate as", 2024-05-04T03:25:28.389 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.390 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.390 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.390 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.390 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.390 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.390 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.390 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.390 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.390 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.390 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.390 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.390 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.390 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.391 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.391 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.391 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.391 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.391 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.391 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "balancer", 2024-05-04T03:25:28.391 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.391 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.391 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.391 INFO:teuthology.orchestra.run.smithi001.stdout: "active": { 2024-05-04T03:25:28.391 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "active", 2024-05-04T03:25:28.391 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.391 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.391 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.392 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.392 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.392 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.392 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.392 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically balance PGs across cluster", 2024-05-04T03:25:28.392 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.392 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.392 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.392 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_time": { 2024-05-04T03:25:28.392 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_time", 2024-05-04T03:25:28.392 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.392 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.392 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.392 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0000", 2024-05-04T03:25:28.392 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.393 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.393 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.393 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "beginning time of day to automatically balance", 2024-05-04T03:25:28.393 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-04T03:25:28.393 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.393 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.393 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_weekday": { 2024-05-04T03:25:28.393 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_weekday", 2024-05-04T03:25:28.393 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-05-04T03:25:28.393 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.393 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.393 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.393 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-05-04T03:25:28.393 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-05-04T03:25:28.394 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.394 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-04T03:25:28.394 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-04T03:25:28.394 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.394 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.394 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_max_iterations": { 2024-05-04T03:25:28.394 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_max_iterations", 2024-05-04T03:25:28.394 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-05-04T03:25:28.394 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.394 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.394 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "25", 2024-05-04T03:25:28.394 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-05-04T03:25:28.394 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "250", 2024-05-04T03:25:28.395 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.395 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-04T03:25:28.395 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.395 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.395 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.395 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_metrics": { 2024-05-04T03:25:28.395 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_metrics", 2024-05-04T03:25:28.395 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.395 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.395 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.395 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "pgs,objects,bytes", 2024-05-04T03:25:28.395 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.395 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.395 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.395 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-04T03:25:28.395 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-04T03:25:28.396 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.396 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.396 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_step": { 2024-05-04T03:25:28.396 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_step", 2024-05-04T03:25:28.396 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-05-04T03:25:28.396 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.396 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.396 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.5", 2024-05-04T03:25:28.396 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0.001", 2024-05-04T03:25:28.396 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "0.999", 2024-05-04T03:25:28.396 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.396 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "aggressiveness of optimization", 2024-05-04T03:25:28.396 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-04T03:25:28.396 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.397 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.397 INFO:teuthology.orchestra.run.smithi001.stdout: "end_time": { 2024-05-04T03:25:28.397 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_time", 2024-05-04T03:25:28.397 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.397 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.397 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.397 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2359", 2024-05-04T03:25:28.397 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.397 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.397 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.397 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ending time of day to automatically balance", 2024-05-04T03:25:28.397 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-04T03:25:28.397 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.397 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.398 INFO:teuthology.orchestra.run.smithi001.stdout: "end_weekday": { 2024-05-04T03:25:28.398 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_weekday", 2024-05-04T03:25:28.398 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-05-04T03:25:28.398 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.398 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.398 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.398 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-05-04T03:25:28.398 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-05-04T03:25:28.398 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.398 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-04T03:25:28.398 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-04T03:25:28.398 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.398 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.399 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.399 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.399 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.399 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.399 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.399 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.399 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.399 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.399 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.399 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.399 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.399 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.399 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.399 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.399 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.400 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.400 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.400 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.400 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.400 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.400 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.400 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.400 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.400 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.400 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.400 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.400 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.400 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.401 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.401 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.401 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.401 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.401 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.401 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.401 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.401 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.401 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.401 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.401 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.401 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.401 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.401 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.401 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.402 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.402 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.402 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.402 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.402 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.402 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.402 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.402 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.402 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.402 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.402 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.402 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.402 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.403 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.403 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.403 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.403 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.403 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.403 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.403 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.403 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.403 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.403 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.403 INFO:teuthology.orchestra.run.smithi001.stdout: "min_score": { 2024-05-04T03:25:28.403 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_score", 2024-05-04T03:25:28.403 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-05-04T03:25:28.404 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.404 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.404 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.404 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.404 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.404 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.404 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-04T03:25:28.404 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.404 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.404 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.404 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-05-04T03:25:28.404 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-05-04T03:25:28.404 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.404 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.405 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.405 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "upmap", 2024-05-04T03:25:28.405 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.405 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.405 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.405 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-compat", 2024-05-04T03:25:28.405 INFO:teuthology.orchestra.run.smithi001.stdout: "none", 2024-05-04T03:25:28.405 INFO:teuthology.orchestra.run.smithi001.stdout: "read", 2024-05-04T03:25:28.405 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap", 2024-05-04T03:25:28.405 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap-read" 2024-05-04T03:25:28.405 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.405 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Balancer mode", 2024-05-04T03:25:28.405 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.405 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.406 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.406 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_ids": { 2024-05-04T03:25:28.406 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_ids", 2024-05-04T03:25:28.406 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.406 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.406 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.406 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.406 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.406 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.406 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.406 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-04T03:25:28.406 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.406 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.406 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.407 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-05-04T03:25:28.407 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-05-04T03:25:28.407 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.407 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.407 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.407 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-05-04T03:25:28.407 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.407 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.407 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.407 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-04T03:25:28.407 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.407 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.407 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.408 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.408 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.408 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.408 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.408 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.408 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.408 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.408 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.408 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.408 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.408 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.408 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.408 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.409 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_deviation": { 2024-05-04T03:25:28.409 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_deviation", 2024-05-04T03:25:28.409 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.409 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.409 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.409 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-05-04T03:25:28.409 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-05-04T03:25:28.409 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.409 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.409 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-04T03:25:28.409 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-04T03:25:28.409 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.409 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.409 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_optimizations": { 2024-05-04T03:25:28.410 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_optimizations", 2024-05-04T03:25:28.410 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-05-04T03:25:28.410 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.410 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.410 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-05-04T03:25:28.410 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.410 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.410 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.410 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-04T03:25:28.410 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.410 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.410 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.410 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.410 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.411 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm", 2024-05-04T03:25:28.411 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.411 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.411 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.411 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_down_multiplier": { 2024-05-04T03:25:28.411 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_down_multiplier", 2024-05-04T03:25:28.411 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-05-04T03:25:28.411 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.411 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.411 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-05-04T03:25:28.411 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.411 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.411 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-04T03:25:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_refresh_rate": { 2024-05-04T03:25:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_refresh_rate", 2024-05-04T03:25:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "20", 2024-05-04T03:25:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.412 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-04T03:25:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_starting_port": { 2024-05-04T03:25:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_starting_port", 2024-05-04T03:25:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4721", 2024-05-04T03:25:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-04T03:25:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.413 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_ptrace": { 2024-05-04T03:25:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_ptrace", 2024-05-04T03:25:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-04T03:25:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-04T03:25:28.414 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_interval": { 2024-05-04T03:25:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_interval", 2024-05-04T03:25:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-05-04T03:25:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-04T03:25:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.415 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_memory_target_ratio": { 2024-05-04T03:25:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_memory_target_ratio", 2024-05-04T03:25:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-05-04T03:25:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.7", 2024-05-04T03:25:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-04T03:25:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.416 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm_log_destination": { 2024-05-04T03:25:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm_log_destination", 2024-05-04T03:25:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: "file", 2024-05-04T03:25:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: "file,syslog", 2024-05-04T03:25:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: "syslog" 2024-05-04T03:25:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-04T03:25:28.417 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: "cgroups_split": { 2024-05-04T03:25:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cgroups_split", 2024-05-04T03:25:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.418 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-04T03:25:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: "config_checks_enabled": { 2024-05-04T03:25:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_checks_enabled", 2024-05-04T03:25:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-04T03:25:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.419 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: "config_dashboard": { 2024-05-04T03:25:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_dashboard", 2024-05-04T03:25:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-04T03:25:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.420 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_alertmanager": { 2024-05-04T03:25:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_alertmanager", 2024-05-04T03:25:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-04T03:25:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-05-04T03:25:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_base": { 2024-05-04T03:25:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_base", 2024-05-04T03:25:28.421 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-04T03:25:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Container image name, without the tag", 2024-05-04T03:25:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_elasticsearch": { 2024-05-04T03:25:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_elasticsearch", 2024-05-04T03:25:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.422 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.423 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.423 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-04T03:25:28.423 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.423 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.423 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.423 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "elasticsearch container image", 2024-05-04T03:25:28.423 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.423 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.423 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.423 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_grafana": { 2024-05-04T03:25:28.423 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_grafana", 2024-05-04T03:25:28.423 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.423 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.423 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.423 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-04T03:25:28.424 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.424 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.424 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.424 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-05-04T03:25:28.424 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.424 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.424 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.424 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_haproxy": { 2024-05-04T03:25:28.424 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_haproxy", 2024-05-04T03:25:28.424 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.424 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.424 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.424 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-04T03:25:28.424 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.424 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.425 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.425 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "HAproxy container image", 2024-05-04T03:25:28.425 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.425 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.425 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.425 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_agent": { 2024-05-04T03:25:28.425 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_agent", 2024-05-04T03:25:28.425 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.425 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.425 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.425 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-04T03:25:28.425 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.425 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.425 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.425 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger agent container image", 2024-05-04T03:25:28.426 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.426 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.426 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.426 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_collector": { 2024-05-04T03:25:28.426 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_collector", 2024-05-04T03:25:28.426 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.426 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.426 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.426 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-04T03:25:28.426 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.426 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.426 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.426 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger collector container image", 2024-05-04T03:25:28.427 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.427 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.427 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.427 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_query": { 2024-05-04T03:25:28.427 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_query", 2024-05-04T03:25:28.427 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.427 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.427 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.427 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-04T03:25:28.427 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.427 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.427 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.427 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger query container image", 2024-05-04T03:25:28.427 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.428 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.428 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.428 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_keepalived": { 2024-05-04T03:25:28.428 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_keepalived", 2024-05-04T03:25:28.428 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.428 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.428 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.428 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-04T03:25:28.428 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.428 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.428 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.428 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Keepalived container image", 2024-05-04T03:25:28.428 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.428 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.429 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.429 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_loki": { 2024-05-04T03:25:28.429 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_loki", 2024-05-04T03:25:28.429 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.429 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.429 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.429 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-04T03:25:28.429 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.429 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.429 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.429 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Loki container image", 2024-05-04T03:25:28.429 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.429 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.429 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.430 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_node_exporter": { 2024-05-04T03:25:28.430 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_node_exporter", 2024-05-04T03:25:28.430 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.430 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.430 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.430 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-04T03:25:28.430 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.430 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.430 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.430 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-05-04T03:25:28.430 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.430 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.430 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.430 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_nvmeof": { 2024-05-04T03:25:28.431 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_nvmeof", 2024-05-04T03:25:28.431 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.431 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.431 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.431 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-04T03:25:28.431 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.431 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.431 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.431 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Nvme-of container image", 2024-05-04T03:25:28.431 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.431 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.431 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.431 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_prometheus": { 2024-05-04T03:25:28.431 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_prometheus", 2024-05-04T03:25:28.432 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.432 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.432 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.432 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-04T03:25:28.432 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.432 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.432 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.432 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-05-04T03:25:28.432 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.432 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.432 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.432 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_promtail": { 2024-05-04T03:25:28.432 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_promtail", 2024-05-04T03:25:28.432 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.432 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.433 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.433 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-04T03:25:28.433 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.433 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.433 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.433 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Promtail container image", 2024-05-04T03:25:28.433 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.433 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.433 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.433 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.433 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_samba": { 2024-05-04T03:25:28.433 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_samba", 2024-05-04T03:25:28.433 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.433 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.433 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.434 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-04T03:25:28.434 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.434 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.434 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.434 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Samba/SMB container image", 2024-05-04T03:25:28.434 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.434 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.434 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.434 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_snmp_gateway": { 2024-05-04T03:25:28.434 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_snmp_gateway", 2024-05-04T03:25:28.434 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.434 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.434 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.435 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-04T03:25:28.435 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.435 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.435 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.435 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SNMP Gateway container image", 2024-05-04T03:25:28.435 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.435 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.435 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.435 INFO:teuthology.orchestra.run.smithi001.stdout: "container_init": { 2024-05-04T03:25:28.435 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_init", 2024-05-04T03:25:28.435 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.435 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.435 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.435 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.436 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.436 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.436 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.436 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Run podman/docker with `--init`", 2024-05-04T03:25:28.436 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.436 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.436 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.436 INFO:teuthology.orchestra.run.smithi001.stdout: "daemon_cache_timeout": { 2024-05-04T03:25:28.436 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "daemon_cache_timeout", 2024-05-04T03:25:28.436 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.436 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.436 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.436 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-05-04T03:25:28.436 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.436 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.437 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.437 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-04T03:25:28.437 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.437 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.437 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.437 INFO:teuthology.orchestra.run.smithi001.stdout: "default_cephadm_command_timeout": { 2024-05-04T03:25:28.437 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_cephadm_command_timeout", 2024-05-04T03:25:28.437 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.437 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.437 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.437 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "900", 2024-05-04T03:25:28.437 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.437 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.437 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.438 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-04T03:25:28.438 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.438 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.438 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.438 INFO:teuthology.orchestra.run.smithi001.stdout: "default_registry": { 2024-05-04T03:25:28.438 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_registry", 2024-05-04T03:25:28.438 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.438 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.438 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.438 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io", 2024-05-04T03:25:28.438 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.438 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.438 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.438 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-04T03:25:28.438 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.439 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.439 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.439 INFO:teuthology.orchestra.run.smithi001.stdout: "device_cache_timeout": { 2024-05-04T03:25:28.439 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_cache_timeout", 2024-05-04T03:25:28.439 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.439 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.439 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.439 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1800", 2024-05-04T03:25:28.439 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.439 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.439 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.439 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache device inventory", 2024-05-04T03:25:28.439 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.440 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.440 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.440 INFO:teuthology.orchestra.run.smithi001.stdout: "device_enhanced_scan": { 2024-05-04T03:25:28.440 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_enhanced_scan", 2024-05-04T03:25:28.440 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.440 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.440 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.440 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.440 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.440 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.440 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.440 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-04T03:25:28.440 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.440 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.441 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.441 INFO:teuthology.orchestra.run.smithi001.stdout: "facts_cache_timeout": { 2024-05-04T03:25:28.441 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "facts_cache_timeout", 2024-05-04T03:25:28.441 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.441 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.441 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.441 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-05-04T03:25:28.441 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.441 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.441 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.441 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache host facts data", 2024-05-04T03:25:28.441 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.441 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.441 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.442 INFO:teuthology.orchestra.run.smithi001.stdout: "grafana_dashboards_path": { 2024-05-04T03:25:28.442 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "grafana_dashboards_path", 2024-05-04T03:25:28.442 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.442 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.442 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.442 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-04T03:25:28.442 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.442 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.442 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.442 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-04T03:25:28.442 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.442 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.442 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.443 INFO:teuthology.orchestra.run.smithi001.stdout: "host_check_interval": { 2024-05-04T03:25:28.443 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "host_check_interval", 2024-05-04T03:25:28.443 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.443 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.443 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.443 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-05-04T03:25:28.443 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.443 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.443 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.443 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to perform a host check", 2024-05-04T03:25:28.443 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.443 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.443 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.443 INFO:teuthology.orchestra.run.smithi001.stdout: "hw_monitoring": { 2024-05-04T03:25:28.443 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hw_monitoring", 2024-05-04T03:25:28.444 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.444 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.444 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.444 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.444 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.444 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.444 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.444 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-04T03:25:28.444 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.444 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.444 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.444 INFO:teuthology.orchestra.run.smithi001.stdout: "inventory_list_all": { 2024-05-04T03:25:28.444 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "inventory_list_all", 2024-05-04T03:25:28.445 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.445 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.445 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.445 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.445 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.445 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.445 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.445 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-04T03:25:28.445 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.445 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.445 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.445 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.445 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.446 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.446 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.446 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.446 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.446 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.446 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.446 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.446 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.446 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.446 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.446 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.446 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.446 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.447 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.447 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.447 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.447 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.447 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.447 INFO:teuthology.orchestra.run.smithi001.stdout: "log_refresh_metadata": { 2024-05-04T03:25:28.447 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_refresh_metadata", 2024-05-04T03:25:28.447 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.447 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.447 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.447 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.447 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.447 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.447 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.448 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-04T03:25:28.448 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.448 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.448 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.448 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.448 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.448 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.448 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.448 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.448 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.448 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.448 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.448 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.448 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-04T03:25:28.449 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.449 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.449 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.449 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.449 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.449 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.449 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.449 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.449 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.449 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.449 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.449 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.449 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.449 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.450 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.450 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.450 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.450 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.450 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.450 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.450 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.450 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.450 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.450 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.450 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.450 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.450 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.451 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.451 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.451 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.451 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.451 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.451 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.451 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.451 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.451 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.451 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-04T03:25:28.451 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-04T03:25:28.451 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.451 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.452 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.452 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.452 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.452 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.452 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.452 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-04T03:25:28.452 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.452 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.452 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.452 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-04T03:25:28.452 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-04T03:25:28.452 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.452 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.453 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.453 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "*", 2024-05-04T03:25:28.453 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.453 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.453 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.453 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-04T03:25:28.453 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.453 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.453 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.453 INFO:teuthology.orchestra.run.smithi001.stdout: "max_count_per_host": { 2024-05-04T03:25:28.453 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_count_per_host", 2024-05-04T03:25:28.453 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.453 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.454 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.454 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-05-04T03:25:28.454 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.454 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.454 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.454 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of daemons per service per host", 2024-05-04T03:25:28.454 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.454 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.454 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.454 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd_draining_count": { 2024-05-04T03:25:28.454 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_osd_draining_count", 2024-05-04T03:25:28.454 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.454 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.455 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.455 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-05-04T03:25:28.455 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.455 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.455 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.455 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-04T03:25:28.455 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.455 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.455 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.455 INFO:teuthology.orchestra.run.smithi001.stdout: "migration_current": { 2024-05-04T03:25:28.455 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "migration_current", 2024-05-04T03:25:28.455 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.455 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.455 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.455 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.456 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.456 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.456 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.456 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "internal - do not modify", 2024-05-04T03:25:28.456 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.456 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.456 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.456 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-05-04T03:25:28.456 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-05-04T03:25:28.456 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.456 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.456 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.456 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "root", 2024-05-04T03:25:28.456 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.456 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.457 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.457 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm-package", 2024-05-04T03:25:28.457 INFO:teuthology.orchestra.run.smithi001.stdout: "root" 2024-05-04T03:25:28.457 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.457 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "mode for remote execution of cephadm", 2024-05-04T03:25:28.457 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.457 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.457 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.457 INFO:teuthology.orchestra.run.smithi001.stdout: "oob_default_addr": { 2024-05-04T03:25:28.457 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "oob_default_addr", 2024-05-04T03:25:28.457 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.457 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.457 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.457 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "169.254.1.1", 2024-05-04T03:25:28.458 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.458 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.458 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.458 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-04T03:25:28.458 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.458 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.458 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.458 INFO:teuthology.orchestra.run.smithi001.stdout: "prometheus_alerts_path": { 2024-05-04T03:25:28.458 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus_alerts_path", 2024-05-04T03:25:28.458 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.458 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.458 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.458 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-04T03:25:28.459 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.459 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.459 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.459 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-04T03:25:28.459 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.459 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.459 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.459 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_insecure": { 2024-05-04T03:25:28.459 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_insecure", 2024-05-04T03:25:28.459 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.459 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.459 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.459 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.460 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.460 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.460 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.460 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-04T03:25:28.460 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.460 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.460 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.460 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_password": { 2024-05-04T03:25:28.460 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_password", 2024-05-04T03:25:28.460 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.460 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.460 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.460 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.460 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.461 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.461 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.461 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-04T03:25:28.461 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.461 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.461 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.461 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_url": { 2024-05-04T03:25:28.461 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_url", 2024-05-04T03:25:28.461 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.461 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.461 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.461 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.461 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.461 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.461 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.462 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-04T03:25:28.462 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.462 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.462 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.462 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_username": { 2024-05-04T03:25:28.462 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_username", 2024-05-04T03:25:28.462 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.462 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.462 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.462 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.462 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.462 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.462 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.463 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-04T03:25:28.463 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.463 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.463 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.463 INFO:teuthology.orchestra.run.smithi001.stdout: "secure_monitoring_stack": { 2024-05-04T03:25:28.463 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "secure_monitoring_stack", 2024-05-04T03:25:28.463 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.463 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.463 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.463 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.463 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.463 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.463 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.463 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-04T03:25:28.463 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.463 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.464 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.464 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.464 INFO:teuthology.orchestra.run.smithi001.stdout: "service_discovery_port": { 2024-05-04T03:25:28.464 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "service_discovery_port", 2024-05-04T03:25:28.464 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.464 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.464 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.464 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8765", 2024-05-04T03:25:28.464 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.464 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.464 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.464 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "cephadm service discovery port", 2024-05-04T03:25:28.464 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.464 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.465 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.465 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.465 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.465 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.465 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.465 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.465 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.465 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.465 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.465 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.465 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.465 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.465 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.465 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.466 INFO:teuthology.orchestra.run.smithi001.stdout: "ssh_config_file": { 2024-05-04T03:25:28.466 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssh_config_file", 2024-05-04T03:25:28.466 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.466 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.466 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.466 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.466 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.466 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.466 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.466 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-04T03:25:28.466 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.466 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.466 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.467 INFO:teuthology.orchestra.run.smithi001.stdout: "use_agent": { 2024-05-04T03:25:28.467 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_agent", 2024-05-04T03:25:28.467 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.467 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.467 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.467 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.467 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.467 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.467 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.467 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-04T03:25:28.467 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.467 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.467 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.467 INFO:teuthology.orchestra.run.smithi001.stdout: "use_repo_digest": { 2024-05-04T03:25:28.467 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_repo_digest", 2024-05-04T03:25:28.468 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.468 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.468 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.468 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.468 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.468 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.468 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.468 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-04T03:25:28.468 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.468 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.468 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.468 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_failed_host_check": { 2024-05-04T03:25:28.468 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_failed_host_check", 2024-05-04T03:25:28.468 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.468 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.469 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.469 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.469 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.469 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.469 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.469 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if the host check fails", 2024-05-04T03:25:28.469 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.469 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.469 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.469 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_daemons": { 2024-05-04T03:25:28.469 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_daemons", 2024-05-04T03:25:28.469 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.469 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.469 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.469 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.470 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.470 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.470 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.470 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-04T03:25:28.470 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.470 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.470 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.470 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_hosts": { 2024-05-04T03:25:28.470 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_hosts", 2024-05-04T03:25:28.470 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.470 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.470 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.470 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.471 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.471 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.471 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.471 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-04T03:25:28.471 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.471 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.471 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.471 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.471 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.471 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.471 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crash", 2024-05-04T03:25:28.471 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.471 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.471 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.471 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.472 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.472 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.472 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.472 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.472 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.472 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.472 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.472 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.472 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.472 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.472 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.472 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.472 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.472 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.472 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.472 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.472 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.473 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.473 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.473 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.473 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.473 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.473 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.473 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.473 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.473 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.473 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.473 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.473 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.473 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.473 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.473 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.474 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.474 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.474 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.474 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.474 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.474 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.474 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.474 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.474 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.474 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.474 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.474 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.474 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.474 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.475 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.475 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.475 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.475 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.475 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.475 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.475 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.475 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.475 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.475 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.475 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.475 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.475 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.475 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.475 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.475 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.476 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.476 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.476 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.476 INFO:teuthology.orchestra.run.smithi001.stdout: "retain_interval": { 2024-05-04T03:25:28.476 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retain_interval", 2024-05-04T03:25:28.476 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.476 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.476 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.476 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "31536000", 2024-05-04T03:25:28.476 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.476 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.476 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.476 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-04T03:25:28.476 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.476 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.477 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.477 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.477 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.477 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.477 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.477 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.477 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.477 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.477 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.477 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.477 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.477 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.477 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.477 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.478 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_recent_interval": { 2024-05-04T03:25:28.478 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_recent_interval", 2024-05-04T03:25:28.478 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.478 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.478 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.478 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1209600", 2024-05-04T03:25:28.478 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.478 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.478 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.478 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-04T03:25:28.478 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.478 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.478 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.478 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.479 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.479 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.479 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dashboard", 2024-05-04T03:25:28.479 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.479 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.479 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.479 INFO:teuthology.orchestra.run.smithi001.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-04T03:25:28.479 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-04T03:25:28.479 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.479 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.479 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.479 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-05-04T03:25:28.479 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.479 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.479 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.480 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.480 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.480 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.480 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.480 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_HOST": { 2024-05-04T03:25:28.480 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-04T03:25:28.480 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.480 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.480 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.480 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.480 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.480 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.480 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.481 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.481 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.481 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.481 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.481 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-04T03:25:28.481 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-04T03:25:28.481 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.481 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.481 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.481 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.481 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.481 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.481 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.482 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.482 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.482 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.482 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.482 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_ENABLED": { 2024-05-04T03:25:28.482 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_ENABLED", 2024-05-04T03:25:28.482 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.482 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.482 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.482 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.482 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.482 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.482 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.482 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.483 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.483 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.483 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.483 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-04T03:25:28.483 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-04T03:25:28.483 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.483 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.483 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.483 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.483 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.483 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.483 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.483 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.483 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.484 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.484 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.484 INFO:teuthology.orchestra.run.smithi001.stdout: "ENABLE_BROWSABLE_API": { 2024-05-04T03:25:28.484 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-04T03:25:28.484 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.484 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.484 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.484 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.484 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.484 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.484 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.484 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.484 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.484 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.485 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.485 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-04T03:25:28.485 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-04T03:25:28.485 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.485 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.485 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.485 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.485 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.485 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.485 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.485 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.485 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.485 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.486 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.486 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-04T03:25:28.486 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-04T03:25:28.486 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.486 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.486 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.486 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.486 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.486 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.486 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.486 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.486 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.486 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.487 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.487 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-04T03:25:28.487 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-04T03:25:28.487 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.487 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.487 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.487 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.487 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.487 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.487 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.487 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.487 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.487 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.487 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.488 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-04T03:25:28.488 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-04T03:25:28.488 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.488 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.488 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.488 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.488 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.488 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.488 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.488 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.488 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.488 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.488 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.489 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-04T03:25:28.489 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-04T03:25:28.489 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.489 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.489 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.489 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.489 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.489 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.489 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.489 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.489 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.489 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.489 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.489 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-04T03:25:28.490 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-04T03:25:28.490 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.490 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.490 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.490 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.490 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.490 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.490 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.490 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.490 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.490 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.490 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.490 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-04T03:25:28.491 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-04T03:25:28.491 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.491 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.491 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.491 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.491 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.491 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.491 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.491 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.491 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.491 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.491 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.491 INFO:teuthology.orchestra.run.smithi001.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-04T03:25:28.491 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-04T03:25:28.492 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.492 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.492 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.492 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.492 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.492 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.492 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.492 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.492 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.492 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.492 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.492 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_PASSWORD": { 2024-05-04T03:25:28.492 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-04T03:25:28.492 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.492 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.493 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.493 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-05-04T03:25:28.493 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.493 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.493 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.493 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.493 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.493 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.493 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.493 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-04T03:25:28.493 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-04T03:25:28.493 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.493 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.493 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.493 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.494 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.494 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.494 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.494 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.494 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.494 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.494 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.494 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_URL": { 2024-05-04T03:25:28.494 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_URL", 2024-05-04T03:25:28.494 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.494 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.494 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.494 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.494 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.495 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.495 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.495 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.495 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.495 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.495 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.495 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_USERNAME": { 2024-05-04T03:25:28.495 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-04T03:25:28.495 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.495 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.495 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.495 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-05-04T03:25:28.495 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.495 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.495 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.495 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-04T03:25:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-04T03:25:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-04T03:25:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-04T03:25:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-04T03:25:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-04T03:25:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-04T03:25:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-04T03:25:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-04T03:25:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-04T03:25:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "[]", 2024-05-04T03:25:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: "MULTICLUSTER_CONFIG": { 2024-05-04T03:25:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-04T03:25:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "{}", 2024-05-04T03:25:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_HOST": { 2024-05-04T03:25:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-04T03:25:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-04T03:25:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-04T03:25:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-04T03:25:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-04T03:25:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-04T03:25:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-04T03:25:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-04T03:25:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-04T03:25:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-04T03:25:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-04T03:25:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-04T03:25:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-04T03:25:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-04T03:25:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-04T03:25:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.508 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.508 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.508 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.508 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.508 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.508 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.508 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.508 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.508 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.508 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.508 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-04T03:25:28.508 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-04T03:25:28.509 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.509 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.509 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.509 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.509 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.509 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.509 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.509 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.509 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.509 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.509 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.509 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_ENABLED": { 2024-05-04T03:25:28.509 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-04T03:25:28.509 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.509 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.510 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.510 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.510 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.510 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.510 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.510 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.510 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.510 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.510 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.510 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-04T03:25:28.510 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-04T03:25:28.510 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.510 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.510 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.510 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-04T03:25:28.511 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.511 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.511 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.511 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.511 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.511 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.511 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.511 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-04T03:25:28.511 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-04T03:25:28.511 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.511 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.511 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.511 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-05-04T03:25:28.511 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.511 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.512 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.512 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.512 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.512 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.512 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.512 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-04T03:25:28.512 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-04T03:25:28.512 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.512 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.512 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.512 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8", 2024-05-04T03:25:28.512 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.512 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.512 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.512 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.513 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.513 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.513 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.513 INFO:teuthology.orchestra.run.smithi001.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-04T03:25:28.513 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-04T03:25:28.513 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.513 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.513 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.513 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "45", 2024-05-04T03:25:28.513 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.513 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.513 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.513 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.513 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.513 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.514 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.514 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ACCESS_KEY": { 2024-05-04T03:25:28.514 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-04T03:25:28.514 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.514 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.514 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.514 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.514 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.514 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.514 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.514 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.514 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.515 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.515 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.515 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-04T03:25:28.515 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-04T03:25:28.515 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.515 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.515 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.515 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-05-04T03:25:28.515 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.515 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.515 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.515 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.515 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.515 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.515 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.516 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SECRET_KEY": { 2024-05-04T03:25:28.516 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-04T03:25:28.516 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.516 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.516 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.516 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.516 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.516 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.516 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.516 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.516 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.516 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.516 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.517 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SSL_VERIFY": { 2024-05-04T03:25:28.517 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-04T03:25:28.517 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.517 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.517 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.517 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.517 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.517 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.517 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.517 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.517 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.517 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.517 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.517 INFO:teuthology.orchestra.run.smithi001.stdout: "UNSAFE_TLS_v1_2": { 2024-05-04T03:25:28.517 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-04T03:25:28.518 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.518 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.518 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.518 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.518 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.518 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.518 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.518 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.518 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.518 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.518 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.518 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-04T03:25:28.519 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-04T03:25:28.519 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.519 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.519 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.519 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.519 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.519 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.519 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.519 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.519 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.519 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.519 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.519 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.519 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-04T03:25:28.519 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-04T03:25:28.519 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.520 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.520 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.520 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-05-04T03:25:28.520 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.520 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.520 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.520 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.520 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.520 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.520 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.520 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-04T03:25:28.520 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-04T03:25:28.520 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.521 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.521 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.521 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-05-04T03:25:28.521 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.521 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.521 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.521 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.521 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.521 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.521 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.521 INFO:teuthology.orchestra.run.smithi001.stdout: "cross_origin_url": { 2024-05-04T03:25:28.521 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cross_origin_url", 2024-05-04T03:25:28.521 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.521 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.521 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.522 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.522 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.522 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.522 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.522 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.522 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.522 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.522 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.522 INFO:teuthology.orchestra.run.smithi001.stdout: "crt_file": { 2024-05-04T03:25:28.522 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crt_file", 2024-05-04T03:25:28.522 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.522 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.522 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.523 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.523 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.523 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.523 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.523 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.523 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.523 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.523 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.523 INFO:teuthology.orchestra.run.smithi001.stdout: "debug": { 2024-05-04T03:25:28.523 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "debug", 2024-05-04T03:25:28.523 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.523 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.523 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.523 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.523 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.524 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.524 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.524 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable/disable debug options", 2024-05-04T03:25:28.524 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.524 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.524 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.524 INFO:teuthology.orchestra.run.smithi001.stdout: "jwt_token_ttl": { 2024-05-04T03:25:28.524 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "jwt_token_ttl", 2024-05-04T03:25:28.524 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.524 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.524 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.524 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "28800", 2024-05-04T03:25:28.524 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.524 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.524 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.525 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.525 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.525 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.525 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.525 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-05-04T03:25:28.525 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-05-04T03:25:28.525 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.525 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.525 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.525 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.525 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.525 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.525 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.525 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.525 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.525 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.526 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.526 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.526 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.526 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.526 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.526 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.526 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.526 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.526 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.526 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.526 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.526 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.526 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.526 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.527 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.527 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.527 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.527 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.527 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.527 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.527 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.527 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.527 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.527 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.527 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.527 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.527 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.527 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.527 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.527 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.528 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.528 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.528 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.528 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.528 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.528 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.528 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.528 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.528 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.528 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.528 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.528 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.528 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.528 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.528 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.528 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.529 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.529 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.529 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.529 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.529 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.529 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.529 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.529 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.529 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.529 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.529 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.529 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.529 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.529 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.529 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.530 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.530 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.530 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.530 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.530 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.530 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.530 INFO:teuthology.orchestra.run.smithi001.stdout: "motd": { 2024-05-04T03:25:28.530 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "motd", 2024-05-04T03:25:28.530 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.530 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.530 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.530 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.530 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.530 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.531 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.531 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "The message of the day", 2024-05-04T03:25:28.531 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.531 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.531 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.531 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect_resolve_ip_addr": { 2024-05-04T03:25:28.531 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "redirect_resolve_ip_addr", 2024-05-04T03:25:28.531 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.531 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.531 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.531 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.531 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.531 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.531 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.531 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.532 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.532 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.532 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.532 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-05-04T03:25:28.532 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-05-04T03:25:28.532 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.532 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.532 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.532 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-05-04T03:25:28.532 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.532 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.532 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.532 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.532 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.532 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.533 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.533 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-05-04T03:25:28.533 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-05-04T03:25:28.533 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.533 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.533 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.533 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8080", 2024-05-04T03:25:28.533 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.533 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.533 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.533 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.533 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.533 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.533 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.534 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.534 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.534 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.534 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.534 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.534 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.534 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.534 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.534 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.534 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.534 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.534 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.534 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.534 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-05-04T03:25:28.535 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-05-04T03:25:28.535 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.535 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.535 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.535 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.535 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.535 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.535 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.535 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.535 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.535 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.535 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.535 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl_server_port": { 2024-05-04T03:25:28.535 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl_server_port", 2024-05-04T03:25:28.535 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.536 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.536 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.536 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8443", 2024-05-04T03:25:28.536 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.536 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.536 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.536 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.536 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.536 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.536 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.536 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-05-04T03:25:28.536 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-05-04T03:25:28.536 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.536 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.537 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.537 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "redirect", 2024-05-04T03:25:28.537 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.537 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.537 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.537 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.537 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect" 2024-05-04T03:25:28.537 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.537 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.537 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.537 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.537 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.537 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-05-04T03:25:28.537 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-05-04T03:25:28.538 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.538 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.538 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.538 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-05-04T03:25:28.538 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-05-04T03:25:28.538 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-05-04T03:25:28.538 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.538 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.538 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.538 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.538 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.538 INFO:teuthology.orchestra.run.smithi001.stdout: "url_prefix": { 2024-05-04T03:25:28.538 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url_prefix", 2024-05-04T03:25:28.539 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.539 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.539 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.539 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.539 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.539 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.539 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.539 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.539 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.539 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.539 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.539 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.539 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.540 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.540 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "devicehealth", 2024-05-04T03:25:28.540 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.540 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.540 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.540 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_monitoring": { 2024-05-04T03:25:28.540 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_monitoring", 2024-05-04T03:25:28.540 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.540 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.540 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.540 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.540 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.540 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.540 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.540 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "monitor device health metrics", 2024-05-04T03:25:28.541 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.541 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.541 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.541 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.541 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.541 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.541 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.541 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.541 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.541 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.541 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.541 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.541 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.548 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.548 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.548 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.548 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.548 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.548 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.548 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.548 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.548 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.548 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.548 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.549 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.549 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.549 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.549 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.549 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.549 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.549 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.549 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.549 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.549 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.549 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.549 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.549 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.549 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.550 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.550 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.550 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.550 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.550 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.550 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.550 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.550 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.550 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.550 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.550 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.550 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.550 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.550 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.550 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.551 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.551 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.551 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.551 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.551 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.551 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.551 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.551 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.551 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.551 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.551 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.551 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.551 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.552 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.552 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.552 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.552 INFO:teuthology.orchestra.run.smithi001.stdout: "mark_out_threshold": { 2024-05-04T03:25:28.552 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mark_out_threshold", 2024-05-04T03:25:28.552 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.552 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.552 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.552 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2419200", 2024-05-04T03:25:28.552 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.552 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.552 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.552 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-04T03:25:28.552 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.553 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.553 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.553 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": { 2024-05-04T03:25:28.553 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_name", 2024-05-04T03:25:28.553 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.553 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.553 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.553 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "device_health_metrics", 2024-05-04T03:25:28.553 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.553 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.553 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.553 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-04T03:25:28.553 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.553 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.554 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.554 INFO:teuthology.orchestra.run.smithi001.stdout: "retention_period": { 2024-05-04T03:25:28.554 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retention_period", 2024-05-04T03:25:28.554 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.554 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.554 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.554 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15552000", 2024-05-04T03:25:28.554 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.554 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.554 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.554 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain device health metrics", 2024-05-04T03:25:28.554 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.554 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.555 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.555 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_frequency": { 2024-05-04T03:25:28.555 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_frequency", 2024-05-04T03:25:28.555 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.555 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.555 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.555 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-05-04T03:25:28.555 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.555 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.555 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.555 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-04T03:25:28.555 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.555 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.555 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.556 INFO:teuthology.orchestra.run.smithi001.stdout: "self_heal": { 2024-05-04T03:25:28.556 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "self_heal", 2024-05-04T03:25:28.556 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.556 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.556 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.556 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.556 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.556 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.556 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.556 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-04T03:25:28.556 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.556 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.556 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.556 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-05-04T03:25:28.556 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-05-04T03:25:28.557 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.557 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.557 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.557 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-05-04T03:25:28.557 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.557 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.557 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.557 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and check device health", 2024-05-04T03:25:28.557 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.557 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.557 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.557 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.557 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.557 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.557 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.558 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.558 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.558 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.558 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.558 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.558 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.558 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.558 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.558 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.558 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_threshold": { 2024-05-04T03:25:28.558 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_threshold", 2024-05-04T03:25:28.558 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.558 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.558 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.558 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7257600", 2024-05-04T03:25:28.559 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.559 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.559 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.559 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-04T03:25:28.559 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.559 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.559 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.559 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.559 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.559 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.559 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "diskprediction_local", 2024-05-04T03:25:28.559 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.559 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.559 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.559 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.560 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.560 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.560 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.560 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.560 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.560 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.560 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.560 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.560 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.560 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.560 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.560 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.560 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.560 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.560 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.560 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.561 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.561 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.561 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.561 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.561 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.561 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.561 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.561 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.561 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.561 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.561 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.561 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.561 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.561 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.561 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.562 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.562 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.562 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.562 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.562 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.562 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.562 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.562 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.562 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.562 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.562 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.562 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.562 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.562 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.562 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.563 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.563 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.563 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.563 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.563 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.563 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.563 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.563 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.563 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.563 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.563 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.563 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.563 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.563 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.563 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.564 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.564 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.564 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.564 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.564 INFO:teuthology.orchestra.run.smithi001.stdout: "predict_interval": { 2024-05-04T03:25:28.564 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predict_interval", 2024-05-04T03:25:28.564 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.564 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.564 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.564 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-05-04T03:25:28.564 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.564 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.564 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.564 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.564 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.565 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.565 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.565 INFO:teuthology.orchestra.run.smithi001.stdout: "predictor_model": { 2024-05-04T03:25:28.565 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predictor_model", 2024-05-04T03:25:28.565 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.565 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.565 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.565 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "prophetstor", 2024-05-04T03:25:28.565 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.565 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.565 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.565 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.565 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.565 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.565 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.566 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-05-04T03:25:28.566 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-05-04T03:25:28.566 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.566 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.566 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.566 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-05-04T03:25:28.566 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.566 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.566 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.566 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.566 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.566 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.566 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.566 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.567 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.567 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.567 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.567 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.567 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.567 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.567 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.567 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.567 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.567 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.567 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.567 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.567 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.567 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.567 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "influx", 2024-05-04T03:25:28.568 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": false, 2024-05-04T03:25:28.568 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "influxdb python module not found", 2024-05-04T03:25:28.568 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.568 INFO:teuthology.orchestra.run.smithi001.stdout: "batch_size": { 2024-05-04T03:25:28.568 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "batch_size", 2024-05-04T03:25:28.568 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.568 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.568 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.568 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5000", 2024-05-04T03:25:28.568 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.568 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.568 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.568 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-04T03:25:28.568 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.568 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.568 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.569 INFO:teuthology.orchestra.run.smithi001.stdout: "database": { 2024-05-04T03:25:28.569 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "database", 2024-05-04T03:25:28.569 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.569 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.569 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.569 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "ceph", 2024-05-04T03:25:28.569 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.569 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.569 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.569 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-04T03:25:28.569 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.569 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.569 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.569 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": { 2024-05-04T03:25:28.570 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hostname", 2024-05-04T03:25:28.570 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.570 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.570 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.570 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.570 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.570 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.570 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.570 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server hostname", 2024-05-04T03:25:28.570 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.570 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.570 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.570 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-05-04T03:25:28.570 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-05-04T03:25:28.570 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.570 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.571 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.571 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "30", 2024-05-04T03:25:28.571 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "5", 2024-05-04T03:25:28.571 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.571 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.571 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-04T03:25:28.571 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.571 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.571 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.571 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.571 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.571 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.571 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.571 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.572 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.572 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.572 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.572 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.572 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.572 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.572 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.572 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.572 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.572 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.572 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.572 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.572 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.572 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.572 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.573 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.573 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.573 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.573 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.573 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.573 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.573 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.573 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.573 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.573 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.573 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.573 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.573 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.573 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.573 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.574 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.575 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "password": { 2024-05-04T03:25:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "password", 2024-05-04T03:25:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "password of InfluxDB server user", 2024-05-04T03:25:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "port": { 2024-05-04T03:25:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "port", 2024-05-04T03:25:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8086", 2024-05-04T03:25:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server port", 2024-05-04T03:25:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.577 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-05-04T03:25:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-05-04T03:25:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.578 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "false", 2024-05-04T03:25:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-04T03:25:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "threads": { 2024-05-04T03:25:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threads", 2024-05-04T03:25:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.579 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-05-04T03:25:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-05-04T03:25:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "32", 2024-05-04T03:25:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-04T03:25:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "username": { 2024-05-04T03:25:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "username", 2024-05-04T03:25:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.580 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "username of InfluxDB server user", 2024-05-04T03:25:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "verify_ssl": { 2024-05-04T03:25:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "verify_ssl", 2024-05-04T03:25:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "true", 2024-05-04T03:25:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.581 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-04T03:25:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "insights", 2024-05-04T03:25:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.583 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.585 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.586 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.587 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "iostat", 2024-05-04T03:25:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.588 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.589 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.590 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.591 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.592 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.593 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "k8sevents", 2024-05-04T03:25:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_event_retention_days": { 2024-05-04T03:25:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ceph_event_retention_days", 2024-05-04T03:25:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7", 2024-05-04T03:25:28.594 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-04T03:25:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: "config_check_secs": { 2024-05-04T03:25:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_check_secs", 2024-05-04T03:25:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-05-04T03:25:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "10", 2024-05-04T03:25:28.595 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-04T03:25:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.596 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.597 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.598 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.599 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.599 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.599 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.599 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.599 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.599 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.599 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.599 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.599 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.599 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.599 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.599 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.599 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.599 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.600 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.601 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "localpool", 2024-05-04T03:25:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: "failure_domain": { 2024-05-04T03:25:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "failure_domain", 2024-05-04T03:25:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.602 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "host", 2024-05-04T03:25:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "failure domain for any created local pool", 2024-05-04T03:25:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-04T03:25:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.603 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.604 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.605 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.606 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.607 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": { 2024-05-04T03:25:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_size", 2024-05-04T03:25:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default min_size for any created local pool", 2024-05-04T03:25:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-04T03:25:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.608 INFO:teuthology.orchestra.run.smithi001.stdout: "num_rep": { 2024-05-04T03:25:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "num_rep", 2024-05-04T03:25:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3", 2024-05-04T03:25:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default replica count for any created local pool", 2024-05-04T03:25:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.609 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": { 2024-05-04T03:25:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_num", 2024-05-04T03:25:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "128", 2024-05-04T03:25:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default pg_num for any created local pool", 2024-05-04T03:25:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.610 INFO:teuthology.orchestra.run.smithi001.stdout: "prefix": { 2024-05-04T03:25:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prefix", 2024-05-04T03:25:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name prefix for any created local pool", 2024-05-04T03:25:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.611 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "subtree": { 2024-05-04T03:25:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "subtree", 2024-05-04T03:25:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.612 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "rack", 2024-05-04T03:25:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-04T03:25:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-04T03:25:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mds_autoscaler", 2024-05-04T03:25:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.613 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.614 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.615 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.616 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.617 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.618 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirroring", 2024-05-04T03:25:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.619 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.620 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.621 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.622 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.623 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.624 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "nfs", 2024-05-04T03:25:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.625 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.626 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.627 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.628 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.629 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.630 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-05-04T03:25:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: "fail_fs": { 2024-05-04T03:25:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "fail_fs", 2024-05-04T03:25:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.631 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-04T03:25:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.632 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.633 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.634 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.635 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator": { 2024-05-04T03:25:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-05-04T03:25:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.636 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm", 2024-05-04T03:25:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: "rook", 2024-05-04T03:25:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: "test_orchestrator" 2024-05-04T03:25:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Orchestrator backend", 2024-05-04T03:25:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.637 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_perf_query", 2024-05-04T03:25:28.638 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.639 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.640 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.641 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.642 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.643 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_support", 2024-05-04T03:25:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.644 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.647 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.648 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.649 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_autoscaler", 2024-05-04T03:25:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.650 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.651 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.652 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.653 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.654 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-05-04T03:25:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-05-04T03:25:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-05-04T03:25:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.655 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.656 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: "threshold": { 2024-05-04T03:25:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threshold", 2024-05-04T03:25:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-05-04T03:25:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-05-04T03:25:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1.0", 2024-05-04T03:25:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "scaling threshold", 2024-05-04T03:25:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-04T03:25:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.657 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "progress", 2024-05-04T03:25:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_pg_recovery_event": { 2024-05-04T03:25:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_pg_recovery_event", 2024-05-04T03:25:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.658 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-04T03:25:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-05-04T03:25:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-05-04T03:25:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.659 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.660 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.661 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.662 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.663 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: "max_completed_events": { 2024-05-04T03:25:28.664 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_completed_events", 2024-05-04T03:25:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "50", 2024-05-04T03:25:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "number of past completed events to remember", 2024-05-04T03:25:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-05-04T03:25:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-05-04T03:25:28.665 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-05-04T03:25:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long the module is going to sleep", 2024-05-04T03:25:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.666 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus", 2024-05-04T03:25:28.667 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "cache": { 2024-05-04T03:25:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cache", 2024-05-04T03:25:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.668 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: "exclude_perf_counters": { 2024-05-04T03:25:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "exclude_perf_counters", 2024-05-04T03:25:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-04T03:25:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-04T03:25:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.669 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.670 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.671 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.672 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.673 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.674 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools": { 2024-05-04T03:25:28.675 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools", 2024-05-04T03:25:28.675 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.675 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.675 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.675 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.675 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.675 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.675 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.675 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.675 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.675 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.675 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.675 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.675 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-04T03:25:28.675 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-04T03:25:28.675 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.676 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.676 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.676 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "300", 2024-05-04T03:25:28.676 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.676 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.676 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.676 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.676 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.676 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.676 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.676 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_interval": { 2024-05-04T03:25:28.676 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_interval", 2024-05-04T03:25:28.676 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-05-04T03:25:28.676 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.677 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.677 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15.0", 2024-05-04T03:25:28.677 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.677 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.677 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.677 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.677 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.677 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.677 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.677 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-05-04T03:25:28.677 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-05-04T03:25:28.677 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.677 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.677 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.677 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-05-04T03:25:28.678 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.678 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.678 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.678 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-04T03:25:28.678 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.678 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.678 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.678 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-05-04T03:25:28.678 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-05-04T03:25:28.678 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.678 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.678 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.678 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "9283", 2024-05-04T03:25:28.679 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.679 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.679 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.679 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-04T03:25:28.679 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.679 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.679 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.679 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.679 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.679 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.679 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.679 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.679 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.679 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: "stale_cache_strategy": { 2024-05-04T03:25:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stale_cache_strategy", 2024-05-04T03:25:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "log", 2024-05-04T03:25:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.681 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.681 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.681 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.681 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.681 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.681 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-05-04T03:25:28.681 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-05-04T03:25:28.681 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.681 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.681 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.681 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "default", 2024-05-04T03:25:28.681 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.681 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.682 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.682 INFO:teuthology.orchestra.run.smithi001.stdout: "default", 2024-05-04T03:25:28.682 INFO:teuthology.orchestra.run.smithi001.stdout: "error" 2024-05-04T03:25:28.682 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.682 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.682 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.682 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.682 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.682 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-05-04T03:25:28.682 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-05-04T03:25:28.682 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.682 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.682 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.682 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-05-04T03:25:28.683 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-05-04T03:25:28.683 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-05-04T03:25:28.683 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.683 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.683 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.683 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.683 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.683 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.683 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.683 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.683 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_support", 2024-05-04T03:25:28.683 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.683 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.684 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.684 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.684 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.684 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.684 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.684 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.684 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.684 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.684 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.684 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.684 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.684 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.684 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.684 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.685 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.685 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.685 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.685 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.685 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.685 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.685 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.685 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.685 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.685 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.685 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.685 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.685 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.685 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.686 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.686 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.686 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.686 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.686 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.686 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.686 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.686 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.686 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.686 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.686 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.686 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.686 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.687 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.687 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.687 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.687 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.687 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.687 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.687 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.687 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.687 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.687 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.687 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.687 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.687 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.687 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.688 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.688 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.688 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.688 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.688 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.688 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.688 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.688 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.688 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.688 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.688 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.688 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.688 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_snap_create": { 2024-05-04T03:25:28.688 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_snap_create", 2024-05-04T03:25:28.689 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.689 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.689 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.689 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-05-04T03:25:28.689 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.689 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.689 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.689 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.689 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.689 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.689 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.689 INFO:teuthology.orchestra.run.smithi001.stdout: "mirror_snapshot_schedule": { 2024-05-04T03:25:28.689 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirror_snapshot_schedule", 2024-05-04T03:25:28.689 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.690 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.690 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.690 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.690 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.690 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.690 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.690 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.690 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.690 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.690 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.690 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.690 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.690 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.691 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.691 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.691 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.691 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.691 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.691 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.691 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.691 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.691 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.691 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.691 INFO:teuthology.orchestra.run.smithi001.stdout: "trash_purge_schedule": { 2024-05-04T03:25:28.691 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "trash_purge_schedule", 2024-05-04T03:25:28.691 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.691 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.692 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.692 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.692 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.692 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.692 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.692 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.692 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.692 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.692 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.692 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.692 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.692 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.692 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "restful", 2024-05-04T03:25:28.693 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.693 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.693 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.693 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_auth": { 2024-05-04T03:25:28.693 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_auth", 2024-05-04T03:25:28.693 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.693 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.693 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.693 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.693 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.693 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.693 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.693 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.693 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.694 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.694 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.694 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-05-04T03:25:28.694 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-05-04T03:25:28.694 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.694 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.694 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.694 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.694 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.694 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.694 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.694 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.694 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.694 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.695 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.695 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.695 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.695 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.695 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.695 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.695 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.695 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.695 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.695 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.695 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.695 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.695 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.695 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.696 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.696 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.696 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.696 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.696 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.696 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.696 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.696 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.696 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.696 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.696 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.696 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.696 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.696 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.697 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.697 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.697 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.697 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.697 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.697 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.697 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.697 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.697 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.697 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.697 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.697 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.698 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.698 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.698 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.698 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.698 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.698 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.698 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.698 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.698 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.698 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.698 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.698 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.698 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.698 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.699 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.699 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.699 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.699 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.699 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.699 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.699 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.699 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.699 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.699 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.699 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.699 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.699 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.700 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-05-04T03:25:28.700 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-05-04T03:25:28.700 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.700 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.700 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.700 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.700 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.700 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.700 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.700 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.700 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.700 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.700 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.700 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-05-04T03:25:28.701 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-05-04T03:25:28.701 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.701 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.701 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.701 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.701 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.701 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.701 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.701 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.701 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.701 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.701 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.701 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.701 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.702 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.702 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.702 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.702 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.702 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.702 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.702 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.702 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.702 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.702 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.702 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.702 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.702 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.703 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rgw", 2024-05-04T03:25:28.703 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.703 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.703 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.703 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.703 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.703 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.703 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.703 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.703 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.703 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.703 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.703 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.704 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.704 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.704 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.704 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.704 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.704 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.704 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.704 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.704 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.704 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.704 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.704 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.704 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.704 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.704 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.705 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.705 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.705 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.705 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.705 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.705 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.705 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.705 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.705 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.705 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.705 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.705 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.705 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.706 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.706 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.706 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.706 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.706 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.706 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.706 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.706 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.706 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.706 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.706 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.706 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.706 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.706 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.707 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.707 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.707 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.707 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.707 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.707 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.707 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.707 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.707 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.707 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.707 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.707 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.707 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.707 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.708 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.708 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.708 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.708 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.708 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.708 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.708 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.708 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.708 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.708 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.708 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.708 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.708 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.708 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.709 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.709 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.709 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.709 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rook", 2024-05-04T03:25:28.709 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.709 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.709 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.709 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.709 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.709 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.709 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.709 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.709 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.709 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.710 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.710 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.710 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.710 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.710 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.710 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.710 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.710 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.710 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.710 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.710 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.710 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.710 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.711 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.711 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.711 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.711 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.711 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.711 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.711 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.711 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.711 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.711 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.711 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.711 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.711 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.712 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.712 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.712 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.712 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.712 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.712 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.712 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.712 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.712 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.712 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.712 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.712 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.712 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.712 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.713 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.713 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.713 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.713 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.713 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.713 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.713 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.713 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.713 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.713 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.713 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.713 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.713 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.713 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.714 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.714 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.714 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.714 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.714 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.714 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.714 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.714 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.714 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.714 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.714 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.714 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.714 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.715 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.715 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.715 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.715 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.715 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.715 INFO:teuthology.orchestra.run.smithi001.stdout: "storage_class": { 2024-05-04T03:25:28.715 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "storage_class", 2024-05-04T03:25:28.715 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.715 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.715 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.715 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "local", 2024-05-04T03:25:28.715 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.715 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.716 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.716 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-04T03:25:28.716 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.716 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.716 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.716 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.716 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.716 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "selftest", 2024-05-04T03:25:28.716 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.716 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.716 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.716 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.716 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.717 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.717 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.717 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.717 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.717 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.717 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.717 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.717 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.717 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.717 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.717 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.717 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.717 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.717 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.717 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.718 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.718 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.718 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.718 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.718 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.718 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.718 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.718 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.718 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.718 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.718 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.718 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.718 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.718 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.719 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.719 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.719 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.719 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.719 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.719 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.719 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.719 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.719 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.719 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.719 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.719 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.719 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.719 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.720 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.720 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.720 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.720 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.720 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.720 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.720 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.720 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.720 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.720 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.720 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.720 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.720 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.721 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.721 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.721 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.721 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.721 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.721 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.721 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.721 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.721 INFO:teuthology.orchestra.run.smithi001.stdout: "roption1": { 2024-05-04T03:25:28.721 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption1", 2024-05-04T03:25:28.721 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.721 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.721 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.721 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.722 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.722 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.722 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.722 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.722 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.722 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.722 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.722 INFO:teuthology.orchestra.run.smithi001.stdout: "roption2": { 2024-05-04T03:25:28.722 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption2", 2024-05-04T03:25:28.722 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.722 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.722 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.722 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "xyz", 2024-05-04T03:25:28.723 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.723 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.723 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.723 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.723 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.723 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.723 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.723 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption1": { 2024-05-04T03:25:28.723 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption1", 2024-05-04T03:25:28.723 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.723 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.723 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.723 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.723 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.724 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.724 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.724 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.724 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.724 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.724 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.724 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption2": { 2024-05-04T03:25:28.724 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption2", 2024-05-04T03:25:28.724 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.724 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.724 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.724 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.725 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.725 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.725 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.725 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.725 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.725 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.725 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.725 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption3": { 2024-05-04T03:25:28.725 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption3", 2024-05-04T03:25:28.725 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-05-04T03:25:28.725 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.725 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.725 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.726 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.726 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.726 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.726 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.726 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.726 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.726 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.726 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption4": { 2024-05-04T03:25:28.726 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption4", 2024-05-04T03:25:28.726 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.726 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.726 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.726 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.726 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.727 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.727 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.727 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.727 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.727 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.727 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.727 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption5": { 2024-05-04T03:25:28.727 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption5", 2024-05-04T03:25:28.727 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.727 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.727 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.727 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.727 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.728 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.728 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.728 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.728 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.728 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.728 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.728 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption6": { 2024-05-04T03:25:28.728 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption6", 2024-05-04T03:25:28.728 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.728 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.728 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.728 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.728 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.729 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.729 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.729 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.729 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.729 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.729 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.729 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption7": { 2024-05-04T03:25:28.729 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption7", 2024-05-04T03:25:28.729 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.729 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.729 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.729 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.729 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-05-04T03:25:28.729 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "42", 2024-05-04T03:25:28.730 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.730 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.730 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.730 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.730 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.730 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.730 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.730 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.730 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.730 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.730 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.730 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.730 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.731 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.731 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.731 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.731 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.731 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.731 INFO:teuthology.orchestra.run.smithi001.stdout: "testkey": { 2024-05-04T03:25:28.731 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testkey", 2024-05-04T03:25:28.731 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.731 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.731 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.731 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.731 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.731 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.731 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.732 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.732 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.732 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.732 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.732 INFO:teuthology.orchestra.run.smithi001.stdout: "testlkey": { 2024-05-04T03:25:28.732 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testlkey", 2024-05-04T03:25:28.732 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.732 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.732 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.732 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.732 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.732 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.732 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.733 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.733 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.733 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.733 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.733 INFO:teuthology.orchestra.run.smithi001.stdout: "testnewline": { 2024-05-04T03:25:28.733 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testnewline", 2024-05-04T03:25:28.733 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.733 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.733 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.733 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.733 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.733 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.733 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.733 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.734 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.734 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.734 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.734 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.734 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.734 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smb", 2024-05-04T03:25:28.734 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.734 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.734 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.734 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.734 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.734 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.735 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.735 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.735 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.735 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.735 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.735 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.735 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.735 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.735 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.735 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.735 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.735 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.735 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.735 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.736 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.736 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.736 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.736 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.736 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.736 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.736 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.736 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.736 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.736 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.736 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.736 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.736 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.736 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.737 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.737 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.737 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.737 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.737 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.737 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.737 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.737 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.737 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.737 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.737 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.737 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.737 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.738 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.738 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.738 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.738 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.738 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.738 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.738 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.738 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.738 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.738 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.738 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.738 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.738 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.738 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.739 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.739 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.739 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.739 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.739 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.739 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.739 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.739 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.739 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.739 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.739 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.739 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.739 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.739 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.740 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.740 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.740 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.740 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.740 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.740 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.740 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.740 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.740 INFO:teuthology.orchestra.run.smithi001.stdout: "update_orchestration": { 2024-05-04T03:25:28.740 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "update_orchestration", 2024-05-04T03:25:28.740 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.740 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.740 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.741 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.741 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.741 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.741 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.741 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-04T03:25:28.741 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.741 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.741 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.741 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.741 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.741 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.741 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2024-05-04T03:25:28.741 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.741 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.742 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.742 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_m_granularity": { 2024-05-04T03:25:28.742 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_m_granularity", 2024-05-04T03:25:28.742 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.742 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.742 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.742 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.742 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.742 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.742 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.742 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow minute scheduled snapshots", 2024-05-04T03:25:28.742 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.742 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.742 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.743 INFO:teuthology.orchestra.run.smithi001.stdout: "dump_on_update": { 2024-05-04T03:25:28.743 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dump_on_update", 2024-05-04T03:25:28.743 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.743 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.743 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.743 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.743 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.743 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.743 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.743 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "dump database to debug log on update", 2024-05-04T03:25:28.743 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.743 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.743 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.744 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.744 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.744 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.744 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.744 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.744 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.744 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.744 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.744 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.744 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.744 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.744 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.744 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.744 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.745 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.745 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.745 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.745 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.745 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.745 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.745 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.745 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.745 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.745 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.745 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.745 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.745 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.745 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.746 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.746 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.746 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.746 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.746 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.746 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.746 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.746 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.746 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.746 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.746 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.746 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.746 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.747 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.747 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.747 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.747 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.747 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.747 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.747 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.747 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.747 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.747 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.747 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.747 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.747 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.748 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.748 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.748 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.748 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.748 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.748 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.748 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.748 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.748 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.748 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.748 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.748 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.748 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.749 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.749 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.749 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.749 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.749 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.749 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.749 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.749 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.749 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.749 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.749 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.749 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.749 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.749 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.750 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.750 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.750 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stats", 2024-05-04T03:25:28.750 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.750 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.750 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.750 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.750 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.750 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.750 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.750 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.750 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.750 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.751 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.751 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.751 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.751 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.751 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.751 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.751 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.751 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.751 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.751 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.751 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.751 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.751 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.752 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.752 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.752 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.752 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.752 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.752 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.752 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.752 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.752 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.752 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.752 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.752 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.752 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.753 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.753 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.753 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.753 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.753 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.753 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.753 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.753 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.753 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.753 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.753 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.753 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.753 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.754 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.754 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.754 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.754 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.754 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.754 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.754 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.754 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.754 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.754 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.754 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.754 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.754 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.755 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.755 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.755 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.755 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.755 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.755 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.755 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.755 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.755 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.755 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.755 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.755 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.756 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.756 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.756 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.756 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.756 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.756 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.756 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.756 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.756 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.756 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.756 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.756 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "status", 2024-05-04T03:25:28.756 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.756 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.757 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.757 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.757 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.757 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.757 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.757 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.757 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.757 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.757 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.757 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.757 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.757 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.757 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.757 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.758 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.758 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.758 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.758 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.758 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.758 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.758 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.758 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.758 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.758 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.758 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.758 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.758 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.759 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.759 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.759 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.759 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.759 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.759 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.759 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.759 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.759 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.759 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.759 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.759 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.759 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.759 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.760 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.760 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.760 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.760 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.760 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.760 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.760 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.760 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.760 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.760 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.760 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.760 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.760 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.760 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.761 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.761 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.761 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.761 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.761 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.761 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.761 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.761 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.761 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.761 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.761 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.761 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.761 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.762 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.762 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.762 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.762 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.762 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.762 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.762 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.762 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.762 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.762 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.762 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.762 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.762 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.762 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.763 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.763 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.763 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telegraf", 2024-05-04T03:25:28.763 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.763 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.763 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.763 INFO:teuthology.orchestra.run.smithi001.stdout: "address": { 2024-05-04T03:25:28.763 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "address", 2024-05-04T03:25:28.763 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.763 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.763 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.763 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-04T03:25:28.763 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.764 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.764 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.764 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.764 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.764 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.764 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.764 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-05-04T03:25:28.764 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-05-04T03:25:28.764 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.764 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.764 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.764 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15", 2024-05-04T03:25:28.764 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.764 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.765 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.765 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.765 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.765 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.765 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.765 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.765 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.765 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.765 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.765 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.765 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.765 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.766 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.766 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.766 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.766 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.766 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.766 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.766 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.766 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.766 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.766 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.766 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.766 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.766 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.767 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.767 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.767 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.767 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.767 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.767 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.767 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.767 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.767 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.767 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.767 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.767 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.767 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.767 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.768 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.768 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.768 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.768 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.768 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.768 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.768 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.768 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.768 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.768 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.768 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.768 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.768 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.768 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.769 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.769 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.769 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.769 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.769 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.769 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.769 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.769 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.769 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.769 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.769 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.769 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.769 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.769 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.770 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.770 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.770 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.770 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.770 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.770 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.770 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.770 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.770 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.770 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.770 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.770 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.770 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.771 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.771 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.771 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.771 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.771 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.771 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.771 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.771 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.771 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.771 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telemetry", 2024-05-04T03:25:28.771 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.771 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.771 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.771 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_basic": { 2024-05-04T03:25:28.772 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_basic", 2024-05-04T03:25:28.772 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.772 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.772 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.772 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.772 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.772 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.772 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.772 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-04T03:25:28.772 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.772 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.772 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.772 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.772 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_crash": { 2024-05-04T03:25:28.773 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_crash", 2024-05-04T03:25:28.773 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.773 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.773 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.773 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.773 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.773 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.773 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.773 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-04T03:25:28.773 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.773 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.773 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.773 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_device": { 2024-05-04T03:25:28.774 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_device", 2024-05-04T03:25:28.774 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.774 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.774 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.774 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.774 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.774 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.774 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.774 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-04T03:25:28.774 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.774 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.774 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.774 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.774 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_ident": { 2024-05-04T03:25:28.774 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_ident", 2024-05-04T03:25:28.775 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.775 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.775 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.775 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.775 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.775 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.775 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.775 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-04T03:25:28.775 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.775 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.775 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.775 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_perf": { 2024-05-04T03:25:28.775 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_perf", 2024-05-04T03:25:28.776 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.776 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.776 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.776 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.776 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.776 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.776 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.776 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-04T03:25:28.776 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.776 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.776 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.776 INFO:teuthology.orchestra.run.smithi001.stdout: "contact": { 2024-05-04T03:25:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "contact", 2024-05-04T03:25:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: "description": { 2024-05-04T03:25:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "description", 2024-05-04T03:25:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.777 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.778 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.778 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.778 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.778 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.778 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.778 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.778 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.778 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.778 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.778 INFO:teuthology.orchestra.run.smithi001.stdout: "device_url": { 2024-05-04T03:25:28.778 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_url", 2024-05-04T03:25:28.778 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.779 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.779 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.779 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-04T03:25:28.779 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.779 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.779 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.779 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.779 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.779 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.779 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.779 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-05-04T03:25:28.779 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-05-04T03:25:28.779 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.779 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.780 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.780 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.780 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.780 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.780 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.780 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.780 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.780 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.780 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.780 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-05-04T03:25:28.780 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-05-04T03:25:28.780 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.780 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.780 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.780 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "24", 2024-05-04T03:25:28.781 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "8", 2024-05-04T03:25:28.781 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.781 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.781 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.781 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.781 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.781 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.781 INFO:teuthology.orchestra.run.smithi001.stdout: "last_opt_revision": { 2024-05-04T03:25:28.781 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "last_opt_revision", 2024-05-04T03:25:28.781 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.781 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.781 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.781 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1", 2024-05-04T03:25:28.781 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.781 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.782 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.782 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.782 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.782 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.782 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.782 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard": { 2024-05-04T03:25:28.782 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard", 2024-05-04T03:25:28.782 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.782 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.782 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.782 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.782 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.783 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.783 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.783 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.783 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.783 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.783 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.783 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.783 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard_description": { 2024-05-04T03:25:28.783 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard_description", 2024-05-04T03:25:28.783 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.783 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.783 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.783 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.783 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.784 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.784 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.784 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.784 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.784 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.784 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.784 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.784 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.784 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.784 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.784 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.784 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.784 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.785 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.785 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.785 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.785 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.785 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.785 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.785 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.785 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.785 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.785 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.785 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.785 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.785 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.785 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.786 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.786 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.786 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.786 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.786 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.786 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.786 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.786 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.786 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.786 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.786 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.786 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.786 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.787 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.787 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.787 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.787 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.787 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.787 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.787 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.787 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.787 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.787 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.787 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.787 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.787 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.787 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.788 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.788 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.788 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.788 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.788 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.788 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.788 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.788 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.788 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.788 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.788 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.788 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.788 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.788 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.788 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.789 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.789 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.789 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.789 INFO:teuthology.orchestra.run.smithi001.stdout: "organization": { 2024-05-04T03:25:28.789 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "organization", 2024-05-04T03:25:28.789 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.789 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.789 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.789 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.789 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.789 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.789 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.789 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.789 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.790 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.790 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.790 INFO:teuthology.orchestra.run.smithi001.stdout: "proxy": { 2024-05-04T03:25:28.790 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "proxy", 2024-05-04T03:25:28.790 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.790 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.790 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.790 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.790 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.790 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.790 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.790 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.790 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.791 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.791 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.791 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.791 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.791 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.791 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.791 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.791 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.791 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.791 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.791 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.791 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.791 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.792 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.792 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.792 INFO:teuthology.orchestra.run.smithi001.stdout: "url": { 2024-05-04T03:25:28.792 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url", 2024-05-04T03:25:28.792 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.792 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.792 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.792 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-04T03:25:28.792 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.792 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.792 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.792 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.793 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.793 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.793 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.793 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.793 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.793 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.793 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "test_orchestrator", 2024-05-04T03:25:28.793 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.793 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.793 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.793 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.793 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.793 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.793 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.794 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.794 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.794 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.794 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.794 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.794 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.794 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.794 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.794 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.794 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.794 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.794 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.794 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.794 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.794 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.796 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.796 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.796 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.796 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.796 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.796 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.796 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.796 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.796 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.796 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.796 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.796 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.796 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.796 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.796 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.796 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.797 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.797 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.797 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.797 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.797 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.797 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.797 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.797 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.797 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.797 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.797 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.797 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.797 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.797 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.797 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.798 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.798 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.798 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.798 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.798 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.798 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.798 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.798 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.798 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.798 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.798 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.798 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.798 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.798 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.799 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.799 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.799 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.799 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.799 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.799 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "volumes", 2024-05-04T03:25:28.799 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.799 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.799 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.799 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.799 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.799 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.799 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.799 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.799 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.799 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.800 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.801 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.801 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.801 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.801 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.801 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.801 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.801 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.801 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.801 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.801 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.801 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.801 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.801 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.801 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.801 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.802 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.802 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.802 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.802 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.802 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.802 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.802 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.802 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.802 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.802 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.802 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.802 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.802 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.802 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.802 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.803 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.803 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.803 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.803 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.803 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.803 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.803 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.803 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.803 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.803 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.803 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.803 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.803 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.803 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_clones": { 2024-05-04T03:25:28.804 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_clones", 2024-05-04T03:25:28.804 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.804 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.804 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.804 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4", 2024-05-04T03:25:28.804 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.804 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.804 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.804 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-04T03:25:28.804 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.804 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.804 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.804 INFO:teuthology.orchestra.run.smithi001.stdout: "periodic_async_work": { 2024-05-04T03:25:28.804 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "periodic_async_work", 2024-05-04T03:25:28.805 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.805 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.805 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.805 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.805 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.805 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.805 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.805 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Periodically check for async work", 2024-05-04T03:25:28.805 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.805 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.805 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.805 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_delay": { 2024-05-04T03:25:28.805 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_delay", 2024-05-04T03:25:28.805 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.805 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.806 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.806 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.806 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.806 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.806 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.806 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-04T03:25:28.806 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.806 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.806 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.806 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_no_wait": { 2024-05-04T03:25:28.806 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_no_wait", 2024-05-04T03:25:28.806 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.806 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.806 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.807 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.807 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.807 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.807 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.807 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-04T03:25:28.807 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.807 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.807 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.807 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.807 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.807 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.807 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.807 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.807 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.807 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.808 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.808 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.808 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.808 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.808 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.808 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.808 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.808 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.808 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.808 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix", 2024-05-04T03:25:28.808 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.808 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.808 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.808 INFO:teuthology.orchestra.run.smithi001.stdout: "discovery_interval": { 2024-05-04T03:25:28.808 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "discovery_interval", 2024-05-04T03:25:28.809 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-05-04T03:25:28.809 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.809 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.809 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "100", 2024-05-04T03:25:28.809 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.809 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.809 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.809 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.809 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.809 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.809 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.809 INFO:teuthology.orchestra.run.smithi001.stdout: "identifier": { 2024-05-04T03:25:28.809 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "identifier", 2024-05-04T03:25:28.809 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.809 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.810 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.810 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.810 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.810 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.810 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.810 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.810 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.810 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.810 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.810 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-05-04T03:25:28.810 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-05-04T03:25:28.810 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.810 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.811 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.811 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-05-04T03:25:28.811 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.811 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.811 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.811 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.811 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.811 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.811 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.811 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.811 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.811 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.811 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.811 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.812 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.812 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.812 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.812 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.812 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.812 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.812 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.812 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.812 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.812 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.812 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.812 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.812 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.812 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.812 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.813 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.813 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.813 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.813 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.813 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.813 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.813 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.813 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.813 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.813 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.813 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.813 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.813 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.814 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.814 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.814 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.814 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.814 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.814 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.814 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.814 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.814 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.814 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.814 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.814 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.814 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.814 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.814 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.815 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.815 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.815 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.815 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.815 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.815 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.815 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.815 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.815 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.815 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.815 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.815 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.815 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.815 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.816 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.816 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.816 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.816 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.816 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.816 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.816 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.816 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.816 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.816 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.816 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.816 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.816 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.817 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.817 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.817 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.817 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.817 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_host": { 2024-05-04T03:25:28.817 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_host", 2024-05-04T03:25:28.817 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.817 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.817 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.817 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.817 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.817 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.817 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.817 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.817 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.818 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.818 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.818 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_port": { 2024-05-04T03:25:28.818 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_port", 2024-05-04T03:25:28.818 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.818 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.818 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.818 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10051", 2024-05-04T03:25:28.818 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.818 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.818 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.818 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_sender": { 2024-05-04T03:25:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_sender", 2024-05-04T03:25:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-04T03:25:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.819 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-04T03:25:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: "gid": 24112, 2024-05-04T03:25:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "y", 2024-05-04T03:25:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_features": 4540701547738038271, 2024-05-04T03:25:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: "available_modules": [ 2024-05-04T03:25:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "alerts", 2024-05-04T03:25:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.820 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.821 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-05-04T03:25:28.821 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-05-04T03:25:28.821 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.821 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.821 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.821 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-05-04T03:25:28.821 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.821 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.821 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.821 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How frequently to reexamine health status", 2024-05-04T03:25:28.821 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.821 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.821 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.821 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.821 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.822 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.822 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.822 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.822 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.822 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.822 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.822 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.822 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.822 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.822 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.822 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.822 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.822 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.822 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.822 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.823 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.823 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.823 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.823 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.823 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.823 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.823 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.823 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.823 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.823 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.823 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.823 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.823 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.824 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.824 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.824 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.824 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.824 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.824 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.824 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.824 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.824 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.824 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.824 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.824 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.824 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.824 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.825 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.826 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.826 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.826 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.826 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.826 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.826 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.826 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_destination": { 2024-05-04T03:25:28.826 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_destination", 2024-05-04T03:25:28.826 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.826 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.826 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.826 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.826 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.826 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email address to send alerts to", 2024-05-04T03:25:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_from_name": { 2024-05-04T03:25:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_from_name", 2024-05-04T03:25:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "Ceph", 2024-05-04T03:25:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.827 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email From: name", 2024-05-04T03:25:28.828 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.828 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.828 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.828 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.828 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_host": { 2024-05-04T03:25:28.828 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_host", 2024-05-04T03:25:28.828 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.828 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.828 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.828 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.828 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.828 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.828 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.828 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP server", 2024-05-04T03:25:28.828 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.829 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.829 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.829 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_password": { 2024-05-04T03:25:28.829 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_password", 2024-05-04T03:25:28.829 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.829 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.829 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.829 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.829 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.829 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.829 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.829 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Password to authenticate with", 2024-05-04T03:25:28.829 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.829 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.829 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.830 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_port": { 2024-05-04T03:25:28.830 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_port", 2024-05-04T03:25:28.830 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.830 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.830 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.830 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "465", 2024-05-04T03:25:28.830 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.830 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.830 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.830 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP port", 2024-05-04T03:25:28.830 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.830 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.830 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.831 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_sender": { 2024-05-04T03:25:28.831 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_sender", 2024-05-04T03:25:28.831 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.831 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.831 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.831 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.831 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.831 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.831 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.831 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP envelope sender", 2024-05-04T03:25:28.831 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.831 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.831 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.831 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_ssl": { 2024-05-04T03:25:28.832 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_ssl", 2024-05-04T03:25:28.832 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.832 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.832 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.832 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.832 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.832 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.832 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.832 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-04T03:25:28.832 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.832 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.832 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.832 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_user": { 2024-05-04T03:25:28.832 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_user", 2024-05-04T03:25:28.832 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.833 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.833 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.833 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.833 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.833 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.833 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.833 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "User to authenticate as", 2024-05-04T03:25:28.833 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.833 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.833 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.833 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.833 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.833 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.833 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.833 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.834 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.834 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.834 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.834 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.834 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.834 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.834 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.834 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.834 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.834 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.834 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.834 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.834 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "balancer", 2024-05-04T03:25:28.834 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.835 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.835 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.835 INFO:teuthology.orchestra.run.smithi001.stdout: "active": { 2024-05-04T03:25:28.835 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "active", 2024-05-04T03:25:28.835 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.835 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.835 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.835 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.835 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.835 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.835 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.835 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically balance PGs across cluster", 2024-05-04T03:25:28.835 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.835 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.835 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.836 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_time": { 2024-05-04T03:25:28.836 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_time", 2024-05-04T03:25:28.836 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.836 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.836 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.836 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0000", 2024-05-04T03:25:28.836 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.836 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.836 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.836 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "beginning time of day to automatically balance", 2024-05-04T03:25:28.836 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-04T03:25:28.836 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.836 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.836 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_weekday": { 2024-05-04T03:25:28.836 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_weekday", 2024-05-04T03:25:28.837 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-05-04T03:25:28.837 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.837 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.837 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.837 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-05-04T03:25:28.837 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-05-04T03:25:28.837 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.837 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-04T03:25:28.837 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-04T03:25:28.837 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.837 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.837 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_max_iterations": { 2024-05-04T03:25:28.837 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_max_iterations", 2024-05-04T03:25:28.837 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-05-04T03:25:28.837 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.838 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.838 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "25", 2024-05-04T03:25:28.838 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-05-04T03:25:28.838 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "250", 2024-05-04T03:25:28.838 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.838 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-04T03:25:28.838 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.838 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.838 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.838 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_metrics": { 2024-05-04T03:25:28.838 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_metrics", 2024-05-04T03:25:28.838 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.838 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.838 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.839 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "pgs,objects,bytes", 2024-05-04T03:25:28.839 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.839 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.839 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.839 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-04T03:25:28.839 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-04T03:25:28.839 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.839 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.839 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_step": { 2024-05-04T03:25:28.839 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_step", 2024-05-04T03:25:28.839 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-05-04T03:25:28.839 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.839 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.839 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.5", 2024-05-04T03:25:28.839 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0.001", 2024-05-04T03:25:28.840 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "0.999", 2024-05-04T03:25:28.840 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.840 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "aggressiveness of optimization", 2024-05-04T03:25:28.840 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-04T03:25:28.840 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.840 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.840 INFO:teuthology.orchestra.run.smithi001.stdout: "end_time": { 2024-05-04T03:25:28.840 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_time", 2024-05-04T03:25:28.840 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.840 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.840 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.840 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2359", 2024-05-04T03:25:28.840 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.840 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.841 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.841 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ending time of day to automatically balance", 2024-05-04T03:25:28.841 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-04T03:25:28.841 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.841 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.841 INFO:teuthology.orchestra.run.smithi001.stdout: "end_weekday": { 2024-05-04T03:25:28.841 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_weekday", 2024-05-04T03:25:28.841 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-05-04T03:25:28.841 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.841 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.841 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.841 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-05-04T03:25:28.841 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-05-04T03:25:28.841 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.841 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-04T03:25:28.841 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-04T03:25:28.842 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.842 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.842 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.842 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.842 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.842 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.842 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.842 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.842 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.842 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.842 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.842 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.842 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.843 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.843 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.843 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.843 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.843 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.843 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.843 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.843 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.843 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.843 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.843 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.843 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.843 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.843 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.843 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.843 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.844 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.844 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.844 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.844 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.844 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.844 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.844 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.844 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.844 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.844 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.844 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.844 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.844 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.844 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.844 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.844 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.845 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.845 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.845 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.845 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.845 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.845 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.845 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.845 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.845 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.845 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.845 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.845 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.845 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.845 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.846 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.846 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.846 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.846 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.846 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.846 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.846 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.846 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.846 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.846 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.846 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.846 INFO:teuthology.orchestra.run.smithi001.stdout: "min_score": { 2024-05-04T03:25:28.846 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_score", 2024-05-04T03:25:28.846 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-05-04T03:25:28.847 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.847 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.847 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.847 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.847 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.847 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.847 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-04T03:25:28.847 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.847 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.847 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.847 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-05-04T03:25:28.847 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-05-04T03:25:28.847 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.848 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.848 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.848 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "upmap", 2024-05-04T03:25:28.848 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.848 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.848 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.848 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-compat", 2024-05-04T03:25:28.848 INFO:teuthology.orchestra.run.smithi001.stdout: "none", 2024-05-04T03:25:28.848 INFO:teuthology.orchestra.run.smithi001.stdout: "read", 2024-05-04T03:25:28.848 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap", 2024-05-04T03:25:28.848 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap-read" 2024-05-04T03:25:28.848 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.848 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Balancer mode", 2024-05-04T03:25:28.848 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.848 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.849 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.849 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_ids": { 2024-05-04T03:25:28.849 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_ids", 2024-05-04T03:25:28.849 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.849 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.849 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.849 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.849 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.849 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.849 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.849 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-04T03:25:28.849 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.850 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.850 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.850 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-05-04T03:25:28.850 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-05-04T03:25:28.850 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.850 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.850 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.850 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-05-04T03:25:28.850 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.850 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.850 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.851 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-04T03:25:28.851 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.851 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.851 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.851 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.851 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.851 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.851 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.851 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.851 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.851 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.851 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.851 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.852 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.852 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.852 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.852 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.852 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_deviation": { 2024-05-04T03:25:28.852 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_deviation", 2024-05-04T03:25:28.852 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.852 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.852 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.852 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-05-04T03:25:28.852 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-05-04T03:25:28.852 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.853 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.853 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-04T03:25:28.853 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-04T03:25:28.853 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.853 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.853 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_optimizations": { 2024-05-04T03:25:28.853 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_optimizations", 2024-05-04T03:25:28.853 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-05-04T03:25:28.853 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.853 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.853 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-05-04T03:25:28.853 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.853 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.854 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.854 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-04T03:25:28.854 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.854 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.854 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.854 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.854 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.854 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm", 2024-05-04T03:25:28.854 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.854 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.854 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.854 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_down_multiplier": { 2024-05-04T03:25:28.855 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_down_multiplier", 2024-05-04T03:25:28.855 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-05-04T03:25:28.855 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.855 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.855 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-05-04T03:25:28.855 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.855 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.855 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.855 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-04T03:25:28.855 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.855 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.855 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.855 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_refresh_rate": { 2024-05-04T03:25:28.855 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_refresh_rate", 2024-05-04T03:25:28.856 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.856 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.856 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.856 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "20", 2024-05-04T03:25:28.856 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.856 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.856 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.856 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-04T03:25:28.856 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.856 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.856 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.856 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.856 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_starting_port": { 2024-05-04T03:25:28.856 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_starting_port", 2024-05-04T03:25:28.857 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.857 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.857 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.857 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4721", 2024-05-04T03:25:28.857 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.857 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.857 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.857 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-04T03:25:28.857 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.857 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.857 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.857 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_ptrace": { 2024-05-04T03:25:28.857 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_ptrace", 2024-05-04T03:25:28.858 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.858 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.858 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.858 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.858 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.858 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.858 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.858 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-04T03:25:28.858 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-04T03:25:28.858 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.858 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.858 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_interval": { 2024-05-04T03:25:28.858 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_interval", 2024-05-04T03:25:28.858 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.859 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.859 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.859 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-05-04T03:25:28.859 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.859 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.859 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.859 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-04T03:25:28.859 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.859 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.859 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.859 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_memory_target_ratio": { 2024-05-04T03:25:28.859 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_memory_target_ratio", 2024-05-04T03:25:28.859 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-05-04T03:25:28.859 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.860 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.860 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.7", 2024-05-04T03:25:28.860 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.860 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.860 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.860 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-04T03:25:28.860 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.860 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.860 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.860 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm_log_destination": { 2024-05-04T03:25:28.860 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm_log_destination", 2024-05-04T03:25:28.860 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.860 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.861 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.861 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.861 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.861 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.861 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.861 INFO:teuthology.orchestra.run.smithi001.stdout: "file", 2024-05-04T03:25:28.861 INFO:teuthology.orchestra.run.smithi001.stdout: "file,syslog", 2024-05-04T03:25:28.861 INFO:teuthology.orchestra.run.smithi001.stdout: "syslog" 2024-05-04T03:25:28.861 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.861 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-04T03:25:28.861 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.861 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.861 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.862 INFO:teuthology.orchestra.run.smithi001.stdout: "cgroups_split": { 2024-05-04T03:25:28.862 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cgroups_split", 2024-05-04T03:25:28.862 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.862 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.862 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.862 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.862 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.862 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.862 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.862 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-04T03:25:28.862 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.862 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.862 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.862 INFO:teuthology.orchestra.run.smithi001.stdout: "config_checks_enabled": { 2024-05-04T03:25:28.863 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_checks_enabled", 2024-05-04T03:25:28.863 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.863 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.863 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.863 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.863 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.863 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.863 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.863 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-04T03:25:28.863 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.863 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.863 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.863 INFO:teuthology.orchestra.run.smithi001.stdout: "config_dashboard": { 2024-05-04T03:25:28.863 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_dashboard", 2024-05-04T03:25:28.864 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.864 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.864 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.864 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.864 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.864 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.864 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.864 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-04T03:25:28.864 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.864 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.864 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.864 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_alertmanager": { 2024-05-04T03:25:28.864 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_alertmanager", 2024-05-04T03:25:28.865 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.865 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.865 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.865 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-04T03:25:28.865 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.865 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.865 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.865 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-05-04T03:25:28.865 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.865 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.865 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.865 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.865 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_base": { 2024-05-04T03:25:28.865 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_base", 2024-05-04T03:25:28.866 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.866 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.866 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.866 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-04T03:25:28.866 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.866 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.866 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.866 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Container image name, without the tag", 2024-05-04T03:25:28.866 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.866 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.866 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.866 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_elasticsearch": { 2024-05-04T03:25:28.866 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_elasticsearch", 2024-05-04T03:25:28.867 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.867 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.867 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.867 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-04T03:25:28.867 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.867 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.867 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.867 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "elasticsearch container image", 2024-05-04T03:25:28.867 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.867 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.867 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.867 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_grafana": { 2024-05-04T03:25:28.867 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_grafana", 2024-05-04T03:25:28.867 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.868 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.868 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.868 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-04T03:25:28.868 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.868 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.868 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.868 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-05-04T03:25:28.868 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.868 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.868 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.868 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_haproxy": { 2024-05-04T03:25:28.868 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_haproxy", 2024-05-04T03:25:28.868 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.868 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.868 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.869 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-04T03:25:28.869 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.869 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.869 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.869 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "HAproxy container image", 2024-05-04T03:25:28.869 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.869 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.869 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.869 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.869 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_agent": { 2024-05-04T03:25:28.869 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_agent", 2024-05-04T03:25:28.869 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.869 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.869 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.869 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-04T03:25:28.869 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.870 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.870 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.870 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger agent container image", 2024-05-04T03:25:28.870 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.870 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.870 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.870 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_collector": { 2024-05-04T03:25:28.870 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_collector", 2024-05-04T03:25:28.870 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.870 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.870 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.870 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-04T03:25:28.870 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.870 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.871 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.871 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger collector container image", 2024-05-04T03:25:28.871 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.871 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.871 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.871 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_query": { 2024-05-04T03:25:28.871 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_query", 2024-05-04T03:25:28.871 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.871 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.871 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.871 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-04T03:25:28.871 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.871 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.871 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.871 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger query container image", 2024-05-04T03:25:28.872 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.872 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.872 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.872 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_keepalived": { 2024-05-04T03:25:28.872 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_keepalived", 2024-05-04T03:25:28.872 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.872 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.872 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.872 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-04T03:25:28.872 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.872 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.872 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.872 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Keepalived container image", 2024-05-04T03:25:28.872 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.872 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.873 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.873 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_loki": { 2024-05-04T03:25:28.873 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_loki", 2024-05-04T03:25:28.873 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.873 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.873 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.873 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-04T03:25:28.873 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.873 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.873 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.873 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Loki container image", 2024-05-04T03:25:28.873 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.873 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.873 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.874 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_node_exporter": { 2024-05-04T03:25:28.874 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_node_exporter", 2024-05-04T03:25:28.874 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.874 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.874 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.874 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-04T03:25:28.874 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.874 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.874 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.874 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-05-04T03:25:28.874 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.874 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.874 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.875 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_nvmeof": { 2024-05-04T03:25:28.875 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_nvmeof", 2024-05-04T03:25:28.875 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.875 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.875 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.875 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-04T03:25:28.875 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.875 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.875 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.875 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Nvme-of container image", 2024-05-04T03:25:28.875 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.875 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.875 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.875 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_prometheus": { 2024-05-04T03:25:28.875 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_prometheus", 2024-05-04T03:25:28.876 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.876 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.876 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.876 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-04T03:25:28.876 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.876 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.876 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.876 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-05-04T03:25:28.876 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.876 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.876 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.876 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_promtail": { 2024-05-04T03:25:28.876 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_promtail", 2024-05-04T03:25:28.876 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.876 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.877 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.877 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-04T03:25:28.877 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.877 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.877 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.877 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Promtail container image", 2024-05-04T03:25:28.877 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.877 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.877 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.877 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_samba": { 2024-05-04T03:25:28.877 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_samba", 2024-05-04T03:25:28.877 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.877 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.877 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.877 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-04T03:25:28.878 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.878 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.878 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.878 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Samba/SMB container image", 2024-05-04T03:25:28.878 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.878 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.878 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.878 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_snmp_gateway": { 2024-05-04T03:25:28.878 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_snmp_gateway", 2024-05-04T03:25:28.878 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.878 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.878 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.878 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-04T03:25:28.879 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.879 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.879 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.879 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SNMP Gateway container image", 2024-05-04T03:25:28.879 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.879 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.879 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.879 INFO:teuthology.orchestra.run.smithi001.stdout: "container_init": { 2024-05-04T03:25:28.879 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_init", 2024-05-04T03:25:28.879 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.879 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.879 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.879 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.879 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.880 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.880 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.880 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Run podman/docker with `--init`", 2024-05-04T03:25:28.880 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.880 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.880 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.880 INFO:teuthology.orchestra.run.smithi001.stdout: "daemon_cache_timeout": { 2024-05-04T03:25:28.880 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "daemon_cache_timeout", 2024-05-04T03:25:28.880 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.880 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.880 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.880 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-05-04T03:25:28.880 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.880 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.880 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.881 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-04T03:25:28.881 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.881 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.881 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.881 INFO:teuthology.orchestra.run.smithi001.stdout: "default_cephadm_command_timeout": { 2024-05-04T03:25:28.881 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_cephadm_command_timeout", 2024-05-04T03:25:28.881 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.881 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.881 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.881 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "900", 2024-05-04T03:25:28.881 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.881 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.881 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.881 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-04T03:25:28.881 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.881 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.882 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.882 INFO:teuthology.orchestra.run.smithi001.stdout: "default_registry": { 2024-05-04T03:25:28.882 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_registry", 2024-05-04T03:25:28.882 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.882 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.882 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.882 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io", 2024-05-04T03:25:28.882 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.882 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.882 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.882 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-04T03:25:28.882 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.882 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.883 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.883 INFO:teuthology.orchestra.run.smithi001.stdout: "device_cache_timeout": { 2024-05-04T03:25:28.883 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_cache_timeout", 2024-05-04T03:25:28.883 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.883 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.883 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.883 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1800", 2024-05-04T03:25:28.883 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.883 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.883 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.883 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache device inventory", 2024-05-04T03:25:28.883 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.883 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.883 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.884 INFO:teuthology.orchestra.run.smithi001.stdout: "device_enhanced_scan": { 2024-05-04T03:25:28.884 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_enhanced_scan", 2024-05-04T03:25:28.884 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.884 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.884 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.884 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.884 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.884 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.884 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.884 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-04T03:25:28.884 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.884 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.884 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.884 INFO:teuthology.orchestra.run.smithi001.stdout: "facts_cache_timeout": { 2024-05-04T03:25:28.885 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "facts_cache_timeout", 2024-05-04T03:25:28.885 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.885 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.885 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.885 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-05-04T03:25:28.885 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.885 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.885 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.885 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache host facts data", 2024-05-04T03:25:28.885 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.885 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.885 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.885 INFO:teuthology.orchestra.run.smithi001.stdout: "grafana_dashboards_path": { 2024-05-04T03:25:28.885 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "grafana_dashboards_path", 2024-05-04T03:25:28.885 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.886 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.886 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.886 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-04T03:25:28.886 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.886 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.886 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.886 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-04T03:25:28.886 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.886 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.886 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.886 INFO:teuthology.orchestra.run.smithi001.stdout: "host_check_interval": { 2024-05-04T03:25:28.886 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "host_check_interval", 2024-05-04T03:25:28.886 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.886 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.886 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.887 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-05-04T03:25:28.887 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.887 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.887 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.887 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to perform a host check", 2024-05-04T03:25:28.887 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.887 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.887 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.887 INFO:teuthology.orchestra.run.smithi001.stdout: "hw_monitoring": { 2024-05-04T03:25:28.887 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hw_monitoring", 2024-05-04T03:25:28.887 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.887 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.887 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.888 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.888 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.888 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.888 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.888 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-04T03:25:28.888 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.888 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.888 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.888 INFO:teuthology.orchestra.run.smithi001.stdout: "inventory_list_all": { 2024-05-04T03:25:28.888 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "inventory_list_all", 2024-05-04T03:25:28.888 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.888 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.888 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.888 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.888 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.888 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.889 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.889 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-04T03:25:28.889 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.889 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.889 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.889 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.889 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.889 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.889 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.889 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.889 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.889 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.889 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.889 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.889 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.890 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.890 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.890 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.890 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.890 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.890 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.890 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.890 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.890 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.890 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.890 INFO:teuthology.orchestra.run.smithi001.stdout: "log_refresh_metadata": { 2024-05-04T03:25:28.890 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_refresh_metadata", 2024-05-04T03:25:28.890 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.891 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.891 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.891 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.891 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.891 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.891 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.891 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-04T03:25:28.891 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.891 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.891 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.891 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.891 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.891 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.891 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.891 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.892 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.892 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.892 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.892 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.892 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-04T03:25:28.892 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.892 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.892 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.892 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.892 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.892 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.892 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.892 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.892 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.892 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.892 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.893 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.893 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.893 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.893 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.893 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.893 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.893 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.893 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.893 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.893 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.893 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.893 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.893 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.893 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.893 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.893 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.894 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.894 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.894 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.894 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.894 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.894 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.894 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.894 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.894 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.894 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-04T03:25:28.894 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-04T03:25:28.894 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.894 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.895 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.895 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.895 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.895 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.895 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.895 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-04T03:25:28.895 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.895 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.895 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.895 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-04T03:25:28.895 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-04T03:25:28.895 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.895 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.896 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.896 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "*", 2024-05-04T03:25:28.896 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.896 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.896 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.896 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-04T03:25:28.896 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.896 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.896 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.896 INFO:teuthology.orchestra.run.smithi001.stdout: "max_count_per_host": { 2024-05-04T03:25:28.896 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_count_per_host", 2024-05-04T03:25:28.896 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.896 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.896 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.897 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-05-04T03:25:28.897 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.897 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.897 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.897 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of daemons per service per host", 2024-05-04T03:25:28.897 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.897 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.897 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.897 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd_draining_count": { 2024-05-04T03:25:28.897 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_osd_draining_count", 2024-05-04T03:25:28.897 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.897 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.897 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.897 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-05-04T03:25:28.897 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.898 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.898 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.898 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-04T03:25:28.898 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.898 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.898 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.898 INFO:teuthology.orchestra.run.smithi001.stdout: "migration_current": { 2024-05-04T03:25:28.898 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "migration_current", 2024-05-04T03:25:28.898 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.898 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.898 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.898 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.899 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.899 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.899 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.899 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "internal - do not modify", 2024-05-04T03:25:28.899 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.899 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.899 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.899 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-05-04T03:25:28.899 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-05-04T03:25:28.899 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.899 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.899 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.899 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "root", 2024-05-04T03:25:28.900 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.900 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.900 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.900 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm-package", 2024-05-04T03:25:28.900 INFO:teuthology.orchestra.run.smithi001.stdout: "root" 2024-05-04T03:25:28.900 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.900 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "mode for remote execution of cephadm", 2024-05-04T03:25:28.900 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.900 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.900 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.900 INFO:teuthology.orchestra.run.smithi001.stdout: "oob_default_addr": { 2024-05-04T03:25:28.900 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "oob_default_addr", 2024-05-04T03:25:28.900 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.901 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.901 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.901 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "169.254.1.1", 2024-05-04T03:25:28.901 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.901 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.901 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.901 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-04T03:25:28.901 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.901 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.901 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.901 INFO:teuthology.orchestra.run.smithi001.stdout: "prometheus_alerts_path": { 2024-05-04T03:25:28.901 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus_alerts_path", 2024-05-04T03:25:28.901 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.901 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.902 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.902 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-04T03:25:28.902 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.902 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.902 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.902 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-04T03:25:28.902 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.902 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.902 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.902 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_insecure": { 2024-05-04T03:25:28.902 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_insecure", 2024-05-04T03:25:28.902 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.902 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.903 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.903 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.903 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.903 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.903 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.903 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-04T03:25:28.903 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.903 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.903 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.903 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_password": { 2024-05-04T03:25:28.903 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_password", 2024-05-04T03:25:28.903 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.903 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.903 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.903 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.904 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.904 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.904 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.904 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-04T03:25:28.904 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.904 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.904 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.904 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_url": { 2024-05-04T03:25:28.904 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_url", 2024-05-04T03:25:28.904 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.904 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.904 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.904 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.904 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.904 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.904 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.905 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-04T03:25:28.905 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.905 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.905 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.905 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_username": { 2024-05-04T03:25:28.905 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_username", 2024-05-04T03:25:28.905 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.905 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.905 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.905 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.905 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.905 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.905 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.905 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-04T03:25:28.906 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.906 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.906 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.906 INFO:teuthology.orchestra.run.smithi001.stdout: "secure_monitoring_stack": { 2024-05-04T03:25:28.906 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "secure_monitoring_stack", 2024-05-04T03:25:28.906 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.906 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.906 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.906 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.906 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.906 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.906 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.906 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-04T03:25:28.907 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.907 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.907 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.907 INFO:teuthology.orchestra.run.smithi001.stdout: "service_discovery_port": { 2024-05-04T03:25:28.907 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "service_discovery_port", 2024-05-04T03:25:28.907 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.907 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.907 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.907 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8765", 2024-05-04T03:25:28.907 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.907 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.907 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.907 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "cephadm service discovery port", 2024-05-04T03:25:28.907 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.907 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.907 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.908 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.908 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.908 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.908 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.908 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.908 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.908 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.908 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.908 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.908 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.908 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.908 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.908 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.908 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.908 INFO:teuthology.orchestra.run.smithi001.stdout: "ssh_config_file": { 2024-05-04T03:25:28.909 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssh_config_file", 2024-05-04T03:25:28.909 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.909 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.909 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.909 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.909 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.909 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.909 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.909 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-04T03:25:28.909 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.909 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.909 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.909 INFO:teuthology.orchestra.run.smithi001.stdout: "use_agent": { 2024-05-04T03:25:28.909 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_agent", 2024-05-04T03:25:28.909 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.910 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.910 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.910 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.910 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.910 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.910 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.910 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-04T03:25:28.910 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.910 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.910 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.910 INFO:teuthology.orchestra.run.smithi001.stdout: "use_repo_digest": { 2024-05-04T03:25:28.910 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_repo_digest", 2024-05-04T03:25:28.910 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.911 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.911 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.911 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.911 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.911 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.911 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.911 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-04T03:25:28.911 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.911 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.911 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.911 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_failed_host_check": { 2024-05-04T03:25:28.911 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_failed_host_check", 2024-05-04T03:25:28.911 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.911 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.911 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.912 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.912 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.912 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.912 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.912 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if the host check fails", 2024-05-04T03:25:28.912 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.912 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.912 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.912 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_daemons": { 2024-05-04T03:25:28.912 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_daemons", 2024-05-04T03:25:28.912 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.912 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.912 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.912 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.912 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.913 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.913 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.913 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-04T03:25:28.913 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.913 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.913 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.913 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_hosts": { 2024-05-04T03:25:28.913 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_hosts", 2024-05-04T03:25:28.913 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.913 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.913 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.913 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.913 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.913 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.913 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.914 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-04T03:25:28.914 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.914 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.914 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.914 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.914 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.914 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.914 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crash", 2024-05-04T03:25:28.914 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.914 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.914 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.914 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.914 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.914 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.915 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.915 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.915 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.915 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.915 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.915 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.915 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.915 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.915 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.915 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.915 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.915 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.915 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.915 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.915 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.915 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.916 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.916 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.916 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.916 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.916 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.916 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.916 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.916 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.916 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.916 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.916 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.916 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.916 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.916 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.917 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.917 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.917 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.917 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.917 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.917 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.917 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.917 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.917 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.917 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.917 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.917 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.917 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.917 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.917 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.918 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.918 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.918 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.918 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.918 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.918 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.918 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.921 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.921 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.921 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.921 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.921 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.921 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.921 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.921 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.921 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.921 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.922 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.922 INFO:teuthology.orchestra.run.smithi001.stdout: "retain_interval": { 2024-05-04T03:25:28.922 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retain_interval", 2024-05-04T03:25:28.922 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.922 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.922 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.922 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "31536000", 2024-05-04T03:25:28.922 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.922 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.922 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.922 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-04T03:25:28.922 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.922 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.922 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.923 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.923 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.923 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.923 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.923 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.923 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.923 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.923 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.923 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.923 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.923 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.923 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.923 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.923 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_recent_interval": { 2024-05-04T03:25:28.924 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_recent_interval", 2024-05-04T03:25:28.924 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.924 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.924 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.924 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1209600", 2024-05-04T03:25:28.924 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.924 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.924 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.924 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-04T03:25:28.924 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.924 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.924 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.924 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.924 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.924 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.925 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dashboard", 2024-05-04T03:25:28.925 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.925 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.925 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.925 INFO:teuthology.orchestra.run.smithi001.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-04T03:25:28.925 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-04T03:25:28.925 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.925 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.925 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.925 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-05-04T03:25:28.925 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.925 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.925 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.925 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.926 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.926 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.926 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.926 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_HOST": { 2024-05-04T03:25:28.926 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-04T03:25:28.926 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.926 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.926 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.926 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.926 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.926 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.926 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.926 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.927 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.927 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.927 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.927 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-04T03:25:28.927 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-04T03:25:28.927 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.927 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.927 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.927 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.927 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.927 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.927 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.927 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.928 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.928 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.928 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.928 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_ENABLED": { 2024-05-04T03:25:28.928 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_ENABLED", 2024-05-04T03:25:28.928 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.928 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.928 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.928 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.928 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.928 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.928 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.928 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.928 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.929 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.929 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.929 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-04T03:25:28.929 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-04T03:25:28.929 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.929 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.929 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.929 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.929 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.929 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.929 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.929 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.929 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.929 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.930 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.930 INFO:teuthology.orchestra.run.smithi001.stdout: "ENABLE_BROWSABLE_API": { 2024-05-04T03:25:28.930 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-04T03:25:28.930 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.930 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.930 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.930 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.930 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.930 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.930 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.930 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.930 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.930 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.930 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.931 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-04T03:25:28.931 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-04T03:25:28.931 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.931 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.931 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.931 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.931 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.931 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.931 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.931 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.931 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.931 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.932 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.932 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-04T03:25:28.932 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-04T03:25:28.932 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.932 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.932 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.932 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.932 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.932 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.932 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.932 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.932 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.932 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.932 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.933 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-04T03:25:28.933 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-04T03:25:28.933 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.933 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.933 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.933 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.933 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.933 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.933 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.933 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.933 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.933 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.933 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.934 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-04T03:25:28.934 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-04T03:25:28.934 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.934 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.934 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.934 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.934 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.934 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.934 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.934 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.934 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.934 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.934 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.935 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-04T03:25:28.935 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-04T03:25:28.935 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.935 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.935 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.935 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.935 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.935 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.935 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.935 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.935 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.935 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.935 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.936 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-04T03:25:28.936 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-04T03:25:28.936 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.936 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.936 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.936 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.936 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.936 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.936 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.936 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.936 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.936 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.936 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.936 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-04T03:25:28.936 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-04T03:25:28.937 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.937 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.937 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.937 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.937 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.937 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.937 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.937 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.937 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.937 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.937 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.937 INFO:teuthology.orchestra.run.smithi001.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-04T03:25:28.938 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-04T03:25:28.938 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.938 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.938 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.938 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.938 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.938 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.938 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.938 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.938 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.938 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.938 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.938 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_PASSWORD": { 2024-05-04T03:25:28.939 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-04T03:25:28.939 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.939 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.939 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.939 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-05-04T03:25:28.939 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.939 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.939 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.939 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.939 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.939 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.939 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.939 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-04T03:25:28.939 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-04T03:25:28.939 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.940 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.940 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.940 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.940 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.940 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.940 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.940 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.940 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.940 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.940 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.940 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_URL": { 2024-05-04T03:25:28.940 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_URL", 2024-05-04T03:25:28.940 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.940 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.940 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.941 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.941 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.941 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.941 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.941 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.941 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.941 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.941 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.941 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_USERNAME": { 2024-05-04T03:25:28.941 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-04T03:25:28.941 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.941 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.941 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.941 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-05-04T03:25:28.942 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.942 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.942 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.942 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.942 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.942 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.942 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.942 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-04T03:25:28.942 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-04T03:25:28.942 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.942 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.942 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.942 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.943 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.943 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.943 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.943 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.943 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.943 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.943 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.943 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-04T03:25:28.943 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-04T03:25:28.943 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.943 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.943 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.943 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.943 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.944 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.944 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.944 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.944 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.944 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.944 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.944 INFO:teuthology.orchestra.run.smithi001.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-04T03:25:28.944 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-04T03:25:28.944 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.944 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.944 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.944 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.944 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.944 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.944 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.945 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.945 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.945 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.945 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.945 INFO:teuthology.orchestra.run.smithi001.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-04T03:25:28.945 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-04T03:25:28.945 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.945 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.945 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.945 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.945 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.945 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.945 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.945 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.946 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.946 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.946 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.946 INFO:teuthology.orchestra.run.smithi001.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-04T03:25:28.946 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-04T03:25:28.946 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.946 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.946 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.946 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "[]", 2024-05-04T03:25:28.946 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.946 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.946 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.946 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.947 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.947 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.947 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.947 INFO:teuthology.orchestra.run.smithi001.stdout: "MULTICLUSTER_CONFIG": { 2024-05-04T03:25:28.947 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-04T03:25:28.947 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.947 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.947 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.947 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "{}", 2024-05-04T03:25:28.947 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.947 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.947 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.947 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.948 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.948 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.948 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.948 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_HOST": { 2024-05-04T03:25:28.948 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-04T03:25:28.948 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.948 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.948 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.948 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.948 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.948 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.948 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.948 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.949 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.949 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.949 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.949 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-04T03:25:28.949 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-04T03:25:28.949 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.949 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.949 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.949 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.949 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.949 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.949 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.949 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.949 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.950 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.950 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.950 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-04T03:25:28.950 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-04T03:25:28.950 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.950 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.950 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.950 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.950 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.950 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.950 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.950 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.950 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.951 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.951 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.951 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-04T03:25:28.951 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-04T03:25:28.951 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.951 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.951 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.951 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.951 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.951 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.951 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.951 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.951 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.952 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.952 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.952 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-04T03:25:28.952 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-04T03:25:28.952 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.952 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.952 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.952 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.952 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.952 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.952 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.952 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.952 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.953 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.953 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.953 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-04T03:25:28.953 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-04T03:25:28.953 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.953 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.953 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.953 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.953 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.953 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.953 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.953 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.954 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.954 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.954 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.954 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-04T03:25:28.954 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-04T03:25:28.954 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.954 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.954 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.954 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.954 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.954 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.954 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.954 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.955 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.955 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.955 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.955 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.955 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-04T03:25:28.955 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-04T03:25:28.955 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.955 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.955 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.955 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.955 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.955 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.955 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.955 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.955 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.956 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.956 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.956 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-04T03:25:28.956 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-04T03:25:28.956 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.956 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.956 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.956 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.956 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.956 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.956 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.956 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.956 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.956 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.957 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.957 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.957 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_ENABLED": { 2024-05-04T03:25:28.957 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-04T03:25:28.957 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.957 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.957 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.957 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.957 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.957 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.957 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.957 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.957 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.957 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.957 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.958 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-04T03:25:28.958 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-04T03:25:28.958 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.958 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.958 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.958 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-04T03:25:28.958 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.958 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.958 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.958 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.958 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.958 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.958 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.959 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-04T03:25:28.959 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-04T03:25:28.959 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.959 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.959 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.959 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-05-04T03:25:28.959 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.959 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.959 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.959 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.959 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.959 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.959 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.959 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.960 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-04T03:25:28.960 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-04T03:25:28.960 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.960 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.960 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.960 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8", 2024-05-04T03:25:28.960 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.960 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.960 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.960 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.960 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.960 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.960 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.960 INFO:teuthology.orchestra.run.smithi001.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-04T03:25:28.961 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-04T03:25:28.961 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.961 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.961 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.961 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "45", 2024-05-04T03:25:28.961 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.961 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.961 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.961 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.961 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.961 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.961 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.961 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.961 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ACCESS_KEY": { 2024-05-04T03:25:28.961 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-04T03:25:28.961 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.962 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.962 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.962 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.962 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.962 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.962 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.962 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.962 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.962 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.962 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.962 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-04T03:25:28.962 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-04T03:25:28.962 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.963 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.963 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.963 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-05-04T03:25:28.963 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.963 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.963 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.963 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.963 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.963 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.963 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.963 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SECRET_KEY": { 2024-05-04T03:25:28.963 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-04T03:25:28.963 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.963 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.963 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.964 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.964 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.964 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.964 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.964 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.964 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.964 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.964 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.964 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.964 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SSL_VERIFY": { 2024-05-04T03:25:28.964 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-04T03:25:28.964 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.964 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.964 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.964 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.965 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.965 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.965 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.965 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.965 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.965 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.965 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.965 INFO:teuthology.orchestra.run.smithi001.stdout: "UNSAFE_TLS_v1_2": { 2024-05-04T03:25:28.965 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-04T03:25:28.965 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.965 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.965 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.965 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.966 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.966 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.966 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.966 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.966 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.966 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.966 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.966 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-04T03:25:28.966 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-04T03:25:28.966 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.966 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.966 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.966 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.966 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.967 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.967 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.967 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.967 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.967 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.967 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.967 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-04T03:25:28.967 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-04T03:25:28.967 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.967 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.967 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.967 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-05-04T03:25:28.967 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.967 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.968 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.968 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.968 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.968 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.968 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.968 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-04T03:25:28.968 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-04T03:25:28.968 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.968 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.968 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.968 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-05-04T03:25:28.968 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.968 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.968 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.969 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.969 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.969 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.969 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.969 INFO:teuthology.orchestra.run.smithi001.stdout: "cross_origin_url": { 2024-05-04T03:25:28.969 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cross_origin_url", 2024-05-04T03:25:28.969 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.969 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.969 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.969 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.969 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.969 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.969 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.969 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.970 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.970 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.970 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.970 INFO:teuthology.orchestra.run.smithi001.stdout: "crt_file": { 2024-05-04T03:25:28.970 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crt_file", 2024-05-04T03:25:28.970 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.970 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.970 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.970 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.970 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.970 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.970 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.970 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.970 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.971 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.971 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.971 INFO:teuthology.orchestra.run.smithi001.stdout: "debug": { 2024-05-04T03:25:28.971 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "debug", 2024-05-04T03:25:28.971 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.971 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.971 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.971 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.971 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.971 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.971 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.971 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable/disable debug options", 2024-05-04T03:25:28.971 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.971 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.972 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.972 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.972 INFO:teuthology.orchestra.run.smithi001.stdout: "jwt_token_ttl": { 2024-05-04T03:25:28.972 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "jwt_token_ttl", 2024-05-04T03:25:28.972 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.972 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.972 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.972 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "28800", 2024-05-04T03:25:28.972 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.972 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.972 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.972 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.972 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.972 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.973 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.973 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-05-04T03:25:28.973 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-05-04T03:25:28.973 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.973 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.973 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.973 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.973 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.973 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.973 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.973 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.973 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.973 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.973 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.974 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.974 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.974 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.974 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.974 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.974 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.974 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.974 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.974 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.974 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.974 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.974 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.974 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.975 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.975 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.975 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.975 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.975 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.975 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.975 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.975 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.975 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.975 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.975 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.975 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.975 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.976 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.976 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.976 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.976 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.976 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.976 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.976 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.976 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.976 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.976 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.976 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.976 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.976 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.976 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.977 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.977 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.977 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.977 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.977 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.977 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.977 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.977 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.977 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.977 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.977 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.977 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.977 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.977 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.977 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.978 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.978 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.978 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.978 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.978 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.978 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.978 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.978 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.978 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.978 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.978 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.978 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.978 INFO:teuthology.orchestra.run.smithi001.stdout: "motd": { 2024-05-04T03:25:28.978 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "motd", 2024-05-04T03:25:28.979 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.979 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.979 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.979 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.979 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.979 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.979 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.979 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "The message of the day", 2024-05-04T03:25:28.979 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.979 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.979 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.979 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect_resolve_ip_addr": { 2024-05-04T03:25:28.979 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "redirect_resolve_ip_addr", 2024-05-04T03:25:28.980 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.980 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.980 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.980 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.980 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.980 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.980 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.980 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.980 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.980 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.980 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.980 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-05-04T03:25:28.980 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-05-04T03:25:28.980 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.980 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.981 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.981 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-05-04T03:25:28.981 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.981 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.981 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.981 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.981 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.981 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.981 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.981 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.981 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-05-04T03:25:28.981 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-05-04T03:25:28.981 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.981 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.981 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.982 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8080", 2024-05-04T03:25:28.982 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.982 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.982 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.982 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.982 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.982 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.982 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.982 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:28.982 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:28.982 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.982 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:28.982 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.982 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:28.983 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.983 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.983 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.983 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.983 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.983 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.983 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.983 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-05-04T03:25:28.983 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-05-04T03:25:28.983 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.983 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.983 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.984 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.984 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.984 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.984 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.984 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.984 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.984 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.984 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.984 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.984 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl_server_port": { 2024-05-04T03:25:28.984 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl_server_port", 2024-05-04T03:25:28.984 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.984 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.984 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.984 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8443", 2024-05-04T03:25:28.985 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.985 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.985 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.985 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.985 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.985 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.985 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.985 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-05-04T03:25:28.985 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-05-04T03:25:28.985 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.985 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.985 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.985 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "redirect", 2024-05-04T03:25:28.985 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.986 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.986 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.986 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.986 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect" 2024-05-04T03:25:28.986 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.986 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.986 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.986 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.986 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.986 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-05-04T03:25:28.986 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-05-04T03:25:28.986 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:28.986 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.986 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.987 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-05-04T03:25:28.987 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-05-04T03:25:28.987 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-05-04T03:25:28.987 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.987 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.987 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.987 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.987 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.987 INFO:teuthology.orchestra.run.smithi001.stdout: "url_prefix": { 2024-05-04T03:25:28.987 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url_prefix", 2024-05-04T03:25:28.987 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.987 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.987 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:28.988 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.988 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.988 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.988 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.988 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.988 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.988 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.988 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.988 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.988 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:28.988 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.988 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:28.988 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "devicehealth", 2024-05-04T03:25:28.989 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:28.989 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:28.989 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:28.989 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_monitoring": { 2024-05-04T03:25:28.989 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_monitoring", 2024-05-04T03:25:28.989 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.989 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.989 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.989 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.989 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.989 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.989 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.989 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "monitor device health metrics", 2024-05-04T03:25:28.989 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.989 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.990 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.990 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.990 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:28.990 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:28.990 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.990 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.990 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.990 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:28.990 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.990 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.990 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.990 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.990 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.990 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.991 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.991 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.991 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.991 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.991 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.991 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.991 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.991 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.991 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:28.991 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:28.991 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.991 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.991 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.991 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.991 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.992 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.992 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.992 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.992 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.992 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.992 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.992 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.992 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:28.992 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:28.992 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.992 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.992 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.992 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:28.992 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.992 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.993 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:28.993 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:28.993 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:28.993 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:28.993 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:28.993 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:28.993 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:28.993 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:28.993 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.993 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.993 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.993 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.993 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:28.993 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:28.993 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.994 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.994 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.994 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:28.994 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.994 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.994 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.994 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:28.994 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.994 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.994 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.994 INFO:teuthology.orchestra.run.smithi001.stdout: "mark_out_threshold": { 2024-05-04T03:25:28.994 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mark_out_threshold", 2024-05-04T03:25:28.994 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.995 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.995 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.995 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2419200", 2024-05-04T03:25:28.995 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.995 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.995 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.995 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-04T03:25:28.995 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.995 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.995 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.995 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": { 2024-05-04T03:25:28.995 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_name", 2024-05-04T03:25:28.995 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:28.995 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.996 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.996 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "device_health_metrics", 2024-05-04T03:25:28.996 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.996 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.996 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.996 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-04T03:25:28.996 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.996 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.996 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.996 INFO:teuthology.orchestra.run.smithi001.stdout: "retention_period": { 2024-05-04T03:25:28.996 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retention_period", 2024-05-04T03:25:28.996 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.996 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.997 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.997 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15552000", 2024-05-04T03:25:28.997 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.997 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.997 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.997 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain device health metrics", 2024-05-04T03:25:28.997 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.997 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.997 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.997 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.997 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_frequency": { 2024-05-04T03:25:28.997 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_frequency", 2024-05-04T03:25:28.997 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.997 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.997 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.998 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-05-04T03:25:28.998 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.998 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.998 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.998 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-04T03:25:28.998 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.998 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.998 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.998 INFO:teuthology.orchestra.run.smithi001.stdout: "self_heal": { 2024-05-04T03:25:28.998 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "self_heal", 2024-05-04T03:25:28.998 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:28.998 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.998 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.998 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:28.999 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:28.999 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:28.999 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:28.999 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-04T03:25:28.999 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:28.999 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:28.999 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:28.999 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:28.999 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-05-04T03:25:28.999 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-05-04T03:25:28.999 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:28.999 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:28.999 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:28.999 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-05-04T03:25:29.000 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.000 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.000 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.000 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and check device health", 2024-05-04T03:25:29.000 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.000 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.000 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.000 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.000 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.000 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.001 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.001 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.001 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.001 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.001 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.001 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.001 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.001 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.001 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.001 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.001 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_threshold": { 2024-05-04T03:25:29.001 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_threshold", 2024-05-04T03:25:29.001 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:29.002 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.002 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.002 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7257600", 2024-05-04T03:25:29.002 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.002 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.002 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.002 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-04T03:25:29.002 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.002 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.002 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.002 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.002 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.002 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.003 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "diskprediction_local", 2024-05-04T03:25:29.003 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.003 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.003 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.003 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.003 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.003 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.003 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.003 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.003 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.003 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.003 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.003 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.003 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.003 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.003 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.004 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.004 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.004 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.004 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.004 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.004 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.004 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.004 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.004 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.004 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.004 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.004 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.004 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.004 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.004 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.005 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.005 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.005 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.005 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.005 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.005 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.005 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.005 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.005 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.005 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.005 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.005 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.005 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.005 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.006 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.006 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.006 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.006 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.006 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.006 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.006 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.006 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.006 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.006 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.006 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.006 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.006 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.006 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.006 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.007 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.007 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.007 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.007 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.007 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.007 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.007 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.007 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.007 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.007 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.007 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.007 INFO:teuthology.orchestra.run.smithi001.stdout: "predict_interval": { 2024-05-04T03:25:29.007 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predict_interval", 2024-05-04T03:25:29.007 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.008 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.008 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.008 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-05-04T03:25:29.008 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.008 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.008 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.008 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.008 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.008 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.008 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.008 INFO:teuthology.orchestra.run.smithi001.stdout: "predictor_model": { 2024-05-04T03:25:29.008 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predictor_model", 2024-05-04T03:25:29.008 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.008 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.008 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.009 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "prophetstor", 2024-05-04T03:25:29.009 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.009 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.009 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.009 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.009 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.009 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.009 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.009 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-05-04T03:25:29.009 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-05-04T03:25:29.009 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.009 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.009 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.009 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-05-04T03:25:29.009 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.010 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.010 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.010 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.010 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.010 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.010 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.010 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.010 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.010 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.010 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.010 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.010 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.010 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.010 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.011 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.011 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.011 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.011 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.011 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.011 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.011 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.011 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.011 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "influx", 2024-05-04T03:25:29.011 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": false, 2024-05-04T03:25:29.011 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "influxdb python module not found", 2024-05-04T03:25:29.011 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.011 INFO:teuthology.orchestra.run.smithi001.stdout: "batch_size": { 2024-05-04T03:25:29.012 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "batch_size", 2024-05-04T03:25:29.012 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.012 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.012 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.012 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5000", 2024-05-04T03:25:29.012 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.012 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.012 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.012 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-04T03:25:29.012 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.012 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.012 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.012 INFO:teuthology.orchestra.run.smithi001.stdout: "database": { 2024-05-04T03:25:29.012 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "database", 2024-05-04T03:25:29.012 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.013 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.013 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.013 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "ceph", 2024-05-04T03:25:29.013 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.013 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.013 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.013 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-04T03:25:29.013 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.013 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.013 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.013 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": { 2024-05-04T03:25:29.013 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hostname", 2024-05-04T03:25:29.013 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.013 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.014 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.014 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.014 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.014 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.014 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.014 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server hostname", 2024-05-04T03:25:29.014 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.014 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.014 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.014 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-05-04T03:25:29.014 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-05-04T03:25:29.014 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:29.014 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.015 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.015 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "30", 2024-05-04T03:25:29.015 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "5", 2024-05-04T03:25:29.015 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.015 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.015 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-04T03:25:29.015 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.015 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.015 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.015 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.015 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.015 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.015 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.015 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.015 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.016 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.016 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.016 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.016 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.016 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.016 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.016 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.016 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.016 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.016 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.016 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.016 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.016 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.016 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.016 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.017 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.017 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.017 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.017 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.017 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.017 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.017 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.017 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.017 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.017 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.017 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.017 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.017 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.018 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.018 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.018 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.018 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.018 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.018 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.018 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.018 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.018 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.018 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.018 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.018 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.018 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.018 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.019 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.019 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.019 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.019 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.019 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.019 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.019 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.019 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.019 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.019 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.019 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.019 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.019 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.019 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.020 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.020 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.020 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.020 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.020 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.020 INFO:teuthology.orchestra.run.smithi001.stdout: "password": { 2024-05-04T03:25:29.020 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "password", 2024-05-04T03:25:29.020 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.020 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.020 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.020 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.020 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.020 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.020 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.020 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "password of InfluxDB server user", 2024-05-04T03:25:29.021 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.021 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.021 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.021 INFO:teuthology.orchestra.run.smithi001.stdout: "port": { 2024-05-04T03:25:29.021 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "port", 2024-05-04T03:25:29.021 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.021 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.021 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.021 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8086", 2024-05-04T03:25:29.021 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.021 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.021 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.021 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server port", 2024-05-04T03:25:29.021 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.022 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.022 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.022 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.022 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.022 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.022 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.022 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.022 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.022 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.022 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.022 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.022 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.022 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.023 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.023 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.023 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-05-04T03:25:29.023 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-05-04T03:25:29.023 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.023 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.023 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.023 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "false", 2024-05-04T03:25:29.023 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.023 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.023 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.023 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-04T03:25:29.023 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.023 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.024 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.024 INFO:teuthology.orchestra.run.smithi001.stdout: "threads": { 2024-05-04T03:25:29.024 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threads", 2024-05-04T03:25:29.024 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.024 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.024 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.024 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-05-04T03:25:29.024 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-05-04T03:25:29.024 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "32", 2024-05-04T03:25:29.024 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.024 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-04T03:25:29.024 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.024 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.024 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.025 INFO:teuthology.orchestra.run.smithi001.stdout: "username": { 2024-05-04T03:25:29.025 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "username", 2024-05-04T03:25:29.025 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.025 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.025 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.025 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.025 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.025 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.025 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.025 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "username of InfluxDB server user", 2024-05-04T03:25:29.025 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.025 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.025 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.026 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.026 INFO:teuthology.orchestra.run.smithi001.stdout: "verify_ssl": { 2024-05-04T03:25:29.026 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "verify_ssl", 2024-05-04T03:25:29.026 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.026 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.026 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.026 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "true", 2024-05-04T03:25:29.026 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.026 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.026 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.026 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-04T03:25:29.026 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.026 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.026 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.027 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.027 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.027 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.027 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "insights", 2024-05-04T03:25:29.027 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.027 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.027 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.027 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.027 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.027 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.027 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.027 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.028 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.028 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.028 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.028 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.028 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.028 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.028 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.028 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.028 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.028 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.028 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.028 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.028 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.028 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.028 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.029 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.029 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.029 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.029 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.029 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.029 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.029 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.029 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.029 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.029 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.029 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.029 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.029 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.030 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.030 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.030 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.030 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.030 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.030 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.030 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.030 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.030 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.030 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.030 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.030 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.030 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.031 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.031 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.031 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.031 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.031 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.031 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.031 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.031 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.031 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.031 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.031 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.031 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.032 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.032 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.032 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.032 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.032 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.032 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.032 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.032 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.032 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.032 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.032 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.032 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.032 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.032 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.033 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.033 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.033 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.033 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.033 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.033 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.033 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.033 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.033 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.033 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.033 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "iostat", 2024-05-04T03:25:29.033 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.033 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.033 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.034 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.034 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.034 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.034 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.034 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.034 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.034 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.034 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.034 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.034 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.034 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.034 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.034 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.034 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.034 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.035 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.035 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.035 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.035 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.035 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.035 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.035 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.035 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.035 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.035 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.035 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.035 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.035 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.035 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.036 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.036 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.036 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.036 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.036 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.036 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.036 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.036 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.036 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.036 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.036 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.036 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.036 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.036 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.036 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.036 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.037 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.037 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.037 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.037 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.037 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.037 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.037 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.037 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.037 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.037 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.037 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.037 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.037 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.037 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.037 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.037 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.038 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.038 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.038 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.038 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.038 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.038 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.038 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.038 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.038 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.038 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.038 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.038 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.038 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.039 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.039 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.039 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.039 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.039 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.039 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.039 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.039 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.039 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.039 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.039 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "k8sevents", 2024-05-04T03:25:29.039 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.039 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.039 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.039 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_event_retention_days": { 2024-05-04T03:25:29.039 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ceph_event_retention_days", 2024-05-04T03:25:29.040 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.040 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.040 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.040 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7", 2024-05-04T03:25:29.040 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.040 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.040 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.040 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-04T03:25:29.040 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.040 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.040 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.040 INFO:teuthology.orchestra.run.smithi001.stdout: "config_check_secs": { 2024-05-04T03:25:29.040 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_check_secs", 2024-05-04T03:25:29.040 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.040 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.041 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.041 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-05-04T03:25:29.041 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "10", 2024-05-04T03:25:29.041 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.041 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.041 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-04T03:25:29.041 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.041 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.041 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.041 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.041 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.041 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.041 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.041 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.041 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.042 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.042 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.042 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.042 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.042 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.042 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.042 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.042 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.042 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.042 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.042 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.042 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.042 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.042 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.043 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.043 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.043 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.043 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.043 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.043 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.043 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.043 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.043 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.043 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.043 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.043 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.043 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.044 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.044 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.044 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.044 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.044 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.044 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.044 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.044 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.044 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.044 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.044 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.044 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.044 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.044 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.044 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.045 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.045 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.045 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.045 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.045 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.045 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.045 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.045 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.045 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.045 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.045 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.045 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.045 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.045 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.046 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.046 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.046 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.046 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.046 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.046 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.046 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.046 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.046 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.046 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.046 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.046 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.046 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.047 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.047 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.047 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.047 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.047 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.047 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.047 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.047 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.047 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "localpool", 2024-05-04T03:25:29.047 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.047 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.047 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.047 INFO:teuthology.orchestra.run.smithi001.stdout: "failure_domain": { 2024-05-04T03:25:29.047 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "failure_domain", 2024-05-04T03:25:29.048 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.048 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.048 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.048 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "host", 2024-05-04T03:25:29.048 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.048 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.048 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.048 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "failure domain for any created local pool", 2024-05-04T03:25:29.048 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-04T03:25:29.048 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.048 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.048 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.048 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.048 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.049 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.049 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.049 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.049 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.049 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.049 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.049 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.049 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.049 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.049 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.049 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.049 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.049 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.049 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.049 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.050 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.050 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.050 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.050 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.050 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.050 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.050 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.050 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.050 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.050 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.050 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.050 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.050 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.050 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.051 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.051 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.051 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.051 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.051 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.051 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.051 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.051 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.051 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.051 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.051 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.051 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.051 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.051 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.052 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.052 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.052 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.052 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.052 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.052 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.052 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.052 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.052 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.052 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.052 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.052 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.052 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.053 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.053 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.053 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.053 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.053 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.053 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.053 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.053 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.053 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": { 2024-05-04T03:25:29.053 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_size", 2024-05-04T03:25:29.053 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.053 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.053 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.053 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.054 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.054 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.054 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.054 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default min_size for any created local pool", 2024-05-04T03:25:29.054 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-04T03:25:29.054 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.054 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.054 INFO:teuthology.orchestra.run.smithi001.stdout: "num_rep": { 2024-05-04T03:25:29.054 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "num_rep", 2024-05-04T03:25:29.054 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.054 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.054 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.054 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3", 2024-05-04T03:25:29.055 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.055 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.055 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.055 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default replica count for any created local pool", 2024-05-04T03:25:29.055 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.055 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.055 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.055 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": { 2024-05-04T03:25:29.055 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_num", 2024-05-04T03:25:29.055 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.055 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.055 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.055 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "128", 2024-05-04T03:25:29.056 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.056 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.056 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.056 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default pg_num for any created local pool", 2024-05-04T03:25:29.056 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.056 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.056 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.056 INFO:teuthology.orchestra.run.smithi001.stdout: "prefix": { 2024-05-04T03:25:29.056 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prefix", 2024-05-04T03:25:29.056 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.056 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.056 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.056 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.057 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.057 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.057 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.057 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name prefix for any created local pool", 2024-05-04T03:25:29.057 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.057 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.057 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.057 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.057 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.057 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.057 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.057 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.057 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.058 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.058 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.058 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.058 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.058 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.058 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.058 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.058 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.058 INFO:teuthology.orchestra.run.smithi001.stdout: "subtree": { 2024-05-04T03:25:29.058 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "subtree", 2024-05-04T03:25:29.058 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.058 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.058 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.058 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "rack", 2024-05-04T03:25:29.059 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.059 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.059 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.059 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-04T03:25:29.059 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-04T03:25:29.059 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.059 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.059 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.059 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.059 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.059 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mds_autoscaler", 2024-05-04T03:25:29.059 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.059 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.060 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.060 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.060 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.060 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.060 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.060 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.060 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.060 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.060 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.060 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.060 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.060 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.060 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.060 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.060 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.061 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.061 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.061 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.061 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.061 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.061 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.061 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.061 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.061 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.061 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.061 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.061 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.061 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.061 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.062 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.062 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.062 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.062 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.062 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.062 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.062 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.062 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.062 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.062 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.062 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.062 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.062 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.062 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.063 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.063 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.063 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.063 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.063 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.063 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.063 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.063 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.063 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.063 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.063 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.063 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.063 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.063 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.064 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.064 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.064 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.064 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.064 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.064 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.064 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.064 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.064 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.064 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.064 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.064 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.064 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.064 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.065 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.065 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.065 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.065 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.065 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.065 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.065 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.065 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.065 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.065 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.065 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.065 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.065 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.066 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirroring", 2024-05-04T03:25:29.066 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.066 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.066 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.066 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.066 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.066 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.066 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.066 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.066 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.066 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.066 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.066 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.066 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.067 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.067 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.067 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.067 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.067 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.067 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.067 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.067 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.067 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.067 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.067 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.067 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.067 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.068 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.068 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.068 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.068 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.068 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.068 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.068 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.068 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.068 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.068 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.068 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.068 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.068 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.068 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.069 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.069 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.069 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.069 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.069 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.069 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.069 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.069 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.069 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.069 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.069 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.069 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.069 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.069 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.069 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.070 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.070 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.070 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.070 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.070 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.070 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.070 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.070 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.070 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.070 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.070 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.070 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.070 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.071 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.071 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.071 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.071 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.071 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.071 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.071 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.071 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.071 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.071 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.071 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.071 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.071 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.072 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.072 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.072 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.072 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.072 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "nfs", 2024-05-04T03:25:29.072 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.072 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.072 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.072 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.072 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.072 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.072 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.072 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.072 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.072 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.073 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.073 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.073 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.073 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.073 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.073 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.073 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.073 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.073 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.073 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.073 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.073 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.073 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.074 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.074 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.074 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.074 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.074 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.074 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.074 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.074 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.074 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.074 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.074 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.074 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.074 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.074 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.075 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.075 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.075 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.075 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.075 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.075 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.075 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.075 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.075 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.075 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.075 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.075 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.075 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.075 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.075 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.076 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.076 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.076 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.076 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.076 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.076 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.076 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.076 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.076 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.076 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.076 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.076 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.076 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.076 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.076 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.076 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.077 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.077 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.077 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.077 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.077 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.077 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.077 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.077 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.077 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.077 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.077 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.077 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.077 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.077 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.077 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.077 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.078 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.078 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.078 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-05-04T03:25:29.078 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.078 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.078 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.078 INFO:teuthology.orchestra.run.smithi001.stdout: "fail_fs": { 2024-05-04T03:25:29.078 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "fail_fs", 2024-05-04T03:25:29.078 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.078 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.078 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.078 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.078 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.079 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.079 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.079 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-04T03:25:29.079 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.079 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.079 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.079 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.079 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.079 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.079 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.079 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.079 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.079 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.079 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.079 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.080 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.080 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.080 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.080 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.080 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.080 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.080 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.080 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.080 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.080 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.080 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.080 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.080 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.080 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.080 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.081 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.081 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.081 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.081 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.081 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.081 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.081 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.081 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.081 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.081 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.081 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.081 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.081 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.081 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.081 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.082 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.082 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.082 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.082 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.082 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.082 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.082 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.082 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.082 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.082 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.082 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.082 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.082 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.082 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.083 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.083 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.083 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.083 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.083 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.083 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.083 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.083 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.083 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.083 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.083 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.083 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.083 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.083 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator": { 2024-05-04T03:25:29.084 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-05-04T03:25:29.084 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.084 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.084 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.084 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.084 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.084 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.084 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.084 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm", 2024-05-04T03:25:29.084 INFO:teuthology.orchestra.run.smithi001.stdout: "rook", 2024-05-04T03:25:29.084 INFO:teuthology.orchestra.run.smithi001.stdout: "test_orchestrator" 2024-05-04T03:25:29.084 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.084 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Orchestrator backend", 2024-05-04T03:25:29.084 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.084 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.085 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.085 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.085 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.085 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.085 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.085 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.085 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.085 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.085 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.085 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.085 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.085 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.085 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.086 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.086 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.086 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.086 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.086 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_perf_query", 2024-05-04T03:25:29.086 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.086 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.086 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.086 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.086 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.086 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.086 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.087 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.087 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.087 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.087 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.087 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.087 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.087 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.087 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.087 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.087 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.087 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.087 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.087 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.087 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.087 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.087 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.088 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.088 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.088 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.088 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.088 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.088 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.088 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.088 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.088 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.088 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.088 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.088 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.088 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.089 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.089 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.089 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.089 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.089 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.089 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.089 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.089 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.089 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.089 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.089 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.089 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.089 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.089 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.090 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.090 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.090 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.090 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.090 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.090 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.090 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.090 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.090 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.090 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.090 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.090 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.090 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.091 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.091 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.091 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.091 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.091 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.091 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.091 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.091 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.091 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.091 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.091 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.091 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.091 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.092 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.092 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.092 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.092 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.092 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.092 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.092 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.092 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.092 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.092 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_support", 2024-05-04T03:25:29.092 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.092 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.092 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.093 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.093 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.093 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.093 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.093 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.093 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.093 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.093 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.093 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.093 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.093 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.093 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.093 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.093 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.093 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.094 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.094 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.094 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.094 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.094 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.094 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.094 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.094 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.094 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.094 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.094 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.094 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.094 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.094 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.095 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.095 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.095 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.095 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.095 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.095 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.095 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.095 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.095 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.095 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.095 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.095 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.095 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.096 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.096 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.096 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.096 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.096 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.096 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.096 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.096 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.096 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.096 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.096 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.096 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.096 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.096 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.096 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.097 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.097 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.097 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.097 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.097 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.097 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.097 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.097 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.097 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.097 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.097 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.097 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.097 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.097 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.097 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.098 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.098 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.098 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.098 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.098 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.098 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.098 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.098 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.098 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.098 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.098 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.098 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.098 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.098 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_autoscaler", 2024-05-04T03:25:29.099 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.099 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.099 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.099 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.099 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.099 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.099 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.099 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.099 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.099 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.099 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.099 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.099 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.099 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.099 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.099 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.100 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.100 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.100 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.100 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.100 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.100 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.100 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.100 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.100 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.100 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.100 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.100 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.100 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.100 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.100 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.100 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.101 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.101 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.101 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.101 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.101 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.101 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.101 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.101 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.101 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.101 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.101 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.101 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.101 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.101 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.101 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.102 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.102 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.102 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.102 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.102 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.102 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.102 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.102 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.102 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.102 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.102 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.102 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.102 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.102 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.103 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.103 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.103 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.103 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.103 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.103 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.103 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.103 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.103 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.103 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-05-04T03:25:29.103 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-05-04T03:25:29.103 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:29.103 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.103 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.103 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-05-04T03:25:29.103 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.104 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.104 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.104 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.104 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.104 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.104 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.104 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.104 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.104 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.104 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.104 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.104 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.104 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.105 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.105 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.105 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.105 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.105 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.105 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.105 INFO:teuthology.orchestra.run.smithi001.stdout: "threshold": { 2024-05-04T03:25:29.105 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threshold", 2024-05-04T03:25:29.105 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-05-04T03:25:29.105 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.105 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.105 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-05-04T03:25:29.105 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1.0", 2024-05-04T03:25:29.106 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.106 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.106 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "scaling threshold", 2024-05-04T03:25:29.106 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-04T03:25:29.106 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.106 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.106 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.106 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.106 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.106 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "progress", 2024-05-04T03:25:29.106 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.106 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.106 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.107 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_pg_recovery_event": { 2024-05-04T03:25:29.107 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_pg_recovery_event", 2024-05-04T03:25:29.107 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.107 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.107 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.107 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.107 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.107 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.107 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.107 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-04T03:25:29.107 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.107 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.107 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.107 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-05-04T03:25:29.108 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-05-04T03:25:29.108 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.108 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.108 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.108 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.108 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.108 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.108 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.108 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.108 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.108 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.108 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.108 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.109 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.109 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.109 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.109 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.109 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.109 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.109 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.109 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.109 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.109 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.109 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.109 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.109 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.109 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.110 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.110 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.110 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.110 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.110 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.110 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.110 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.110 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.110 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.110 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.110 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.110 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.110 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.110 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.111 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.111 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.111 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.111 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.111 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.111 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.111 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.111 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.111 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.111 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.111 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.111 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.111 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.111 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.112 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.112 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.112 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.112 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.112 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.112 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.112 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.112 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.112 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.112 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.112 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.112 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.112 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.112 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.113 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.113 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.113 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.113 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.113 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.113 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.113 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.113 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.113 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.113 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.113 INFO:teuthology.orchestra.run.smithi001.stdout: "max_completed_events": { 2024-05-04T03:25:29.113 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_completed_events", 2024-05-04T03:25:29.113 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.113 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.113 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.114 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "50", 2024-05-04T03:25:29.114 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.114 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.114 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.114 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "number of past completed events to remember", 2024-05-04T03:25:29.114 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.114 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.114 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.114 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-05-04T03:25:29.114 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-05-04T03:25:29.114 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:29.114 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.114 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.115 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-05-04T03:25:29.115 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.115 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.115 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.115 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long the module is going to sleep", 2024-05-04T03:25:29.115 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.115 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.115 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.115 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.115 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.115 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.115 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.115 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.115 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.116 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.116 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.116 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.116 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.116 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.116 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.116 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.116 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.116 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.116 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.116 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus", 2024-05-04T03:25:29.116 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.116 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.116 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.117 INFO:teuthology.orchestra.run.smithi001.stdout: "cache": { 2024-05-04T03:25:29.117 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cache", 2024-05-04T03:25:29.117 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.117 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.117 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.117 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.117 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.117 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.117 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.117 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.117 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.117 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.117 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.117 INFO:teuthology.orchestra.run.smithi001.stdout: "exclude_perf_counters": { 2024-05-04T03:25:29.118 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "exclude_perf_counters", 2024-05-04T03:25:29.118 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.118 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.118 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.118 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.118 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.118 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.118 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.118 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-04T03:25:29.118 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-04T03:25:29.118 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.118 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.118 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.118 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.118 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.119 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.119 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.119 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.119 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.119 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.119 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.119 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.119 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.119 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.119 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.119 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.119 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.119 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.120 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.120 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.120 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.120 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.120 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.120 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.120 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.120 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.120 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.120 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.120 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.120 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.120 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.120 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.120 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.120 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.120 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.121 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.121 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.121 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.121 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.121 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.121 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.121 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.121 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.121 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.121 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.121 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.121 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.121 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.121 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.121 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.122 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.122 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.122 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.122 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.122 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.122 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.122 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.122 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.122 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.122 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.122 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.122 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.122 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.122 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.123 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.123 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.123 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.123 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.123 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.123 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.123 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools": { 2024-05-04T03:25:29.123 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools", 2024-05-04T03:25:29.123 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.123 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.123 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.123 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.123 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.123 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.124 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.124 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.124 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.124 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.124 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.124 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-04T03:25:29.124 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-04T03:25:29.124 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.124 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.124 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.124 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "300", 2024-05-04T03:25:29.124 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.124 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.124 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.124 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.125 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.125 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.125 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.125 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.125 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_interval": { 2024-05-04T03:25:29.125 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_interval", 2024-05-04T03:25:29.125 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-05-04T03:25:29.125 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.125 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.125 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15.0", 2024-05-04T03:25:29.125 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.125 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.125 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.125 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.125 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.125 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.126 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.126 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-05-04T03:25:29.126 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-05-04T03:25:29.126 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.126 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.126 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.126 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-05-04T03:25:29.126 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.126 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.126 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.126 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-04T03:25:29.126 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.126 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.126 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.127 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-05-04T03:25:29.127 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-05-04T03:25:29.127 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.127 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.127 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.127 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "9283", 2024-05-04T03:25:29.127 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.127 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.127 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.127 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-04T03:25:29.127 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.127 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.127 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.127 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.128 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.128 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.128 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.128 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.128 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.128 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.128 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.128 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.128 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.128 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.128 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.128 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.128 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.128 INFO:teuthology.orchestra.run.smithi001.stdout: "stale_cache_strategy": { 2024-05-04T03:25:29.128 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stale_cache_strategy", 2024-05-04T03:25:29.128 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.129 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.129 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.129 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "log", 2024-05-04T03:25:29.129 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.129 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.376+0000 7f32753ce640 1 -- 172.21.15.1:0/1996677302 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f325c0b2920 msgr2=0x7f325c0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:29.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.376+0000 7f32753ce640 1 --2- 172.21.15.1:0/1996677302 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f325c0b2920 0x7f325c0b4e30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3270103730 tx=0x7f3260005f90 comp rx=0 tx=0).stop 2024-05-04T03:25:29.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.376+0000 7f32753ce640 1 -- 172.21.15.1:0/1996677302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3270103d70 msgr2=0x7f327019a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:25:29.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.376+0000 7f32753ce640 1 --2- 172.21.15.1:0/1996677302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3270103d70 0x7f327019a7a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f3258006010 tx=0x7f3258031a50 comp rx=0 tx=0).stop 2024-05-04T03:25:29.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.376+0000 7f32753ce640 1 -- 172.21.15.1:0/1996677302 shutdown_connections 2024-05-04T03:25:29.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.376+0000 7f32753ce640 1 --2- 172.21.15.1:0/1996677302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3270108a70 0x7f32701a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:29.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.376+0000 7f32753ce640 1 --2- 172.21.15.1:0/1996677302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3270103d70 0x7f327019a7a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:29.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.376+0000 7f32753ce640 1 --2- 172.21.15.1:0/1996677302 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f325c0b2920 0x7f325c0b4e30 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:29.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.376+0000 7f32753ce640 1 --2- 172.21.15.1:0/1996677302 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3270102b30 0x7f327019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:25:29.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.376+0000 7f32753ce640 1 -- 172.21.15.1:0/1996677302 >> 172.21.15.1:0/1996677302 conn(0x7f32700fdd20 msgr2=0x7f3270109ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:25:29.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.376+0000 7f32753ce640 1 -- 172.21.15.1:0/1996677302 shutdown_connections 2024-05-04T03:25:29.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:25:28.376+0000 7f32753ce640 1 -- 172.21.15.1:0/1996677302 wait complete. 2024-05-04T03:25:29.131 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:28 smithi077 ceph-mon[33531]: pgmap v580: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 804 KiB/s wr, 335 op/s 2024-05-04T03:25:29.131 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:28 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/4228828802' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-04T03:25:29.131 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:28 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/1593336909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-04T03:25:29.131 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:28 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/1996677302' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-04T03:25:29.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:28 smithi072 ceph-mon[33332]: pgmap v580: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 804 KiB/s wr, 335 op/s 2024-05-04T03:25:29.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:28 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/4228828802' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-04T03:25:29.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:28 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/1593336909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-04T03:25:29.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:28 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/1996677302' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-04T03:25:29.134 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.134 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.134 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.135 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.135 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.135 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-05-04T03:25:29.135 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-05-04T03:25:29.135 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.135 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.135 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.135 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "default", 2024-05-04T03:25:29.135 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.135 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.135 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.135 INFO:teuthology.orchestra.run.smithi001.stdout: "default", 2024-05-04T03:25:29.135 INFO:teuthology.orchestra.run.smithi001.stdout: "error" 2024-05-04T03:25:29.135 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.135 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.136 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.136 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.136 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.136 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-05-04T03:25:29.136 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-05-04T03:25:29.136 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.136 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.136 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.136 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-05-04T03:25:29.136 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-05-04T03:25:29.136 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-05-04T03:25:29.136 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.136 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.136 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.136 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.137 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.137 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.137 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.137 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.137 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_support", 2024-05-04T03:25:29.137 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.137 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.137 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.137 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.137 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.137 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.137 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.137 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.137 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.137 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.138 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.138 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.138 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.138 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.138 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.138 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.138 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.138 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.138 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.138 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.138 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.138 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.138 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.138 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.139 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.139 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.139 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.139 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.139 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.139 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.139 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.139 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.139 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.139 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.139 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.139 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.139 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.139 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.139 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.139 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.140 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.140 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.140 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.140 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.140 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.140 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.140 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.140 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.140 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.140 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.140 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.140 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.140 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.140 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.140 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.140 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.141 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.141 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.141 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.141 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.141 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.141 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.141 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.141 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.141 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.141 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.141 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.141 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.141 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.141 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.141 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_snap_create": { 2024-05-04T03:25:29.142 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_snap_create", 2024-05-04T03:25:29.142 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.142 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.142 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.142 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-05-04T03:25:29.142 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.142 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.142 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.142 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.142 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.142 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.142 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.142 INFO:teuthology.orchestra.run.smithi001.stdout: "mirror_snapshot_schedule": { 2024-05-04T03:25:29.142 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirror_snapshot_schedule", 2024-05-04T03:25:29.142 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.143 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.143 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.143 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.143 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.143 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.143 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.143 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.143 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.143 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.143 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.143 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.143 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.143 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.143 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.143 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.144 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.144 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.144 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.144 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.144 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.144 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.144 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.144 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.144 INFO:teuthology.orchestra.run.smithi001.stdout: "trash_purge_schedule": { 2024-05-04T03:25:29.144 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "trash_purge_schedule", 2024-05-04T03:25:29.144 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.144 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.144 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.144 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.145 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.145 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.145 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.145 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.145 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.145 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.145 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.145 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.145 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.145 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.145 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "restful", 2024-05-04T03:25:29.145 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.145 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.145 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.145 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_auth": { 2024-05-04T03:25:29.146 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_auth", 2024-05-04T03:25:29.146 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.146 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.146 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.146 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.146 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.146 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.146 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.146 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.146 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.146 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.146 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.146 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.146 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-05-04T03:25:29.146 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-05-04T03:25:29.147 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.147 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.147 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.147 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.147 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.147 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.147 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.147 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.147 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.147 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.147 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.147 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.148 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.148 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.148 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.148 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.148 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.148 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.148 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.148 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.148 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.148 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.148 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.148 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.148 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.148 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.148 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.149 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.149 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.149 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.149 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.149 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.149 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.149 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.149 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.149 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.149 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.149 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.149 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.149 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.149 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.150 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.150 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.150 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.150 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.150 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.150 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.150 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.150 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.150 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.150 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.150 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.150 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.150 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.150 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.150 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.151 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.151 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.151 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.151 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.151 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.151 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.151 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.151 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.151 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.151 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.151 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.151 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.151 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.151 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.151 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.152 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.152 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.152 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.152 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.152 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.152 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.152 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.152 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-05-04T03:25:29.152 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-05-04T03:25:29.152 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.152 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.152 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.152 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.152 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.152 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.153 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.153 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.153 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.153 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.153 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.153 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.153 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-05-04T03:25:29.153 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-05-04T03:25:29.153 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.153 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.153 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.153 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.153 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.153 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.153 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.154 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.154 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.154 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.154 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.154 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.154 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.154 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.154 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.154 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.154 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.154 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.154 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.154 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.155 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.155 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.155 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.155 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.155 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.155 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.155 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.155 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.155 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rgw", 2024-05-04T03:25:29.155 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.155 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.155 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.155 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.155 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.155 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.156 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.156 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.156 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.156 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.156 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.156 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.156 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.156 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.156 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.156 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.156 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.156 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.156 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.156 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.156 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.157 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.157 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.157 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.157 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.157 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.157 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.157 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.157 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.157 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.157 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.157 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.157 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.157 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.157 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.157 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.157 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.158 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.158 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.158 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.158 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.158 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.158 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.158 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.158 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.158 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.158 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.158 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.158 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.158 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.159 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.159 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.159 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.159 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.159 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.159 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.159 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.159 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.159 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.159 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.159 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.159 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.159 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.159 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.160 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.160 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.160 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.160 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.160 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.160 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.160 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.160 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.160 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.160 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.160 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.160 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.160 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.160 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.161 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.161 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.161 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.161 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.161 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.161 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.161 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.161 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.161 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.161 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rook", 2024-05-04T03:25:29.161 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.161 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.161 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.162 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.162 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.162 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.162 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.162 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.162 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.162 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.162 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.162 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.162 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.162 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.162 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.162 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.162 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.163 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.163 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.163 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.163 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.163 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.163 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.163 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.163 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.163 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.163 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.163 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.163 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.163 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.163 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.164 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.164 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.164 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.164 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.164 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.164 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.164 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.164 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.164 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.164 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.164 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.164 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.164 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.164 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.165 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.165 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.165 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.165 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.165 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.165 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.165 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.165 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.165 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.165 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.165 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.165 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.165 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.166 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.166 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.166 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.166 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.166 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.166 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.166 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.166 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.166 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.166 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.166 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.166 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.166 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.166 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.167 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.167 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.167 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.167 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.167 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.167 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.167 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.167 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.167 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.167 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.167 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.167 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.167 INFO:teuthology.orchestra.run.smithi001.stdout: "storage_class": { 2024-05-04T03:25:29.168 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "storage_class", 2024-05-04T03:25:29.168 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.168 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.168 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.168 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "local", 2024-05-04T03:25:29.168 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.168 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.168 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.168 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-04T03:25:29.168 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.168 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.168 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.168 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.168 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.168 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.169 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "selftest", 2024-05-04T03:25:29.169 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.169 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.169 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.169 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.169 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.169 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.169 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.169 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.169 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.169 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.169 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.169 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.169 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.169 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.170 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.170 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.170 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.170 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.170 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.170 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.170 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.170 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.170 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.170 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.170 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.170 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.170 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.170 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.170 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.171 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.171 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.171 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.171 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.171 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.171 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.171 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.171 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.171 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.171 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.171 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.171 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.171 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.171 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.171 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.172 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.172 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.172 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.172 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.172 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.172 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.172 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.172 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.172 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.172 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.172 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.172 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.172 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.172 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.172 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.172 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.173 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.173 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.173 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.173 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.173 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.173 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.173 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.173 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.173 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.173 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.173 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.173 INFO:teuthology.orchestra.run.smithi001.stdout: "roption1": { 2024-05-04T03:25:29.173 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption1", 2024-05-04T03:25:29.173 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.173 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.174 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.174 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.174 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.174 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.174 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.174 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.174 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.174 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.174 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.174 INFO:teuthology.orchestra.run.smithi001.stdout: "roption2": { 2024-05-04T03:25:29.174 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption2", 2024-05-04T03:25:29.174 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.174 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.174 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.175 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "xyz", 2024-05-04T03:25:29.175 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.175 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.175 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.175 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.175 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.175 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.175 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.175 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption1": { 2024-05-04T03:25:29.175 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption1", 2024-05-04T03:25:29.175 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.175 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.175 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.175 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.175 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.176 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.176 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.176 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.176 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.176 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.176 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.176 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption2": { 2024-05-04T03:25:29.176 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption2", 2024-05-04T03:25:29.176 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.176 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.176 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.176 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.176 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.177 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.177 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.177 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.177 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.177 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.177 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.177 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption3": { 2024-05-04T03:25:29.177 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption3", 2024-05-04T03:25:29.177 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-05-04T03:25:29.177 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.177 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.177 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.177 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.177 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.177 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.177 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.178 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.178 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.178 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.178 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption4": { 2024-05-04T03:25:29.178 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption4", 2024-05-04T03:25:29.178 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.178 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.178 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.178 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.178 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.178 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.178 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.179 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.179 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.179 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.179 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.179 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption5": { 2024-05-04T03:25:29.179 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption5", 2024-05-04T03:25:29.179 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.179 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.179 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.179 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.179 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.179 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.179 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.179 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.179 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.179 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.180 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.180 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption6": { 2024-05-04T03:25:29.180 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption6", 2024-05-04T03:25:29.180 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.180 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.180 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.180 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.180 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.180 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.180 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.180 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.180 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.180 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.180 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.180 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.181 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption7": { 2024-05-04T03:25:29.181 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption7", 2024-05-04T03:25:29.181 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.181 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.181 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.181 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.181 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-05-04T03:25:29.181 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "42", 2024-05-04T03:25:29.181 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.181 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.181 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.181 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.181 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.181 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.181 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.182 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.182 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.182 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.182 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.182 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.182 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.182 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.182 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.182 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.182 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.182 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.182 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.182 INFO:teuthology.orchestra.run.smithi001.stdout: "testkey": { 2024-05-04T03:25:29.183 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testkey", 2024-05-04T03:25:29.183 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.183 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.183 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.183 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.183 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.183 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.183 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.183 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.183 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.183 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.183 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.183 INFO:teuthology.orchestra.run.smithi001.stdout: "testlkey": { 2024-05-04T03:25:29.184 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testlkey", 2024-05-04T03:25:29.184 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.184 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.184 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.184 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.184 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.184 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.184 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.184 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.184 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.184 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.184 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.184 INFO:teuthology.orchestra.run.smithi001.stdout: "testnewline": { 2024-05-04T03:25:29.184 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testnewline", 2024-05-04T03:25:29.185 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.185 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.185 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.185 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.185 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.185 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.185 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.185 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.185 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.185 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.185 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.185 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.185 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.186 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.186 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smb", 2024-05-04T03:25:29.186 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.186 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.186 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.186 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.186 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.186 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.186 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.186 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.186 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.186 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.186 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.186 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.187 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.187 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.187 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.187 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.187 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.187 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.187 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.187 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.187 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.187 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.187 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.187 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.187 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.188 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.188 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.188 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.188 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.188 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.188 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.188 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.188 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.188 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.188 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.188 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.188 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.188 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.189 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.189 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.189 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.189 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.189 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.189 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.189 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.189 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.189 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.189 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.189 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.189 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.189 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.189 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.189 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.190 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.190 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.190 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.190 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.190 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.190 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.190 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.190 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.190 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.190 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.190 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.190 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.190 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.191 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.191 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.191 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.191 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.191 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.191 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.191 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.191 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.191 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.191 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.191 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.191 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.191 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.191 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.191 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.192 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.192 INFO:teuthology.orchestra.run.smithi001.stdout: "update_orchestration": { 2024-05-04T03:25:29.192 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "update_orchestration", 2024-05-04T03:25:29.192 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.192 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.192 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.192 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.192 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.192 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.192 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.192 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-04T03:25:29.192 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.192 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.192 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.193 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.193 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.193 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.193 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.193 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2024-05-04T03:25:29.193 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.193 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.193 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.193 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_m_granularity": { 2024-05-04T03:25:29.193 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_m_granularity", 2024-05-04T03:25:29.193 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.193 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.193 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.193 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.193 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.193 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.193 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.194 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow minute scheduled snapshots", 2024-05-04T03:25:29.194 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.194 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.194 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.194 INFO:teuthology.orchestra.run.smithi001.stdout: "dump_on_update": { 2024-05-04T03:25:29.194 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dump_on_update", 2024-05-04T03:25:29.194 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.194 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.194 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.194 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.194 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.194 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.194 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.194 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "dump database to debug log on update", 2024-05-04T03:25:29.195 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.195 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.195 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.195 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.195 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.195 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.195 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.195 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.195 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.195 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.195 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.195 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.195 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.195 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.195 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.195 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.196 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.196 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.196 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.196 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.196 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.196 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.196 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.196 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.196 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.196 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.196 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.196 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.196 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.196 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.197 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.197 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.197 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.197 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.197 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.197 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.197 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.197 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.197 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.197 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.197 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.197 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.197 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.197 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.197 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.198 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.198 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.198 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.198 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.198 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.198 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.198 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.198 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.198 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.198 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.198 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.198 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.198 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.198 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.199 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.199 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.199 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.199 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.199 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.199 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.199 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.199 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.199 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.199 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.199 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.199 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.199 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.199 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.200 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.200 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.200 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.200 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.200 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.200 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.200 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.200 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.200 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.200 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.200 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.200 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.200 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.200 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stats", 2024-05-04T03:25:29.201 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.201 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.201 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.201 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.201 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.201 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.201 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.201 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.201 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.201 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.201 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.201 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.201 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.201 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.201 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.201 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.202 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.202 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.202 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.202 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.202 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.202 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.202 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.202 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.202 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.202 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.202 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.202 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.202 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.203 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.203 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.203 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.203 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.203 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.203 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.203 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.203 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.203 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.203 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.203 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.203 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.203 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.203 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.204 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.204 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.204 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.204 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.204 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.204 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.204 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.204 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.204 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.204 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.204 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.204 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.204 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.204 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.205 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.205 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.205 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.205 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.205 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.205 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.205 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.205 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.205 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.205 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.205 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.205 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.205 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.205 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.206 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.206 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.206 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.206 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.206 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.206 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.206 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.206 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.206 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.206 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.206 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.206 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.206 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.206 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.206 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.207 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "status", 2024-05-04T03:25:29.207 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.207 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.207 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.207 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.207 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.207 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.207 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.207 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.207 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.207 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.207 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.207 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.207 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.207 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.208 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.208 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.208 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.208 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.208 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.208 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.208 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.208 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.208 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.208 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.208 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.208 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.208 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.208 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.209 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.209 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.209 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.209 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.209 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.209 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.209 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.209 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.209 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.209 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.209 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.209 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.209 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.209 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.210 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.210 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.210 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.210 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.210 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.210 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.210 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.210 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.210 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.210 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.210 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.210 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.210 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.210 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.210 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.211 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.211 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.211 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.211 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.211 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.211 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.211 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.211 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.211 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.211 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.211 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.211 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.211 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.212 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.212 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.212 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.212 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.212 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.212 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.212 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.212 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.212 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.212 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.212 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.212 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.212 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.213 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.213 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.213 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.213 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telegraf", 2024-05-04T03:25:29.213 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.213 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.213 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.213 INFO:teuthology.orchestra.run.smithi001.stdout: "address": { 2024-05-04T03:25:29.213 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "address", 2024-05-04T03:25:29.213 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.213 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.213 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.213 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-04T03:25:29.213 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.214 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.214 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.214 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.214 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.214 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.214 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.214 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-05-04T03:25:29.214 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-05-04T03:25:29.214 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:29.214 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.214 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.214 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15", 2024-05-04T03:25:29.214 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.215 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.215 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.215 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.215 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.215 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.215 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.215 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.215 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.215 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.215 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.215 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.215 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.215 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.216 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.216 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.216 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.216 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.216 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.216 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.216 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.216 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.216 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.216 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.216 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.216 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.216 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.217 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.217 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.217 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.217 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.217 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.217 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.217 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.217 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.217 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.217 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.217 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.217 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.217 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.218 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.218 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.218 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.218 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.218 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.218 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.218 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.218 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.218 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.218 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.218 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.218 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.218 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.218 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.218 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.219 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.219 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.219 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.219 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.219 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.219 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.219 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.219 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.219 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.219 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.219 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.219 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.219 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.220 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.220 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.220 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.220 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.220 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.220 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.220 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.220 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.220 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.220 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.220 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.220 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.220 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.221 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.221 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.221 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.221 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.221 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.221 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.221 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.221 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.221 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telemetry", 2024-05-04T03:25:29.221 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.221 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.221 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.221 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_basic": { 2024-05-04T03:25:29.221 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_basic", 2024-05-04T03:25:29.222 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.222 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.222 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.222 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.222 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.222 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.222 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.222 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-04T03:25:29.222 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.222 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.222 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.222 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_crash": { 2024-05-04T03:25:29.222 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_crash", 2024-05-04T03:25:29.223 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.223 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.223 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.223 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.223 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.223 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.223 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.223 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-04T03:25:29.223 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.223 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.223 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.223 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_device": { 2024-05-04T03:25:29.223 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_device", 2024-05-04T03:25:29.224 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.224 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.224 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.224 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.224 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.224 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.224 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.224 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-04T03:25:29.224 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.224 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.224 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.224 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_ident": { 2024-05-04T03:25:29.224 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_ident", 2024-05-04T03:25:29.225 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.225 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.225 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.225 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.225 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.225 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.225 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.225 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-04T03:25:29.225 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.225 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.225 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.225 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_perf": { 2024-05-04T03:25:29.226 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_perf", 2024-05-04T03:25:29.226 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.226 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.226 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.226 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.226 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.226 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.226 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.226 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-04T03:25:29.226 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.226 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.226 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.226 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.226 INFO:teuthology.orchestra.run.smithi001.stdout: "contact": { 2024-05-04T03:25:29.227 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "contact", 2024-05-04T03:25:29.227 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.227 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.227 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.227 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.227 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.227 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.227 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.227 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.227 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.227 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.227 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.227 INFO:teuthology.orchestra.run.smithi001.stdout: "description": { 2024-05-04T03:25:29.228 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "description", 2024-05-04T03:25:29.228 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.228 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.228 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.228 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.228 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.228 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.228 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.228 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.228 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.228 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.228 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.228 INFO:teuthology.orchestra.run.smithi001.stdout: "device_url": { 2024-05-04T03:25:29.228 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_url", 2024-05-04T03:25:29.228 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.229 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.229 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.229 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-04T03:25:29.229 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.229 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.229 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.229 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.229 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.229 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.229 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.229 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.229 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-05-04T03:25:29.229 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-05-04T03:25:29.229 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.229 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.230 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.230 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.230 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.230 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.230 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.230 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.230 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.230 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.230 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.230 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-05-04T03:25:29.230 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-05-04T03:25:29.230 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.230 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.231 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.231 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "24", 2024-05-04T03:25:29.231 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "8", 2024-05-04T03:25:29.231 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.231 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.231 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.231 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.231 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.231 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.231 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.231 INFO:teuthology.orchestra.run.smithi001.stdout: "last_opt_revision": { 2024-05-04T03:25:29.231 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "last_opt_revision", 2024-05-04T03:25:29.231 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.231 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.232 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.232 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1", 2024-05-04T03:25:29.232 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.232 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.232 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.232 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.232 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.232 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.232 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.232 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard": { 2024-05-04T03:25:29.232 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard", 2024-05-04T03:25:29.232 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.232 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.233 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.233 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.233 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.233 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.233 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.233 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.233 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.233 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.233 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.233 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard_description": { 2024-05-04T03:25:29.233 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard_description", 2024-05-04T03:25:29.233 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.233 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.233 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.234 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.234 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.234 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.234 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.234 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.234 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.234 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.234 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.234 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.234 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.234 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.234 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.234 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.235 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.235 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.235 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.235 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.235 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.235 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.235 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.235 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.235 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.235 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.235 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.235 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.235 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.235 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.236 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.236 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.236 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.236 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.236 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.236 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.236 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.236 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.236 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.236 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.236 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.236 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.236 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.236 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.237 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.237 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.237 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.237 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.237 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.237 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.237 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.237 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.237 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.237 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.237 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.237 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.237 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.237 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.237 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.238 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.238 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.238 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.238 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.238 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.238 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.238 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.238 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.238 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.238 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.238 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.238 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.238 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.238 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.238 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.239 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.239 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.239 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.239 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.239 INFO:teuthology.orchestra.run.smithi001.stdout: "organization": { 2024-05-04T03:25:29.239 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "organization", 2024-05-04T03:25:29.239 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.239 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.239 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.239 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.239 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.239 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.239 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.240 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.240 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.240 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.240 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.240 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.240 INFO:teuthology.orchestra.run.smithi001.stdout: "proxy": { 2024-05-04T03:25:29.240 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "proxy", 2024-05-04T03:25:29.240 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.240 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.240 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.240 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.240 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.240 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.240 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.241 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.241 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.241 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.241 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.241 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.241 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.241 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.241 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.241 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.241 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.241 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.241 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.241 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.241 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.242 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.242 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.242 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.242 INFO:teuthology.orchestra.run.smithi001.stdout: "url": { 2024-05-04T03:25:29.242 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url", 2024-05-04T03:25:29.242 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.242 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.242 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.242 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-04T03:25:29.242 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.242 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.242 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.242 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.243 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.243 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.243 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.243 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.243 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.243 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.243 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.243 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "test_orchestrator", 2024-05-04T03:25:29.243 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.243 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.243 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.243 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.243 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.243 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.243 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.244 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.244 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.244 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.244 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.244 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.244 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.244 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.244 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.244 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.244 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.244 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.244 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.244 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.244 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.245 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.245 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.245 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.245 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.245 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.245 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.245 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.245 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.245 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.245 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.245 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.245 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.245 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.245 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.246 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.246 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.246 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.246 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.246 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.246 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.246 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.246 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.246 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.246 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.246 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.246 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.246 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.247 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.247 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.247 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.247 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.247 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.247 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.247 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.247 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.247 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.247 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.247 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.247 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.247 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.247 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.247 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.248 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.248 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.248 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.248 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.248 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.248 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.248 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.248 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.248 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.248 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.248 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.248 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.248 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.248 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.249 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.249 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.249 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.249 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.249 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.249 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.249 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.249 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.249 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.249 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.249 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "volumes", 2024-05-04T03:25:29.249 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.249 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.249 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.250 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.250 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.250 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.250 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.250 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.250 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.250 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.250 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.250 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.250 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.250 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.250 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.250 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.250 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.250 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.251 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.251 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.251 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.251 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.251 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.251 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.251 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.251 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.251 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.251 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.251 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.251 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.251 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.251 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.252 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.252 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.252 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.252 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.252 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.252 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.252 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.252 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.252 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.252 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.252 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.252 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.252 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.252 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.253 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.253 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.253 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.253 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.253 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.253 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.253 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.253 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.253 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.253 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.253 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.253 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.253 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.254 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.254 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.254 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.254 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.254 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.254 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.254 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.254 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.254 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.254 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.254 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.254 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_clones": { 2024-05-04T03:25:29.254 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_clones", 2024-05-04T03:25:29.255 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.255 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.255 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.255 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4", 2024-05-04T03:25:29.255 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.255 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.255 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.255 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-04T03:25:29.255 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.255 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.255 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.255 INFO:teuthology.orchestra.run.smithi001.stdout: "periodic_async_work": { 2024-05-04T03:25:29.255 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "periodic_async_work", 2024-05-04T03:25:29.256 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.256 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.256 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.256 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.256 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.256 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.256 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.256 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Periodically check for async work", 2024-05-04T03:25:29.256 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.256 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.256 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.256 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.256 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_delay": { 2024-05-04T03:25:29.256 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_delay", 2024-05-04T03:25:29.257 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.257 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.257 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.257 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.257 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.257 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.257 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.257 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-04T03:25:29.257 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.257 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.257 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.257 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_no_wait": { 2024-05-04T03:25:29.257 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_no_wait", 2024-05-04T03:25:29.258 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.258 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.258 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.258 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.258 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.258 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.258 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.258 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-04T03:25:29.258 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.258 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.258 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.258 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.258 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.258 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.259 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.259 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.259 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.259 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.259 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.259 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.259 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.259 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.259 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.259 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.259 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.259 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.259 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.259 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.259 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.260 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix", 2024-05-04T03:25:29.260 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.260 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.260 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.260 INFO:teuthology.orchestra.run.smithi001.stdout: "discovery_interval": { 2024-05-04T03:25:29.260 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "discovery_interval", 2024-05-04T03:25:29.260 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-05-04T03:25:29.260 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.260 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.260 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "100", 2024-05-04T03:25:29.260 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.260 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.260 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.260 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.260 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.261 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.261 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.261 INFO:teuthology.orchestra.run.smithi001.stdout: "identifier": { 2024-05-04T03:25:29.261 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "identifier", 2024-05-04T03:25:29.261 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.261 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.261 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.261 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.261 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.261 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.261 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.261 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.261 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.261 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.262 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.262 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.262 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-05-04T03:25:29.262 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-05-04T03:25:29.262 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:29.262 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.262 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.262 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-05-04T03:25:29.262 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.262 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.262 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.262 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.262 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.263 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.263 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.263 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.263 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.263 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.263 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.263 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.263 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.263 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.263 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.263 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.263 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.263 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.263 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.264 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.264 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.264 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.264 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.264 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.264 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.264 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.264 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.264 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.264 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.264 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.264 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.264 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.264 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.265 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.265 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.265 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.265 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.265 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.265 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.265 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.265 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.265 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.265 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.265 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.265 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.265 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.265 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.266 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.266 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.266 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.266 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.266 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.266 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.266 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.266 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.266 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.266 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.266 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.266 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.266 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.266 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.267 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.267 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.267 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.267 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.267 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.267 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.267 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.267 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.267 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.267 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.267 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.267 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.267 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.268 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.268 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.268 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.268 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.268 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.268 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.268 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.268 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.268 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.268 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.268 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.268 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.268 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.269 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_host": { 2024-05-04T03:25:29.269 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_host", 2024-05-04T03:25:29.269 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.269 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.269 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.269 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.269 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.269 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.269 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.269 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.269 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.269 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.269 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.269 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_port": { 2024-05-04T03:25:29.270 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_port", 2024-05-04T03:25:29.270 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.270 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.270 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.270 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10051", 2024-05-04T03:25:29.270 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.270 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.270 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.270 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.270 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.270 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.270 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.270 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.270 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_sender": { 2024-05-04T03:25:29.271 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_sender", 2024-05-04T03:25:29.271 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.271 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.271 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.271 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-04T03:25:29.271 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.271 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.271 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.271 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.271 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.271 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.271 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.271 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.271 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.272 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.272 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-04T03:25:29.272 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.272 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.272 INFO:teuthology.orchestra.run.smithi001.stdout: "modules": [ 2024-05-04T03:25:29.272 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm", 2024-05-04T03:25:29.272 INFO:teuthology.orchestra.run.smithi001.stdout: "dashboard", 2024-05-04T03:25:29.272 INFO:teuthology.orchestra.run.smithi001.stdout: "iostat", 2024-05-04T03:25:29.272 INFO:teuthology.orchestra.run.smithi001.stdout: "nfs", 2024-05-04T03:25:29.272 INFO:teuthology.orchestra.run.smithi001.stdout: "restful" 2024-05-04T03:25:29.272 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.272 INFO:teuthology.orchestra.run.smithi001.stdout: "available_modules": [ 2024-05-04T03:25:29.272 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.272 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "alerts", 2024-05-04T03:25:29.273 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.273 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.273 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.273 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-05-04T03:25:29.273 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-05-04T03:25:29.273 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:29.273 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.273 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.273 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-05-04T03:25:29.273 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.273 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.273 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.273 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How frequently to reexamine health status", 2024-05-04T03:25:29.273 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.274 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.274 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.274 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.274 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.274 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.274 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.274 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.274 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.274 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.274 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.274 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.274 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.274 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.275 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.275 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.275 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.275 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.275 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.275 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.275 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.275 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.275 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.275 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.275 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.275 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.275 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.275 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.276 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.276 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.276 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.276 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.276 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.276 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.276 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.276 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.276 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.276 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.276 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.276 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.276 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.276 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.276 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.277 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.277 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.277 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.277 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.277 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.277 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.277 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.277 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.277 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.277 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.277 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.277 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.277 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.277 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.278 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.278 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.278 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.278 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.278 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.278 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.278 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.278 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.278 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.278 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.278 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.278 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.278 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.279 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_destination": { 2024-05-04T03:25:29.279 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_destination", 2024-05-04T03:25:29.279 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.279 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.279 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.279 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.279 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.279 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.279 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.279 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email address to send alerts to", 2024-05-04T03:25:29.279 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.279 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.279 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.279 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.280 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_from_name": { 2024-05-04T03:25:29.280 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_from_name", 2024-05-04T03:25:29.280 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.280 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.280 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.280 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "Ceph", 2024-05-04T03:25:29.280 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.280 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.280 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.280 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email From: name", 2024-05-04T03:25:29.280 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.280 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.280 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.281 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_host": { 2024-05-04T03:25:29.281 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_host", 2024-05-04T03:25:29.281 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.281 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.281 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.281 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.281 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.281 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.281 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.281 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP server", 2024-05-04T03:25:29.281 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.281 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.281 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.281 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.282 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_password": { 2024-05-04T03:25:29.282 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_password", 2024-05-04T03:25:29.282 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.282 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.282 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.282 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.282 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.282 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.282 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.282 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Password to authenticate with", 2024-05-04T03:25:29.282 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.282 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.282 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.282 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_port": { 2024-05-04T03:25:29.283 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_port", 2024-05-04T03:25:29.283 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.283 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.283 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.283 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "465", 2024-05-04T03:25:29.283 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.283 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.283 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.283 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP port", 2024-05-04T03:25:29.283 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.283 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.283 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.283 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.283 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_sender": { 2024-05-04T03:25:29.283 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_sender", 2024-05-04T03:25:29.284 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.284 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.284 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.284 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.284 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.284 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.284 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.284 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP envelope sender", 2024-05-04T03:25:29.284 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.284 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.284 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.284 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_ssl": { 2024-05-04T03:25:29.284 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_ssl", 2024-05-04T03:25:29.285 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.285 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.285 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.285 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.285 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.285 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.285 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.285 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-04T03:25:29.285 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.285 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.285 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.285 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_user": { 2024-05-04T03:25:29.285 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_user", 2024-05-04T03:25:29.286 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.286 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.286 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.286 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.286 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.286 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.286 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.286 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "User to authenticate as", 2024-05-04T03:25:29.286 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.286 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.286 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.286 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.286 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.287 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.287 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.287 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.287 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.287 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.287 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.287 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.287 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.287 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.287 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.287 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.287 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.287 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.287 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.288 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.288 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "balancer", 2024-05-04T03:25:29.288 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.288 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.288 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.288 INFO:teuthology.orchestra.run.smithi001.stdout: "active": { 2024-05-04T03:25:29.288 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "active", 2024-05-04T03:25:29.288 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.288 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.288 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.288 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.288 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.288 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.288 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.289 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically balance PGs across cluster", 2024-05-04T03:25:29.289 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.289 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.289 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.289 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.289 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_time": { 2024-05-04T03:25:29.289 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_time", 2024-05-04T03:25:29.289 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.289 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.289 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.289 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0000", 2024-05-04T03:25:29.289 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.289 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.289 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.289 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "beginning time of day to automatically balance", 2024-05-04T03:25:29.290 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-04T03:25:29.290 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.290 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.290 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_weekday": { 2024-05-04T03:25:29.290 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_weekday", 2024-05-04T03:25:29.290 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-05-04T03:25:29.290 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.290 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.290 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.290 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-05-04T03:25:29.290 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-05-04T03:25:29.290 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.290 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-04T03:25:29.291 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-04T03:25:29.291 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.291 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.291 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_max_iterations": { 2024-05-04T03:25:29.291 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_max_iterations", 2024-05-04T03:25:29.291 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-05-04T03:25:29.291 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.291 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.291 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "25", 2024-05-04T03:25:29.291 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-05-04T03:25:29.291 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "250", 2024-05-04T03:25:29.291 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.291 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-04T03:25:29.292 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.292 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.292 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.292 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_metrics": { 2024-05-04T03:25:29.292 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_metrics", 2024-05-04T03:25:29.292 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.292 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.292 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.292 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "pgs,objects,bytes", 2024-05-04T03:25:29.292 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.292 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.292 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.292 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-04T03:25:29.292 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-04T03:25:29.293 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.293 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.293 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_step": { 2024-05-04T03:25:29.293 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_step", 2024-05-04T03:25:29.293 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-05-04T03:25:29.293 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.293 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.293 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.5", 2024-05-04T03:25:29.293 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0.001", 2024-05-04T03:25:29.293 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "0.999", 2024-05-04T03:25:29.293 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.293 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "aggressiveness of optimization", 2024-05-04T03:25:29.293 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-04T03:25:29.293 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.294 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.294 INFO:teuthology.orchestra.run.smithi001.stdout: "end_time": { 2024-05-04T03:25:29.294 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_time", 2024-05-04T03:25:29.294 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.294 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.294 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.294 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2359", 2024-05-04T03:25:29.294 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.294 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.294 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.294 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ending time of day to automatically balance", 2024-05-04T03:25:29.294 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-04T03:25:29.294 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.295 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.295 INFO:teuthology.orchestra.run.smithi001.stdout: "end_weekday": { 2024-05-04T03:25:29.295 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_weekday", 2024-05-04T03:25:29.295 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-05-04T03:25:29.295 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.295 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.295 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.295 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-05-04T03:25:29.295 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-05-04T03:25:29.295 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.295 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-04T03:25:29.295 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-04T03:25:29.295 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.296 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.296 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.296 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.296 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.296 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.296 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.296 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.296 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.296 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.296 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.296 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.296 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.296 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.296 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.297 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.297 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.297 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.297 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.297 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.297 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.297 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.297 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.297 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.297 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.297 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.297 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.297 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.298 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.298 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.298 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.298 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.298 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.298 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.298 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.298 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.298 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.298 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.298 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.298 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.298 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.298 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.299 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.299 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.299 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.299 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.299 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.299 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.299 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.299 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.299 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.299 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.299 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.299 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.299 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.299 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.299 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.300 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.300 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.300 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.300 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.300 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.300 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.300 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.300 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.300 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.300 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.300 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.300 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.300 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.301 INFO:teuthology.orchestra.run.smithi001.stdout: "min_score": { 2024-05-04T03:25:29.301 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_score", 2024-05-04T03:25:29.301 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-05-04T03:25:29.301 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.301 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.301 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.301 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.301 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.301 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.301 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-04T03:25:29.301 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.301 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.301 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.302 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.302 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-05-04T03:25:29.302 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-05-04T03:25:29.302 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.302 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.302 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.302 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "upmap", 2024-05-04T03:25:29.302 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.302 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.302 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.302 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-compat", 2024-05-04T03:25:29.302 INFO:teuthology.orchestra.run.smithi001.stdout: "none", 2024-05-04T03:25:29.302 INFO:teuthology.orchestra.run.smithi001.stdout: "read", 2024-05-04T03:25:29.302 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap", 2024-05-04T03:25:29.303 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap-read" 2024-05-04T03:25:29.303 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.303 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Balancer mode", 2024-05-04T03:25:29.303 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.303 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.303 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.303 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_ids": { 2024-05-04T03:25:29.303 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_ids", 2024-05-04T03:25:29.303 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.303 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.303 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.303 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.303 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.303 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.304 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.304 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-04T03:25:29.304 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.304 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.304 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.304 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-05-04T03:25:29.304 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-05-04T03:25:29.304 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:29.304 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.304 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.304 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-05-04T03:25:29.304 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.304 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.305 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.305 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-04T03:25:29.305 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.305 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.305 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.305 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.305 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.305 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.305 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.305 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.305 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.305 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.305 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.306 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.306 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.306 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.306 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.306 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.306 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_deviation": { 2024-05-04T03:25:29.306 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_deviation", 2024-05-04T03:25:29.306 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.306 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.306 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.306 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-05-04T03:25:29.306 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-05-04T03:25:29.306 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.307 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.307 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-04T03:25:29.307 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-04T03:25:29.307 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.307 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.307 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_optimizations": { 2024-05-04T03:25:29.307 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_optimizations", 2024-05-04T03:25:29.307 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-05-04T03:25:29.307 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.307 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.307 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-05-04T03:25:29.307 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.307 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.307 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.308 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-04T03:25:29.308 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.308 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.308 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.308 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.308 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.308 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.308 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm", 2024-05-04T03:25:29.308 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.308 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.308 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.308 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_down_multiplier": { 2024-05-04T03:25:29.308 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_down_multiplier", 2024-05-04T03:25:29.308 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-05-04T03:25:29.309 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.309 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.309 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-05-04T03:25:29.309 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.309 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.309 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.309 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-04T03:25:29.309 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.309 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.309 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.309 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_refresh_rate": { 2024-05-04T03:25:29.309 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_refresh_rate", 2024-05-04T03:25:29.309 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:29.309 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.310 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.310 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "20", 2024-05-04T03:25:29.310 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.310 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.310 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.310 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-04T03:25:29.310 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.310 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.310 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.310 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.310 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_starting_port": { 2024-05-04T03:25:29.310 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_starting_port", 2024-05-04T03:25:29.310 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.310 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.311 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.311 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4721", 2024-05-04T03:25:29.311 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.311 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.311 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.311 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-04T03:25:29.311 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.311 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.311 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.311 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_ptrace": { 2024-05-04T03:25:29.311 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_ptrace", 2024-05-04T03:25:29.311 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.311 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.312 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.312 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.312 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.312 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.312 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.312 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-04T03:25:29.312 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-04T03:25:29.312 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.312 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.312 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.312 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_interval": { 2024-05-04T03:25:29.312 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_interval", 2024-05-04T03:25:29.312 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:29.312 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.312 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.313 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-05-04T03:25:29.313 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.313 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.313 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.313 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-04T03:25:29.313 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.313 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.313 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.313 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_memory_target_ratio": { 2024-05-04T03:25:29.313 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_memory_target_ratio", 2024-05-04T03:25:29.313 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-05-04T03:25:29.313 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.313 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.314 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.7", 2024-05-04T03:25:29.314 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.314 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.314 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.314 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-04T03:25:29.314 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.314 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.314 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.314 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.314 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm_log_destination": { 2024-05-04T03:25:29.314 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm_log_destination", 2024-05-04T03:25:29.314 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.314 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.315 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.315 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.315 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.315 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.315 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.315 INFO:teuthology.orchestra.run.smithi001.stdout: "file", 2024-05-04T03:25:29.315 INFO:teuthology.orchestra.run.smithi001.stdout: "file,syslog", 2024-05-04T03:25:29.315 INFO:teuthology.orchestra.run.smithi001.stdout: "syslog" 2024-05-04T03:25:29.315 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.315 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-04T03:25:29.315 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.315 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.315 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.315 INFO:teuthology.orchestra.run.smithi001.stdout: "cgroups_split": { 2024-05-04T03:25:29.315 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cgroups_split", 2024-05-04T03:25:29.316 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.316 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.316 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.316 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.316 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.316 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.316 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.316 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-04T03:25:29.316 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.316 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.316 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.316 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.316 INFO:teuthology.orchestra.run.smithi001.stdout: "config_checks_enabled": { 2024-05-04T03:25:29.316 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_checks_enabled", 2024-05-04T03:25:29.316 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.316 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.317 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.317 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.317 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.317 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.317 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.317 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-04T03:25:29.317 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.317 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.317 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.317 INFO:teuthology.orchestra.run.smithi001.stdout: "config_dashboard": { 2024-05-04T03:25:29.317 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_dashboard", 2024-05-04T03:25:29.317 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.317 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.317 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.318 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.318 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.318 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.318 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.318 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-04T03:25:29.318 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.318 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.318 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.318 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_alertmanager": { 2024-05-04T03:25:29.318 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_alertmanager", 2024-05-04T03:25:29.318 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.318 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.318 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.319 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-04T03:25:29.319 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.319 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.319 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.319 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-05-04T03:25:29.319 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.319 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.319 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.319 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_base": { 2024-05-04T03:25:29.319 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_base", 2024-05-04T03:25:29.319 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.319 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.319 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.319 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-04T03:25:29.319 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.320 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.320 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.320 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Container image name, without the tag", 2024-05-04T03:25:29.320 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.320 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.320 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.320 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_elasticsearch": { 2024-05-04T03:25:29.320 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_elasticsearch", 2024-05-04T03:25:29.320 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.320 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.320 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.320 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-04T03:25:29.320 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.321 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.321 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.321 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "elasticsearch container image", 2024-05-04T03:25:29.321 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.321 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.321 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.321 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_grafana": { 2024-05-04T03:25:29.321 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_grafana", 2024-05-04T03:25:29.321 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.321 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.321 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.321 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-04T03:25:29.321 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.321 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.321 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.322 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-05-04T03:25:29.322 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.322 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.322 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.322 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.322 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_haproxy": { 2024-05-04T03:25:29.322 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_haproxy", 2024-05-04T03:25:29.322 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.322 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.322 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.322 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-04T03:25:29.322 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.322 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.322 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.322 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "HAproxy container image", 2024-05-04T03:25:29.323 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.323 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.323 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.323 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_agent": { 2024-05-04T03:25:29.323 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_agent", 2024-05-04T03:25:29.323 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.323 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.323 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.323 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-04T03:25:29.323 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.323 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.323 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.323 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger agent container image", 2024-05-04T03:25:29.323 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.323 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.324 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.324 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_collector": { 2024-05-04T03:25:29.324 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_collector", 2024-05-04T03:25:29.324 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.324 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.324 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.324 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-04T03:25:29.324 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.324 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.324 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.324 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger collector container image", 2024-05-04T03:25:29.324 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.324 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.324 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.325 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_query": { 2024-05-04T03:25:29.325 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_query", 2024-05-04T03:25:29.325 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.325 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.325 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.325 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-04T03:25:29.325 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.325 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.325 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.325 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger query container image", 2024-05-04T03:25:29.325 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.325 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.325 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.326 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_keepalived": { 2024-05-04T03:25:29.326 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_keepalived", 2024-05-04T03:25:29.326 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.326 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.326 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.326 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-04T03:25:29.326 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.329 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.329 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.329 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Keepalived container image", 2024-05-04T03:25:29.329 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.329 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.329 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.330 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_loki": { 2024-05-04T03:25:29.330 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_loki", 2024-05-04T03:25:29.330 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.330 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.330 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.330 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-04T03:25:29.330 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.330 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.330 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.330 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Loki container image", 2024-05-04T03:25:29.330 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.330 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.331 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.331 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_node_exporter": { 2024-05-04T03:25:29.331 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_node_exporter", 2024-05-04T03:25:29.331 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.331 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.331 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.331 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-04T03:25:29.331 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.331 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.331 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.331 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-05-04T03:25:29.331 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.331 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.331 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.332 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.332 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_nvmeof": { 2024-05-04T03:25:29.332 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_nvmeof", 2024-05-04T03:25:29.332 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.332 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.332 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.332 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-04T03:25:29.332 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.332 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.332 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.332 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Nvme-of container image", 2024-05-04T03:25:29.332 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.332 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.333 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.333 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_prometheus": { 2024-05-04T03:25:29.333 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_prometheus", 2024-05-04T03:25:29.333 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.333 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.333 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.333 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-04T03:25:29.333 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.333 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.333 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.333 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-05-04T03:25:29.333 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.333 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.333 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.334 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_promtail": { 2024-05-04T03:25:29.334 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_promtail", 2024-05-04T03:25:29.334 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.334 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.334 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.334 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-04T03:25:29.334 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.334 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.334 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.334 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Promtail container image", 2024-05-04T03:25:29.334 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.334 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.334 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.335 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_samba": { 2024-05-04T03:25:29.335 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_samba", 2024-05-04T03:25:29.335 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.335 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.335 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.335 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-04T03:25:29.335 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.335 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.335 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.335 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Samba/SMB container image", 2024-05-04T03:25:29.335 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.335 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.335 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.336 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_snmp_gateway": { 2024-05-04T03:25:29.336 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_snmp_gateway", 2024-05-04T03:25:29.336 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.336 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.336 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.336 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-04T03:25:29.336 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.336 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.336 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.336 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SNMP Gateway container image", 2024-05-04T03:25:29.336 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.336 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.336 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.336 INFO:teuthology.orchestra.run.smithi001.stdout: "container_init": { 2024-05-04T03:25:29.337 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_init", 2024-05-04T03:25:29.337 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.337 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.337 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.337 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.337 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.337 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.337 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.337 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Run podman/docker with `--init`", 2024-05-04T03:25:29.337 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.337 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.337 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.337 INFO:teuthology.orchestra.run.smithi001.stdout: "daemon_cache_timeout": { 2024-05-04T03:25:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "daemon_cache_timeout", 2024-05-04T03:25:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-05-04T03:25:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-04T03:25:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: "default_cephadm_command_timeout": { 2024-05-04T03:25:29.338 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_cephadm_command_timeout", 2024-05-04T03:25:29.339 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.339 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.339 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.339 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "900", 2024-05-04T03:25:29.339 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.339 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.339 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.339 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-04T03:25:29.339 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.339 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.339 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.339 INFO:teuthology.orchestra.run.smithi001.stdout: "default_registry": { 2024-05-04T03:25:29.339 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_registry", 2024-05-04T03:25:29.339 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.340 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.340 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.340 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io", 2024-05-04T03:25:29.340 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.340 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.340 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.340 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-04T03:25:29.340 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.340 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.340 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.340 INFO:teuthology.orchestra.run.smithi001.stdout: "device_cache_timeout": { 2024-05-04T03:25:29.340 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_cache_timeout", 2024-05-04T03:25:29.340 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:29.340 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.341 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.341 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1800", 2024-05-04T03:25:29.341 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.341 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.341 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.341 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache device inventory", 2024-05-04T03:25:29.341 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.341 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.341 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.341 INFO:teuthology.orchestra.run.smithi001.stdout: "device_enhanced_scan": { 2024-05-04T03:25:29.341 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_enhanced_scan", 2024-05-04T03:25:29.341 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.342 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.342 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.342 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.342 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.342 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.342 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.342 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-04T03:25:29.342 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.342 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.342 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.342 INFO:teuthology.orchestra.run.smithi001.stdout: "facts_cache_timeout": { 2024-05-04T03:25:29.342 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "facts_cache_timeout", 2024-05-04T03:25:29.342 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-05-04T03:25:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache host facts data", 2024-05-04T03:25:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: "grafana_dashboards_path": { 2024-05-04T03:25:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "grafana_dashboards_path", 2024-05-04T03:25:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.343 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.344 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.344 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-04T03:25:29.344 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.344 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.344 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.344 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-04T03:25:29.344 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.344 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.344 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.344 INFO:teuthology.orchestra.run.smithi001.stdout: "host_check_interval": { 2024-05-04T03:25:29.344 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "host_check_interval", 2024-05-04T03:25:29.344 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:29.344 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.344 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.345 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-05-04T03:25:29.345 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.345 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.345 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.345 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to perform a host check", 2024-05-04T03:25:29.345 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.345 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.345 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.345 INFO:teuthology.orchestra.run.smithi001.stdout: "hw_monitoring": { 2024-05-04T03:25:29.345 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hw_monitoring", 2024-05-04T03:25:29.345 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.345 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.345 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.346 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.346 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.346 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.346 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.346 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-04T03:25:29.346 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.346 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.346 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.346 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.346 INFO:teuthology.orchestra.run.smithi001.stdout: "inventory_list_all": { 2024-05-04T03:25:29.346 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "inventory_list_all", 2024-05-04T03:25:29.346 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.346 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.346 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.347 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.347 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.347 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.347 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.347 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-04T03:25:29.347 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.347 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.347 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.347 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.347 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.347 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.347 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.348 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.348 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.348 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.348 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.348 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.348 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.348 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.348 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.348 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.348 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.348 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.348 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.348 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.348 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.349 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.349 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.349 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.349 INFO:teuthology.orchestra.run.smithi001.stdout: "log_refresh_metadata": { 2024-05-04T03:25:29.349 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_refresh_metadata", 2024-05-04T03:25:29.349 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.349 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.349 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.349 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.349 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.349 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.349 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.349 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-04T03:25:29.349 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.350 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.350 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.350 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.350 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.350 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.350 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.350 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.350 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.350 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.350 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.350 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.350 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-04T03:25:29.351 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.351 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.351 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.351 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.351 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.351 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.351 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.351 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.351 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.351 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.351 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.351 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.351 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.352 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.353 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.353 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.353 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.353 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.353 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.353 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.353 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.353 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.353 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.353 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-04T03:25:29.353 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-04T03:25:29.353 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.353 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.354 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.354 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.354 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.354 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.354 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.354 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-04T03:25:29.354 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.354 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.354 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.354 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-04T03:25:29.354 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-04T03:25:29.354 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.354 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.354 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "*", 2024-05-04T03:25:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-04T03:25:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: "max_count_per_host": { 2024-05-04T03:25:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_count_per_host", 2024-05-04T03:25:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.355 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.356 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-05-04T03:25:29.356 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.356 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.356 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.356 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of daemons per service per host", 2024-05-04T03:25:29.356 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.356 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.356 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.356 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd_draining_count": { 2024-05-04T03:25:29.356 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_osd_draining_count", 2024-05-04T03:25:29.356 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.356 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.356 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.356 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-05-04T03:25:29.357 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.357 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.357 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.357 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-04T03:25:29.357 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.357 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.357 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.357 INFO:teuthology.orchestra.run.smithi001.stdout: "migration_current": { 2024-05-04T03:25:29.357 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "migration_current", 2024-05-04T03:25:29.357 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.357 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.357 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.357 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "internal - do not modify", 2024-05-04T03:25:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-05-04T03:25:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-05-04T03:25:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.358 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "root", 2024-05-04T03:25:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm-package", 2024-05-04T03:25:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: "root" 2024-05-04T03:25:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "mode for remote execution of cephadm", 2024-05-04T03:25:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: "oob_default_addr": { 2024-05-04T03:25:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "oob_default_addr", 2024-05-04T03:25:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.359 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "169.254.1.1", 2024-05-04T03:25:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-04T03:25:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: "prometheus_alerts_path": { 2024-05-04T03:25:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus_alerts_path", 2024-05-04T03:25:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.360 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-04T03:25:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-04T03:25:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_insecure": { 2024-05-04T03:25:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_insecure", 2024-05-04T03:25:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.361 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-04T03:25:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_password": { 2024-05-04T03:25:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_password", 2024-05-04T03:25:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.362 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-04T03:25:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_url": { 2024-05-04T03:25:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_url", 2024-05-04T03:25:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.363 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-04T03:25:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_username": { 2024-05-04T03:25:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_username", 2024-05-04T03:25:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.364 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-04T03:25:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: "secure_monitoring_stack": { 2024-05-04T03:25:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "secure_monitoring_stack", 2024-05-04T03:25:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.365 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-04T03:25:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: "service_discovery_port": { 2024-05-04T03:25:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "service_discovery_port", 2024-05-04T03:25:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8765", 2024-05-04T03:25:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.366 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "cephadm service discovery port", 2024-05-04T03:25:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.367 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: "ssh_config_file": { 2024-05-04T03:25:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssh_config_file", 2024-05-04T03:25:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-04T03:25:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.368 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: "use_agent": { 2024-05-04T03:25:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_agent", 2024-05-04T03:25:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-04T03:25:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.369 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: "use_repo_digest": { 2024-05-04T03:25:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_repo_digest", 2024-05-04T03:25:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-04T03:25:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.370 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.371 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_failed_host_check": { 2024-05-04T03:25:29.371 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_failed_host_check", 2024-05-04T03:25:29.371 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.371 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.371 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.371 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.371 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.371 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.371 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.371 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if the host check fails", 2024-05-04T03:25:29.371 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.371 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.371 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_daemons": { 2024-05-04T03:25:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_daemons", 2024-05-04T03:25:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-04T03:25:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_hosts": { 2024-05-04T03:25:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_hosts", 2024-05-04T03:25:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-04T03:25:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.373 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crash", 2024-05-04T03:25:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.374 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.375 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.375 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.375 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.375 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.375 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.375 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.375 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.375 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.375 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.375 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.375 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.375 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.375 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.376 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.378 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: "retain_interval": { 2024-05-04T03:25:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retain_interval", 2024-05-04T03:25:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "31536000", 2024-05-04T03:25:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-04T03:25:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.379 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_recent_interval": { 2024-05-04T03:25:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_recent_interval", 2024-05-04T03:25:29.380 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:29.381 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.381 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.381 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1209600", 2024-05-04T03:25:29.381 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.381 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.381 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.381 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-04T03:25:29.381 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.381 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.381 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.381 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.381 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.381 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.381 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dashboard", 2024-05-04T03:25:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-04T03:25:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-04T03:25:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-05-04T03:25:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.382 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_HOST": { 2024-05-04T03:25:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-04T03:25:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.383 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-04T03:25:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-04T03:25:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.384 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_ENABLED": { 2024-05-04T03:25:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_ENABLED", 2024-05-04T03:25:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-04T03:25:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-04T03:25:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.386 INFO:teuthology.orchestra.run.smithi001.stdout: "ENABLE_BROWSABLE_API": { 2024-05-04T03:25:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-04T03:25:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-04T03:25:29.387 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-04T03:25:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-04T03:25:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-04T03:25:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.388 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.389 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.389 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.389 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.389 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.389 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.389 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.389 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.389 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.389 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.389 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.389 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-04T03:25:29.389 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-04T03:25:29.389 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.389 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.389 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.389 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-04T03:25:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-04T03:25:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.390 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-04T03:25:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-04T03:25:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.391 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.392 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.392 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.392 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.392 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.392 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.392 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-04T03:25:29.392 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-04T03:25:29.392 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.392 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.392 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.392 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.392 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.392 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.392 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-04T03:25:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-04T03:25:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.393 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-04T03:25:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-04T03:25:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.394 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_PASSWORD": { 2024-05-04T03:25:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-04T03:25:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-05-04T03:25:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.395 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-04T03:25:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-04T03:25:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.396 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_URL": { 2024-05-04T03:25:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_URL", 2024-05-04T03:25:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_USERNAME": { 2024-05-04T03:25:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-04T03:25:29.397 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-05-04T03:25:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-04T03:25:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-04T03:25:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.398 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-04T03:25:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-04T03:25:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.399 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-04T03:25:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-04T03:25:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.400 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-04T03:25:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-04T03:25:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.401 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.402 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.402 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.402 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.402 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.402 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.402 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.402 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.402 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.402 INFO:teuthology.orchestra.run.smithi001.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-04T03:25:29.402 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-04T03:25:29.402 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.402 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.402 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "[]", 2024-05-04T03:25:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: "MULTICLUSTER_CONFIG": { 2024-05-04T03:25:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-04T03:25:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.403 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "{}", 2024-05-04T03:25:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_HOST": { 2024-05-04T03:25:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-04T03:25:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.404 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-04T03:25:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-04T03:25:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.405 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.406 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.406 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.406 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.406 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.406 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.406 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-04T03:25:29.406 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-04T03:25:29.406 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.406 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.406 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.406 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.406 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.406 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-04T03:25:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-04T03:25:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.407 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-04T03:25:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-04T03:25:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.408 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-04T03:25:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-04T03:25:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.409 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-04T03:25:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-04T03:25:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.410 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-04T03:25:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-04T03:25:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-04T03:25:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-04T03:25:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.412 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_ENABLED": { 2024-05-04T03:25:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-04T03:25:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-04T03:25:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-04T03:25:29.413 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-04T03:25:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-04T03:25:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-04T03:25:29.414 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.415 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.415 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.415 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-05-04T03:25:29.415 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.415 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.415 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.415 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.415 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.415 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.415 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.415 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-04T03:25:29.415 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-04T03:25:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8", 2024-05-04T03:25:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-04T03:25:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-04T03:25:29.416 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.417 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.417 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.417 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "45", 2024-05-04T03:25:29.417 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.417 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.417 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.417 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.417 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.417 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.417 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.417 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ACCESS_KEY": { 2024-05-04T03:25:29.417 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-04T03:25:29.417 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.417 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-04T03:25:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-04T03:25:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.418 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-05-04T03:25:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SECRET_KEY": { 2024-05-04T03:25:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-04T03:25:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.419 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SSL_VERIFY": { 2024-05-04T03:25:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-04T03:25:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.420 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: "UNSAFE_TLS_v1_2": { 2024-05-04T03:25:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-04T03:25:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.421 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-04T03:25:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-04T03:25:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.422 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-04T03:25:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-04T03:25:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-05-04T03:25:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.423 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-04T03:25:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-04T03:25:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-05-04T03:25:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.424 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "cross_origin_url": { 2024-05-04T03:25:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cross_origin_url", 2024-05-04T03:25:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.425 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: "crt_file": { 2024-05-04T03:25:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crt_file", 2024-05-04T03:25:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.426 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "debug": { 2024-05-04T03:25:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "debug", 2024-05-04T03:25:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.427 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable/disable debug options", 2024-05-04T03:25:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "jwt_token_ttl": { 2024-05-04T03:25:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "jwt_token_ttl", 2024-05-04T03:25:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "28800", 2024-05-04T03:25:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.428 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-05-04T03:25:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-05-04T03:25:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.429 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.430 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.431 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.432 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.433 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.434 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "motd": { 2024-05-04T03:25:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "motd", 2024-05-04T03:25:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "The message of the day", 2024-05-04T03:25:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.435 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect_resolve_ip_addr": { 2024-05-04T03:25:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "redirect_resolve_ip_addr", 2024-05-04T03:25:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.436 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-05-04T03:25:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-05-04T03:25:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-05-04T03:25:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.437 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-05-04T03:25:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-05-04T03:25:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8080", 2024-05-04T03:25:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.438 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.439 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-05-04T03:25:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-05-04T03:25:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.440 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.441 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.441 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl_server_port": { 2024-05-04T03:25:29.441 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl_server_port", 2024-05-04T03:25:29.441 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.441 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.441 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.441 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8443", 2024-05-04T03:25:29.441 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.441 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.441 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.441 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.441 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.441 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.441 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.442 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-05-04T03:25:29.442 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-05-04T03:25:29.442 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.442 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.442 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.442 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "redirect", 2024-05-04T03:25:29.442 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.442 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.442 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.442 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.442 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect" 2024-05-04T03:25:29.442 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.442 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.443 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.443 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.443 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.443 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.443 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-05-04T03:25:29.443 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-05-04T03:25:29.443 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.443 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.443 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.443 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-05-04T03:25:29.443 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-05-04T03:25:29.443 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-05-04T03:25:29.443 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.443 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.444 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.444 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.444 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.444 INFO:teuthology.orchestra.run.smithi001.stdout: "url_prefix": { 2024-05-04T03:25:29.444 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url_prefix", 2024-05-04T03:25:29.444 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.444 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.444 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.444 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.444 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.444 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.444 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.444 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.444 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.445 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.445 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.445 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.445 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.445 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.445 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "devicehealth", 2024-05-04T03:25:29.445 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.445 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.445 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.445 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_monitoring": { 2024-05-04T03:25:29.445 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_monitoring", 2024-05-04T03:25:29.445 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.445 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.445 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.446 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.446 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.446 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.446 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.446 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "monitor device health metrics", 2024-05-04T03:25:29.446 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.446 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.446 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.446 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.446 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.446 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.446 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.446 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.447 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.447 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.447 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.447 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.447 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.447 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.447 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.447 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.447 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.447 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.448 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.448 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.448 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.448 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.448 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.448 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.448 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.448 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.449 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.449 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.449 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.449 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.449 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.449 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.449 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.449 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.449 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.449 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.450 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.450 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.450 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.450 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.450 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.450 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.450 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.450 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.450 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.450 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.451 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.451 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.451 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.451 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.451 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.451 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.451 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.451 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.451 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.452 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.452 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.452 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.452 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.452 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.452 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.452 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.452 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.452 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.452 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.452 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.452 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.453 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.453 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.453 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.453 INFO:teuthology.orchestra.run.smithi001.stdout: "mark_out_threshold": { 2024-05-04T03:25:29.453 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mark_out_threshold", 2024-05-04T03:25:29.453 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:29.453 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.453 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.453 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2419200", 2024-05-04T03:25:29.453 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.453 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.453 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.453 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-04T03:25:29.454 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.454 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.454 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.454 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.454 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": { 2024-05-04T03:25:29.454 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_name", 2024-05-04T03:25:29.454 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.454 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.454 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.454 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "device_health_metrics", 2024-05-04T03:25:29.454 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.454 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.454 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-04T03:25:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: "retention_period": { 2024-05-04T03:25:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retention_period", 2024-05-04T03:25:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15552000", 2024-05-04T03:25:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.455 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain device health metrics", 2024-05-04T03:25:29.456 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.456 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.456 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.456 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_frequency": { 2024-05-04T03:25:29.456 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_frequency", 2024-05-04T03:25:29.456 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:29.456 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.456 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.456 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-05-04T03:25:29.456 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.456 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.456 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.456 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-04T03:25:29.457 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.457 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.457 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.457 INFO:teuthology.orchestra.run.smithi001.stdout: "self_heal": { 2024-05-04T03:25:29.457 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "self_heal", 2024-05-04T03:25:29.457 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.457 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.457 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.457 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.457 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.457 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.457 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.457 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-04T03:25:29.457 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.457 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.458 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.458 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-05-04T03:25:29.458 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-05-04T03:25:29.458 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:29.458 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.458 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.458 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-05-04T03:25:29.458 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.458 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.458 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.458 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and check device health", 2024-05-04T03:25:29.458 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.458 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.459 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.459 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.459 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.459 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.459 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.459 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.459 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.459 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.459 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.459 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.459 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.459 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.459 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.459 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.460 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_threshold": { 2024-05-04T03:25:29.460 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_threshold", 2024-05-04T03:25:29.460 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:29.460 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.460 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.460 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7257600", 2024-05-04T03:25:29.460 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.460 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.460 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.460 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-04T03:25:29.460 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.460 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.460 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.460 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.460 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.461 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.461 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "diskprediction_local", 2024-05-04T03:25:29.461 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.461 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.461 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.461 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.461 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.461 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.461 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.461 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.461 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.461 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.461 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.461 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.462 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.462 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.462 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.462 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.462 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.462 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.462 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.462 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.462 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.462 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.462 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.462 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.462 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.462 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.462 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.463 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.463 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.463 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.463 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.463 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.463 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.463 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.463 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.463 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.463 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.463 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.463 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.463 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.464 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.464 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.464 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.464 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.464 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.464 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.464 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.464 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.464 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.464 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.464 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.464 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.464 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.464 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.464 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.465 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.465 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.465 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.465 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.465 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.465 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.465 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.465 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.465 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.465 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.465 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.465 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.465 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.466 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.466 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.466 INFO:teuthology.orchestra.run.smithi001.stdout: "predict_interval": { 2024-05-04T03:25:29.466 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predict_interval", 2024-05-04T03:25:29.466 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.466 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.466 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.466 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-05-04T03:25:29.466 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.466 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.466 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.466 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.467 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.467 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.467 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.467 INFO:teuthology.orchestra.run.smithi001.stdout: "predictor_model": { 2024-05-04T03:25:29.467 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predictor_model", 2024-05-04T03:25:29.467 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.467 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.467 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.467 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "prophetstor", 2024-05-04T03:25:29.467 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.467 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.467 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.467 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.467 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.468 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.468 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.468 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-05-04T03:25:29.468 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-05-04T03:25:29.468 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.468 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.468 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.468 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-05-04T03:25:29.468 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.468 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.468 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.468 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.468 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.468 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.469 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.469 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.469 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.469 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.469 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.469 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.469 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.469 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.469 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.469 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.469 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.469 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.469 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.469 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.469 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.470 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.470 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.470 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.470 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "influx", 2024-05-04T03:25:29.470 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": false, 2024-05-04T03:25:29.470 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "influxdb python module not found", 2024-05-04T03:25:29.470 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.470 INFO:teuthology.orchestra.run.smithi001.stdout: "batch_size": { 2024-05-04T03:25:29.470 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "batch_size", 2024-05-04T03:25:29.470 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.470 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.470 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.470 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5000", 2024-05-04T03:25:29.470 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.471 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.471 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.471 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-04T03:25:29.471 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.471 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.471 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.471 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.471 INFO:teuthology.orchestra.run.smithi001.stdout: "database": { 2024-05-04T03:25:29.471 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "database", 2024-05-04T03:25:29.471 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.471 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.471 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.471 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "ceph", 2024-05-04T03:25:29.471 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.471 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.472 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.472 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-04T03:25:29.472 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.472 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.472 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.472 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": { 2024-05-04T03:25:29.472 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hostname", 2024-05-04T03:25:29.472 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.472 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.472 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.472 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.472 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.472 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.473 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.473 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server hostname", 2024-05-04T03:25:29.473 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.473 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.473 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.473 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-05-04T03:25:29.473 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-05-04T03:25:29.473 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:29.473 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.473 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.473 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "30", 2024-05-04T03:25:29.473 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "5", 2024-05-04T03:25:29.473 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.474 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.474 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-04T03:25:29.474 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.474 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.474 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.474 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.474 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.474 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.474 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.474 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.474 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.474 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.474 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.474 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.475 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.475 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.475 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.475 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.475 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.475 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.475 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.475 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.475 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.475 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.475 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.475 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.475 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.476 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.476 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.476 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.476 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.476 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.476 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.476 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.476 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.476 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.476 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.476 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.476 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.476 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.476 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.476 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.477 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.477 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.477 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.477 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.477 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.477 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.477 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.477 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.477 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.477 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.477 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.477 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.477 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.477 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.477 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.478 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.478 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.478 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.478 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.478 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.478 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.478 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.478 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.478 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.478 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.478 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.478 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.478 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.478 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.479 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.479 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.479 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.479 INFO:teuthology.orchestra.run.smithi001.stdout: "password": { 2024-05-04T03:25:29.479 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "password", 2024-05-04T03:25:29.479 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.479 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.479 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.479 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.479 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.479 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.479 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.479 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "password of InfluxDB server user", 2024-05-04T03:25:29.479 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.480 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.480 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.480 INFO:teuthology.orchestra.run.smithi001.stdout: "port": { 2024-05-04T03:25:29.480 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "port", 2024-05-04T03:25:29.480 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.480 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.480 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.480 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8086", 2024-05-04T03:25:29.480 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.480 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.480 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.480 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server port", 2024-05-04T03:25:29.480 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.480 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.482 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.482 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-05-04T03:25:29.482 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-05-04T03:25:29.482 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.482 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.482 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.482 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "false", 2024-05-04T03:25:29.482 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.482 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.482 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.482 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-04T03:25:29.482 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.482 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.483 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.483 INFO:teuthology.orchestra.run.smithi001.stdout: "threads": { 2024-05-04T03:25:29.483 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threads", 2024-05-04T03:25:29.483 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.483 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.483 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.483 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-05-04T03:25:29.483 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-05-04T03:25:29.483 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "32", 2024-05-04T03:25:29.483 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.483 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-04T03:25:29.483 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.483 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.483 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.484 INFO:teuthology.orchestra.run.smithi001.stdout: "username": { 2024-05-04T03:25:29.484 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "username", 2024-05-04T03:25:29.484 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.484 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.484 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.484 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.484 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.484 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.484 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.484 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "username of InfluxDB server user", 2024-05-04T03:25:29.484 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.484 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.484 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.485 INFO:teuthology.orchestra.run.smithi001.stdout: "verify_ssl": { 2024-05-04T03:25:29.485 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "verify_ssl", 2024-05-04T03:25:29.485 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.485 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.485 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.485 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "true", 2024-05-04T03:25:29.485 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.485 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.485 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.485 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-04T03:25:29.485 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.485 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.485 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.485 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.486 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.486 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.486 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.486 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "insights", 2024-05-04T03:25:29.486 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.486 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.486 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.486 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.486 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.486 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.486 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.486 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.486 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.487 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.487 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.487 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.487 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.487 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.487 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.487 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.487 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.487 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.487 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.487 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.487 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.487 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.487 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.488 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.488 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.488 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.488 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.488 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.488 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.488 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.488 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.488 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.488 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.488 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.488 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.488 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.489 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.489 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.489 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.489 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.489 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.489 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.489 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.489 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.489 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.489 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.489 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.489 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.489 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.489 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.490 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.490 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.490 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.490 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.490 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.490 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.490 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.490 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.490 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.490 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.490 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.490 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.491 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.492 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.492 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.492 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.492 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.492 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.492 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.492 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.492 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.492 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.492 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.492 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "iostat", 2024-05-04T03:25:29.492 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.492 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.492 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.493 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.494 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.495 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.495 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.495 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.495 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.495 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.495 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.495 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.495 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.495 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.495 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.495 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.495 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.495 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.495 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.496 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.496 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.496 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.496 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.496 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.496 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.496 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.496 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.496 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.496 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.496 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.496 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.496 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.497 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.497 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.497 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.497 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.497 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.497 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.497 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.497 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.497 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.497 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.497 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.497 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.497 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.498 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.498 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.498 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.498 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.498 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.498 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.498 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.498 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.498 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.498 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.498 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.498 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.498 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "k8sevents", 2024-05-04T03:25:29.499 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.499 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.499 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.499 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_event_retention_days": { 2024-05-04T03:25:29.499 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ceph_event_retention_days", 2024-05-04T03:25:29.499 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.499 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.499 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.499 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7", 2024-05-04T03:25:29.499 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.499 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.499 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.499 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-04T03:25:29.499 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.500 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.500 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.500 INFO:teuthology.orchestra.run.smithi001.stdout: "config_check_secs": { 2024-05-04T03:25:29.500 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_check_secs", 2024-05-04T03:25:29.500 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.500 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.500 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.500 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-05-04T03:25:29.500 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "10", 2024-05-04T03:25:29.500 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.500 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.500 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-04T03:25:29.500 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.500 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.500 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.501 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.501 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.501 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.501 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.501 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.501 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.501 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.501 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.501 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.501 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.501 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.501 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.501 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.502 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.502 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.502 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.502 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.502 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.502 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.502 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.502 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.502 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.502 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.502 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.502 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.502 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.503 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.503 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.503 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.503 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.503 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.503 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.503 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.503 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.503 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.503 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.503 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.503 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.503 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.503 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.504 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.504 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.504 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.504 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.504 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.504 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.504 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.504 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.504 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.504 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.504 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.504 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.504 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.504 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.504 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.505 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.505 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.505 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.505 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.505 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.505 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.505 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.505 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.505 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.505 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.505 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.505 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.505 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.505 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.505 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.505 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.506 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.506 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.506 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.506 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.506 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.506 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.506 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.506 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.506 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.506 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.506 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.506 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.506 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "localpool", 2024-05-04T03:25:29.507 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.507 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.507 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.507 INFO:teuthology.orchestra.run.smithi001.stdout: "failure_domain": { 2024-05-04T03:25:29.507 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "failure_domain", 2024-05-04T03:25:29.507 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.507 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.507 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.507 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "host", 2024-05-04T03:25:29.507 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.507 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.507 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.507 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "failure domain for any created local pool", 2024-05-04T03:25:29.507 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-04T03:25:29.507 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.507 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.508 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.508 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.508 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.508 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.508 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.508 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.508 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.508 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.508 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.508 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.508 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.508 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.508 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.508 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.508 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.509 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.509 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.509 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.509 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.509 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.509 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.509 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.509 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.509 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.509 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.509 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.509 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.509 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.509 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.510 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.510 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.510 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.510 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.510 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.510 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.510 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.510 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.510 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.510 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.510 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.510 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.510 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.510 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.511 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.511 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.511 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.511 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.511 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.511 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.511 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.511 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.511 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.511 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.511 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.511 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.511 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.511 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.511 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.512 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.512 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.512 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.512 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.512 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.512 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.512 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.512 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.512 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.512 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": { 2024-05-04T03:25:29.512 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_size", 2024-05-04T03:25:29.512 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.512 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.513 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.513 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.513 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.513 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.513 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.513 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default min_size for any created local pool", 2024-05-04T03:25:29.513 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-04T03:25:29.513 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.513 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.513 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.513 INFO:teuthology.orchestra.run.smithi001.stdout: "num_rep": { 2024-05-04T03:25:29.513 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "num_rep", 2024-05-04T03:25:29.513 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.513 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.513 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.514 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3", 2024-05-04T03:25:29.514 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.514 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.514 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.514 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default replica count for any created local pool", 2024-05-04T03:25:29.514 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.514 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.514 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.514 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": { 2024-05-04T03:25:29.514 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_num", 2024-05-04T03:25:29.514 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.514 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.514 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.515 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "128", 2024-05-04T03:25:29.515 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.515 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.515 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.515 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default pg_num for any created local pool", 2024-05-04T03:25:29.515 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.515 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.515 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.515 INFO:teuthology.orchestra.run.smithi001.stdout: "prefix": { 2024-05-04T03:25:29.515 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prefix", 2024-05-04T03:25:29.515 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.515 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.515 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.516 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.516 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.516 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.516 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.516 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name prefix for any created local pool", 2024-05-04T03:25:29.516 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.516 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.516 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.516 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.516 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.516 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.516 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.516 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.517 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.517 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.517 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.517 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.517 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.517 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.517 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.517 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.517 INFO:teuthology.orchestra.run.smithi001.stdout: "subtree": { 2024-05-04T03:25:29.517 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "subtree", 2024-05-04T03:25:29.517 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.517 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.517 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.517 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "rack", 2024-05-04T03:25:29.517 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.518 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.518 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.518 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-04T03:25:29.518 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-04T03:25:29.518 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.518 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.518 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.518 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.518 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.518 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mds_autoscaler", 2024-05-04T03:25:29.518 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.518 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.518 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.519 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.519 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.519 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.519 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.519 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.519 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.519 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.519 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.519 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.519 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.519 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.519 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.519 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.519 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.520 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.520 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.520 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.520 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.520 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.520 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.520 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.520 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.520 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.520 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.520 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.520 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.520 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.521 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.521 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.521 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.521 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.521 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.521 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.521 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.521 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.521 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.521 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.521 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.521 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.521 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.522 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.522 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.522 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.522 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.522 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.522 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.522 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.522 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.522 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.522 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.522 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.522 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.522 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.522 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.523 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.523 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.523 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.523 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.523 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.523 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.523 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.523 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.523 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.523 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.523 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.523 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.523 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.523 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.524 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.524 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.524 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.524 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.524 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.524 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.524 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.524 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.524 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.524 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.524 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.524 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.524 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.524 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirroring", 2024-05-04T03:25:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.525 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.526 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.526 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.526 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.526 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.526 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.526 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.526 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.526 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.526 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.526 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.526 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.526 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.526 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.527 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.527 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.527 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.527 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.527 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.527 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.527 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.527 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.527 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.527 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.527 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.527 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.527 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.527 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.527 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.528 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.528 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.528 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.528 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.528 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.528 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.528 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.528 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.528 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.528 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.528 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.528 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.528 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.528 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.528 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.529 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.529 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.529 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.529 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.529 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.529 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.529 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.529 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.529 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.529 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.529 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.529 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.529 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.529 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.530 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.530 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.530 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.530 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.530 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.530 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.530 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.530 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.530 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.530 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.530 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.530 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.530 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.530 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.531 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.531 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "nfs", 2024-05-04T03:25:29.531 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.531 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.531 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.531 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.531 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.531 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.531 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.531 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.531 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.531 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.531 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.532 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.532 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.532 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.532 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.532 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.532 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.532 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.532 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.532 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.532 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.532 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.532 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.532 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.532 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.533 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.533 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.533 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.533 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.533 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.533 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.533 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.533 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.533 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.533 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.533 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.533 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.533 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.533 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.534 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.534 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.534 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.534 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.534 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.534 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.534 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.534 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.534 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.534 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.534 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.534 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.534 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.534 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.535 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.536 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.536 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.536 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.536 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.536 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.536 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.536 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.536 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.536 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.536 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.536 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.536 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.536 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.537 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.537 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.537 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.537 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.537 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.537 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-05-04T03:25:29.537 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.537 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.537 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.537 INFO:teuthology.orchestra.run.smithi001.stdout: "fail_fs": { 2024-05-04T03:25:29.537 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "fail_fs", 2024-05-04T03:25:29.537 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.537 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.537 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.537 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.538 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.538 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.538 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.538 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-04T03:25:29.538 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.538 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.538 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.538 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.538 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.538 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.538 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.538 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.538 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.539 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.539 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.539 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.539 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.539 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.539 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.539 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.539 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.539 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.539 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.539 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.539 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.539 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.539 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.539 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.540 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.540 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.540 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.540 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.540 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.540 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.540 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.540 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.540 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.540 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.540 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.540 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.540 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.540 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.541 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.541 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.541 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.541 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.541 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.541 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.541 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.541 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.541 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.541 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.541 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.541 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.541 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.541 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.541 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.541 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.542 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.542 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.542 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.542 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.542 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.542 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.542 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.542 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.542 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.542 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.542 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.542 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.542 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.542 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.543 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.543 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.543 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator": { 2024-05-04T03:25:29.543 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-05-04T03:25:29.543 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.543 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.543 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.543 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.543 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.543 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.543 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.543 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm", 2024-05-04T03:25:29.543 INFO:teuthology.orchestra.run.smithi001.stdout: "rook", 2024-05-04T03:25:29.543 INFO:teuthology.orchestra.run.smithi001.stdout: "test_orchestrator" 2024-05-04T03:25:29.543 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.544 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Orchestrator backend", 2024-05-04T03:25:29.544 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.544 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.544 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.544 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.544 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.544 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.544 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.544 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.544 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.544 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.544 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.544 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.544 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.544 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.544 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.545 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.545 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.545 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.545 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.545 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_perf_query", 2024-05-04T03:25:29.545 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.545 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.545 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.545 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.545 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.545 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.545 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.545 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.545 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.546 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.546 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.546 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.546 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.546 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.546 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.546 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.546 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.546 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.546 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.546 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.546 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.546 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.546 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.547 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.547 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.547 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.547 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.547 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.547 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.547 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.547 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.547 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.547 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.547 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.547 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.547 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.547 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.548 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.548 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.548 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.548 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.548 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.548 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.548 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.548 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.548 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.548 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.548 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.548 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.548 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.548 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.548 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.549 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.549 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.549 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.549 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.549 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.549 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.549 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.549 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.549 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.549 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.549 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.549 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.549 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.549 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.550 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.550 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.550 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.550 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.550 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.550 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.550 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.550 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.550 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.550 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.550 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.550 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.550 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.550 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.551 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.551 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.551 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.551 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.551 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.551 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.551 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.551 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_support", 2024-05-04T03:25:29.551 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.551 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.551 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.551 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.551 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.551 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.552 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.552 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.552 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.552 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.552 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.552 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.552 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.552 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.552 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.552 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.552 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.552 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.552 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.552 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.553 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.553 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.553 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.553 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.553 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.553 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.553 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.553 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.553 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.553 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.553 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.553 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.553 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.553 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.554 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.554 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.554 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.554 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.554 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.554 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.554 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.554 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.554 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.554 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.554 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.554 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.554 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.554 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.554 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.555 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.555 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.555 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.555 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.555 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.555 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.555 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.555 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.555 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.555 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.555 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.555 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.555 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.555 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.555 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.556 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.556 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.556 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.556 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.556 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.556 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.556 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.556 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.556 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.556 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.556 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.556 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.556 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.556 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.557 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.557 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.557 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.557 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.557 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.557 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.557 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.557 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.557 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.557 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_autoscaler", 2024-05-04T03:25:29.557 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.557 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.557 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.557 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.557 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.557 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.558 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.558 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.558 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.558 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.558 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.558 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.558 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.558 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.558 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.558 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.558 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.558 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.558 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.558 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.558 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.558 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.559 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.559 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.559 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.559 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.559 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.559 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.559 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.559 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.559 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.559 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.559 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.559 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.559 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.559 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.560 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.560 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.560 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.560 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.560 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.560 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.560 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.560 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.560 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.560 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.560 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.560 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.560 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.560 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.560 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.560 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.561 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.561 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.561 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.561 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.561 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.561 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.561 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.561 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.561 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.561 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.561 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.561 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.561 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.561 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.561 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.562 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.562 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.562 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-05-04T03:25:29.562 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-05-04T03:25:29.562 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:29.562 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.562 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.562 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-05-04T03:25:29.562 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.562 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.562 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.562 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.562 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.562 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.563 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.563 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.563 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.563 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.563 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.563 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.563 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.563 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.563 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.563 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.563 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.563 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.563 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.563 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.564 INFO:teuthology.orchestra.run.smithi001.stdout: "threshold": { 2024-05-04T03:25:29.564 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threshold", 2024-05-04T03:25:29.564 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-05-04T03:25:29.564 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.564 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.564 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-05-04T03:25:29.564 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1.0", 2024-05-04T03:25:29.564 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.564 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.564 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "scaling threshold", 2024-05-04T03:25:29.564 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-04T03:25:29.564 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.564 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.564 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.564 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.564 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.564 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.565 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "progress", 2024-05-04T03:25:29.565 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.565 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.565 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.565 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_pg_recovery_event": { 2024-05-04T03:25:29.565 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_pg_recovery_event", 2024-05-04T03:25:29.565 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.565 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.565 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.565 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.565 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.565 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.565 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.565 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-04T03:25:29.565 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.565 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.566 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.566 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.566 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-05-04T03:25:29.566 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-05-04T03:25:29.566 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.566 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.566 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.566 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.566 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.566 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.566 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.566 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.566 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.567 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.567 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.567 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.567 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.567 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.567 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.567 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.567 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.567 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.567 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.567 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.567 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.567 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.567 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.568 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.568 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.568 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.568 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.568 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.568 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.568 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.568 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.568 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.568 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.568 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.568 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.568 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.568 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.568 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.569 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.569 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.569 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.569 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.569 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.569 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.569 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.569 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.569 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.569 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.569 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.569 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.569 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.569 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.569 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.569 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.570 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.570 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.570 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.570 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.570 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.570 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.570 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.570 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.570 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.570 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.570 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.570 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.570 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.570 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.571 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.571 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.571 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.571 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.571 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.571 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.571 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.571 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.571 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.571 INFO:teuthology.orchestra.run.smithi001.stdout: "max_completed_events": { 2024-05-04T03:25:29.571 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_completed_events", 2024-05-04T03:25:29.571 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.571 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.571 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.571 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "50", 2024-05-04T03:25:29.571 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.572 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.572 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.572 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "number of past completed events to remember", 2024-05-04T03:25:29.572 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.572 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.572 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.572 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-05-04T03:25:29.572 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-05-04T03:25:29.572 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:29.572 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.572 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.572 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-05-04T03:25:29.572 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.572 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.572 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.573 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long the module is going to sleep", 2024-05-04T03:25:29.573 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.573 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.573 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.573 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.573 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.573 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.573 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.573 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.573 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.573 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.573 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.573 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.574 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.574 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.574 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.574 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.574 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.574 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.574 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.574 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus", 2024-05-04T03:25:29.574 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.574 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.574 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.574 INFO:teuthology.orchestra.run.smithi001.stdout: "cache": { 2024-05-04T03:25:29.574 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cache", 2024-05-04T03:25:29.574 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.574 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.575 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.575 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.575 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.575 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.575 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.575 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.575 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.575 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.575 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.575 INFO:teuthology.orchestra.run.smithi001.stdout: "exclude_perf_counters": { 2024-05-04T03:25:29.575 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "exclude_perf_counters", 2024-05-04T03:25:29.575 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.575 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.575 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.575 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.575 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.576 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.576 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.576 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-04T03:25:29.576 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-04T03:25:29.576 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.576 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.576 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.576 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.576 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.576 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.576 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.576 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.576 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.576 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.577 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.577 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.577 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.577 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.577 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.577 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.577 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.577 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.577 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.577 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.577 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.577 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.577 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.577 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.577 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.578 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.578 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.578 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.578 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.578 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.578 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.578 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.578 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.578 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.578 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.578 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.578 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.578 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.578 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.578 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.579 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.579 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.579 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.579 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.579 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.579 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.579 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.579 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.579 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.579 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.579 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.579 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.579 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.579 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.579 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.579 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.580 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.580 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.580 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.580 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.580 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.580 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.580 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.580 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.580 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.580 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.580 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.580 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.580 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.580 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.581 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools": { 2024-05-04T03:25:29.581 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools", 2024-05-04T03:25:29.581 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.581 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.581 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.581 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.581 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.581 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.581 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.581 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.581 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.581 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.581 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.581 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-04T03:25:29.581 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-04T03:25:29.582 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.582 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.582 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.582 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "300", 2024-05-04T03:25:29.582 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.582 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.582 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.582 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.582 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.582 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.582 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.582 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_interval": { 2024-05-04T03:25:29.582 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_interval", 2024-05-04T03:25:29.582 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-05-04T03:25:29.582 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.583 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.583 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15.0", 2024-05-04T03:25:29.583 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.583 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.583 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.583 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.583 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.583 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.583 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.583 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-05-04T03:25:29.583 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-05-04T03:25:29.583 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.583 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.583 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.583 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-05-04T03:25:29.584 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.584 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.584 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.584 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-04T03:25:29.584 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.584 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.584 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.584 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-05-04T03:25:29.584 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-05-04T03:25:29.584 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.584 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.584 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.584 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "9283", 2024-05-04T03:25:29.584 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.584 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.584 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.585 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-04T03:25:29.585 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.585 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.585 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.585 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.585 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.585 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.585 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.585 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.585 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.585 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.585 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.585 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.585 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.586 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.586 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.586 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.586 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.586 INFO:teuthology.orchestra.run.smithi001.stdout: "stale_cache_strategy": { 2024-05-04T03:25:29.586 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stale_cache_strategy", 2024-05-04T03:25:29.586 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.586 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.586 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.586 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "log", 2024-05-04T03:25:29.586 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.586 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.586 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.586 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.586 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.586 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.587 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.587 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-05-04T03:25:29.587 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-05-04T03:25:29.587 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.587 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.587 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.587 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "default", 2024-05-04T03:25:29.587 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.587 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.587 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.587 INFO:teuthology.orchestra.run.smithi001.stdout: "default", 2024-05-04T03:25:29.587 INFO:teuthology.orchestra.run.smithi001.stdout: "error" 2024-05-04T03:25:29.587 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.587 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.587 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.588 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.588 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.588 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-05-04T03:25:29.588 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-05-04T03:25:29.588 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.588 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.588 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.588 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-05-04T03:25:29.588 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-05-04T03:25:29.588 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-05-04T03:25:29.588 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.588 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.588 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.588 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.589 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.589 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.589 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.589 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.589 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_support", 2024-05-04T03:25:29.589 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.589 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.589 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.589 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.589 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.589 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.589 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.589 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.589 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.590 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.590 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.590 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.590 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.590 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.590 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.590 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.590 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.590 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.590 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.590 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.590 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.590 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.590 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.591 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.591 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.591 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.591 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.591 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.591 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.591 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.591 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.591 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.591 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.591 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.591 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.591 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.591 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.591 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.592 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.592 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.592 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.592 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.592 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.592 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.592 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.592 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.592 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.592 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.592 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.592 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.592 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.592 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.592 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.592 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.593 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.593 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.593 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.593 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.593 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.593 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.593 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.593 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.593 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.593 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.593 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.593 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.593 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.593 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.594 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.594 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_snap_create": { 2024-05-04T03:25:29.594 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_snap_create", 2024-05-04T03:25:29.594 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.594 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.594 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.594 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-05-04T03:25:29.594 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.594 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.594 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.594 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.594 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.594 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.594 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.595 INFO:teuthology.orchestra.run.smithi001.stdout: "mirror_snapshot_schedule": { 2024-05-04T03:25:29.595 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirror_snapshot_schedule", 2024-05-04T03:25:29.595 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.595 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.595 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.595 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.595 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.595 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.595 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.595 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.595 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.595 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.595 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.596 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.596 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.596 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.596 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.596 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.596 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.596 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.596 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.596 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.596 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.596 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.596 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.596 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.596 INFO:teuthology.orchestra.run.smithi001.stdout: "trash_purge_schedule": { 2024-05-04T03:25:29.597 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "trash_purge_schedule", 2024-05-04T03:25:29.597 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.597 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.597 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.597 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.597 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.597 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.597 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.597 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.597 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.597 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.597 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.597 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.597 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.598 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.598 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "restful", 2024-05-04T03:25:29.598 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.598 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.598 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.598 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_auth": { 2024-05-04T03:25:29.598 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_auth", 2024-05-04T03:25:29.598 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.598 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.598 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.598 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.598 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.598 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.598 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.599 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.599 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.599 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.599 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.599 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-05-04T03:25:29.599 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-05-04T03:25:29.599 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.599 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.599 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.599 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.599 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.599 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.599 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.599 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.599 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.600 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.600 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.600 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.600 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.600 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.600 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.600 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.600 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.600 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.600 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.600 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.600 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.600 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.600 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.600 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.601 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.601 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.601 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.601 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.601 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.601 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.601 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.601 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.601 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.601 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.601 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.601 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.601 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.601 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.601 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.601 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.602 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.602 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.602 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.602 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.602 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.602 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.602 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.602 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.602 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.602 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.602 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.602 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.602 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.602 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.603 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.603 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.603 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.603 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.603 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.603 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.603 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.603 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.603 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.603 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.603 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.603 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.603 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.603 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.603 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.604 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.604 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.604 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.604 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.604 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.604 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.604 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.604 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.604 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-05-04T03:25:29.604 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-05-04T03:25:29.604 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.604 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.604 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.604 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.604 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.604 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.605 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.605 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.605 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.605 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.605 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.605 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-05-04T03:25:29.605 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-05-04T03:25:29.605 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.605 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.605 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.605 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.605 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.605 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.605 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.605 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.606 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.606 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.606 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.606 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.606 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.606 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.606 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.606 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.606 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.606 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.606 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.606 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.606 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.606 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.607 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.607 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.607 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.607 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.607 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.607 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rgw", 2024-05-04T03:25:29.607 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.607 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.607 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.607 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.607 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.607 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.607 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.607 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.607 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.608 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.608 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.608 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.608 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.608 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.608 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.608 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.608 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.608 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.608 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.608 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.608 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.608 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.608 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.608 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.609 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.609 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.609 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.609 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.609 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.609 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.609 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.609 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.609 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.609 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.609 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.609 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.609 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.609 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.610 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.610 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.610 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.610 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.610 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.610 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.610 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.610 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.610 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.610 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.610 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.610 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.610 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.610 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.610 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.610 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.611 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.611 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.611 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.611 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.611 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.611 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.611 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.611 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.611 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.611 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.611 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.611 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.611 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.611 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.611 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.612 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.612 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.612 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.612 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.612 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.612 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.612 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.612 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.612 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.612 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.612 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.612 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.612 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.612 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.612 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.613 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.613 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.613 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rook", 2024-05-04T03:25:29.613 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.613 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.613 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.613 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.613 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.613 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.613 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.613 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.613 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.613 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.613 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.613 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.613 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.614 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.614 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.614 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.614 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.614 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.614 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.614 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.614 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.614 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.614 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.614 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.614 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.614 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.614 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.614 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.615 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.615 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.615 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.615 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.615 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.615 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.615 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.615 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.615 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.615 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.615 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.615 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.615 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.615 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.616 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.616 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.616 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.616 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.616 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.616 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.616 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.616 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.616 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.616 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.616 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.616 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.616 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.616 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.617 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.617 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.617 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.617 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.617 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.617 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.617 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.617 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.617 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.617 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.617 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.617 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.617 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.618 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.618 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.618 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.618 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.618 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.618 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.618 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.618 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.618 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.618 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.618 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.618 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.618 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.618 INFO:teuthology.orchestra.run.smithi001.stdout: "storage_class": { 2024-05-04T03:25:29.619 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "storage_class", 2024-05-04T03:25:29.619 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.619 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.619 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.619 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "local", 2024-05-04T03:25:29.619 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.619 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.619 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.619 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-04T03:25:29.619 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.619 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.619 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.619 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.619 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.620 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.620 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "selftest", 2024-05-04T03:25:29.620 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.620 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.620 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.620 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.620 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.620 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.620 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.620 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.620 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.620 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.620 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.620 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.620 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.621 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.621 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.621 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.621 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.621 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.621 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.621 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.621 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.621 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.621 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.621 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.621 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.621 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.622 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.622 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.622 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.622 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.622 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.622 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.622 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.622 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.622 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.622 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.622 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.622 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.622 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.623 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.623 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.623 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.623 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.623 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.623 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.623 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.623 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.623 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.623 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.623 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.623 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.623 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.623 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.623 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.624 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.624 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.624 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.624 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.624 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.624 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.624 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.624 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.624 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.624 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.624 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.624 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.624 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.624 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.625 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.625 INFO:teuthology.orchestra.run.smithi001.stdout: "roption1": { 2024-05-04T03:25:29.625 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption1", 2024-05-04T03:25:29.625 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.625 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.625 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.625 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.625 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.625 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.625 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.625 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.625 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.625 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.625 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.626 INFO:teuthology.orchestra.run.smithi001.stdout: "roption2": { 2024-05-04T03:25:29.626 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption2", 2024-05-04T03:25:29.626 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.626 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.626 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.626 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "xyz", 2024-05-04T03:25:29.626 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.626 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.626 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.626 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.626 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.626 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.626 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.626 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption1": { 2024-05-04T03:25:29.627 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption1", 2024-05-04T03:25:29.627 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.627 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.627 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.627 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.627 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.627 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.627 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.627 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.627 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.627 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.627 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.627 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption2": { 2024-05-04T03:25:29.628 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption2", 2024-05-04T03:25:29.628 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.628 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.628 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.628 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.628 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.628 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.628 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.628 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.628 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.628 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.628 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.628 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.628 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption3": { 2024-05-04T03:25:29.628 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption3", 2024-05-04T03:25:29.629 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-05-04T03:25:29.629 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.629 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.629 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.629 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.629 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.629 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.629 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.629 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.629 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.629 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.629 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption4": { 2024-05-04T03:25:29.629 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption4", 2024-05-04T03:25:29.630 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.630 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.630 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.630 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.630 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.630 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.630 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.630 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.630 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.630 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.630 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.630 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.630 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption5": { 2024-05-04T03:25:29.630 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption5", 2024-05-04T03:25:29.631 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.631 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.631 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.631 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.631 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.631 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.631 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.631 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.631 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.631 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.631 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.631 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption6": { 2024-05-04T03:25:29.631 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption6", 2024-05-04T03:25:29.631 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.632 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.632 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.632 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.632 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.632 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.632 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.632 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.632 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.632 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.632 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.632 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.632 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption7": { 2024-05-04T03:25:29.632 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption7", 2024-05-04T03:25:29.632 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.632 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.632 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.633 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.633 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-05-04T03:25:29.633 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "42", 2024-05-04T03:25:29.633 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.633 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.633 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.633 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.633 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.633 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.633 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.633 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.633 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.633 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.634 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.634 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.634 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.634 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.634 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.634 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.634 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.634 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.634 INFO:teuthology.orchestra.run.smithi001.stdout: "testkey": { 2024-05-04T03:25:29.634 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testkey", 2024-05-04T03:25:29.634 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.634 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.634 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.635 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.635 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.635 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.635 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.635 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.635 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.635 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.635 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.635 INFO:teuthology.orchestra.run.smithi001.stdout: "testlkey": { 2024-05-04T03:25:29.636 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testlkey", 2024-05-04T03:25:29.636 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.636 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.636 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.636 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.636 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.636 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.636 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.637 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.637 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.637 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.637 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.637 INFO:teuthology.orchestra.run.smithi001.stdout: "testnewline": { 2024-05-04T03:25:29.637 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testnewline", 2024-05-04T03:25:29.638 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.638 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.638 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.638 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.638 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.638 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.638 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.638 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.639 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.639 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.639 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.639 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.639 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.639 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.639 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.639 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smb", 2024-05-04T03:25:29.640 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.640 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.640 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.640 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.640 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.640 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.640 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.640 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.641 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.641 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.641 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.641 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.641 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.641 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.641 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.642 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.642 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.642 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.642 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.642 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.642 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.643 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.643 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.643 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.643 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.643 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.643 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.643 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.643 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.643 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.644 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.644 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.644 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.644 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.644 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.644 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.644 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.644 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.644 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.644 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.644 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.644 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.644 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.644 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.645 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.645 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.645 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.645 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.645 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.645 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.645 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.645 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.645 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.645 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.645 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.645 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.645 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.645 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.646 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.646 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.646 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.646 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.646 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.646 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.646 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.646 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.646 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.646 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.646 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.646 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.646 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.647 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.647 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.647 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.647 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.647 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.647 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.647 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.647 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.647 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.647 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.647 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.647 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.647 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.647 INFO:teuthology.orchestra.run.smithi001.stdout: "update_orchestration": { 2024-05-04T03:25:29.648 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "update_orchestration", 2024-05-04T03:25:29.648 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.648 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.648 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.648 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.648 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.648 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.648 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.648 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-04T03:25:29.648 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.648 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.648 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.648 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.648 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.649 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.649 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2024-05-04T03:25:29.649 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.649 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.649 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.649 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_m_granularity": { 2024-05-04T03:25:29.649 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_m_granularity", 2024-05-04T03:25:29.649 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.649 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.649 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.649 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.649 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.649 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.650 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.650 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow minute scheduled snapshots", 2024-05-04T03:25:29.650 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.650 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.650 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.650 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.650 INFO:teuthology.orchestra.run.smithi001.stdout: "dump_on_update": { 2024-05-04T03:25:29.650 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dump_on_update", 2024-05-04T03:25:29.650 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.650 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.650 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.650 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.650 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.650 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.651 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.651 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "dump database to debug log on update", 2024-05-04T03:25:29.651 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.651 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.651 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.651 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.651 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.651 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.651 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.651 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.651 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.651 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.651 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.651 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.652 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.652 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.652 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.652 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.652 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.652 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.652 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.652 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.652 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.652 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.652 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.652 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.652 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.652 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.653 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.653 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.653 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.653 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.653 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.653 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.653 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.653 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.653 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.653 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.653 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.653 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.653 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.654 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.654 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.654 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.654 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.654 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.654 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.654 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.654 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.654 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.654 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.654 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.654 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.654 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.654 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.655 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.655 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.655 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.655 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.655 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.655 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.655 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.655 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.655 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.655 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.655 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.655 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.655 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.655 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.656 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.656 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.656 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.656 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.656 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.656 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.656 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.656 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.656 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.656 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.656 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.656 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.656 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.657 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.657 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.657 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.657 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.657 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.657 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.657 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stats", 2024-05-04T03:25:29.657 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.657 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.657 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.657 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.657 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.657 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.657 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.658 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.658 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.658 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.658 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.658 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.658 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.658 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.658 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.658 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.658 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.658 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.658 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.658 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.658 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.658 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.659 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.659 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.659 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.659 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.659 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.659 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.659 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.659 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.659 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.659 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.659 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.659 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.659 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.660 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.660 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.660 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.660 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.660 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.660 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.660 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.660 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.660 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.660 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.660 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.660 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.660 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.661 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.661 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.661 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.661 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.661 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.661 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.661 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.661 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.661 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.661 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.661 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.661 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.661 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.661 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.661 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.662 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.662 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.662 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.662 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.662 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.662 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.662 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.662 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.662 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.662 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.662 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.662 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.663 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.663 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.663 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.663 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.663 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.663 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.663 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.663 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.663 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.663 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.663 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.663 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.663 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "status", 2024-05-04T03:25:29.663 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.663 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.664 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.664 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.664 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.664 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.664 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.664 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.664 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.664 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.664 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.664 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.664 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.664 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.664 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.664 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.665 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.665 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.665 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.665 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.665 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.665 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.665 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.665 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.665 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.665 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.665 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.665 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.665 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.665 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.666 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.666 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.666 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.666 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.666 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.666 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.666 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.666 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.666 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.666 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.666 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.666 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.666 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.667 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.667 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.667 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.667 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.667 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.667 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.667 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.667 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.667 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.667 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.667 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.667 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.667 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.667 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.668 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.668 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.668 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.668 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.668 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.668 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.668 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.668 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.668 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.668 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.668 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.668 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.668 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.668 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.669 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.669 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.669 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.669 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.669 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.669 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.669 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.669 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.669 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.669 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.669 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.669 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.669 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.669 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.670 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.670 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.670 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telegraf", 2024-05-04T03:25:29.670 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.670 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.670 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.670 INFO:teuthology.orchestra.run.smithi001.stdout: "address": { 2024-05-04T03:25:29.670 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "address", 2024-05-04T03:25:29.670 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.670 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.670 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.670 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-04T03:25:29.670 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.670 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.671 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.671 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.671 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.671 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.671 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.671 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-05-04T03:25:29.671 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-05-04T03:25:29.671 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:29.671 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.671 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.671 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15", 2024-05-04T03:25:29.671 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.671 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.672 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.672 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.672 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.672 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.672 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.672 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.672 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.672 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.672 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.672 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.672 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.672 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.672 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.672 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.673 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.673 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.673 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.673 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.673 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.673 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.673 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.673 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.673 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.673 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.673 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.673 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.673 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.674 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.674 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.674 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.674 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.674 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.674 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.674 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.674 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.674 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.674 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.674 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.674 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.674 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.674 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.675 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.675 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.675 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.675 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.675 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.675 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.675 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.675 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.675 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.675 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.675 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.675 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.675 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.675 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.676 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.676 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.676 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.676 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.676 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.676 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.676 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.676 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.676 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.676 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.676 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.676 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.676 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.677 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.677 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.677 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.677 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.677 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.677 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.677 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.677 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.677 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.677 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.677 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.677 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.677 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.678 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.678 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.678 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.678 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.678 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.678 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.678 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.678 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telemetry", 2024-05-04T03:25:29.678 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.678 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.678 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.678 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_basic": { 2024-05-04T03:25:29.678 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_basic", 2024-05-04T03:25:29.679 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.679 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.679 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.679 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.679 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.679 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.679 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.679 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-04T03:25:29.679 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.679 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.679 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.679 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_crash": { 2024-05-04T03:25:29.679 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_crash", 2024-05-04T03:25:29.680 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.680 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.680 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.680 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.680 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.680 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.680 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.680 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-04T03:25:29.680 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.680 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.680 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.680 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.680 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_device": { 2024-05-04T03:25:29.680 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_device", 2024-05-04T03:25:29.681 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.681 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.681 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.681 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.681 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.681 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.681 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.681 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-04T03:25:29.681 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.681 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.681 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.681 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_ident": { 2024-05-04T03:25:29.681 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_ident", 2024-05-04T03:25:29.682 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.682 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.682 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.682 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.682 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.682 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.682 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.682 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-04T03:25:29.682 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.682 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.682 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.682 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_perf": { 2024-05-04T03:25:29.682 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_perf", 2024-05-04T03:25:29.683 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.683 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.683 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.683 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.683 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.683 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.683 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.683 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-04T03:25:29.683 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.683 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.683 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.683 INFO:teuthology.orchestra.run.smithi001.stdout: "contact": { 2024-05-04T03:25:29.683 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "contact", 2024-05-04T03:25:29.684 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.684 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.684 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.684 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.684 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.684 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.684 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.684 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.684 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.684 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.684 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.684 INFO:teuthology.orchestra.run.smithi001.stdout: "description": { 2024-05-04T03:25:29.684 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "description", 2024-05-04T03:25:29.685 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.685 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.685 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.685 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.685 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.685 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.685 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.685 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.685 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.685 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.685 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.685 INFO:teuthology.orchestra.run.smithi001.stdout: "device_url": { 2024-05-04T03:25:29.685 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_url", 2024-05-04T03:25:29.686 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.686 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.686 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.686 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-04T03:25:29.686 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.686 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.686 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.686 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.686 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.686 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.686 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.686 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-05-04T03:25:29.686 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-05-04T03:25:29.687 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.687 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.687 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.687 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.687 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.687 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.687 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.687 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.687 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.687 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.687 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.687 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.687 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-05-04T03:25:29.687 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-05-04T03:25:29.687 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.688 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.688 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.688 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "24", 2024-05-04T03:25:29.688 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "8", 2024-05-04T03:25:29.688 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.688 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.688 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.688 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.688 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.688 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.688 INFO:teuthology.orchestra.run.smithi001.stdout: "last_opt_revision": { 2024-05-04T03:25:29.688 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "last_opt_revision", 2024-05-04T03:25:29.688 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.688 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.689 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.689 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1", 2024-05-04T03:25:29.689 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.689 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.689 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.689 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.689 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.689 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.689 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.689 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard": { 2024-05-04T03:25:29.689 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard", 2024-05-04T03:25:29.689 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.689 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.690 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.690 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.690 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.690 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.690 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.690 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.690 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.690 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.690 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.690 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard_description": { 2024-05-04T03:25:29.690 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard_description", 2024-05-04T03:25:29.691 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.691 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.691 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.691 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.691 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.691 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.691 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.691 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.691 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.691 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.691 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.691 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.691 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.691 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.692 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.692 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.692 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.692 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.692 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.692 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.692 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.692 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.692 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.692 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.692 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.692 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.692 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.692 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.693 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.693 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.693 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.693 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.693 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.693 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.693 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.693 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.693 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.693 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.693 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.693 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.694 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.694 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.694 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.694 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.694 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.694 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.694 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.694 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.694 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.694 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.694 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.694 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.695 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.695 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.695 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.695 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.695 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.695 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.695 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.695 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.695 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.695 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.695 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.695 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.695 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.695 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.695 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.696 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.696 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.696 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.696 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.696 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.696 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.696 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.696 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.696 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.696 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.696 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.696 INFO:teuthology.orchestra.run.smithi001.stdout: "organization": { 2024-05-04T03:25:29.696 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "organization", 2024-05-04T03:25:29.697 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.697 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.697 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.697 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.697 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.697 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.697 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.697 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.697 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.697 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.697 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.697 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.697 INFO:teuthology.orchestra.run.smithi001.stdout: "proxy": { 2024-05-04T03:25:29.697 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "proxy", 2024-05-04T03:25:29.698 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.698 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.698 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.698 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.698 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.698 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.698 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.698 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.698 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.698 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.698 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.698 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.698 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.699 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.699 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.699 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.699 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.699 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.699 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.699 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.699 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.699 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.699 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.699 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.699 INFO:teuthology.orchestra.run.smithi001.stdout: "url": { 2024-05-04T03:25:29.699 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url", 2024-05-04T03:25:29.700 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.700 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.700 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.700 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-04T03:25:29.700 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.700 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.700 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.700 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.700 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.700 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.700 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.700 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.700 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.700 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.701 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "test_orchestrator", 2024-05-04T03:25:29.701 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.701 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.701 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.701 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.701 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.701 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.701 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.701 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.701 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.701 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.701 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.701 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.701 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.702 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.702 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.702 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.702 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.702 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.702 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.702 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.702 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.702 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.702 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.702 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.702 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.702 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.703 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.703 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.703 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.703 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.703 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.703 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.703 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.703 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.703 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.703 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.703 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.703 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.703 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.704 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.704 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.704 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.704 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.704 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.704 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.704 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.704 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.704 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.704 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.704 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.704 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.704 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.704 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.704 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.705 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.705 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.705 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.705 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.705 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.705 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.705 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.705 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.705 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.705 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.705 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.705 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.705 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.705 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.706 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.706 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.706 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.706 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.706 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.706 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.706 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.706 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.706 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.706 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.706 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.706 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.706 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.706 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.707 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.707 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.707 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.707 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.707 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "volumes", 2024-05-04T03:25:29.707 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.707 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.707 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.707 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.707 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.707 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.707 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.707 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.707 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.708 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.708 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.708 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.708 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.708 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.708 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.708 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.708 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.708 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.708 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.708 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.708 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.708 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.708 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.709 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.709 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.709 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.709 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.709 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.709 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.709 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.709 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.709 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.709 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.709 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.709 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.709 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.709 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.709 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.710 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.710 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.710 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.710 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.710 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.710 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.710 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.710 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.710 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.710 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.710 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.710 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.710 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.710 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.710 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.711 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.711 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.711 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.711 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.711 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.711 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.711 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.711 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.711 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.711 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.711 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.711 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.711 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.711 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.712 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.712 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.712 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_clones": { 2024-05-04T03:25:29.712 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_clones", 2024-05-04T03:25:29.712 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.712 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.712 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.712 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4", 2024-05-04T03:25:29.712 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.712 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.712 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.712 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-04T03:25:29.712 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.712 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.712 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.713 INFO:teuthology.orchestra.run.smithi001.stdout: "periodic_async_work": { 2024-05-04T03:25:29.713 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "periodic_async_work", 2024-05-04T03:25:29.713 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.713 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.713 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.713 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.713 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.713 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.713 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.713 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Periodically check for async work", 2024-05-04T03:25:29.713 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.713 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.713 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.713 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_delay": { 2024-05-04T03:25:29.714 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_delay", 2024-05-04T03:25:29.714 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.714 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.714 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.714 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.714 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.714 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.714 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.714 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-04T03:25:29.714 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.714 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.714 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.714 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_no_wait": { 2024-05-04T03:25:29.714 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_no_wait", 2024-05-04T03:25:29.715 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.715 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.715 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.715 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-05-04T03:25:29.715 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.715 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.715 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.715 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-04T03:25:29.715 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.715 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.715 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.715 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.715 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.715 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.716 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.716 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.716 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.716 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.716 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.716 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.716 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.716 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.716 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.716 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.716 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.716 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.716 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix", 2024-05-04T03:25:29.717 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-05-04T03:25:29.717 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-05-04T03:25:29.717 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-05-04T03:25:29.717 INFO:teuthology.orchestra.run.smithi001.stdout: "discovery_interval": { 2024-05-04T03:25:29.717 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "discovery_interval", 2024-05-04T03:25:29.717 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-05-04T03:25:29.717 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.717 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.717 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "100", 2024-05-04T03:25:29.717 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.717 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.717 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.717 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.717 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.718 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.718 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.718 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.718 INFO:teuthology.orchestra.run.smithi001.stdout: "identifier": { 2024-05-04T03:25:29.718 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "identifier", 2024-05-04T03:25:29.718 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.718 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.718 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.718 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.718 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.718 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.718 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.718 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.718 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.719 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.719 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.719 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-05-04T03:25:29.719 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-05-04T03:25:29.719 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-05-04T03:25:29.719 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.719 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.719 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-05-04T03:25:29.719 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.719 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.719 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.719 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.719 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.719 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.719 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.720 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-05-04T03:25:29.720 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-05-04T03:25:29.720 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.720 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.720 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.720 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.720 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.720 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.720 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.720 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.720 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.720 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.720 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.720 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.721 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.721 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.721 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.721 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.721 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.721 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.721 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.721 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-05-04T03:25:29.721 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-05-04T03:25:29.721 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.721 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.721 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.721 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.721 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.721 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.722 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.722 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.722 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.722 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.722 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.722 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-05-04T03:25:29.722 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-05-04T03:25:29.722 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.722 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.722 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.722 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-05-04T03:25:29.722 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.722 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.722 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-05-04T03:25:29.723 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-05-04T03:25:29.723 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-05-04T03:25:29.723 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-05-04T03:25:29.723 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-05-04T03:25:29.723 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-05-04T03:25:29.723 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-05-04T03:25:29.723 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.723 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.723 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.723 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.723 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.723 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-05-04T03:25:29.723 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-05-04T03:25:29.723 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-05-04T03:25:29.723 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.724 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.724 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-05-04T03:25:29.724 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.724 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.724 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.724 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.724 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.724 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.724 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.724 INFO:teuthology.orchestra.run.smithi001.stdout: "sqlite3_killpoint": { 2024-05-04T03:25:29.724 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sqlite3_killpoint", 2024-05-04T03:25:29.724 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.724 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "dev", 2024-05-04T03:25:29.725 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-05-04T03:25:29.725 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-05-04T03:25:29.725 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.725 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.725 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.725 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.725 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.725 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.725 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.725 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_host": { 2024-05-04T03:25:29.725 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_host", 2024-05-04T03:25:29.725 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.725 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.725 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.725 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-05-04T03:25:29.726 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.726 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.726 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.726 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.726 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.726 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.726 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.726 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_port": { 2024-05-04T03:25:29.726 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_port", 2024-05-04T03:25:29.726 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-05-04T03:25:29.726 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.726 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.726 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10051", 2024-05-04T03:25:29.726 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.727 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.727 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.727 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.727 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.727 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.727 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.727 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_sender": { 2024-05-04T03:25:29.727 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_sender", 2024-05-04T03:25:29.727 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-05-04T03:25:29.727 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-05-04T03:25:29.727 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-05-04T03:25:29.727 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-04T03:25:29.727 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-05-04T03:25:29.727 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-05-04T03:25:29.728 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-05-04T03:25:29.728 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-05-04T03:25:29.728 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-05-04T03:25:29.728 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-05-04T03:25:29.728 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-05-04T03:25:29.728 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.728 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.728 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.728 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.728 INFO:teuthology.orchestra.run.smithi001.stdout: "services": { 2024-05-04T03:25:29.728 INFO:teuthology.orchestra.run.smithi001.stdout: "dashboard": "https://172.21.15.1:8443/" 2024-05-04T03:25:29.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.728 INFO:teuthology.orchestra.run.smithi001.stdout: "always_on_modules": { 2024-05-04T03:25:29.728 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus": [ 2024-05-04T03:25:29.728 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-05-04T03:25:29.729 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-05-04T03:25:29.729 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-05-04T03:25:29.729 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-05-04T03:25:29.729 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-05-04T03:25:29.729 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-05-04T03:25:29.729 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-05-04T03:25:29.729 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-05-04T03:25:29.729 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-05-04T03:25:29.729 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-05-04T03:25:29.729 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.729 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific": [ 2024-05-04T03:25:29.729 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-05-04T03:25:29.729 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-05-04T03:25:29.729 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-05-04T03:25:29.729 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-05-04T03:25:29.729 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-05-04T03:25:29.730 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-05-04T03:25:29.730 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-05-04T03:25:29.730 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-05-04T03:25:29.730 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-05-04T03:25:29.730 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-05-04T03:25:29.730 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.730 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy": [ 2024-05-04T03:25:29.730 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-05-04T03:25:29.730 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-05-04T03:25:29.730 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-05-04T03:25:29.730 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-05-04T03:25:29.730 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-05-04T03:25:29.730 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-05-04T03:25:29.730 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-05-04T03:25:29.731 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-05-04T03:25:29.731 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-05-04T03:25:29.731 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-05-04T03:25:29.731 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.731 INFO:teuthology.orchestra.run.smithi001.stdout: "reef": [ 2024-05-04T03:25:29.731 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-05-04T03:25:29.731 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-05-04T03:25:29.731 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-05-04T03:25:29.731 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-05-04T03:25:29.731 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-05-04T03:25:29.731 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-05-04T03:25:29.731 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-05-04T03:25:29.731 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-05-04T03:25:29.731 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-05-04T03:25:29.731 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-05-04T03:25:29.732 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-05-04T03:25:29.732 INFO:teuthology.orchestra.run.smithi001.stdout: "squid": [ 2024-05-04T03:25:29.732 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-05-04T03:25:29.732 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-05-04T03:25:29.732 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-05-04T03:25:29.732 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-05-04T03:25:29.732 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-05-04T03:25:29.732 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-05-04T03:25:29.732 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-05-04T03:25:29.732 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-05-04T03:25:29.732 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-05-04T03:25:29.732 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-05-04T03:25:29.732 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-04T03:25:29.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.732 INFO:teuthology.orchestra.run.smithi001.stdout: "last_failure_osd_epoch": 3, 2024-05-04T03:25:29.732 INFO:teuthology.orchestra.run.smithi001.stdout: "active_clients": [ 2024-05-04T03:25:29.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.733 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "devicehealth", 2024-05-04T03:25:29.733 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-04T03:25:29.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.733 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-04T03:25:29.733 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-05-04T03:25:29.733 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2426286568 2024-05-04T03:25:29.733 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.733 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-04T03:25:29.733 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.733 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "libcephsqlite", 2024-05-04T03:25:29.733 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-04T03:25:29.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.733 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-04T03:25:29.734 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-05-04T03:25:29.734 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 82390108 2024-05-04T03:25:29.734 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.734 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-04T03:25:29.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.734 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_support", 2024-05-04T03:25:29.734 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-04T03:25:29.734 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.734 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-04T03:25:29.734 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-05-04T03:25:29.734 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1105801409 2024-05-04T03:25:29.734 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.734 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-04T03:25:29.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:29.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.735 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "volumes", 2024-05-04T03:25:29.735 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-05-04T03:25:29.735 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-05-04T03:25:29.735 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-05-04T03:25:29.735 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-05-04T03:25:29.735 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 629779919 2024-05-04T03:25:29.735 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.735 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-04T03:25:29.735 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:29.735 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-05-04T03:25:29.735 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-04T03:25:29.741 DEBUG:tasks.check_counter:Getting stats from a 2024-05-04T03:25:29.742 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-04T03:25:29.742 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:28 smithi001 ceph-mon[29731]: pgmap v580: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 804 KiB/s wr, 335 op/s 2024-05-04T03:25:29.742 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:28 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/4228828802' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-04T03:25:29.742 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:28 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/1593336909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-04T03:25:29.742 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:28 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/1996677302' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-04T03:25:30.004 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:25:30.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:30 smithi072 ceph-mon[33332]: pgmap v581: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 806 KiB/s wr, 303 op/s 2024-05-04T03:25:30.855 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:30 smithi001 ceph-mon[29731]: pgmap v581: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 806 KiB/s wr, 303 op/s 2024-05-04T03:25:30.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:30 smithi077 ceph-mon[33531]: pgmap v581: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 806 KiB/s wr, 303 op/s 2024-05-04T03:25:31.722 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-04T03:25:31.722 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-0": { 2024-05-04T03:25:31.722 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 4, 2024-05-04T03:25:31.722 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 4, 2024-05-04T03:25:31.722 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 2992, 2024-05-04T03:25:31.722 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 448, 2024-05-04T03:25:31.723 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 4, 2024-05-04T03:25:31.723 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 0, 2024-05-04T03:25:31.723 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.001844357, 2024-05-04T03:25:31.723 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.000042888, 2024-05-04T03:25:31.723 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.000703619, 2024-05-04T03:25:31.723 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-04T03:25:31.723 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:31.723 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 3, 2024-05-04T03:25:31.723 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000844294, 2024-05-04T03:25:31.723 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000281431 2024-05-04T03:25:31.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:31.723 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:31.723 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-05-04T03:25:31.723 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-05-04T03:25:31.723 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-05-04T03:25:31.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:31.724 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-04T03:25:31.724 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 448 2024-05-04T03:25:31.724 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:31.724 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-1": { 2024-05-04T03:25:31.724 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 1, 2024-05-04T03:25:31.724 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 124, 2024-05-04T03:25:31.724 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 160, 2024-05-04T03:25:31.724 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 100416, 2024-05-04T03:25:31.724 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 3, 2024-05-04T03:25:31.724 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 2, 2024-05-04T03:25:31.724 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.009911464, 2024-05-04T03:25:31.724 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.007245565, 2024-05-04T03:25:31.724 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.001470773, 2024-05-04T03:25:31.724 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-04T03:25:31.724 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:31.725 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 122, 2024-05-04T03:25:31.725 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.005215160, 2024-05-04T03:25:31.725 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000042747 2024-05-04T03:25:31.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:31.725 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:31.725 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-05-04T03:25:31.725 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-05-04T03:25:31.725 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-05-04T03:25:31.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:31.725 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-04T03:25:31.725 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 100416 2024-05-04T03:25:31.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:31.725 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-2": { 2024-05-04T03:25:31.725 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 169, 2024-05-04T03:25:31.725 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 164, 2024-05-04T03:25:31.726 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 798512, 2024-05-04T03:25:31.726 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 100048, 2024-05-04T03:25:31.726 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 2, 2024-05-04T03:25:31.726 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 1, 2024-05-04T03:25:31.726 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.027293672, 2024-05-04T03:25:31.726 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.009112106, 2024-05-04T03:25:31.726 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.019565647, 2024-05-04T03:25:31.726 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.003104292, 2024-05-04T03:25:31.726 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:31.726 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 163, 2024-05-04T03:25:31.726 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.006772087, 2024-05-04T03:25:31.726 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000041546 2024-05-04T03:25:31.726 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:31.726 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:31.727 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-05-04T03:25:31.727 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-05-04T03:25:31.727 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-05-04T03:25:31.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:31.727 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 798512, 2024-05-04T03:25:31.727 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 100048 2024-05-04T03:25:31.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:31.727 INFO:teuthology.orchestra.run.smithi001.stdout: "cct": { 2024-05-04T03:25:31.727 INFO:teuthology.orchestra.run.smithi001.stdout: "total_workers": 0, 2024-05-04T03:25:31.727 INFO:teuthology.orchestra.run.smithi001.stdout: "unhealthy_workers": 0 2024-05-04T03:25:31.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:31.727 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool": { 2024-05-04T03:25:31.727 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_bytes": 0, 2024-05-04T03:25:31.727 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_items": 0, 2024-05-04T03:25:31.727 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_bytes": 0, 2024-05-04T03:25:31.728 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_items": 0, 2024-05-04T03:25:31.728 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_bytes": 0, 2024-05-04T03:25:31.728 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_items": 0, 2024-05-04T03:25:31.728 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-04T03:25:31.728 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_items": 0, 2024-05-04T03:25:31.728 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-04T03:25:31.728 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_items": 0, 2024-05-04T03:25:31.728 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_bytes": 0, 2024-05-04T03:25:31.728 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_items": 0, 2024-05-04T03:25:31.728 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-04T03:25:31.728 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_items": 0, 2024-05-04T03:25:31.728 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_bytes": 0, 2024-05-04T03:25:31.728 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_items": 0, 2024-05-04T03:25:31.728 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_bytes": 0, 2024-05-04T03:25:31.728 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_items": 0, 2024-05-04T03:25:31.729 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-04T03:25:31.729 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_items": 0, 2024-05-04T03:25:31.729 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-04T03:25:31.729 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_items": 0, 2024-05-04T03:25:31.729 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_bytes": 0, 2024-05-04T03:25:31.729 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_items": 0, 2024-05-04T03:25:31.729 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_bytes": 0, 2024-05-04T03:25:31.729 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_items": 0, 2024-05-04T03:25:31.729 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-04T03:25:31.729 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_items": 0, 2024-05-04T03:25:31.729 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_bytes": 0, 2024-05-04T03:25:31.729 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_items": 0, 2024-05-04T03:25:31.729 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_bytes": 0, 2024-05-04T03:25:31.729 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_items": 0, 2024-05-04T03:25:31.729 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_bytes": 0, 2024-05-04T03:25:31.729 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_items": 0, 2024-05-04T03:25:31.730 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_bytes": 0, 2024-05-04T03:25:31.730 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_items": 0, 2024-05-04T03:25:31.730 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_bytes": 149007, 2024-05-04T03:25:31.730 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_items": 30, 2024-05-04T03:25:31.730 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_bytes": 0, 2024-05-04T03:25:31.730 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_items": 0, 2024-05-04T03:25:31.730 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bytes": 0, 2024-05-04T03:25:31.730 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_items": 0, 2024-05-04T03:25:31.730 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_bytes": 0, 2024-05-04T03:25:31.730 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_items": 0, 2024-05-04T03:25:31.730 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_bytes": 0, 2024-05-04T03:25:31.730 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_items": 0, 2024-05-04T03:25:31.730 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_bytes": 0, 2024-05-04T03:25:31.730 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_items": 0, 2024-05-04T03:25:31.730 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_bytes": 0, 2024-05-04T03:25:31.731 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_items": 0, 2024-05-04T03:25:31.731 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_bytes": 0, 2024-05-04T03:25:31.731 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_items": 0, 2024-05-04T03:25:31.731 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_bytes": 928, 2024-05-04T03:25:31.731 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_items": 2, 2024-05-04T03:25:31.731 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_bytes": 0, 2024-05-04T03:25:31.731 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_items": 0, 2024-05-04T03:25:31.731 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_bytes": 0, 2024-05-04T03:25:31.731 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_items": 0 2024-05-04T03:25:31.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:31.731 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-04T03:25:31.731 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-05-04T03:25:31.731 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-05-04T03:25:31.731 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-05-04T03:25:31.731 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 170, 2024-05-04T03:25:31.731 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 774642, 2024-05-04T03:25:31.732 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:31.732 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 170, 2024-05-04T03:25:31.732 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-05-04T03:25:31.732 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-05-04T03:25:31.732 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 170, 2024-05-04T03:25:31.732 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 774642, 2024-05-04T03:25:31.732 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-05-04T03:25:31.732 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-05-04T03:25:31.732 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-05-04T03:25:31.732 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-05-04T03:25:31.732 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:31.732 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:31.732 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-04T03:25:32.317 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-05-04T03:25:32.317 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-05-04T03:25:32.317 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-05-04T03:25:32.317 DEBUG:tasks.check_counter:Getting stats from d 2024-05-04T03:25:32.318 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-04T03:25:32.567 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:25:32.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:32 smithi001 ceph-mon[29731]: pgmap v582: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 782 KiB/s wr, 192 op/s 2024-05-04T03:25:32.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:32 smithi072 ceph-mon[33332]: pgmap v582: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 782 KiB/s wr, 192 op/s 2024-05-04T03:25:32.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:32 smithi077 ceph-mon[33531]: pgmap v582: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 772 KiB/s rd, 782 KiB/s wr, 192 op/s 2024-05-04T03:25:34.133 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-04T03:25:34.134 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-0": { 2024-05-04T03:25:34.134 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 4, 2024-05-04T03:25:34.134 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 4, 2024-05-04T03:25:34.134 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 2992, 2024-05-04T03:25:34.134 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 448, 2024-05-04T03:25:34.134 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 4, 2024-05-04T03:25:34.134 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 0, 2024-05-04T03:25:34.134 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.001772490, 2024-05-04T03:25:34.134 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.000038738, 2024-05-04T03:25:34.134 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.000580081, 2024-05-04T03:25:34.134 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-04T03:25:34.134 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:34.134 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 3, 2024-05-04T03:25:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000896086, 2024-05-04T03:25:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000298695 2024-05-04T03:25:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-05-04T03:25:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-05-04T03:25:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-05-04T03:25:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-04T03:25:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 448 2024-05-04T03:25:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-1": { 2024-05-04T03:25:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 174, 2024-05-04T03:25:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 169, 2024-05-04T03:25:34.135 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 799872, 2024-05-04T03:25:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 103248, 2024-05-04T03:25:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 2, 2024-05-04T03:25:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 1, 2024-05-04T03:25:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.029041028, 2024-05-04T03:25:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.009720008, 2024-05-04T03:25:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.021067021, 2024-05-04T03:25:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.003228860, 2024-05-04T03:25:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 168, 2024-05-04T03:25:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.007363304, 2024-05-04T03:25:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000043829 2024-05-04T03:25:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:34.136 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-05-04T03:25:34.137 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-05-04T03:25:34.137 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-05-04T03:25:34.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:34.137 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 799872, 2024-05-04T03:25:34.137 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 103248 2024-05-04T03:25:34.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:34.137 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-2": { 2024-05-04T03:25:34.137 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 1, 2024-05-04T03:25:34.137 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 128, 2024-05-04T03:25:34.137 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 160, 2024-05-04T03:25:34.137 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 101120, 2024-05-04T03:25:34.137 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 3, 2024-05-04T03:25:34.137 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 2, 2024-05-04T03:25:34.137 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.009974113, 2024-05-04T03:25:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.007223761, 2024-05-04T03:25:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.001464327, 2024-05-04T03:25:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-04T03:25:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 126, 2024-05-04T03:25:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.005763282, 2024-05-04T03:25:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000045740 2024-05-04T03:25:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-05-04T03:25:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-05-04T03:25:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-05-04T03:25:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-04T03:25:34.138 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 101120 2024-05-04T03:25:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: "cct": { 2024-05-04T03:25:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: "total_workers": 0, 2024-05-04T03:25:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: "unhealthy_workers": 0 2024-05-04T03:25:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool": { 2024-05-04T03:25:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_bytes": 0, 2024-05-04T03:25:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_items": 0, 2024-05-04T03:25:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_bytes": 0, 2024-05-04T03:25:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_items": 0, 2024-05-04T03:25:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_bytes": 0, 2024-05-04T03:25:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_items": 0, 2024-05-04T03:25:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-04T03:25:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_items": 0, 2024-05-04T03:25:34.139 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-04T03:25:34.140 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_items": 0, 2024-05-04T03:25:34.140 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_bytes": 0, 2024-05-04T03:25:34.140 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_items": 0, 2024-05-04T03:25:34.140 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-04T03:25:34.140 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_items": 0, 2024-05-04T03:25:34.140 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_bytes": 0, 2024-05-04T03:25:34.140 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_items": 0, 2024-05-04T03:25:34.140 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_bytes": 0, 2024-05-04T03:25:34.140 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_items": 0, 2024-05-04T03:25:34.140 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-04T03:25:34.140 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_items": 0, 2024-05-04T03:25:34.140 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-04T03:25:34.140 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_items": 0, 2024-05-04T03:25:34.140 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_bytes": 0, 2024-05-04T03:25:34.140 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_items": 0, 2024-05-04T03:25:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_bytes": 0, 2024-05-04T03:25:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_items": 0, 2024-05-04T03:25:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-04T03:25:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_items": 0, 2024-05-04T03:25:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_bytes": 0, 2024-05-04T03:25:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_items": 0, 2024-05-04T03:25:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_bytes": 0, 2024-05-04T03:25:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_items": 0, 2024-05-04T03:25:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_bytes": 0, 2024-05-04T03:25:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_items": 0, 2024-05-04T03:25:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_bytes": 0, 2024-05-04T03:25:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_items": 0, 2024-05-04T03:25:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_bytes": 148984, 2024-05-04T03:25:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_items": 30, 2024-05-04T03:25:34.141 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_bytes": 0, 2024-05-04T03:25:34.142 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_items": 0, 2024-05-04T03:25:34.142 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bytes": 0, 2024-05-04T03:25:34.142 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_items": 0, 2024-05-04T03:25:34.142 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_bytes": 0, 2024-05-04T03:25:34.142 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_items": 0, 2024-05-04T03:25:34.142 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_bytes": 0, 2024-05-04T03:25:34.142 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_items": 0, 2024-05-04T03:25:34.142 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_bytes": 0, 2024-05-04T03:25:34.142 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_items": 0, 2024-05-04T03:25:34.142 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_bytes": 0, 2024-05-04T03:25:34.142 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_items": 0, 2024-05-04T03:25:34.142 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_bytes": 0, 2024-05-04T03:25:34.142 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_items": 0, 2024-05-04T03:25:34.142 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_bytes": 928, 2024-05-04T03:25:34.143 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_items": 2, 2024-05-04T03:25:34.143 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_bytes": 0, 2024-05-04T03:25:34.143 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_items": 0, 2024-05-04T03:25:34.143 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_bytes": 0, 2024-05-04T03:25:34.143 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_items": 0 2024-05-04T03:25:34.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:34.143 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-04T03:25:34.143 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-05-04T03:25:34.143 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-05-04T03:25:34.143 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-05-04T03:25:34.143 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 175, 2024-05-04T03:25:34.143 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 775292, 2024-05-04T03:25:34.143 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:34.143 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 175, 2024-05-04T03:25:34.143 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-05-04T03:25:34.144 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-05-04T03:25:34.144 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 175, 2024-05-04T03:25:34.144 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 775292, 2024-05-04T03:25:34.144 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-05-04T03:25:34.144 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-05-04T03:25:34.144 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-05-04T03:25:34.144 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-05-04T03:25:34.144 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:34.144 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:34.144 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-04T03:25:34.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:34 smithi072 ceph-mon[33332]: pgmap v583: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 437 KiB/s wr, 124 op/s 2024-05-04T03:25:34.811 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:34 smithi001 ceph-mon[29731]: pgmap v583: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 437 KiB/s wr, 124 op/s 2024-05-04T03:25:34.843 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-04T03:25:34.844 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-04T03:25:34.844 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-05-04T03:25:34.844 DEBUG:tasks.check_counter:Getting stats from g 2024-05-04T03:25:34.844 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-04T03:25:34.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:34 smithi077 ceph-mon[33531]: pgmap v583: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 437 KiB/s wr, 124 op/s 2024-05-04T03:25:35.098 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:25:36.746 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:36 smithi001 ceph-mon[29731]: pgmap v584: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 64 op/s 2024-05-04T03:25:36.807 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-04T03:25:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-0": { 2024-05-04T03:25:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 4, 2024-05-04T03:25:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 4, 2024-05-04T03:25:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 2992, 2024-05-04T03:25:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 448, 2024-05-04T03:25:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 4, 2024-05-04T03:25:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 0, 2024-05-04T03:25:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.001714027, 2024-05-04T03:25:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.000031898, 2024-05-04T03:25:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.000612253, 2024-05-04T03:25:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-04T03:25:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 3, 2024-05-04T03:25:36.808 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000775554, 2024-05-04T03:25:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000258518 2024-05-04T03:25:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-05-04T03:25:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-05-04T03:25:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-05-04T03:25:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-04T03:25:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 448 2024-05-04T03:25:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-1": { 2024-05-04T03:25:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 1, 2024-05-04T03:25:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 125, 2024-05-04T03:25:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 160, 2024-05-04T03:25:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 100592, 2024-05-04T03:25:36.809 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 3, 2024-05-04T03:25:36.810 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 2, 2024-05-04T03:25:36.810 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.010061281, 2024-05-04T03:25:36.810 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.007432026, 2024-05-04T03:25:36.810 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.001229066, 2024-05-04T03:25:36.810 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-04T03:25:36.810 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:36.810 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 123, 2024-05-04T03:25:36.810 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.005896517, 2024-05-04T03:25:36.810 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000047939 2024-05-04T03:25:36.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:36.810 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:36.810 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-05-04T03:25:36.810 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-05-04T03:25:36.810 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-05-04T03:25:36.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:36.811 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-04T03:25:36.811 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 100592 2024-05-04T03:25:36.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:36.811 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-2": { 2024-05-04T03:25:36.811 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 171, 2024-05-04T03:25:36.811 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 166, 2024-05-04T03:25:36.811 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 799056, 2024-05-04T03:25:36.811 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 101328, 2024-05-04T03:25:36.811 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 2, 2024-05-04T03:25:36.811 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 1, 2024-05-04T03:25:36.811 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.029527825, 2024-05-04T03:25:36.811 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.009619367, 2024-05-04T03:25:36.811 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.021528201, 2024-05-04T03:25:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.003180437, 2024-05-04T03:25:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 165, 2024-05-04T03:25:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.007262869, 2024-05-04T03:25:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000044017 2024-05-04T03:25:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-05-04T03:25:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-05-04T03:25:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-05-04T03:25:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 799056, 2024-05-04T03:25:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 101328 2024-05-04T03:25:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: "cct": { 2024-05-04T03:25:36.812 INFO:teuthology.orchestra.run.smithi001.stdout: "total_workers": 0, 2024-05-04T03:25:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: "unhealthy_workers": 0 2024-05-04T03:25:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool": { 2024-05-04T03:25:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_bytes": 0, 2024-05-04T03:25:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_items": 0, 2024-05-04T03:25:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_bytes": 0, 2024-05-04T03:25:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_items": 0, 2024-05-04T03:25:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_bytes": 0, 2024-05-04T03:25:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_items": 0, 2024-05-04T03:25:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-04T03:25:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_items": 0, 2024-05-04T03:25:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-04T03:25:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_items": 0, 2024-05-04T03:25:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_bytes": 0, 2024-05-04T03:25:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_items": 0, 2024-05-04T03:25:36.813 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-04T03:25:36.814 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_items": 0, 2024-05-04T03:25:36.814 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_bytes": 0, 2024-05-04T03:25:36.814 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_items": 0, 2024-05-04T03:25:36.814 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_bytes": 0, 2024-05-04T03:25:36.814 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_items": 0, 2024-05-04T03:25:36.814 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-04T03:25:36.814 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_items": 0, 2024-05-04T03:25:36.814 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-04T03:25:36.814 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_items": 0, 2024-05-04T03:25:36.814 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_bytes": 0, 2024-05-04T03:25:36.814 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_items": 0, 2024-05-04T03:25:36.814 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_bytes": 0, 2024-05-04T03:25:36.814 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_items": 0, 2024-05-04T03:25:36.814 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-04T03:25:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_items": 0, 2024-05-04T03:25:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_bytes": 0, 2024-05-04T03:25:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_items": 0, 2024-05-04T03:25:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_bytes": 0, 2024-05-04T03:25:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_items": 0, 2024-05-04T03:25:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_bytes": 0, 2024-05-04T03:25:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_items": 0, 2024-05-04T03:25:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_bytes": 0, 2024-05-04T03:25:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_items": 0, 2024-05-04T03:25:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_bytes": 148967, 2024-05-04T03:25:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_items": 29, 2024-05-04T03:25:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_bytes": 0, 2024-05-04T03:25:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_items": 0, 2024-05-04T03:25:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bytes": 0, 2024-05-04T03:25:36.815 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_items": 0, 2024-05-04T03:25:36.816 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_bytes": 0, 2024-05-04T03:25:36.816 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_items": 0, 2024-05-04T03:25:36.816 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_bytes": 0, 2024-05-04T03:25:36.816 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_items": 0, 2024-05-04T03:25:36.816 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_bytes": 0, 2024-05-04T03:25:36.816 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_items": 0, 2024-05-04T03:25:36.816 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_bytes": 0, 2024-05-04T03:25:36.816 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_items": 0, 2024-05-04T03:25:36.816 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_bytes": 0, 2024-05-04T03:25:36.816 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_items": 0, 2024-05-04T03:25:36.816 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_bytes": 928, 2024-05-04T03:25:36.816 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_items": 2, 2024-05-04T03:25:36.816 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_bytes": 0, 2024-05-04T03:25:36.816 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_items": 0, 2024-05-04T03:25:36.816 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_bytes": 0, 2024-05-04T03:25:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_items": 0 2024-05-04T03:25:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-04T03:25:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-05-04T03:25:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-05-04T03:25:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-05-04T03:25:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 172, 2024-05-04T03:25:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 774902, 2024-05-04T03:25:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 172, 2024-05-04T03:25:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-05-04T03:25:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-05-04T03:25:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 172, 2024-05-04T03:25:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 774902, 2024-05-04T03:25:36.817 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-05-04T03:25:36.818 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-05-04T03:25:36.818 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-05-04T03:25:36.818 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-05-04T03:25:36.818 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:36.818 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:36.818 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-04T03:25:36.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:36 smithi077 ceph-mon[33531]: pgmap v584: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 64 op/s 2024-05-04T03:25:36.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:36 smithi072 ceph-mon[33332]: pgmap v584: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 64 op/s 2024-05-04T03:25:37.472 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-05-04T03:25:37.473 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-05-04T03:25:37.473 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-05-04T03:25:37.473 DEBUG:tasks.check_counter:Getting stats from j 2024-05-04T03:25:37.473 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-04T03:25:37.725 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:25:38.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:38 smithi001 ceph-mon[29731]: pgmap v585: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-04T03:25:38.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:38 smithi077 ceph-mon[33531]: pgmap v585: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-04T03:25:38.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:38 smithi072 ceph-mon[33332]: pgmap v585: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-04T03:25:39.479 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-05-04T03:25:39.479 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-0": { 2024-05-04T03:25:39.479 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 0, 2024-05-04T03:25:39.479 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 2, 2024-05-04T03:25:39.479 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 0, 2024-05-04T03:25:39.479 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 0, 2024-05-04T03:25:39.480 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 4, 2024-05-04T03:25:39.480 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 0, 2024-05-04T03:25:39.480 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.001383956, 2024-05-04T03:25:39.480 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.000000000, 2024-05-04T03:25:39.480 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.000340406, 2024-05-04T03:25:39.480 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-04T03:25:39.480 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:39.480 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-05-04T03:25:39.480 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-05-04T03:25:39.480 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-05-04T03:25:39.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:39.480 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:39.480 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-05-04T03:25:39.480 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-05-04T03:25:39.480 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-05-04T03:25:39.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:39.480 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-04T03:25:39.481 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 0 2024-05-04T03:25:39.481 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:39.481 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-1": { 2024-05-04T03:25:39.481 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 181, 2024-05-04T03:25:39.481 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 174, 2024-05-04T03:25:39.481 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 803680, 2024-05-04T03:25:39.481 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 105616, 2024-05-04T03:25:39.481 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 2, 2024-05-04T03:25:39.481 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 1, 2024-05-04T03:25:39.481 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.030604631, 2024-05-04T03:25:39.481 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.010328397, 2024-05-04T03:25:39.481 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.022420533, 2024-05-04T03:25:39.481 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.003277365, 2024-05-04T03:25:39.481 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:39.481 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 174, 2024-05-04T03:25:39.482 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.008847231, 2024-05-04T03:25:39.482 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000050846 2024-05-04T03:25:39.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:39.482 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:39.482 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-05-04T03:25:39.482 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-05-04T03:25:39.482 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-05-04T03:25:39.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:39.482 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 803680, 2024-05-04T03:25:39.482 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 105616 2024-05-04T03:25:39.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:39.482 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-2": { 2024-05-04T03:25:39.482 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 1, 2024-05-04T03:25:39.482 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 130, 2024-05-04T03:25:39.483 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 160, 2024-05-04T03:25:39.483 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 101472, 2024-05-04T03:25:39.483 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 3, 2024-05-04T03:25:39.483 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 2, 2024-05-04T03:25:39.483 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.010606588, 2024-05-04T03:25:39.483 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.007490238, 2024-05-04T03:25:39.483 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.001816282, 2024-05-04T03:25:39.483 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-04T03:25:39.483 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:39.483 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 128, 2024-05-04T03:25:39.483 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.005697379, 2024-05-04T03:25:39.483 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000044510 2024-05-04T03:25:39.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:39.483 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:39.483 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-05-04T03:25:39.483 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-05-04T03:25:39.484 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-05-04T03:25:39.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:39.484 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-04T03:25:39.484 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 101472 2024-05-04T03:25:39.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:39.484 INFO:teuthology.orchestra.run.smithi001.stdout: "cct": { 2024-05-04T03:25:39.484 INFO:teuthology.orchestra.run.smithi001.stdout: "total_workers": 0, 2024-05-04T03:25:39.484 INFO:teuthology.orchestra.run.smithi001.stdout: "unhealthy_workers": 0 2024-05-04T03:25:39.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:39.484 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool": { 2024-05-04T03:25:39.484 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_bytes": 0, 2024-05-04T03:25:39.484 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_items": 0, 2024-05-04T03:25:39.484 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_bytes": 0, 2024-05-04T03:25:39.484 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_items": 0, 2024-05-04T03:25:39.484 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_bytes": 0, 2024-05-04T03:25:39.484 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_items": 0, 2024-05-04T03:25:39.484 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-04T03:25:39.485 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_items": 0, 2024-05-04T03:25:39.485 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-04T03:25:39.485 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_items": 0, 2024-05-04T03:25:39.485 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_bytes": 0, 2024-05-04T03:25:39.485 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_items": 0, 2024-05-04T03:25:39.485 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-04T03:25:39.485 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_items": 0, 2024-05-04T03:25:39.485 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_bytes": 0, 2024-05-04T03:25:39.485 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_items": 0, 2024-05-04T03:25:39.485 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_bytes": 0, 2024-05-04T03:25:39.485 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_items": 0, 2024-05-04T03:25:39.485 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-04T03:25:39.485 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_items": 0, 2024-05-04T03:25:39.485 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-04T03:25:39.485 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_items": 0, 2024-05-04T03:25:39.485 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_bytes": 0, 2024-05-04T03:25:39.485 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_items": 0, 2024-05-04T03:25:39.486 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_bytes": 0, 2024-05-04T03:25:39.486 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_items": 0, 2024-05-04T03:25:39.486 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-04T03:25:39.486 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_items": 0, 2024-05-04T03:25:39.486 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_bytes": 0, 2024-05-04T03:25:39.486 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_items": 0, 2024-05-04T03:25:39.486 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_bytes": 0, 2024-05-04T03:25:39.486 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_items": 0, 2024-05-04T03:25:39.486 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_bytes": 0, 2024-05-04T03:25:39.486 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_items": 0, 2024-05-04T03:25:39.486 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_bytes": 0, 2024-05-04T03:25:39.486 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_items": 0, 2024-05-04T03:25:39.486 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_bytes": 148984, 2024-05-04T03:25:39.486 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_items": 30, 2024-05-04T03:25:39.486 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_bytes": 0, 2024-05-04T03:25:39.487 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_items": 0, 2024-05-04T03:25:39.487 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bytes": 0, 2024-05-04T03:25:39.487 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_items": 0, 2024-05-04T03:25:39.487 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_bytes": 0, 2024-05-04T03:25:39.487 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_items": 0, 2024-05-04T03:25:39.487 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_bytes": 0, 2024-05-04T03:25:39.487 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_items": 0, 2024-05-04T03:25:39.487 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_bytes": 0, 2024-05-04T03:25:39.487 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_items": 0, 2024-05-04T03:25:39.487 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_bytes": 0, 2024-05-04T03:25:39.487 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_items": 0, 2024-05-04T03:25:39.487 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_bytes": 0, 2024-05-04T03:25:39.487 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_items": 0, 2024-05-04T03:25:39.487 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_bytes": 928, 2024-05-04T03:25:39.487 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_items": 2, 2024-05-04T03:25:39.487 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_bytes": 0, 2024-05-04T03:25:39.488 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_items": 0, 2024-05-04T03:25:39.488 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_bytes": 0, 2024-05-04T03:25:39.488 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_items": 0 2024-05-04T03:25:39.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-05-04T03:25:39.488 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-04T03:25:39.488 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-05-04T03:25:39.488 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-05-04T03:25:39.488 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-05-04T03:25:39.488 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 178, 2024-05-04T03:25:39.488 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 775682, 2024-05-04T03:25:39.488 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:39.488 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 178, 2024-05-04T03:25:39.488 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-05-04T03:25:39.488 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-05-04T03:25:39.488 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 178, 2024-05-04T03:25:39.488 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 775682, 2024-05-04T03:25:39.489 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-05-04T03:25:39.489 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-05-04T03:25:39.489 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-05-04T03:25:39.489 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-05-04T03:25:39.489 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:39.489 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-05-04T03:25:39.489 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-05-04T03:25:40.113 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-04T03:25:40.113 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-04T03:25:40.113 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-05-04T03:25:40.113 DEBUG:tasks.check_counter:Getting stats from b 2024-05-04T03:25:40.114 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-04T03:25:40.365 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.b/config 2024-05-04T03:25:40.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:40 smithi072 ceph-mon[33332]: pgmap v586: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-05-04T03:25:40.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:40 smithi001 ceph-mon[29731]: pgmap v586: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-05-04T03:25:40.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:40 smithi077 ceph-mon[33531]: pgmap v586: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-05-04T03:25:42.209 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-05-04T03:25:42.210 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-0": { 2024-05-04T03:25:42.210 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 2268, 2024-05-04T03:25:42.210 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 2242, 2024-05-04T03:25:42.210 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 1530675, 2024-05-04T03:25:42.210 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 2052828, 2024-05-04T03:25:42.210 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 81, 2024-05-04T03:25:42.210 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 9, 2024-05-04T03:25:42.210 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.217074309, 2024-05-04T03:25:42.210 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.058241260, 2024-05-04T03:25:42.210 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.114639468, 2024-05-04T03:25:42.210 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.045426715, 2024-05-04T03:25:42.211 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:42.211 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 2238, 2024-05-04T03:25:42.211 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.237549470, 2024-05-04T03:25:42.211 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000106143 2024-05-04T03:25:42.211 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.211 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:42.211 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 1840, 2024-05-04T03:25:42.211 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000441273, 2024-05-04T03:25:42.211 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000239 2024-05-04T03:25:42.211 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.211 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 974992, 2024-05-04T03:25:42.211 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 140064 2024-05-04T03:25:42.212 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.212 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-1": { 2024-05-04T03:25:42.212 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 1597, 2024-05-04T03:25:42.212 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 1540, 2024-05-04T03:25:42.212 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 577967, 2024-05-04T03:25:42.212 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 1101171, 2024-05-04T03:25:42.212 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 22, 2024-05-04T03:25:42.212 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 5, 2024-05-04T03:25:42.212 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.122730467, 2024-05-04T03:25:42.212 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.050412974, 2024-05-04T03:25:42.212 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.073692222, 2024-05-04T03:25:42.212 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.010673868, 2024-05-04T03:25:42.212 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:42.212 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 1542, 2024-05-04T03:25:42.213 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.983679717, 2024-05-04T03:25:42.213 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000637924 2024-05-04T03:25:42.213 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.213 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:42.213 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 2167, 2024-05-04T03:25:42.213 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000433012, 2024-05-04T03:25:42.213 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000199 2024-05-04T03:25:42.213 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.213 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-05-04T03:25:42.213 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 302912 2024-05-04T03:25:42.213 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.213 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-2": { 2024-05-04T03:25:42.213 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 1225, 2024-05-04T03:25:42.213 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 1158, 2024-05-04T03:25:42.214 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 377267, 2024-05-04T03:25:42.214 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 1850244, 2024-05-04T03:25:42.214 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 8, 2024-05-04T03:25:42.214 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 6, 2024-05-04T03:25:42.214 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.095564659, 2024-05-04T03:25:42.214 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.032845034, 2024-05-04T03:25:42.214 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.048091154, 2024-05-04T03:25:42.214 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.018898445, 2024-05-04T03:25:42.214 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:42.214 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 1156, 2024-05-04T03:25:42.214 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.331432527, 2024-05-04T03:25:42.214 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000286706 2024-05-04T03:25:42.215 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.215 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:42.215 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 919, 2024-05-04T03:25:42.215 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000219722, 2024-05-04T03:25:42.215 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000239 2024-05-04T03:25:42.215 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.215 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-04T03:25:42.215 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 0 2024-05-04T03:25:42.215 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.215 INFO:teuthology.orchestra.run.smithi072.stdout: "cct": { 2024-05-04T03:25:42.215 INFO:teuthology.orchestra.run.smithi072.stdout: "total_workers": 1, 2024-05-04T03:25:42.215 INFO:teuthology.orchestra.run.smithi072.stdout: "unhealthy_workers": 0 2024-05-04T03:25:42.215 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.216 INFO:teuthology.orchestra.run.smithi072.stdout: "finisher-MDSRank": { 2024-05-04T03:25:42.216 INFO:teuthology.orchestra.run.smithi072.stdout: "queue_len": 0, 2024-05-04T03:25:42.216 INFO:teuthology.orchestra.run.smithi072.stdout: "complete_latency": { 2024-05-04T03:25:42.216 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 788, 2024-05-04T03:25:42.216 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.365637388, 2024-05-04T03:25:42.216 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000464006 2024-05-04T03:25:42.216 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:42.216 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.216 INFO:teuthology.orchestra.run.smithi072.stdout: "finisher-PurgeQueue": { 2024-05-04T03:25:42.216 INFO:teuthology.orchestra.run.smithi072.stdout: "queue_len": 0, 2024-05-04T03:25:42.216 INFO:teuthology.orchestra.run.smithi072.stdout: "complete_latency": { 2024-05-04T03:25:42.216 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 254, 2024-05-04T03:25:42.217 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.017262241, 2024-05-04T03:25:42.217 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000067961 2024-05-04T03:25:42.217 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:42.217 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.217 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": { 2024-05-04T03:25:42.217 INFO:teuthology.orchestra.run.smithi072.stdout: "request": 792, 2024-05-04T03:25:42.217 INFO:teuthology.orchestra.run.smithi072.stdout: "reply": 782, 2024-05-04T03:25:42.217 INFO:teuthology.orchestra.run.smithi072.stdout: "reply_latency": { 2024-05-04T03:25:42.217 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 782, 2024-05-04T03:25:42.218 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 19.733721174, 2024-05-04T03:25:42.218 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.025234937 2024-05-04T03:25:42.218 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.218 INFO:teuthology.orchestra.run.smithi072.stdout: "slow_reply": 0, 2024-05-04T03:25:42.218 INFO:teuthology.orchestra.run.smithi072.stdout: "forward": 10, 2024-05-04T03:25:42.218 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_fetch_complete": 0, 2024-05-04T03:25:42.218 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_fetch_keys": 0, 2024-05-04T03:25:42.218 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_commit": 146, 2024-05-04T03:25:42.218 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_split": 0, 2024-05-04T03:25:42.218 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_merge": 0, 2024-05-04T03:25:42.218 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes": 125, 2024-05-04T03:25:42.219 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_top": 0, 2024-05-04T03:25:42.219 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_bottom": 115, 2024-05-04T03:25:42.219 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_pin_tail": 10, 2024-05-04T03:25:42.219 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_pinned": 122, 2024-05-04T03:25:42.219 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_expired": 314, 2024-05-04T03:25:42.219 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_with_caps": 2, 2024-05-04T03:25:42.219 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": 2, 2024-05-04T03:25:42.219 INFO:teuthology.orchestra.run.smithi072.stdout: "subtrees": 3, 2024-05-04T03:25:42.219 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse": 1343, 2024-05-04T03:25:42.219 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_hit": 662, 2024-05-04T03:25:42.219 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_forward": 10, 2024-05-04T03:25:42.219 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_discover": 3, 2024-05-04T03:25:42.219 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_dir_fetch": 0, 2024-05-04T03:25:42.220 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_remote_ino": 0, 2024-05-04T03:25:42.220 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_lock": 4, 2024-05-04T03:25:42.220 INFO:teuthology.orchestra.run.smithi072.stdout: "load_cent": 0, 2024-05-04T03:25:42.220 INFO:teuthology.orchestra.run.smithi072.stdout: "q": 0, 2024-05-04T03:25:42.220 INFO:teuthology.orchestra.run.smithi072.stdout: "exported": 3, 2024-05-04T03:25:42.220 INFO:teuthology.orchestra.run.smithi072.stdout: "exported_inodes": 298, 2024-05-04T03:25:42.220 INFO:teuthology.orchestra.run.smithi072.stdout: "imported": 3, 2024-05-04T03:25:42.220 INFO:teuthology.orchestra.run.smithi072.stdout: "imported_inodes": 66, 2024-05-04T03:25:42.220 INFO:teuthology.orchestra.run.smithi072.stdout: "openino_dir_fetch": 0, 2024-05-04T03:25:42.220 INFO:teuthology.orchestra.run.smithi072.stdout: "openino_backtrace_fetch": 0, 2024-05-04T03:25:42.220 INFO:teuthology.orchestra.run.smithi072.stdout: "openino_peer_discover": 0, 2024-05-04T03:25:42.220 INFO:teuthology.orchestra.run.smithi072.stdout: "root_rfiles": 4, 2024-05-04T03:25:42.220 INFO:teuthology.orchestra.run.smithi072.stdout: "root_rbytes": 234, 2024-05-04T03:25:42.221 INFO:teuthology.orchestra.run.smithi072.stdout: "root_rsnaps": 0, 2024-05-04T03:25:42.221 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_backtrace_fetch": 0, 2024-05-04T03:25:42.221 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_set_tag": 0, 2024-05-04T03:25:42.221 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_backtrace_repaired": 0, 2024-05-04T03:25:42.221 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_inotable_repaired": 0, 2024-05-04T03:25:42.221 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_dir_inodes": 0, 2024-05-04T03:25:42.221 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_dir_base_inodes": 0, 2024-05-04T03:25:42.221 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_dirfrag_rstats": 0, 2024-05-04T03:25:42.221 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_file_inodes": 0, 2024-05-04T03:25:42.221 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_inode_file_caps": 6, 2024-05-04T03:25:42.221 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_revoke": 144, 2024-05-04T03:25:42.221 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_grant": 258, 2024-05-04T03:25:42.221 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_trunc": 2, 2024-05-04T03:25:42.221 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-04T03:25:42.222 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-04T03:25:42.222 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_caps": 275, 2024-05-04T03:25:42.222 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_caps_dirty": 121, 2024-05-04T03:25:42.222 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_cap_release": 12, 2024-05-04T03:25:42.222 INFO:teuthology.orchestra.run.smithi072.stdout: "process_request_cap_release": 340 2024-05-04T03:25:42.222 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.222 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_cache": { 2024-05-04T03:25:42.222 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_update": 0, 2024-05-04T03:25:42.222 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_update_receipt": 0, 2024-05-04T03:25:42.222 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_try_discover": 0, 2024-05-04T03:25:42.222 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_send_discover": 2, 2024-05-04T03:25:42.222 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_handle_discover": 8, 2024-05-04T03:25:42.223 INFO:teuthology.orchestra.run.smithi072.stdout: "num_strays": 0, 2024-05-04T03:25:42.223 INFO:teuthology.orchestra.run.smithi072.stdout: "num_strays_delayed": 0, 2024-05-04T03:25:42.223 INFO:teuthology.orchestra.run.smithi072.stdout: "num_strays_enqueuing": 0, 2024-05-04T03:25:42.223 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_created": 119, 2024-05-04T03:25:42.223 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_enqueued": 119, 2024-05-04T03:25:42.223 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_reintegrated": 0, 2024-05-04T03:25:42.223 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_migrated": 0, 2024-05-04T03:25:42.223 INFO:teuthology.orchestra.run.smithi072.stdout: "num_recovering_processing": 0, 2024-05-04T03:25:42.223 INFO:teuthology.orchestra.run.smithi072.stdout: "num_recovering_enqueued": 0, 2024-05-04T03:25:42.223 INFO:teuthology.orchestra.run.smithi072.stdout: "num_recovering_prioritized": 0, 2024-05-04T03:25:42.223 INFO:teuthology.orchestra.run.smithi072.stdout: "recovery_started": 0, 2024-05-04T03:25:42.223 INFO:teuthology.orchestra.run.smithi072.stdout: "recovery_completed": 0, 2024-05-04T03:25:42.223 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_quiesce_path": 4, 2024-05-04T03:25:42.224 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_quiesce_inode": 258, 2024-05-04T03:25:42.224 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_enqueue_scrub": 0, 2024-05-04T03:25:42.224 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_exportdir": 3, 2024-05-04T03:25:42.224 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_flush": 0, 2024-05-04T03:25:42.224 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_fragmentdir": 0, 2024-05-04T03:25:42.224 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_fragstats": 0, 2024-05-04T03:25:42.224 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_inodestats": 0 2024-05-04T03:25:42.224 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.224 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_log": { 2024-05-04T03:25:42.224 INFO:teuthology.orchestra.run.smithi072.stdout: "evlrg": 0, 2024-05-04T03:25:42.224 INFO:teuthology.orchestra.run.smithi072.stdout: "evadd": 630, 2024-05-04T03:25:42.224 INFO:teuthology.orchestra.run.smithi072.stdout: "evex": 514, 2024-05-04T03:25:42.224 INFO:teuthology.orchestra.run.smithi072.stdout: "evtrm": 514, 2024-05-04T03:25:42.225 INFO:teuthology.orchestra.run.smithi072.stdout: "ev": 116, 2024-05-04T03:25:42.225 INFO:teuthology.orchestra.run.smithi072.stdout: "evexg": 0, 2024-05-04T03:25:42.225 INFO:teuthology.orchestra.run.smithi072.stdout: "evexd": 119, 2024-05-04T03:25:42.225 INFO:teuthology.orchestra.run.smithi072.stdout: "segadd": 32, 2024-05-04T03:25:42.225 INFO:teuthology.orchestra.run.smithi072.stdout: "segex": 31, 2024-05-04T03:25:42.225 INFO:teuthology.orchestra.run.smithi072.stdout: "segtrm": 31, 2024-05-04T03:25:42.225 INFO:teuthology.orchestra.run.smithi072.stdout: "seg": 1, 2024-05-04T03:25:42.225 INFO:teuthology.orchestra.run.smithi072.stdout: "segmjr": 1, 2024-05-04T03:25:42.225 INFO:teuthology.orchestra.run.smithi072.stdout: "segexg": 0, 2024-05-04T03:25:42.225 INFO:teuthology.orchestra.run.smithi072.stdout: "segexd": 1, 2024-05-04T03:25:42.225 INFO:teuthology.orchestra.run.smithi072.stdout: "expos": 5398119, 2024-05-04T03:25:42.225 INFO:teuthology.orchestra.run.smithi072.stdout: "wrpos": 5480943, 2024-05-04T03:25:42.225 INFO:teuthology.orchestra.run.smithi072.stdout: "rdpos": 0, 2024-05-04T03:25:42.225 INFO:teuthology.orchestra.run.smithi072.stdout: "jlat": { 2024-05-04T03:25:42.226 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 172, 2024-05-04T03:25:42.226 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.819444789, 2024-05-04T03:25:42.226 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.004764213 2024-05-04T03:25:42.226 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.226 INFO:teuthology.orchestra.run.smithi072.stdout: "replayed": 0 2024-05-04T03:25:42.226 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.226 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_mem": { 2024-05-04T03:25:42.226 INFO:teuthology.orchestra.run.smithi072.stdout: "ino": 20, 2024-05-04T03:25:42.226 INFO:teuthology.orchestra.run.smithi072.stdout: "ino+": 180, 2024-05-04T03:25:42.226 INFO:teuthology.orchestra.run.smithi072.stdout: "ino-": 160, 2024-05-04T03:25:42.226 INFO:teuthology.orchestra.run.smithi072.stdout: "dir": 19, 2024-05-04T03:25:42.226 INFO:teuthology.orchestra.run.smithi072.stdout: "dir+": 38, 2024-05-04T03:25:42.226 INFO:teuthology.orchestra.run.smithi072.stdout: "dir-": 19, 2024-05-04T03:25:42.227 INFO:teuthology.orchestra.run.smithi072.stdout: "dn": 125, 2024-05-04T03:25:42.227 INFO:teuthology.orchestra.run.smithi072.stdout: "dn+": 604, 2024-05-04T03:25:42.227 INFO:teuthology.orchestra.run.smithi072.stdout: "dn-": 479, 2024-05-04T03:25:42.227 INFO:teuthology.orchestra.run.smithi072.stdout: "cap": 2, 2024-05-04T03:25:42.227 INFO:teuthology.orchestra.run.smithi072.stdout: "cap+": 277, 2024-05-04T03:25:42.227 INFO:teuthology.orchestra.run.smithi072.stdout: "cap-": 275, 2024-05-04T03:25:42.227 INFO:teuthology.orchestra.run.smithi072.stdout: "rss": 132624, 2024-05-04T03:25:42.227 INFO:teuthology.orchestra.run.smithi072.stdout: "heap": 231824 2024-05-04T03:25:42.227 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.227 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_server": { 2024-05-04T03:25:42.227 INFO:teuthology.orchestra.run.smithi072.stdout: "dispatch_client_request": 1056, 2024-05-04T03:25:42.227 INFO:teuthology.orchestra.run.smithi072.stdout: "dispatch_server_request": 45, 2024-05-04T03:25:42.228 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_request": 792, 2024-05-04T03:25:42.228 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_session": 17, 2024-05-04T03:25:42.228 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_peer_request": 239, 2024-05-04T03:25:42.228 INFO:teuthology.orchestra.run.smithi072.stdout: "req_create_latency": { 2024-05-04T03:25:42.228 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 76, 2024-05-04T03:25:42.228 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.140258924, 2024-05-04T03:25:42.228 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.001845512 2024-05-04T03:25:42.228 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.228 INFO:teuthology.orchestra.run.smithi072.stdout: "req_getattr_latency": { 2024-05-04T03:25:42.228 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 18, 2024-05-04T03:25:42.228 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.010301353, 2024-05-04T03:25:42.228 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000572297 2024-05-04T03:25:42.228 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.229 INFO:teuthology.orchestra.run.smithi072.stdout: "req_getfilelock_latency": { 2024-05-04T03:25:42.229 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:42.229 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:42.229 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:42.229 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.229 INFO:teuthology.orchestra.run.smithi072.stdout: "req_link_latency": { 2024-05-04T03:25:42.229 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:42.229 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:42.229 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:42.229 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.229 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookup_latency": { 2024-05-04T03:25:42.229 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 230, 2024-05-04T03:25:42.229 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.094586172, 2024-05-04T03:25:42.230 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000411244 2024-05-04T03:25:42.230 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.230 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookuphash_latency": { 2024-05-04T03:25:42.230 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:42.230 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:42.230 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:42.230 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.230 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupino_latency": { 2024-05-04T03:25:42.230 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 1, 2024-05-04T03:25:42.230 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000182856, 2024-05-04T03:25:42.230 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000182856 2024-05-04T03:25:42.230 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.230 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupname_latency": { 2024-05-04T03:25:42.230 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:42.230 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:42.231 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:42.231 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.231 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupparent_latency": { 2024-05-04T03:25:42.231 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:42.231 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:42.231 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:42.231 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.231 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupsnap_latency": { 2024-05-04T03:25:42.231 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:42.231 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:42.231 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:42.231 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.231 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lssnap_latency": { 2024-05-04T03:25:42.232 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:42.232 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:42.232 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:42.232 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.232 INFO:teuthology.orchestra.run.smithi072.stdout: "req_mkdir_latency": { 2024-05-04T03:25:42.232 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 20, 2024-05-04T03:25:42.233 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 19.058840007, 2024-05-04T03:25:42.233 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.952942000 2024-05-04T03:25:42.233 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.233 INFO:teuthology.orchestra.run.smithi072.stdout: "req_mknod_latency": { 2024-05-04T03:25:42.233 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:42.233 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:42.233 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:42.233 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.233 INFO:teuthology.orchestra.run.smithi072.stdout: "req_mksnap_latency": { 2024-05-04T03:25:42.233 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:42.233 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:42.233 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:42.233 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.233 INFO:teuthology.orchestra.run.smithi072.stdout: "req_open_latency": { 2024-05-04T03:25:42.234 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 278, 2024-05-04T03:25:42.234 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.049847675, 2024-05-04T03:25:42.234 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000179308 2024-05-04T03:25:42.234 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.234 INFO:teuthology.orchestra.run.smithi072.stdout: "req_readdir_latency": { 2024-05-04T03:25:42.234 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 10, 2024-05-04T03:25:42.234 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.013590547, 2024-05-04T03:25:42.234 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.001359054 2024-05-04T03:25:42.234 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.234 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rename_latency": { 2024-05-04T03:25:42.234 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 23, 2024-05-04T03:25:42.234 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.043175340, 2024-05-04T03:25:42.234 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.001877188 2024-05-04T03:25:42.235 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.235 INFO:teuthology.orchestra.run.smithi072.stdout: "req_renamesnap_latency": { 2024-05-04T03:25:42.235 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:42.235 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:42.235 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:42.235 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.235 INFO:teuthology.orchestra.run.smithi072.stdout: "req_snapdiff_latency": { 2024-05-04T03:25:42.235 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:42.235 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:42.235 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:42.235 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.235 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rmdir_latency": { 2024-05-04T03:25:42.235 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 2, 2024-05-04T03:25:42.235 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.075898245, 2024-05-04T03:25:42.236 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.037949122 2024-05-04T03:25:42.236 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.236 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rmsnap_latency": { 2024-05-04T03:25:42.236 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:42.236 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:42.236 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:42.236 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.236 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rmxattr_latency": { 2024-05-04T03:25:42.236 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:42.236 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:42.236 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:42.236 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.236 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setattr_latency": { 2024-05-04T03:25:42.237 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 3, 2024-05-04T03:25:42.237 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.010354678, 2024-05-04T03:25:42.237 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.003451559 2024-05-04T03:25:42.237 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.237 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setdirlayout_latency": { 2024-05-04T03:25:42.237 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:42.237 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:42.237 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:42.237 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.237 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setfilelock_latency": { 2024-05-04T03:25:42.237 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:42.237 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:42.237 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:42.237 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.238 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setlayout_latency": { 2024-05-04T03:25:42.238 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:42.238 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:42.238 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:42.238 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.238 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setxattr_latency": { 2024-05-04T03:25:42.238 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 8, 2024-05-04T03:25:42.238 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.021193041, 2024-05-04T03:25:42.238 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.002649130 2024-05-04T03:25:42.238 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.238 INFO:teuthology.orchestra.run.smithi072.stdout: "req_symlink_latency": { 2024-05-04T03:25:42.238 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 2, 2024-05-04T03:25:42.238 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.001275017, 2024-05-04T03:25:42.239 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000637508 2024-05-04T03:25:42.239 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.239 INFO:teuthology.orchestra.run.smithi072.stdout: "req_unlink_latency": { 2024-05-04T03:25:42.239 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 111, 2024-05-04T03:25:42.239 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.214217319, 2024-05-04T03:25:42.239 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.001929885 2024-05-04T03:25:42.239 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.239 INFO:teuthology.orchestra.run.smithi072.stdout: "cap_revoke_eviction": 0, 2024-05-04T03:25:42.239 INFO:teuthology.orchestra.run.smithi072.stdout: "cap_acquisition_throttle": 0, 2024-05-04T03:25:42.239 INFO:teuthology.orchestra.run.smithi072.stdout: "req_getvxattr_latency": { 2024-05-04T03:25:42.239 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:42.239 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:42.239 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:42.239 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:42.240 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.240 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_sessions": { 2024-05-04T03:25:42.240 INFO:teuthology.orchestra.run.smithi072.stdout: "session_count": 1, 2024-05-04T03:25:42.240 INFO:teuthology.orchestra.run.smithi072.stdout: "session_add": 3, 2024-05-04T03:25:42.240 INFO:teuthology.orchestra.run.smithi072.stdout: "session_remove": 2, 2024-05-04T03:25:42.240 INFO:teuthology.orchestra.run.smithi072.stdout: "sessions_open": 1, 2024-05-04T03:25:42.240 INFO:teuthology.orchestra.run.smithi072.stdout: "sessions_stale": 0, 2024-05-04T03:25:42.240 INFO:teuthology.orchestra.run.smithi072.stdout: "total_load": 245, 2024-05-04T03:25:42.240 INFO:teuthology.orchestra.run.smithi072.stdout: "average_load": 245, 2024-05-04T03:25:42.240 INFO:teuthology.orchestra.run.smithi072.stdout: "avg_session_uptime": 415, 2024-05-04T03:25:42.240 INFO:teuthology.orchestra.run.smithi072.stdout: "mdthresh_evicted": 0 2024-05-04T03:25:42.240 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.240 INFO:teuthology.orchestra.run.smithi072.stdout: "mempool": { 2024-05-04T03:25:42.241 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_bytes": 360, 2024-05-04T03:25:42.241 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_items": 360, 2024-05-04T03:25:42.241 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_bytes": 0, 2024-05-04T03:25:42.241 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_items": 0, 2024-05-04T03:25:42.241 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_bytes": 0, 2024-05-04T03:25:42.241 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_items": 0, 2024-05-04T03:25:42.241 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-04T03:25:42.241 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_items": 0, 2024-05-04T03:25:42.241 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-04T03:25:42.241 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_items": 0, 2024-05-04T03:25:42.241 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_bytes": 0, 2024-05-04T03:25:42.241 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_items": 0, 2024-05-04T03:25:42.241 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-04T03:25:42.241 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_items": 0, 2024-05-04T03:25:42.242 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_bytes": 0, 2024-05-04T03:25:42.242 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_items": 0, 2024-05-04T03:25:42.242 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_bytes": 0, 2024-05-04T03:25:42.242 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_items": 0, 2024-05-04T03:25:42.242 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-04T03:25:42.242 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_items": 0, 2024-05-04T03:25:42.242 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-04T03:25:42.242 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_items": 0, 2024-05-04T03:25:42.242 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_bytes": 0, 2024-05-04T03:25:42.242 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_items": 0, 2024-05-04T03:25:42.242 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_bytes": 0, 2024-05-04T03:25:42.242 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_items": 0, 2024-05-04T03:25:42.242 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-04T03:25:42.242 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_items": 0, 2024-05-04T03:25:42.243 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_bytes": 0, 2024-05-04T03:25:42.243 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_items": 0, 2024-05-04T03:25:42.243 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_bytes": 0, 2024-05-04T03:25:42.243 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_items": 0, 2024-05-04T03:25:42.243 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_bytes": 0, 2024-05-04T03:25:42.243 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_items": 0, 2024-05-04T03:25:42.243 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_bytes": 0, 2024-05-04T03:25:42.243 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_items": 0, 2024-05-04T03:25:42.243 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_bytes": 279013, 2024-05-04T03:25:42.243 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_items": 99, 2024-05-04T03:25:42.243 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_bytes": 0, 2024-05-04T03:25:42.243 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_items": 0, 2024-05-04T03:25:42.243 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_bytes": 0, 2024-05-04T03:25:42.244 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_items": 0, 2024-05-04T03:25:42.244 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_bytes": 0, 2024-05-04T03:25:42.244 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_items": 0, 2024-05-04T03:25:42.244 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_bytes": 0, 2024-05-04T03:25:42.244 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_items": 0, 2024-05-04T03:25:42.244 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_bytes": 7240, 2024-05-04T03:25:42.244 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_items": 152, 2024-05-04T03:25:42.244 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_bytes": 0, 2024-05-04T03:25:42.244 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_items": 0, 2024-05-04T03:25:42.244 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_bytes": 0, 2024-05-04T03:25:42.244 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_items": 0, 2024-05-04T03:25:42.244 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_bytes": 164181, 2024-05-04T03:25:42.244 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_items": 1345, 2024-05-04T03:25:42.244 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_bytes": 0, 2024-05-04T03:25:42.244 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_items": 0, 2024-05-04T03:25:42.245 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_bytes": 0, 2024-05-04T03:25:42.245 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_items": 0 2024-05-04T03:25:42.245 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.245 INFO:teuthology.orchestra.run.smithi072.stdout: "objecter": { 2024-05-04T03:25:42.245 INFO:teuthology.orchestra.run.smithi072.stdout: "op_active": 0, 2024-05-04T03:25:42.245 INFO:teuthology.orchestra.run.smithi072.stdout: "op_laggy": 0, 2024-05-04T03:25:42.245 INFO:teuthology.orchestra.run.smithi072.stdout: "op_send": 1396, 2024-05-04T03:25:42.245 INFO:teuthology.orchestra.run.smithi072.stdout: "op_send_bytes": 1726987, 2024-05-04T03:25:42.245 INFO:teuthology.orchestra.run.smithi072.stdout: "op_resend": 37, 2024-05-04T03:25:42.245 INFO:teuthology.orchestra.run.smithi072.stdout: "op_reply": 1396, 2024-05-04T03:25:42.245 INFO:teuthology.orchestra.run.smithi072.stdout: "op_latency": { 2024-05-04T03:25:42.245 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 1396, 2024-05-04T03:25:42.245 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 24.932464055, 2024-05-04T03:25:42.246 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.017859931 2024-05-04T03:25:42.246 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.246 INFO:teuthology.orchestra.run.smithi072.stdout: "op_inflight": 0, 2024-05-04T03:25:42.246 INFO:teuthology.orchestra.run.smithi072.stdout: "oplen_avg": { 2024-05-04T03:25:42.246 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 1396, 2024-05-04T03:25:42.246 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 2191 2024-05-04T03:25:42.246 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.246 INFO:teuthology.orchestra.run.smithi072.stdout: "op": 1396, 2024-05-04T03:25:42.246 INFO:teuthology.orchestra.run.smithi072.stdout: "op_r": 8, 2024-05-04T03:25:42.246 INFO:teuthology.orchestra.run.smithi072.stdout: "op_w": 1388, 2024-05-04T03:25:42.246 INFO:teuthology.orchestra.run.smithi072.stdout: "op_rmw": 0, 2024-05-04T03:25:42.246 INFO:teuthology.orchestra.run.smithi072.stdout: "op_pg": 0, 2024-05-04T03:25:42.246 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_stat": 124, 2024-05-04T03:25:42.246 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_create": 249, 2024-05-04T03:25:42.247 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_read": 8, 2024-05-04T03:25:42.247 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_write": 180, 2024-05-04T03:25:42.247 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_writefull": 112, 2024-05-04T03:25:42.247 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_writesame": 0, 2024-05-04T03:25:42.247 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_append": 0, 2024-05-04T03:25:42.247 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_zero": 0, 2024-05-04T03:25:42.247 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_truncate": 0, 2024-05-04T03:25:42.247 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_delete": 631, 2024-05-04T03:25:42.247 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_mapext": 0, 2024-05-04T03:25:42.247 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_sparse_read": 0, 2024-05-04T03:25:42.247 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_clonerange": 0, 2024-05-04T03:25:42.247 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_getxattr": 0, 2024-05-04T03:25:42.247 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_setxattr": 500, 2024-05-04T03:25:42.248 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_cmpxattr": 0, 2024-05-04T03:25:42.248 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_rmxattr": 0, 2024-05-04T03:25:42.248 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_resetxattrs": 0, 2024-05-04T03:25:42.248 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_call": 0, 2024-05-04T03:25:42.248 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_watch": 0, 2024-05-04T03:25:42.248 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_notify": 0, 2024-05-04T03:25:42.248 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_src_cmpxattr": 0, 2024-05-04T03:25:42.248 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_pgls": 0, 2024-05-04T03:25:42.248 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_pgls_filter": 0, 2024-05-04T03:25:42.248 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_other": 2, 2024-05-04T03:25:42.248 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_active": 0, 2024-05-04T03:25:42.248 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_send": 0, 2024-05-04T03:25:42.248 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_resend": 0, 2024-05-04T03:25:42.248 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_ping": 0, 2024-05-04T03:25:42.249 INFO:teuthology.orchestra.run.smithi072.stdout: "poolop_active": 0, 2024-05-04T03:25:42.249 INFO:teuthology.orchestra.run.smithi072.stdout: "poolop_send": 0, 2024-05-04T03:25:42.249 INFO:teuthology.orchestra.run.smithi072.stdout: "poolop_resend": 0, 2024-05-04T03:25:42.249 INFO:teuthology.orchestra.run.smithi072.stdout: "poolstat_active": 0, 2024-05-04T03:25:42.249 INFO:teuthology.orchestra.run.smithi072.stdout: "poolstat_send": 0, 2024-05-04T03:25:42.249 INFO:teuthology.orchestra.run.smithi072.stdout: "poolstat_resend": 0, 2024-05-04T03:25:42.249 INFO:teuthology.orchestra.run.smithi072.stdout: "statfs_active": 0, 2024-05-04T03:25:42.249 INFO:teuthology.orchestra.run.smithi072.stdout: "statfs_send": 0, 2024-05-04T03:25:42.249 INFO:teuthology.orchestra.run.smithi072.stdout: "statfs_resend": 0, 2024-05-04T03:25:42.249 INFO:teuthology.orchestra.run.smithi072.stdout: "command_active": 0, 2024-05-04T03:25:42.249 INFO:teuthology.orchestra.run.smithi072.stdout: "command_send": 0, 2024-05-04T03:25:42.249 INFO:teuthology.orchestra.run.smithi072.stdout: "command_resend": 0, 2024-05-04T03:25:42.249 INFO:teuthology.orchestra.run.smithi072.stdout: "map_epoch": 83, 2024-05-04T03:25:42.249 INFO:teuthology.orchestra.run.smithi072.stdout: "map_full": 0, 2024-05-04T03:25:42.249 INFO:teuthology.orchestra.run.smithi072.stdout: "map_inc": 2, 2024-05-04T03:25:42.250 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_sessions": 12, 2024-05-04T03:25:42.250 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_session_open": 12, 2024-05-04T03:25:42.250 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_session_close": 0, 2024-05-04T03:25:42.250 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_laggy": 0, 2024-05-04T03:25:42.250 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_wr": 362, 2024-05-04T03:25:42.250 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_rd": 0, 2024-05-04T03:25:42.250 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_del": 23 2024-05-04T03:25:42.250 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.250 INFO:teuthology.orchestra.run.smithi072.stdout: "oft": { 2024-05-04T03:25:42.250 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_objs": 1, 2024-05-04T03:25:42.250 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_kv_pairs": 0, 2024-05-04T03:25:42.250 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_updates": 0, 2024-05-04T03:25:42.250 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_removes": 0 2024-05-04T03:25:42.251 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.251 INFO:teuthology.orchestra.run.smithi072.stdout: "osd-slow-ops": { 2024-05-04T03:25:42.251 INFO:teuthology.orchestra.run.smithi072.stdout: "slow_ops_count": 303 2024-05-04T03:25:42.251 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.251 INFO:teuthology.orchestra.run.smithi072.stdout: "purge_queue": { 2024-05-04T03:25:42.251 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing_ops": 0, 2024-05-04T03:25:42.251 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing_ops_high_water": 10, 2024-05-04T03:25:42.251 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing": 0, 2024-05-04T03:25:42.251 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing_high_water": 10, 2024-05-04T03:25:42.251 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executed_ops": 119, 2024-05-04T03:25:42.251 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executed": 119, 2024-05-04T03:25:42.251 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_item_in_journal": 0 2024-05-04T03:25:42.251 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.252 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-04T03:25:42.252 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-04T03:25:42.252 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-05-04T03:25:42.252 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-04T03:25:42.252 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 5086, 2024-05-04T03:25:42.252 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 2027077, 2024-05-04T03:25:42.252 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:42.252 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 5086, 2024-05-04T03:25:42.252 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-05-04T03:25:42.252 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-05-04T03:25:42.252 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 5086, 2024-05-04T03:25:42.252 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 2027077, 2024-05-04T03:25:42.252 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-04T03:25:42.253 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:42.253 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:42.253 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:42.253 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:42.253 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.253 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-objecter_bytes": { 2024-05-04T03:25:42.253 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-04T03:25:42.253 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-05-04T03:25:42.253 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-04T03:25:42.253 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 0, 2024-05-04T03:25:42.253 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 0, 2024-05-04T03:25:42.253 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:42.253 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 0, 2024-05-04T03:25:42.253 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 1396, 2024-05-04T03:25:42.254 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 1739006, 2024-05-04T03:25:42.254 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 1396, 2024-05-04T03:25:42.254 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 1739006, 2024-05-04T03:25:42.254 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-04T03:25:42.254 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:42.254 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:42.254 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:42.254 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:42.254 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.254 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-objecter_ops": { 2024-05-04T03:25:42.254 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-04T03:25:42.254 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 1024, 2024-05-04T03:25:42.254 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-04T03:25:42.255 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 0, 2024-05-04T03:25:42.255 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 0, 2024-05-04T03:25:42.255 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:42.255 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 0, 2024-05-04T03:25:42.255 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 1396, 2024-05-04T03:25:42.255 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 1396, 2024-05-04T03:25:42.255 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 1396, 2024-05-04T03:25:42.255 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 1396, 2024-05-04T03:25:42.255 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-04T03:25:42.255 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:42.255 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:42.255 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:42.255 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:42.255 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.256 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-write_buf_throttle": { 2024-05-04T03:25:42.256 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-04T03:25:42.256 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 3758096384, 2024-05-04T03:25:42.256 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-04T03:25:42.256 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 119, 2024-05-04T03:25:42.256 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 12019, 2024-05-04T03:25:42.256 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:42.256 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 119, 2024-05-04T03:25:42.256 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-05-04T03:25:42.256 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-05-04T03:25:42.256 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 8, 2024-05-04T03:25:42.256 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 12019, 2024-05-04T03:25:42.256 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-04T03:25:42.256 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:42.257 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:42.257 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:42.257 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:42.257 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:42.257 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-write_buf_throttle-0x55ef0a7be680": { 2024-05-04T03:25:42.257 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-04T03:25:42.257 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 3758096384, 2024-05-04T03:25:42.257 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-04T03:25:42.257 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 630, 2024-05-04T03:25:42.257 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 1286639, 2024-05-04T03:25:42.257 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:42.257 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 630, 2024-05-04T03:25:42.257 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-05-04T03:25:42.257 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-05-04T03:25:42.258 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 172, 2024-05-04T03:25:42.258 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 1286639, 2024-05-04T03:25:42.258 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-04T03:25:42.258 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:42.258 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:42.258 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:42.258 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:42.258 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:42.258 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-05-04T03:25:42.837 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:42 smithi072 ceph-mon[33332]: pgmap v587: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-04T03:25:42.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:42 smithi001 ceph-mon[29731]: pgmap v587: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-04T03:25:42.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:42 smithi077 ceph-mon[33531]: pgmap v587: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-04T03:25:43.012 INFO:tasks.check_counter:Daemon mds.b mds.exported=3 2024-05-04T03:25:43.013 INFO:tasks.check_counter:Daemon mds.b mds.imported=3 2024-05-04T03:25:43.013 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=0 2024-05-04T03:25:43.013 DEBUG:tasks.check_counter:Getting stats from e 2024-05-04T03:25:43.013 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-04T03:25:43.251 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.b/config 2024-05-04T03:25:44.712 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:44 smithi072 ceph-mon[33332]: pgmap v588: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-04T03:25:44.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:44 smithi001 ceph-mon[29731]: pgmap v588: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-04T03:25:44.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:44 smithi077 ceph-mon[33531]: pgmap v588: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-04T03:25:45.098 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-05-04T03:25:45.099 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-0": { 2024-05-04T03:25:45.099 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 713, 2024-05-04T03:25:45.099 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 713, 2024-05-04T03:25:45.099 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 30056077, 2024-05-04T03:25:45.099 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 225974, 2024-05-04T03:25:45.099 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 7, 2024-05-04T03:25:45.099 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 3, 2024-05-04T03:25:45.099 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.077333657, 2024-05-04T03:25:45.099 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.020713787, 2024-05-04T03:25:45.099 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.030861608, 2024-05-04T03:25:45.100 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.019601786, 2024-05-04T03:25:45.100 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:45.100 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 712, 2024-05-04T03:25:45.100 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.027205305, 2024-05-04T03:25:45.100 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000038209 2024-05-04T03:25:45.100 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.100 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:45.100 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.100 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.100 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.100 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.100 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-04T03:25:45.101 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 448 2024-05-04T03:25:45.101 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.101 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-1": { 2024-05-04T03:25:45.101 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 476, 2024-05-04T03:25:45.101 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 605, 2024-05-04T03:25:45.101 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 37868072, 2024-05-04T03:25:45.101 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 449263, 2024-05-04T03:25:45.101 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 6, 2024-05-04T03:25:45.101 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 5, 2024-05-04T03:25:45.101 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.067863365, 2024-05-04T03:25:45.101 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.016223648, 2024-05-04T03:25:45.101 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.032898279, 2024-05-04T03:25:45.102 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.014006871, 2024-05-04T03:25:45.102 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:45.102 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 603, 2024-05-04T03:25:45.102 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.023756907, 2024-05-04T03:25:45.102 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000039397 2024-05-04T03:25:45.102 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.102 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:45.102 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.102 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.102 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.102 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.103 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-04T03:25:45.103 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 298208 2024-05-04T03:25:45.103 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.103 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-2": { 2024-05-04T03:25:45.103 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 935, 2024-05-04T03:25:45.103 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 919, 2024-05-04T03:25:45.103 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 26269904, 2024-05-04T03:25:45.103 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 341350, 2024-05-04T03:25:45.103 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 5, 2024-05-04T03:25:45.103 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 4, 2024-05-04T03:25:45.103 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.122220135, 2024-05-04T03:25:45.103 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.034008600, 2024-05-04T03:25:45.103 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.056403148, 2024-05-04T03:25:45.103 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.030350264, 2024-05-04T03:25:45.104 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:45.104 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 918, 2024-05-04T03:25:45.104 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.042785588, 2024-05-04T03:25:45.104 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000046607 2024-05-04T03:25:45.104 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.104 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:45.104 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.104 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.104 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.104 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.104 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 862272, 2024-05-04T03:25:45.105 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 105712 2024-05-04T03:25:45.105 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.105 INFO:teuthology.orchestra.run.smithi072.stdout: "cct": { 2024-05-04T03:25:45.105 INFO:teuthology.orchestra.run.smithi072.stdout: "total_workers": 1, 2024-05-04T03:25:45.105 INFO:teuthology.orchestra.run.smithi072.stdout: "unhealthy_workers": 0 2024-05-04T03:25:45.105 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.105 INFO:teuthology.orchestra.run.smithi072.stdout: "finisher-MDSRank": { 2024-05-04T03:25:45.105 INFO:teuthology.orchestra.run.smithi072.stdout: "queue_len": 0, 2024-05-04T03:25:45.105 INFO:teuthology.orchestra.run.smithi072.stdout: "complete_latency": { 2024-05-04T03:25:45.105 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 3237, 2024-05-04T03:25:45.105 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.092320962, 2024-05-04T03:25:45.105 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000028520 2024-05-04T03:25:45.105 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:45.105 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.106 INFO:teuthology.orchestra.run.smithi072.stdout: "finisher-PurgeQueue": { 2024-05-04T03:25:45.106 INFO:teuthology.orchestra.run.smithi072.stdout: "queue_len": 0, 2024-05-04T03:25:45.106 INFO:teuthology.orchestra.run.smithi072.stdout: "complete_latency": { 2024-05-04T03:25:45.106 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.106 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.106 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.106 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:45.106 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.106 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": { 2024-05-04T03:25:45.106 INFO:teuthology.orchestra.run.smithi072.stdout: "request": 0, 2024-05-04T03:25:45.106 INFO:teuthology.orchestra.run.smithi072.stdout: "reply": 0, 2024-05-04T03:25:45.106 INFO:teuthology.orchestra.run.smithi072.stdout: "reply_latency": { 2024-05-04T03:25:45.107 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.107 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.107 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.107 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.107 INFO:teuthology.orchestra.run.smithi072.stdout: "slow_reply": 0, 2024-05-04T03:25:45.107 INFO:teuthology.orchestra.run.smithi072.stdout: "forward": 0, 2024-05-04T03:25:45.107 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_fetch_complete": 0, 2024-05-04T03:25:45.107 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_fetch_keys": 0, 2024-05-04T03:25:45.107 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_commit": 0, 2024-05-04T03:25:45.107 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_split": 0, 2024-05-04T03:25:45.107 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_merge": 0, 2024-05-04T03:25:45.107 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes": 5048, 2024-05-04T03:25:45.108 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_top": 0, 2024-05-04T03:25:45.108 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_bottom": 5048, 2024-05-04T03:25:45.108 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_pin_tail": 0, 2024-05-04T03:25:45.108 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_pinned": 5048, 2024-05-04T03:25:45.108 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_expired": 0, 2024-05-04T03:25:45.108 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_with_caps": 0, 2024-05-04T03:25:45.108 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": 0, 2024-05-04T03:25:45.108 INFO:teuthology.orchestra.run.smithi072.stdout: "subtrees": 1, 2024-05-04T03:25:45.108 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse": 0, 2024-05-04T03:25:45.108 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_hit": 0, 2024-05-04T03:25:45.108 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_forward": 0, 2024-05-04T03:25:45.108 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_discover": 0, 2024-05-04T03:25:45.108 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_dir_fetch": 0, 2024-05-04T03:25:45.108 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_remote_ino": 0, 2024-05-04T03:25:45.109 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_lock": 0, 2024-05-04T03:25:45.109 INFO:teuthology.orchestra.run.smithi072.stdout: "load_cent": 0, 2024-05-04T03:25:45.109 INFO:teuthology.orchestra.run.smithi072.stdout: "q": 0, 2024-05-04T03:25:45.109 INFO:teuthology.orchestra.run.smithi072.stdout: "exported": 0, 2024-05-04T03:25:45.109 INFO:teuthology.orchestra.run.smithi072.stdout: "exported_inodes": 0, 2024-05-04T03:25:45.109 INFO:teuthology.orchestra.run.smithi072.stdout: "imported": 0, 2024-05-04T03:25:45.109 INFO:teuthology.orchestra.run.smithi072.stdout: "imported_inodes": 0, 2024-05-04T03:25:45.109 INFO:teuthology.orchestra.run.smithi072.stdout: "openino_dir_fetch": 0, 2024-05-04T03:25:45.109 INFO:teuthology.orchestra.run.smithi072.stdout: "openino_backtrace_fetch": 0, 2024-05-04T03:25:45.109 INFO:teuthology.orchestra.run.smithi072.stdout: "openino_peer_discover": 0, 2024-05-04T03:25:45.109 INFO:teuthology.orchestra.run.smithi072.stdout: "root_rfiles": 0, 2024-05-04T03:25:45.110 INFO:teuthology.orchestra.run.smithi072.stdout: "root_rbytes": 0, 2024-05-04T03:25:45.110 INFO:teuthology.orchestra.run.smithi072.stdout: "root_rsnaps": 0, 2024-05-04T03:25:45.110 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_backtrace_fetch": 0, 2024-05-04T03:25:45.110 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_set_tag": 0, 2024-05-04T03:25:45.110 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_backtrace_repaired": 0, 2024-05-04T03:25:45.110 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_inotable_repaired": 0, 2024-05-04T03:25:45.110 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_dir_inodes": 0, 2024-05-04T03:25:45.110 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_dir_base_inodes": 0, 2024-05-04T03:25:45.110 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_dirfrag_rstats": 0, 2024-05-04T03:25:45.110 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_file_inodes": 0, 2024-05-04T03:25:45.110 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_inode_file_caps": 0, 2024-05-04T03:25:45.110 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_revoke": 0, 2024-05-04T03:25:45.110 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_grant": 0, 2024-05-04T03:25:45.111 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_trunc": 0, 2024-05-04T03:25:45.111 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-04T03:25:45.111 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-04T03:25:45.111 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_caps": 0, 2024-05-04T03:25:45.111 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_caps_dirty": 0, 2024-05-04T03:25:45.111 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_cap_release": 0, 2024-05-04T03:25:45.111 INFO:teuthology.orchestra.run.smithi072.stdout: "process_request_cap_release": 0 2024-05-04T03:25:45.111 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.111 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_cache": { 2024-05-04T03:25:45.111 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_update": 0, 2024-05-04T03:25:45.111 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_update_receipt": 0, 2024-05-04T03:25:45.111 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_try_discover": 0, 2024-05-04T03:25:45.111 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_send_discover": 0, 2024-05-04T03:25:45.112 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_handle_discover": 0, 2024-05-04T03:25:45.112 INFO:teuthology.orchestra.run.smithi072.stdout: "num_strays": 0, 2024-05-04T03:25:45.112 INFO:teuthology.orchestra.run.smithi072.stdout: "num_strays_delayed": 0, 2024-05-04T03:25:45.112 INFO:teuthology.orchestra.run.smithi072.stdout: "num_strays_enqueuing": 0, 2024-05-04T03:25:45.112 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_created": 0, 2024-05-04T03:25:45.112 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_enqueued": 0, 2024-05-04T03:25:45.112 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_reintegrated": 0, 2024-05-04T03:25:45.112 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_migrated": 0, 2024-05-04T03:25:45.112 INFO:teuthology.orchestra.run.smithi072.stdout: "num_recovering_processing": 0, 2024-05-04T03:25:45.112 INFO:teuthology.orchestra.run.smithi072.stdout: "num_recovering_enqueued": 0, 2024-05-04T03:25:45.112 INFO:teuthology.orchestra.run.smithi072.stdout: "num_recovering_prioritized": 0, 2024-05-04T03:25:45.112 INFO:teuthology.orchestra.run.smithi072.stdout: "recovery_started": 0, 2024-05-04T03:25:45.112 INFO:teuthology.orchestra.run.smithi072.stdout: "recovery_completed": 0, 2024-05-04T03:25:45.113 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_quiesce_path": 0, 2024-05-04T03:25:45.113 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_quiesce_inode": 0, 2024-05-04T03:25:45.113 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_enqueue_scrub": 0, 2024-05-04T03:25:45.113 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_exportdir": 0, 2024-05-04T03:25:45.113 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_flush": 0, 2024-05-04T03:25:45.113 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_fragmentdir": 0, 2024-05-04T03:25:45.113 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_fragstats": 0, 2024-05-04T03:25:45.113 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_inodestats": 0 2024-05-04T03:25:45.113 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.113 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_log": { 2024-05-04T03:25:45.113 INFO:teuthology.orchestra.run.smithi072.stdout: "evlrg": 0, 2024-05-04T03:25:45.113 INFO:teuthology.orchestra.run.smithi072.stdout: "evadd": 0, 2024-05-04T03:25:45.113 INFO:teuthology.orchestra.run.smithi072.stdout: "evex": 0, 2024-05-04T03:25:45.114 INFO:teuthology.orchestra.run.smithi072.stdout: "evtrm": 0, 2024-05-04T03:25:45.114 INFO:teuthology.orchestra.run.smithi072.stdout: "ev": 44989, 2024-05-04T03:25:45.114 INFO:teuthology.orchestra.run.smithi072.stdout: "evexg": 0, 2024-05-04T03:25:45.114 INFO:teuthology.orchestra.run.smithi072.stdout: "evexd": 0, 2024-05-04T03:25:45.114 INFO:teuthology.orchestra.run.smithi072.stdout: "segadd": 0, 2024-05-04T03:25:45.114 INFO:teuthology.orchestra.run.smithi072.stdout: "segex": 0, 2024-05-04T03:25:45.114 INFO:teuthology.orchestra.run.smithi072.stdout: "segtrm": 0, 2024-05-04T03:25:45.114 INFO:teuthology.orchestra.run.smithi072.stdout: "seg": 53, 2024-05-04T03:25:45.114 INFO:teuthology.orchestra.run.smithi072.stdout: "segmjr": 5, 2024-05-04T03:25:45.114 INFO:teuthology.orchestra.run.smithi072.stdout: "segexg": 0, 2024-05-04T03:25:45.114 INFO:teuthology.orchestra.run.smithi072.stdout: "segexd": 0, 2024-05-04T03:25:45.115 INFO:teuthology.orchestra.run.smithi072.stdout: "expos": 4194304, 2024-05-04T03:25:45.115 INFO:teuthology.orchestra.run.smithi072.stdout: "wrpos": 96978816, 2024-05-04T03:25:45.115 INFO:teuthology.orchestra.run.smithi072.stdout: "rdpos": 96978217, 2024-05-04T03:25:45.115 INFO:teuthology.orchestra.run.smithi072.stdout: "jlat": { 2024-05-04T03:25:45.115 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.115 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.115 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.115 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.115 INFO:teuthology.orchestra.run.smithi072.stdout: "replayed": 44989 2024-05-04T03:25:45.115 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.115 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_mem": { 2024-05-04T03:25:45.116 INFO:teuthology.orchestra.run.smithi072.stdout: "ino": 13, 2024-05-04T03:25:45.116 INFO:teuthology.orchestra.run.smithi072.stdout: "ino+": 5129, 2024-05-04T03:25:45.116 INFO:teuthology.orchestra.run.smithi072.stdout: "ino-": 5116, 2024-05-04T03:25:45.116 INFO:teuthology.orchestra.run.smithi072.stdout: "dir": 11, 2024-05-04T03:25:45.116 INFO:teuthology.orchestra.run.smithi072.stdout: "dir+": 278, 2024-05-04T03:25:45.116 INFO:teuthology.orchestra.run.smithi072.stdout: "dir-": 267, 2024-05-04T03:25:45.116 INFO:teuthology.orchestra.run.smithi072.stdout: "dn": 5048, 2024-05-04T03:25:45.116 INFO:teuthology.orchestra.run.smithi072.stdout: "dn+": 9959, 2024-05-04T03:25:45.116 INFO:teuthology.orchestra.run.smithi072.stdout: "dn-": 4911, 2024-05-04T03:25:45.116 INFO:teuthology.orchestra.run.smithi072.stdout: "cap": 0, 2024-05-04T03:25:45.116 INFO:teuthology.orchestra.run.smithi072.stdout: "cap+": 0, 2024-05-04T03:25:45.116 INFO:teuthology.orchestra.run.smithi072.stdout: "cap-": 0, 2024-05-04T03:25:45.116 INFO:teuthology.orchestra.run.smithi072.stdout: "rss": 138616, 2024-05-04T03:25:45.117 INFO:teuthology.orchestra.run.smithi072.stdout: "heap": 215440 2024-05-04T03:25:45.117 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.117 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_server": { 2024-05-04T03:25:45.117 INFO:teuthology.orchestra.run.smithi072.stdout: "dispatch_client_request": 0, 2024-05-04T03:25:45.117 INFO:teuthology.orchestra.run.smithi072.stdout: "dispatch_server_request": 0, 2024-05-04T03:25:45.117 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_request": 0, 2024-05-04T03:25:45.117 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_session": 0, 2024-05-04T03:25:45.117 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_peer_request": 0, 2024-05-04T03:25:45.117 INFO:teuthology.orchestra.run.smithi072.stdout: "req_create_latency": { 2024-05-04T03:25:45.117 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.117 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.117 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.118 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.118 INFO:teuthology.orchestra.run.smithi072.stdout: "req_getattr_latency": { 2024-05-04T03:25:45.118 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.118 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.118 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.118 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.118 INFO:teuthology.orchestra.run.smithi072.stdout: "req_getfilelock_latency": { 2024-05-04T03:25:45.118 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.118 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.118 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.118 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.118 INFO:teuthology.orchestra.run.smithi072.stdout: "req_link_latency": { 2024-05-04T03:25:45.119 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.119 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.119 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.119 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.119 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookup_latency": { 2024-05-04T03:25:45.119 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.119 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.119 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.119 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.119 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookuphash_latency": { 2024-05-04T03:25:45.119 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.120 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.120 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.120 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.120 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupino_latency": { 2024-05-04T03:25:45.120 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.120 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.120 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.120 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.120 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupname_latency": { 2024-05-04T03:25:45.120 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.120 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.120 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.121 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.121 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupparent_latency": { 2024-05-04T03:25:45.121 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.121 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.121 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.121 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.121 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupsnap_latency": { 2024-05-04T03:25:45.121 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.121 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.121 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.121 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.122 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lssnap_latency": { 2024-05-04T03:25:45.122 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.122 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.122 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.122 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.122 INFO:teuthology.orchestra.run.smithi072.stdout: "req_mkdir_latency": { 2024-05-04T03:25:45.123 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.123 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.123 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.123 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.123 INFO:teuthology.orchestra.run.smithi072.stdout: "req_mknod_latency": { 2024-05-04T03:25:45.123 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.123 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.123 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.123 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.123 INFO:teuthology.orchestra.run.smithi072.stdout: "req_mksnap_latency": { 2024-05-04T03:25:45.124 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.124 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.124 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.124 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.124 INFO:teuthology.orchestra.run.smithi072.stdout: "req_open_latency": { 2024-05-04T03:25:45.124 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.124 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.124 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.124 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.124 INFO:teuthology.orchestra.run.smithi072.stdout: "req_readdir_latency": { 2024-05-04T03:25:45.124 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.125 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.125 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.125 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.125 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rename_latency": { 2024-05-04T03:25:45.125 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.125 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.125 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.125 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.125 INFO:teuthology.orchestra.run.smithi072.stdout: "req_renamesnap_latency": { 2024-05-04T03:25:45.125 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.125 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.125 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.126 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.126 INFO:teuthology.orchestra.run.smithi072.stdout: "req_snapdiff_latency": { 2024-05-04T03:25:45.126 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.126 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.126 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.126 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.126 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rmdir_latency": { 2024-05-04T03:25:45.126 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.126 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.126 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.126 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.126 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rmsnap_latency": { 2024-05-04T03:25:45.127 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.127 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.127 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.127 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.127 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rmxattr_latency": { 2024-05-04T03:25:45.127 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.127 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.127 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.127 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.127 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setattr_latency": { 2024-05-04T03:25:45.127 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.127 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.128 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.128 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.128 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setdirlayout_latency": { 2024-05-04T03:25:45.128 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.128 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.128 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.128 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.128 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setfilelock_latency": { 2024-05-04T03:25:45.128 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.128 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.128 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.128 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.129 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setlayout_latency": { 2024-05-04T03:25:45.129 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.129 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.129 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.129 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.129 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setxattr_latency": { 2024-05-04T03:25:45.129 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.129 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.129 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.129 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.129 INFO:teuthology.orchestra.run.smithi072.stdout: "req_symlink_latency": { 2024-05-04T03:25:45.129 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.130 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.130 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.130 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.130 INFO:teuthology.orchestra.run.smithi072.stdout: "req_unlink_latency": { 2024-05-04T03:25:45.130 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.130 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.130 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.130 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.130 INFO:teuthology.orchestra.run.smithi072.stdout: "cap_revoke_eviction": 0, 2024-05-04T03:25:45.130 INFO:teuthology.orchestra.run.smithi072.stdout: "cap_acquisition_throttle": 0, 2024-05-04T03:25:45.130 INFO:teuthology.orchestra.run.smithi072.stdout: "req_getvxattr_latency": { 2024-05-04T03:25:45.131 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.131 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.131 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.131 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:45.131 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.131 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_sessions": { 2024-05-04T03:25:45.131 INFO:teuthology.orchestra.run.smithi072.stdout: "session_count": 1, 2024-05-04T03:25:45.131 INFO:teuthology.orchestra.run.smithi072.stdout: "session_add": 3, 2024-05-04T03:25:45.131 INFO:teuthology.orchestra.run.smithi072.stdout: "session_remove": 2, 2024-05-04T03:25:45.131 INFO:teuthology.orchestra.run.smithi072.stdout: "sessions_open": 1, 2024-05-04T03:25:45.131 INFO:teuthology.orchestra.run.smithi072.stdout: "sessions_stale": 0, 2024-05-04T03:25:45.131 INFO:teuthology.orchestra.run.smithi072.stdout: "total_load": 0, 2024-05-04T03:25:45.132 INFO:teuthology.orchestra.run.smithi072.stdout: "average_load": 0, 2024-05-04T03:25:45.132 INFO:teuthology.orchestra.run.smithi072.stdout: "avg_session_uptime": 1649, 2024-05-04T03:25:45.132 INFO:teuthology.orchestra.run.smithi072.stdout: "mdthresh_evicted": 0 2024-05-04T03:25:45.132 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.132 INFO:teuthology.orchestra.run.smithi072.stdout: "mempool": { 2024-05-04T03:25:45.132 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_bytes": 0, 2024-05-04T03:25:45.132 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_items": 0, 2024-05-04T03:25:45.132 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_bytes": 0, 2024-05-04T03:25:45.132 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_items": 0, 2024-05-04T03:25:45.132 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_bytes": 0, 2024-05-04T03:25:45.132 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_items": 0, 2024-05-04T03:25:45.132 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-04T03:25:45.133 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_items": 0, 2024-05-04T03:25:45.133 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-04T03:25:45.133 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_items": 0, 2024-05-04T03:25:45.133 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_bytes": 0, 2024-05-04T03:25:45.133 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_items": 0, 2024-05-04T03:25:45.133 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-04T03:25:45.133 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_items": 0, 2024-05-04T03:25:45.133 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_bytes": 0, 2024-05-04T03:25:45.133 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_items": 0, 2024-05-04T03:25:45.133 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_bytes": 0, 2024-05-04T03:25:45.133 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_items": 0, 2024-05-04T03:25:45.133 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-04T03:25:45.134 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_items": 0, 2024-05-04T03:25:45.134 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-04T03:25:45.134 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_items": 0, 2024-05-04T03:25:45.134 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_bytes": 0, 2024-05-04T03:25:45.134 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_items": 0, 2024-05-04T03:25:45.134 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_bytes": 0, 2024-05-04T03:25:45.134 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_items": 0, 2024-05-04T03:25:45.134 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-04T03:25:45.134 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_items": 0, 2024-05-04T03:25:45.134 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_bytes": 0, 2024-05-04T03:25:45.134 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_items": 0, 2024-05-04T03:25:45.135 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_bytes": 0, 2024-05-04T03:25:45.135 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_items": 0, 2024-05-04T03:25:45.135 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_bytes": 0, 2024-05-04T03:25:45.135 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_items": 0, 2024-05-04T03:25:45.135 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_bytes": 0, 2024-05-04T03:25:45.135 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_items": 0, 2024-05-04T03:25:45.135 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_bytes": 223288, 2024-05-04T03:25:45.135 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_items": 71, 2024-05-04T03:25:45.135 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_bytes": 0, 2024-05-04T03:25:45.135 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_items": 0, 2024-05-04T03:25:45.135 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_bytes": 0, 2024-05-04T03:25:45.135 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_items": 0, 2024-05-04T03:25:45.135 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_bytes": 0, 2024-05-04T03:25:45.136 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_items": 0, 2024-05-04T03:25:45.136 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_bytes": 0, 2024-05-04T03:25:45.136 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_items": 0, 2024-05-04T03:25:45.136 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_bytes": 7240, 2024-05-04T03:25:45.136 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_items": 152, 2024-05-04T03:25:45.136 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_bytes": 0, 2024-05-04T03:25:45.136 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_items": 0, 2024-05-04T03:25:45.136 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_bytes": 0, 2024-05-04T03:25:45.136 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_items": 0, 2024-05-04T03:25:45.136 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_bytes": 3597384, 2024-05-04T03:25:45.136 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_items": 20320, 2024-05-04T03:25:45.136 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_bytes": 0, 2024-05-04T03:25:45.137 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_items": 0, 2024-05-04T03:25:45.137 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_bytes": 0, 2024-05-04T03:25:45.137 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_items": 0 2024-05-04T03:25:45.137 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.137 INFO:teuthology.orchestra.run.smithi072.stdout: "objecter": { 2024-05-04T03:25:45.137 INFO:teuthology.orchestra.run.smithi072.stdout: "op_active": 0, 2024-05-04T03:25:45.137 INFO:teuthology.orchestra.run.smithi072.stdout: "op_laggy": 0, 2024-05-04T03:25:45.137 INFO:teuthology.orchestra.run.smithi072.stdout: "op_send": 1925, 2024-05-04T03:25:45.137 INFO:teuthology.orchestra.run.smithi072.stdout: "op_send_bytes": 21, 2024-05-04T03:25:45.137 INFO:teuthology.orchestra.run.smithi072.stdout: "op_resend": 3, 2024-05-04T03:25:45.137 INFO:teuthology.orchestra.run.smithi072.stdout: "op_reply": 1925, 2024-05-04T03:25:45.137 INFO:teuthology.orchestra.run.smithi072.stdout: "op_latency": { 2024-05-04T03:25:45.137 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 1925, 2024-05-04T03:25:45.138 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 1.216973711, 2024-05-04T03:25:45.138 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000632194 2024-05-04T03:25:45.138 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.138 INFO:teuthology.orchestra.run.smithi072.stdout: "op_inflight": 0, 2024-05-04T03:25:45.138 INFO:teuthology.orchestra.run.smithi072.stdout: "oplen_avg": { 2024-05-04T03:25:45.138 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 1925, 2024-05-04T03:25:45.138 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 1926 2024-05-04T03:25:45.138 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.138 INFO:teuthology.orchestra.run.smithi072.stdout: "op": 1925, 2024-05-04T03:25:45.138 INFO:teuthology.orchestra.run.smithi072.stdout: "op_r": 1925, 2024-05-04T03:25:45.138 INFO:teuthology.orchestra.run.smithi072.stdout: "op_w": 0, 2024-05-04T03:25:45.139 INFO:teuthology.orchestra.run.smithi072.stdout: "op_rmw": 0, 2024-05-04T03:25:45.139 INFO:teuthology.orchestra.run.smithi072.stdout: "op_pg": 0, 2024-05-04T03:25:45.139 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_stat": 1126, 2024-05-04T03:25:45.139 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_create": 0, 2024-05-04T03:25:45.139 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_read": 797, 2024-05-04T03:25:45.139 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_write": 0, 2024-05-04T03:25:45.139 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_writefull": 0, 2024-05-04T03:25:45.139 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_writesame": 0, 2024-05-04T03:25:45.139 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_append": 0, 2024-05-04T03:25:45.139 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_zero": 0, 2024-05-04T03:25:45.139 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_truncate": 0, 2024-05-04T03:25:45.139 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_delete": 0, 2024-05-04T03:25:45.139 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_mapext": 0, 2024-05-04T03:25:45.140 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_sparse_read": 0, 2024-05-04T03:25:45.140 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_clonerange": 0, 2024-05-04T03:25:45.140 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_getxattr": 1, 2024-05-04T03:25:45.140 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_setxattr": 0, 2024-05-04T03:25:45.140 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_cmpxattr": 0, 2024-05-04T03:25:45.140 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_rmxattr": 0, 2024-05-04T03:25:45.140 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_resetxattrs": 0, 2024-05-04T03:25:45.140 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_call": 0, 2024-05-04T03:25:45.140 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_watch": 0, 2024-05-04T03:25:45.140 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_notify": 0, 2024-05-04T03:25:45.140 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_src_cmpxattr": 0, 2024-05-04T03:25:45.140 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_pgls": 0, 2024-05-04T03:25:45.141 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_pgls_filter": 0, 2024-05-04T03:25:45.141 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_other": 0, 2024-05-04T03:25:45.141 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_active": 0, 2024-05-04T03:25:45.141 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_send": 0, 2024-05-04T03:25:45.141 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_resend": 0, 2024-05-04T03:25:45.141 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_ping": 0, 2024-05-04T03:25:45.141 INFO:teuthology.orchestra.run.smithi072.stdout: "poolop_active": 0, 2024-05-04T03:25:45.141 INFO:teuthology.orchestra.run.smithi072.stdout: "poolop_send": 0, 2024-05-04T03:25:45.141 INFO:teuthology.orchestra.run.smithi072.stdout: "poolop_resend": 0, 2024-05-04T03:25:45.141 INFO:teuthology.orchestra.run.smithi072.stdout: "poolstat_active": 0, 2024-05-04T03:25:45.141 INFO:teuthology.orchestra.run.smithi072.stdout: "poolstat_send": 0, 2024-05-04T03:25:45.141 INFO:teuthology.orchestra.run.smithi072.stdout: "poolstat_resend": 0, 2024-05-04T03:25:45.141 INFO:teuthology.orchestra.run.smithi072.stdout: "statfs_active": 0, 2024-05-04T03:25:45.142 INFO:teuthology.orchestra.run.smithi072.stdout: "statfs_send": 0, 2024-05-04T03:25:45.142 INFO:teuthology.orchestra.run.smithi072.stdout: "statfs_resend": 0, 2024-05-04T03:25:45.142 INFO:teuthology.orchestra.run.smithi072.stdout: "command_active": 0, 2024-05-04T03:25:45.142 INFO:teuthology.orchestra.run.smithi072.stdout: "command_send": 0, 2024-05-04T03:25:45.142 INFO:teuthology.orchestra.run.smithi072.stdout: "command_resend": 0, 2024-05-04T03:25:45.142 INFO:teuthology.orchestra.run.smithi072.stdout: "map_epoch": 83, 2024-05-04T03:25:45.142 INFO:teuthology.orchestra.run.smithi072.stdout: "map_full": 0, 2024-05-04T03:25:45.142 INFO:teuthology.orchestra.run.smithi072.stdout: "map_inc": 2, 2024-05-04T03:25:45.142 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_sessions": 9, 2024-05-04T03:25:45.142 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_session_open": 9, 2024-05-04T03:25:45.142 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_session_close": 0, 2024-05-04T03:25:45.142 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_laggy": 0, 2024-05-04T03:25:45.143 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_wr": 0, 2024-05-04T03:25:45.143 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_rd": 2, 2024-05-04T03:25:45.143 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_del": 0 2024-05-04T03:25:45.143 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.143 INFO:teuthology.orchestra.run.smithi072.stdout: "oft": { 2024-05-04T03:25:45.143 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_objs": 0, 2024-05-04T03:25:45.143 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_kv_pairs": 0, 2024-05-04T03:25:45.143 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_updates": 0, 2024-05-04T03:25:45.143 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_removes": 0 2024-05-04T03:25:45.143 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.143 INFO:teuthology.orchestra.run.smithi072.stdout: "osd-slow-ops": { 2024-05-04T03:25:45.143 INFO:teuthology.orchestra.run.smithi072.stdout: "slow_ops_count": 0 2024-05-04T03:25:45.143 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.144 INFO:teuthology.orchestra.run.smithi072.stdout: "purge_queue": { 2024-05-04T03:25:45.144 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing_ops": 0, 2024-05-04T03:25:45.144 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing_ops_high_water": 0, 2024-05-04T03:25:45.144 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing": 0, 2024-05-04T03:25:45.144 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing_high_water": 0, 2024-05-04T03:25:45.144 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executed_ops": 0, 2024-05-04T03:25:45.144 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executed": 0, 2024-05-04T03:25:45.144 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_item_in_journal": 0 2024-05-04T03:25:45.144 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.144 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-04T03:25:45.144 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-04T03:25:45.144 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-05-04T03:25:45.145 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-04T03:25:45.145 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 2120, 2024-05-04T03:25:45.145 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 93990311, 2024-05-04T03:25:45.145 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:45.145 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 2120, 2024-05-04T03:25:45.145 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-05-04T03:25:45.145 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-05-04T03:25:45.145 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 2120, 2024-05-04T03:25:45.145 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 93990311, 2024-05-04T03:25:45.145 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-04T03:25:45.145 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.145 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.145 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.146 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:45.146 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.146 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-objecter_bytes": { 2024-05-04T03:25:45.146 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-04T03:25:45.146 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-05-04T03:25:45.146 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-04T03:25:45.146 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 0, 2024-05-04T03:25:45.146 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 0, 2024-05-04T03:25:45.146 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:45.146 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 0, 2024-05-04T03:25:45.146 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 1925, 2024-05-04T03:25:45.146 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 92790801, 2024-05-04T03:25:45.147 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 1925, 2024-05-04T03:25:45.147 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 92790801, 2024-05-04T03:25:45.147 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-04T03:25:45.147 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.147 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.147 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.147 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:45.147 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.147 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-objecter_ops": { 2024-05-04T03:25:45.147 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-04T03:25:45.147 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 1024, 2024-05-04T03:25:45.147 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-04T03:25:45.147 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 0, 2024-05-04T03:25:45.148 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 0, 2024-05-04T03:25:45.148 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:45.148 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 0, 2024-05-04T03:25:45.148 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 1925, 2024-05-04T03:25:45.148 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 1925, 2024-05-04T03:25:45.148 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 1925, 2024-05-04T03:25:45.148 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 1925, 2024-05-04T03:25:45.148 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-04T03:25:45.148 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.148 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.148 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.148 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:45.149 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.149 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-write_buf_throttle": { 2024-05-04T03:25:45.149 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-04T03:25:45.149 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 3758096384, 2024-05-04T03:25:45.149 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-04T03:25:45.149 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 0, 2024-05-04T03:25:45.149 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 0, 2024-05-04T03:25:45.149 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:45.149 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 0, 2024-05-04T03:25:45.149 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-05-04T03:25:45.149 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-05-04T03:25:45.149 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 0, 2024-05-04T03:25:45.150 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 0, 2024-05-04T03:25:45.150 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-04T03:25:45.150 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.150 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.150 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.150 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:45.150 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:45.150 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-write_buf_throttle-0x5601c29c68f0": { 2024-05-04T03:25:45.150 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-04T03:25:45.150 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 3758096384, 2024-05-04T03:25:45.150 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-04T03:25:45.151 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 0, 2024-05-04T03:25:45.151 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 0, 2024-05-04T03:25:45.151 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:45.151 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 0, 2024-05-04T03:25:45.151 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-05-04T03:25:45.151 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-05-04T03:25:45.151 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 0, 2024-05-04T03:25:45.151 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 0, 2024-05-04T03:25:45.151 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-04T03:25:45.151 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:45.151 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:45.151 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:45.152 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:45.152 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:45.152 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-05-04T03:25:45.958 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-04T03:25:45.958 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-04T03:25:45.958 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-05-04T03:25:45.958 DEBUG:tasks.check_counter:Getting stats from h 2024-05-04T03:25:45.958 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-04T03:25:46.200 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.b/config 2024-05-04T03:25:46.592 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:46 smithi072 ceph-mon[33332]: pgmap v589: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-04T03:25:46.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:46 smithi001 ceph-mon[29731]: pgmap v589: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-04T03:25:46.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:46 smithi077 ceph-mon[33531]: pgmap v589: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-04T03:25:47.934 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-05-04T03:25:47.934 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-0": { 2024-05-04T03:25:47.934 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 58110, 2024-05-04T03:25:47.934 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 110822, 2024-05-04T03:25:47.934 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 16556614, 2024-05-04T03:25:47.934 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 78028849, 2024-05-04T03:25:47.935 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 16, 2024-05-04T03:25:47.935 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 4, 2024-05-04T03:25:47.935 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 3.789234545, 2024-05-04T03:25:47.935 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 1.697010315, 2024-05-04T03:25:47.935 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 2.447335299, 2024-05-04T03:25:47.935 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.598065408, 2024-05-04T03:25:47.935 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:47.935 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 110820, 2024-05-04T03:25:47.935 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 3.085690136, 2024-05-04T03:25:47.935 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000027844 2024-05-04T03:25:47.935 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.935 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:47.935 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 79043, 2024-05-04T03:25:47.935 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.017550988, 2024-05-04T03:25:47.935 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000222 2024-05-04T03:25:47.936 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.936 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 4960, 2024-05-04T03:25:47.936 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 5968 2024-05-04T03:25:47.936 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.936 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-1": { 2024-05-04T03:25:47.936 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 35979, 2024-05-04T03:25:47.936 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 36252, 2024-05-04T03:25:47.936 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 9050117, 2024-05-04T03:25:47.936 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 48234579, 2024-05-04T03:25:47.936 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 10, 2024-05-04T03:25:47.936 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 6, 2024-05-04T03:25:47.936 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 2.520683388, 2024-05-04T03:25:47.936 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.606567467, 2024-05-04T03:25:47.936 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.561855143, 2024-05-04T03:25:47.937 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 1.211456205, 2024-05-04T03:25:47.937 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:47.937 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 36250, 2024-05-04T03:25:47.937 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 1.034242243, 2024-05-04T03:25:47.937 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000028530 2024-05-04T03:25:47.937 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.937 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:47.937 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 601, 2024-05-04T03:25:47.937 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000100924, 2024-05-04T03:25:47.937 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000167 2024-05-04T03:25:47.937 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.937 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-04T03:25:47.937 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 299648 2024-05-04T03:25:47.937 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.938 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-2": { 2024-05-04T03:25:47.938 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 13415, 2024-05-04T03:25:47.938 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 13577, 2024-05-04T03:25:47.938 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 4813511, 2024-05-04T03:25:47.938 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 17336073, 2024-05-04T03:25:47.938 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 7, 2024-05-04T03:25:47.938 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 6, 2024-05-04T03:25:47.938 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 1.148650536, 2024-05-04T03:25:47.938 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.391789920, 2024-05-04T03:25:47.938 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.294146086, 2024-05-04T03:25:47.938 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.425290521, 2024-05-04T03:25:47.938 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:47.938 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 13577, 2024-05-04T03:25:47.938 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.583372625, 2024-05-04T03:25:47.939 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000042967 2024-05-04T03:25:47.939 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.939 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:47.939 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 2135, 2024-05-04T03:25:47.939 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000454617, 2024-05-04T03:25:47.939 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000212 2024-05-04T03:25:47.939 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.939 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 877872, 2024-05-04T03:25:47.939 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 111824 2024-05-04T03:25:47.939 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.939 INFO:teuthology.orchestra.run.smithi072.stdout: "cct": { 2024-05-04T03:25:47.939 INFO:teuthology.orchestra.run.smithi072.stdout: "total_workers": 1, 2024-05-04T03:25:47.939 INFO:teuthology.orchestra.run.smithi072.stdout: "unhealthy_workers": 0 2024-05-04T03:25:47.939 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.940 INFO:teuthology.orchestra.run.smithi072.stdout: "finisher-MDSRank": { 2024-05-04T03:25:47.940 INFO:teuthology.orchestra.run.smithi072.stdout: "queue_len": 0, 2024-05-04T03:25:47.940 INFO:teuthology.orchestra.run.smithi072.stdout: "complete_latency": { 2024-05-04T03:25:47.940 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 86485, 2024-05-04T03:25:47.940 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 28.150851208, 2024-05-04T03:25:47.940 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000325499 2024-05-04T03:25:47.940 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:47.940 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.940 INFO:teuthology.orchestra.run.smithi072.stdout: "finisher-PurgeQueue": { 2024-05-04T03:25:47.940 INFO:teuthology.orchestra.run.smithi072.stdout: "queue_len": 0, 2024-05-04T03:25:47.940 INFO:teuthology.orchestra.run.smithi072.stdout: "complete_latency": { 2024-05-04T03:25:47.940 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 7857, 2024-05-04T03:25:47.940 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 2.263711436, 2024-05-04T03:25:47.940 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000288113 2024-05-04T03:25:47.941 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:47.941 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.941 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": { 2024-05-04T03:25:47.941 INFO:teuthology.orchestra.run.smithi072.stdout: "request": 20100, 2024-05-04T03:25:47.941 INFO:teuthology.orchestra.run.smithi072.stdout: "reply": 20039, 2024-05-04T03:25:47.941 INFO:teuthology.orchestra.run.smithi072.stdout: "reply_latency": { 2024-05-04T03:25:47.941 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 20039, 2024-05-04T03:25:47.941 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 99.558660449, 2024-05-04T03:25:47.941 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.004968244 2024-05-04T03:25:47.941 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.941 INFO:teuthology.orchestra.run.smithi072.stdout: "slow_reply": 0, 2024-05-04T03:25:47.941 INFO:teuthology.orchestra.run.smithi072.stdout: "forward": 61, 2024-05-04T03:25:47.941 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_fetch_complete": 0, 2024-05-04T03:25:47.941 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_fetch_keys": 0, 2024-05-04T03:25:47.942 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_commit": 211, 2024-05-04T03:25:47.942 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_split": 24, 2024-05-04T03:25:47.942 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_merge": 2, 2024-05-04T03:25:47.942 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes": 307, 2024-05-04T03:25:47.942 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_top": 204, 2024-05-04T03:25:47.942 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_bottom": 91, 2024-05-04T03:25:47.942 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_pin_tail": 12, 2024-05-04T03:25:47.942 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_pinned": 15, 2024-05-04T03:25:47.942 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_expired": 89, 2024-05-04T03:25:47.942 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_with_caps": 0, 2024-05-04T03:25:47.942 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": 0, 2024-05-04T03:25:47.942 INFO:teuthology.orchestra.run.smithi072.stdout: "subtrees": 3, 2024-05-04T03:25:47.942 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse": 21193, 2024-05-04T03:25:47.942 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_hit": 20823, 2024-05-04T03:25:47.943 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_forward": 56, 2024-05-04T03:25:47.943 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_discover": 3, 2024-05-04T03:25:47.943 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_dir_fetch": 0, 2024-05-04T03:25:47.943 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_remote_ino": 0, 2024-05-04T03:25:47.943 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_lock": 1, 2024-05-04T03:25:47.943 INFO:teuthology.orchestra.run.smithi072.stdout: "load_cent": 0, 2024-05-04T03:25:47.943 INFO:teuthology.orchestra.run.smithi072.stdout: "q": 0, 2024-05-04T03:25:47.943 INFO:teuthology.orchestra.run.smithi072.stdout: "exported": 5, 2024-05-04T03:25:47.943 INFO:teuthology.orchestra.run.smithi072.stdout: "exported_inodes": 79, 2024-05-04T03:25:47.943 INFO:teuthology.orchestra.run.smithi072.stdout: "imported": 5, 2024-05-04T03:25:47.943 INFO:teuthology.orchestra.run.smithi072.stdout: "imported_inodes": 298, 2024-05-04T03:25:47.943 INFO:teuthology.orchestra.run.smithi072.stdout: "openino_dir_fetch": 0, 2024-05-04T03:25:47.943 INFO:teuthology.orchestra.run.smithi072.stdout: "openino_backtrace_fetch": 0, 2024-05-04T03:25:47.943 INFO:teuthology.orchestra.run.smithi072.stdout: "openino_peer_discover": 0, 2024-05-04T03:25:47.943 INFO:teuthology.orchestra.run.smithi072.stdout: "root_rfiles": 4, 2024-05-04T03:25:47.944 INFO:teuthology.orchestra.run.smithi072.stdout: "root_rbytes": 234, 2024-05-04T03:25:47.944 INFO:teuthology.orchestra.run.smithi072.stdout: "root_rsnaps": 0, 2024-05-04T03:25:47.944 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_backtrace_fetch": 0, 2024-05-04T03:25:47.944 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_set_tag": 0, 2024-05-04T03:25:47.944 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_backtrace_repaired": 0, 2024-05-04T03:25:47.944 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_inotable_repaired": 0, 2024-05-04T03:25:47.944 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_dir_inodes": 0, 2024-05-04T03:25:47.944 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_dir_base_inodes": 0, 2024-05-04T03:25:47.944 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_dirfrag_rstats": 0, 2024-05-04T03:25:47.944 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_file_inodes": 0, 2024-05-04T03:25:47.944 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_inode_file_caps": 18, 2024-05-04T03:25:47.944 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_revoke": 13003, 2024-05-04T03:25:47.944 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_grant": 19030, 2024-05-04T03:25:47.944 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_trunc": 7861, 2024-05-04T03:25:47.945 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-04T03:25:47.945 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-04T03:25:47.945 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_caps": 18455, 2024-05-04T03:25:47.945 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_caps_dirty": 12222, 2024-05-04T03:25:47.945 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_cap_release": 374, 2024-05-04T03:25:47.945 INFO:teuthology.orchestra.run.smithi072.stdout: "process_request_cap_release": 14697 2024-05-04T03:25:47.945 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.945 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_cache": { 2024-05-04T03:25:47.945 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_update": 0, 2024-05-04T03:25:47.945 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_update_receipt": 0, 2024-05-04T03:25:47.945 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_try_discover": 0, 2024-05-04T03:25:47.945 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_send_discover": 4, 2024-05-04T03:25:47.945 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_handle_discover": 5, 2024-05-04T03:25:47.945 INFO:teuthology.orchestra.run.smithi072.stdout: "num_strays": 0, 2024-05-04T03:25:47.945 INFO:teuthology.orchestra.run.smithi072.stdout: "num_strays_delayed": 0, 2024-05-04T03:25:47.946 INFO:teuthology.orchestra.run.smithi072.stdout: "num_strays_enqueuing": 0, 2024-05-04T03:25:47.946 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_created": 5041, 2024-05-04T03:25:47.946 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_enqueued": 5038, 2024-05-04T03:25:47.946 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_reintegrated": 3, 2024-05-04T03:25:47.946 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_migrated": 0, 2024-05-04T03:25:47.946 INFO:teuthology.orchestra.run.smithi072.stdout: "num_recovering_processing": 0, 2024-05-04T03:25:47.946 INFO:teuthology.orchestra.run.smithi072.stdout: "num_recovering_enqueued": 0, 2024-05-04T03:25:47.946 INFO:teuthology.orchestra.run.smithi072.stdout: "num_recovering_prioritized": 0, 2024-05-04T03:25:47.946 INFO:teuthology.orchestra.run.smithi072.stdout: "recovery_started": 0, 2024-05-04T03:25:47.946 INFO:teuthology.orchestra.run.smithi072.stdout: "recovery_completed": 0, 2024-05-04T03:25:47.946 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_quiesce_path": 4, 2024-05-04T03:25:47.947 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_quiesce_inode": 9035, 2024-05-04T03:25:47.947 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_enqueue_scrub": 0, 2024-05-04T03:25:47.947 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_exportdir": 5, 2024-05-04T03:25:47.947 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_flush": 0, 2024-05-04T03:25:47.947 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_fragmentdir": 26, 2024-05-04T03:25:47.947 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_fragstats": 0, 2024-05-04T03:25:47.947 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_inodestats": 0 2024-05-04T03:25:47.947 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.947 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_log": { 2024-05-04T03:25:47.947 INFO:teuthology.orchestra.run.smithi072.stdout: "evlrg": 0, 2024-05-04T03:25:47.948 INFO:teuthology.orchestra.run.smithi072.stdout: "evadd": 44989, 2024-05-04T03:25:47.948 INFO:teuthology.orchestra.run.smithi072.stdout: "evex": 0, 2024-05-04T03:25:47.948 INFO:teuthology.orchestra.run.smithi072.stdout: "evtrm": 0, 2024-05-04T03:25:47.948 INFO:teuthology.orchestra.run.smithi072.stdout: "ev": 44989, 2024-05-04T03:25:47.948 INFO:teuthology.orchestra.run.smithi072.stdout: "evexg": 0, 2024-05-04T03:25:47.948 INFO:teuthology.orchestra.run.smithi072.stdout: "evexd": 0, 2024-05-04T03:25:47.948 INFO:teuthology.orchestra.run.smithi072.stdout: "segadd": 53, 2024-05-04T03:25:47.948 INFO:teuthology.orchestra.run.smithi072.stdout: "segex": 0, 2024-05-04T03:25:47.948 INFO:teuthology.orchestra.run.smithi072.stdout: "segtrm": 0, 2024-05-04T03:25:47.948 INFO:teuthology.orchestra.run.smithi072.stdout: "seg": 53, 2024-05-04T03:25:47.948 INFO:teuthology.orchestra.run.smithi072.stdout: "segmjr": 5, 2024-05-04T03:25:47.948 INFO:teuthology.orchestra.run.smithi072.stdout: "segexg": 0, 2024-05-04T03:25:47.948 INFO:teuthology.orchestra.run.smithi072.stdout: "segexd": 0, 2024-05-04T03:25:47.949 INFO:teuthology.orchestra.run.smithi072.stdout: "expos": 4194304, 2024-05-04T03:25:47.949 INFO:teuthology.orchestra.run.smithi072.stdout: "wrpos": 96978816, 2024-05-04T03:25:47.949 INFO:teuthology.orchestra.run.smithi072.stdout: "rdpos": 0, 2024-05-04T03:25:47.949 INFO:teuthology.orchestra.run.smithi072.stdout: "jlat": { 2024-05-04T03:25:47.949 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 24695, 2024-05-04T03:25:47.949 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 51.830090057, 2024-05-04T03:25:47.949 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.002098809 2024-05-04T03:25:47.949 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.949 INFO:teuthology.orchestra.run.smithi072.stdout: "replayed": 0 2024-05-04T03:25:47.949 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.949 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_mem": { 2024-05-04T03:25:47.949 INFO:teuthology.orchestra.run.smithi072.stdout: "ino": 18, 2024-05-04T03:25:47.949 INFO:teuthology.orchestra.run.smithi072.stdout: "ino+": 5129, 2024-05-04T03:25:47.949 INFO:teuthology.orchestra.run.smithi072.stdout: "ino-": 5111, 2024-05-04T03:25:47.949 INFO:teuthology.orchestra.run.smithi072.stdout: "dir": 17, 2024-05-04T03:25:47.950 INFO:teuthology.orchestra.run.smithi072.stdout: "dir+": 279, 2024-05-04T03:25:47.950 INFO:teuthology.orchestra.run.smithi072.stdout: "dir-": 262, 2024-05-04T03:25:47.950 INFO:teuthology.orchestra.run.smithi072.stdout: "dn": 307, 2024-05-04T03:25:47.950 INFO:teuthology.orchestra.run.smithi072.stdout: "dn+": 10114, 2024-05-04T03:25:47.950 INFO:teuthology.orchestra.run.smithi072.stdout: "dn-": 9807, 2024-05-04T03:25:47.950 INFO:teuthology.orchestra.run.smithi072.stdout: "cap": 0, 2024-05-04T03:25:47.950 INFO:teuthology.orchestra.run.smithi072.stdout: "cap+": 9571, 2024-05-04T03:25:47.950 INFO:teuthology.orchestra.run.smithi072.stdout: "cap-": 9571, 2024-05-04T03:25:47.950 INFO:teuthology.orchestra.run.smithi072.stdout: "rss": 164044, 2024-05-04T03:25:47.950 INFO:teuthology.orchestra.run.smithi072.stdout: "heap": 223632 2024-05-04T03:25:47.950 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.950 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_server": { 2024-05-04T03:25:47.950 INFO:teuthology.orchestra.run.smithi072.stdout: "dispatch_client_request": 38385, 2024-05-04T03:25:47.950 INFO:teuthology.orchestra.run.smithi072.stdout: "dispatch_server_request": 168, 2024-05-04T03:25:47.951 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_request": 20100, 2024-05-04T03:25:47.951 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_session": 18, 2024-05-04T03:25:47.951 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_peer_request": 365, 2024-05-04T03:25:47.951 INFO:teuthology.orchestra.run.smithi072.stdout: "req_create_latency": { 2024-05-04T03:25:47.951 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 5020, 2024-05-04T03:25:47.951 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 15.336086218, 2024-05-04T03:25:47.951 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.003054997 2024-05-04T03:25:47.951 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.951 INFO:teuthology.orchestra.run.smithi072.stdout: "req_getattr_latency": { 2024-05-04T03:25:47.951 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 60, 2024-05-04T03:25:47.951 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.265913657, 2024-05-04T03:25:47.951 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.004431894 2024-05-04T03:25:47.951 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.951 INFO:teuthology.orchestra.run.smithi072.stdout: "req_getfilelock_latency": { 2024-05-04T03:25:47.951 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:47.952 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:47.952 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:47.952 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.952 INFO:teuthology.orchestra.run.smithi072.stdout: "req_link_latency": { 2024-05-04T03:25:47.952 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 3, 2024-05-04T03:25:47.952 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.008172600, 2024-05-04T03:25:47.952 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.002724200 2024-05-04T03:25:47.952 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.952 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookup_latency": { 2024-05-04T03:25:47.952 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 205, 2024-05-04T03:25:47.952 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.068181962, 2024-05-04T03:25:47.952 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000332594 2024-05-04T03:25:47.952 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.952 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookuphash_latency": { 2024-05-04T03:25:47.952 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:47.953 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:47.953 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:47.953 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.953 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupino_latency": { 2024-05-04T03:25:47.953 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:47.953 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:47.953 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:47.953 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.953 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupname_latency": { 2024-05-04T03:25:47.953 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:47.953 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:47.953 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:47.953 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.953 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupparent_latency": { 2024-05-04T03:25:47.953 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:47.954 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:47.954 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:47.954 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.954 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupsnap_latency": { 2024-05-04T03:25:47.954 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:47.954 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:47.954 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:47.954 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.954 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lssnap_latency": { 2024-05-04T03:25:47.954 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:47.954 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:47.954 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:47.954 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.954 INFO:teuthology.orchestra.run.smithi072.stdout: "req_mkdir_latency": { 2024-05-04T03:25:47.955 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 53, 2024-05-04T03:25:47.955 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.041963181, 2024-05-04T03:25:47.955 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000791758 2024-05-04T03:25:47.955 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.955 INFO:teuthology.orchestra.run.smithi072.stdout: "req_mknod_latency": { 2024-05-04T03:25:47.955 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:47.955 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:47.955 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:47.955 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.955 INFO:teuthology.orchestra.run.smithi072.stdout: "req_mksnap_latency": { 2024-05-04T03:25:47.955 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:47.955 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:47.955 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:47.955 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.956 INFO:teuthology.orchestra.run.smithi072.stdout: "req_open_latency": { 2024-05-04T03:25:47.956 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 138, 2024-05-04T03:25:47.956 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.045190427, 2024-05-04T03:25:47.956 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000327466 2024-05-04T03:25:47.956 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.956 INFO:teuthology.orchestra.run.smithi072.stdout: "req_readdir_latency": { 2024-05-04T03:25:47.956 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 464, 2024-05-04T03:25:47.956 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 1.337133534, 2024-05-04T03:25:47.956 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.002881753 2024-05-04T03:25:47.956 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.956 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rename_latency": { 2024-05-04T03:25:47.956 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 69, 2024-05-04T03:25:47.956 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.142758343, 2024-05-04T03:25:47.956 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.002068961 2024-05-04T03:25:47.956 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.957 INFO:teuthology.orchestra.run.smithi072.stdout: "req_renamesnap_latency": { 2024-05-04T03:25:47.957 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:47.957 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:47.957 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:47.957 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.957 INFO:teuthology.orchestra.run.smithi072.stdout: "req_snapdiff_latency": { 2024-05-04T03:25:47.957 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:47.957 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:47.957 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:47.957 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.957 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rmdir_latency": { 2024-05-04T03:25:47.957 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 65, 2024-05-04T03:25:47.957 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 3.383736714, 2024-05-04T03:25:47.957 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.052057487 2024-05-04T03:25:47.958 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.958 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rmsnap_latency": { 2024-05-04T03:25:47.958 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:47.958 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:47.958 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:47.958 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.958 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rmxattr_latency": { 2024-05-04T03:25:47.958 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:47.958 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:47.958 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:47.958 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.958 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setattr_latency": { 2024-05-04T03:25:47.958 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 9027, 2024-05-04T03:25:47.958 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 71.356396209, 2024-05-04T03:25:47.958 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.007904774 2024-05-04T03:25:47.959 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.959 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setdirlayout_latency": { 2024-05-04T03:25:47.959 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:47.959 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:47.959 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:47.959 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.959 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setfilelock_latency": { 2024-05-04T03:25:47.959 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 8, 2024-05-04T03:25:47.959 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.002227779, 2024-05-04T03:25:47.959 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000278472 2024-05-04T03:25:47.959 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.959 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setlayout_latency": { 2024-05-04T03:25:47.959 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:47.959 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:47.960 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:47.960 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.960 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setxattr_latency": { 2024-05-04T03:25:47.960 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:47.960 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:47.960 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:47.960 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.960 INFO:teuthology.orchestra.run.smithi072.stdout: "req_symlink_latency": { 2024-05-04T03:25:47.960 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 4, 2024-05-04T03:25:47.960 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.002688686, 2024-05-04T03:25:47.960 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000672171 2024-05-04T03:25:47.960 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.960 INFO:teuthology.orchestra.run.smithi072.stdout: "req_unlink_latency": { 2024-05-04T03:25:47.960 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 4923, 2024-05-04T03:25:47.961 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 7.568211139, 2024-05-04T03:25:47.961 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.001537316 2024-05-04T03:25:47.961 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.961 INFO:teuthology.orchestra.run.smithi072.stdout: "cap_revoke_eviction": 0, 2024-05-04T03:25:47.961 INFO:teuthology.orchestra.run.smithi072.stdout: "cap_acquisition_throttle": 0, 2024-05-04T03:25:47.961 INFO:teuthology.orchestra.run.smithi072.stdout: "req_getvxattr_latency": { 2024-05-04T03:25:47.961 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:47.961 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:47.961 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:47.961 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:47.961 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.961 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_sessions": { 2024-05-04T03:25:47.961 INFO:teuthology.orchestra.run.smithi072.stdout: "session_count": 1, 2024-05-04T03:25:47.961 INFO:teuthology.orchestra.run.smithi072.stdout: "session_add": 3, 2024-05-04T03:25:47.962 INFO:teuthology.orchestra.run.smithi072.stdout: "session_remove": 2, 2024-05-04T03:25:47.962 INFO:teuthology.orchestra.run.smithi072.stdout: "sessions_open": 1, 2024-05-04T03:25:47.962 INFO:teuthology.orchestra.run.smithi072.stdout: "sessions_stale": 0, 2024-05-04T03:25:47.962 INFO:teuthology.orchestra.run.smithi072.stdout: "total_load": 7396, 2024-05-04T03:25:47.962 INFO:teuthology.orchestra.run.smithi072.stdout: "average_load": 7396, 2024-05-04T03:25:47.962 INFO:teuthology.orchestra.run.smithi072.stdout: "avg_session_uptime": 1652, 2024-05-04T03:25:47.962 INFO:teuthology.orchestra.run.smithi072.stdout: "mdthresh_evicted": 0 2024-05-04T03:25:47.962 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.962 INFO:teuthology.orchestra.run.smithi072.stdout: "mempool": { 2024-05-04T03:25:47.962 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_bytes": 0, 2024-05-04T03:25:47.962 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_items": 0, 2024-05-04T03:25:47.962 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_bytes": 0, 2024-05-04T03:25:47.962 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_items": 0, 2024-05-04T03:25:47.962 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_bytes": 0, 2024-05-04T03:25:47.963 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_items": 0, 2024-05-04T03:25:47.963 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-04T03:25:47.963 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_items": 0, 2024-05-04T03:25:47.963 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-04T03:25:47.963 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_items": 0, 2024-05-04T03:25:47.963 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_bytes": 0, 2024-05-04T03:25:47.963 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_items": 0, 2024-05-04T03:25:47.963 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-04T03:25:47.963 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_items": 0, 2024-05-04T03:25:47.963 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_bytes": 0, 2024-05-04T03:25:47.963 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_items": 0, 2024-05-04T03:25:47.963 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_bytes": 0, 2024-05-04T03:25:47.963 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_items": 0, 2024-05-04T03:25:47.963 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-04T03:25:47.963 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_items": 0, 2024-05-04T03:25:47.964 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-04T03:25:47.964 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_items": 0, 2024-05-04T03:25:47.964 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_bytes": 0, 2024-05-04T03:25:47.964 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_items": 0, 2024-05-04T03:25:47.964 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_bytes": 0, 2024-05-04T03:25:47.964 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_items": 0, 2024-05-04T03:25:47.964 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-04T03:25:47.964 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_items": 0, 2024-05-04T03:25:47.964 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_bytes": 0, 2024-05-04T03:25:47.964 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_items": 0, 2024-05-04T03:25:47.964 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_bytes": 0, 2024-05-04T03:25:47.964 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_items": 0, 2024-05-04T03:25:47.964 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_bytes": 0, 2024-05-04T03:25:47.964 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_items": 0, 2024-05-04T03:25:47.964 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_bytes": 0, 2024-05-04T03:25:47.965 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_items": 0, 2024-05-04T03:25:47.965 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_bytes": 295429, 2024-05-04T03:25:47.965 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_items": 101, 2024-05-04T03:25:47.965 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_bytes": 0, 2024-05-04T03:25:47.965 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_items": 0, 2024-05-04T03:25:47.965 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_bytes": 0, 2024-05-04T03:25:47.965 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_items": 0, 2024-05-04T03:25:47.965 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_bytes": 0, 2024-05-04T03:25:47.965 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_items": 0, 2024-05-04T03:25:47.965 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_bytes": 0, 2024-05-04T03:25:47.965 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_items": 0, 2024-05-04T03:25:47.965 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_bytes": 7240, 2024-05-04T03:25:47.965 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_items": 152, 2024-05-04T03:25:47.965 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_bytes": 0, 2024-05-04T03:25:47.965 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_items": 0, 2024-05-04T03:25:47.966 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_bytes": 0, 2024-05-04T03:25:47.966 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_items": 0, 2024-05-04T03:25:47.966 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_bytes": 296656, 2024-05-04T03:25:47.966 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_items": 3590, 2024-05-04T03:25:47.966 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_bytes": 0, 2024-05-04T03:25:47.966 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_items": 0, 2024-05-04T03:25:47.966 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_bytes": 0, 2024-05-04T03:25:47.966 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_items": 0 2024-05-04T03:25:47.966 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.966 INFO:teuthology.orchestra.run.smithi072.stdout: "objecter": { 2024-05-04T03:25:47.966 INFO:teuthology.orchestra.run.smithi072.stdout: "op_active": 0, 2024-05-04T03:25:47.966 INFO:teuthology.orchestra.run.smithi072.stdout: "op_laggy": 0, 2024-05-04T03:25:47.966 INFO:teuthology.orchestra.run.smithi072.stdout: "op_send": 67239, 2024-05-04T03:25:47.966 INFO:teuthology.orchestra.run.smithi072.stdout: "op_send_bytes": 96255916, 2024-05-04T03:25:47.966 INFO:teuthology.orchestra.run.smithi072.stdout: "op_resend": 33, 2024-05-04T03:25:47.967 INFO:teuthology.orchestra.run.smithi072.stdout: "op_reply": 67239, 2024-05-04T03:25:47.967 INFO:teuthology.orchestra.run.smithi072.stdout: "op_latency": { 2024-05-04T03:25:47.967 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 67239, 2024-05-04T03:25:47.967 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 212.722394252, 2024-05-04T03:25:47.967 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.003163675 2024-05-04T03:25:47.967 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.967 INFO:teuthology.orchestra.run.smithi072.stdout: "op_inflight": 0, 2024-05-04T03:25:47.967 INFO:teuthology.orchestra.run.smithi072.stdout: "oplen_avg": { 2024-05-04T03:25:47.967 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 67239, 2024-05-04T03:25:47.967 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 67556 2024-05-04T03:25:47.967 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.967 INFO:teuthology.orchestra.run.smithi072.stdout: "op": 67239, 2024-05-04T03:25:47.967 INFO:teuthology.orchestra.run.smithi072.stdout: "op_r": 63, 2024-05-04T03:25:47.967 INFO:teuthology.orchestra.run.smithi072.stdout: "op_w": 67176, 2024-05-04T03:25:47.968 INFO:teuthology.orchestra.run.smithi072.stdout: "op_rmw": 0, 2024-05-04T03:25:47.968 INFO:teuthology.orchestra.run.smithi072.stdout: "op_pg": 0, 2024-05-04T03:25:47.968 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_stat": 0, 2024-05-04T03:25:47.968 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_create": 10, 2024-05-04T03:25:47.968 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_read": 63, 2024-05-04T03:25:47.968 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_write": 24758, 2024-05-04T03:25:47.968 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_writefull": 58, 2024-05-04T03:25:47.968 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_writesame": 0, 2024-05-04T03:25:47.968 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_append": 0, 2024-05-04T03:25:47.968 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_zero": 0, 2024-05-04T03:25:47.968 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_truncate": 8, 2024-05-04T03:25:47.968 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_delete": 5319, 2024-05-04T03:25:47.968 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_mapext": 0, 2024-05-04T03:25:47.968 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_sparse_read": 0, 2024-05-04T03:25:47.968 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_clonerange": 0, 2024-05-04T03:25:47.969 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_getxattr": 0, 2024-05-04T03:25:47.969 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_setxattr": 20, 2024-05-04T03:25:47.969 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_cmpxattr": 0, 2024-05-04T03:25:47.969 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_rmxattr": 0, 2024-05-04T03:25:47.969 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_resetxattrs": 0, 2024-05-04T03:25:47.969 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_call": 0, 2024-05-04T03:25:47.969 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_watch": 0, 2024-05-04T03:25:47.969 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_notify": 0, 2024-05-04T03:25:47.969 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_src_cmpxattr": 0, 2024-05-04T03:25:47.969 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_pgls": 0, 2024-05-04T03:25:47.969 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_pgls_filter": 0, 2024-05-04T03:25:47.969 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_other": 36715, 2024-05-04T03:25:47.969 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_active": 0, 2024-05-04T03:25:47.969 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_send": 0, 2024-05-04T03:25:47.969 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_resend": 0, 2024-05-04T03:25:47.970 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_ping": 0, 2024-05-04T03:25:47.970 INFO:teuthology.orchestra.run.smithi072.stdout: "poolop_active": 0, 2024-05-04T03:25:47.970 INFO:teuthology.orchestra.run.smithi072.stdout: "poolop_send": 0, 2024-05-04T03:25:47.970 INFO:teuthology.orchestra.run.smithi072.stdout: "poolop_resend": 0, 2024-05-04T03:25:47.970 INFO:teuthology.orchestra.run.smithi072.stdout: "poolstat_active": 0, 2024-05-04T03:25:47.970 INFO:teuthology.orchestra.run.smithi072.stdout: "poolstat_send": 0, 2024-05-04T03:25:47.970 INFO:teuthology.orchestra.run.smithi072.stdout: "poolstat_resend": 0, 2024-05-04T03:25:47.970 INFO:teuthology.orchestra.run.smithi072.stdout: "statfs_active": 0, 2024-05-04T03:25:47.970 INFO:teuthology.orchestra.run.smithi072.stdout: "statfs_send": 0, 2024-05-04T03:25:47.970 INFO:teuthology.orchestra.run.smithi072.stdout: "statfs_resend": 0, 2024-05-04T03:25:47.970 INFO:teuthology.orchestra.run.smithi072.stdout: "command_active": 0, 2024-05-04T03:25:47.970 INFO:teuthology.orchestra.run.smithi072.stdout: "command_send": 0, 2024-05-04T03:25:47.970 INFO:teuthology.orchestra.run.smithi072.stdout: "command_resend": 0, 2024-05-04T03:25:47.970 INFO:teuthology.orchestra.run.smithi072.stdout: "map_epoch": 83, 2024-05-04T03:25:47.971 INFO:teuthology.orchestra.run.smithi072.stdout: "map_full": 0, 2024-05-04T03:25:47.971 INFO:teuthology.orchestra.run.smithi072.stdout: "map_inc": 2, 2024-05-04T03:25:47.971 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_sessions": 12, 2024-05-04T03:25:47.971 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_session_open": 12, 2024-05-04T03:25:47.971 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_session_close": 0, 2024-05-04T03:25:47.971 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_laggy": 0, 2024-05-04T03:25:47.971 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_wr": 555, 2024-05-04T03:25:47.971 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_rd": 0, 2024-05-04T03:25:47.971 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_del": 50 2024-05-04T03:25:47.971 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.971 INFO:teuthology.orchestra.run.smithi072.stdout: "oft": { 2024-05-04T03:25:47.971 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_objs": 1, 2024-05-04T03:25:47.971 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_kv_pairs": 0, 2024-05-04T03:25:47.971 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_updates": 0, 2024-05-04T03:25:47.971 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_removes": 0 2024-05-04T03:25:47.972 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.972 INFO:teuthology.orchestra.run.smithi072.stdout: "osd-slow-ops": { 2024-05-04T03:25:47.972 INFO:teuthology.orchestra.run.smithi072.stdout: "slow_ops_count": 9058 2024-05-04T03:25:47.972 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.972 INFO:teuthology.orchestra.run.smithi072.stdout: "purge_queue": { 2024-05-04T03:25:47.972 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing_ops": 0, 2024-05-04T03:25:47.972 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing_ops_high_water": 74, 2024-05-04T03:25:47.972 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing": 0, 2024-05-04T03:25:47.972 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing_high_water": 10, 2024-05-04T03:25:47.972 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executed_ops": 5253, 2024-05-04T03:25:47.972 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executed": 5038, 2024-05-04T03:25:47.972 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_item_in_journal": 0 2024-05-04T03:25:47.972 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.972 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-04T03:25:47.972 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-04T03:25:47.973 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-05-04T03:25:47.973 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-04T03:25:47.973 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 107500, 2024-05-04T03:25:47.973 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 21350244, 2024-05-04T03:25:47.973 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:47.973 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 107500, 2024-05-04T03:25:47.973 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-05-04T03:25:47.973 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-05-04T03:25:47.973 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 107500, 2024-05-04T03:25:47.973 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 21350244, 2024-05-04T03:25:47.973 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-04T03:25:47.973 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:47.973 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:47.973 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:47.973 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:47.974 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.974 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-objecter_bytes": { 2024-05-04T03:25:47.974 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-04T03:25:47.974 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-05-04T03:25:47.974 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-04T03:25:47.974 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 0, 2024-05-04T03:25:47.975 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 0, 2024-05-04T03:25:47.975 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:47.975 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 0, 2024-05-04T03:25:47.975 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 67239, 2024-05-04T03:25:47.975 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 96764802, 2024-05-04T03:25:47.975 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 67239, 2024-05-04T03:25:47.975 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 96764802, 2024-05-04T03:25:47.975 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-04T03:25:47.975 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:47.975 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:47.975 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:47.975 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:47.975 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.975 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-objecter_ops": { 2024-05-04T03:25:47.976 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-04T03:25:47.976 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 1024, 2024-05-04T03:25:47.976 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-04T03:25:47.976 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 0, 2024-05-04T03:25:47.976 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 0, 2024-05-04T03:25:47.976 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:47.976 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 0, 2024-05-04T03:25:47.976 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 67239, 2024-05-04T03:25:47.976 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 67239, 2024-05-04T03:25:47.976 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 67239, 2024-05-04T03:25:47.976 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 67239, 2024-05-04T03:25:47.976 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-04T03:25:47.976 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:47.977 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:47.977 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:47.977 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:47.977 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.977 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-write_buf_throttle": { 2024-05-04T03:25:47.977 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-04T03:25:47.977 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 3758096384, 2024-05-04T03:25:47.977 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-04T03:25:47.977 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 5038, 2024-05-04T03:25:47.977 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 508886, 2024-05-04T03:25:47.977 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:47.977 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 5038, 2024-05-04T03:25:47.977 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-05-04T03:25:47.977 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-05-04T03:25:47.977 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 63, 2024-05-04T03:25:47.978 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 508886, 2024-05-04T03:25:47.978 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-04T03:25:47.978 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:47.978 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:47.978 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:47.978 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:47.978 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:47.978 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-write_buf_throttle-0x564412b4e4e0": { 2024-05-04T03:25:47.978 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-04T03:25:47.978 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 3758096384, 2024-05-04T03:25:47.978 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-04T03:25:47.978 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 44989, 2024-05-04T03:25:47.978 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 92784512, 2024-05-04T03:25:47.979 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:47.979 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 44989, 2024-05-04T03:25:47.979 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-05-04T03:25:47.979 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-05-04T03:25:47.979 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 24695, 2024-05-04T03:25:47.979 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 92784512, 2024-05-04T03:25:47.979 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-04T03:25:47.979 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:47.979 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:47.979 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:47.979 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:47.979 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:47.979 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-05-04T03:25:48.750 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:48 smithi072 ceph-mon[33332]: pgmap v590: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-04T03:25:48.839 INFO:tasks.check_counter:Daemon mds.h mds.exported=5 2024-05-04T03:25:48.839 INFO:tasks.check_counter:Daemon mds.h mds.imported=5 2024-05-04T03:25:48.839 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=24 2024-05-04T03:25:48.839 DEBUG:tasks.check_counter:Getting stats from k 2024-05-04T03:25:48.840 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-04T03:25:48.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:48 smithi001 ceph-mon[29731]: pgmap v590: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-04T03:25:48.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:48 smithi077 ceph-mon[33531]: pgmap v590: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-04T03:25:49.080 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.b/config 2024-05-04T03:25:50.734 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:50 smithi072 ceph-mon[33332]: pgmap v591: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-04T03:25:50.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:50 smithi001 ceph-mon[29731]: pgmap v591: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-04T03:25:50.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:50 smithi077 ceph-mon[33531]: pgmap v591: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-04T03:25:50.998 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-05-04T03:25:50.998 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-0": { 2024-05-04T03:25:50.999 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 4, 2024-05-04T03:25:50.999 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 4, 2024-05-04T03:25:50.999 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 2992, 2024-05-04T03:25:50.999 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 448, 2024-05-04T03:25:50.999 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 4, 2024-05-04T03:25:50.999 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 0, 2024-05-04T03:25:50.999 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.001412543, 2024-05-04T03:25:50.999 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.000031225, 2024-05-04T03:25:50.999 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.000634040, 2024-05-04T03:25:50.999 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-04T03:25:50.999 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:50.999 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 3, 2024-05-04T03:25:50.999 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000733489, 2024-05-04T03:25:50.999 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000244496 2024-05-04T03:25:51.000 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:51.000 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:51.000 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:51.000 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:51.000 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:51.000 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:51.000 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-04T03:25:51.000 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 448 2024-05-04T03:25:51.000 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:51.000 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-1": { 2024-05-04T03:25:51.000 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 1, 2024-05-04T03:25:51.000 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 133, 2024-05-04T03:25:51.000 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 160, 2024-05-04T03:25:51.000 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 102000, 2024-05-04T03:25:51.001 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 3, 2024-05-04T03:25:51.001 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 2, 2024-05-04T03:25:51.001 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.008392106, 2024-05-04T03:25:51.001 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.005908486, 2024-05-04T03:25:51.001 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.001036887, 2024-05-04T03:25:51.001 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-04T03:25:51.001 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:51.001 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 131, 2024-05-04T03:25:51.001 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.007619470, 2024-05-04T03:25:51.001 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000058163 2024-05-04T03:25:51.001 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:51.001 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:51.001 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:51.001 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:51.002 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:51.002 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:51.002 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-04T03:25:51.002 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 102000 2024-05-04T03:25:51.002 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:51.002 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-2": { 2024-05-04T03:25:51.002 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 181, 2024-05-04T03:25:51.002 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 176, 2024-05-04T03:25:51.002 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 801776, 2024-05-04T03:25:51.002 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 107728, 2024-05-04T03:25:51.002 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 2, 2024-05-04T03:25:51.002 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 1, 2024-05-04T03:25:51.002 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.032865728, 2024-05-04T03:25:51.002 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.010600605, 2024-05-04T03:25:51.002 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.022355958, 2024-05-04T03:25:51.003 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.004083944, 2024-05-04T03:25:51.003 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:51.003 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 175, 2024-05-04T03:25:51.003 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.009291543, 2024-05-04T03:25:51.003 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000053094 2024-05-04T03:25:51.003 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:51.003 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:51.003 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:51.003 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:51.003 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:51.003 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:51.003 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 801776, 2024-05-04T03:25:51.003 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 107728 2024-05-04T03:25:51.003 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:51.003 INFO:teuthology.orchestra.run.smithi072.stdout: "cct": { 2024-05-04T03:25:51.004 INFO:teuthology.orchestra.run.smithi072.stdout: "total_workers": 0, 2024-05-04T03:25:51.004 INFO:teuthology.orchestra.run.smithi072.stdout: "unhealthy_workers": 0 2024-05-04T03:25:51.004 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:51.004 INFO:teuthology.orchestra.run.smithi072.stdout: "mempool": { 2024-05-04T03:25:51.004 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_bytes": 0, 2024-05-04T03:25:51.004 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_items": 0, 2024-05-04T03:25:51.004 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_bytes": 0, 2024-05-04T03:25:51.004 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_items": 0, 2024-05-04T03:25:51.004 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_bytes": 0, 2024-05-04T03:25:51.004 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_items": 0, 2024-05-04T03:25:51.004 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-04T03:25:51.004 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_items": 0, 2024-05-04T03:25:51.004 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-04T03:25:51.004 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_items": 0, 2024-05-04T03:25:51.005 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_bytes": 0, 2024-05-04T03:25:51.005 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_items": 0, 2024-05-04T03:25:51.005 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-04T03:25:51.005 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_items": 0, 2024-05-04T03:25:51.005 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_bytes": 0, 2024-05-04T03:25:51.005 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_items": 0, 2024-05-04T03:25:51.005 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_bytes": 0, 2024-05-04T03:25:51.005 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_items": 0, 2024-05-04T03:25:51.005 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-04T03:25:51.005 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_items": 0, 2024-05-04T03:25:51.005 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-04T03:25:51.005 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_items": 0, 2024-05-04T03:25:51.005 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_bytes": 0, 2024-05-04T03:25:51.005 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_items": 0, 2024-05-04T03:25:51.005 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_bytes": 0, 2024-05-04T03:25:51.006 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_items": 0, 2024-05-04T03:25:51.006 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-04T03:25:51.006 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_items": 0, 2024-05-04T03:25:51.006 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_bytes": 0, 2024-05-04T03:25:51.006 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_items": 0, 2024-05-04T03:25:51.006 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_bytes": 0, 2024-05-04T03:25:51.006 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_items": 0, 2024-05-04T03:25:51.006 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_bytes": 0, 2024-05-04T03:25:51.006 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_items": 0, 2024-05-04T03:25:51.006 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_bytes": 0, 2024-05-04T03:25:51.006 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_items": 0, 2024-05-04T03:25:51.006 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_bytes": 149024, 2024-05-04T03:25:51.006 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_items": 31, 2024-05-04T03:25:51.006 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_bytes": 0, 2024-05-04T03:25:51.007 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_items": 0, 2024-05-04T03:25:51.007 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_bytes": 0, 2024-05-04T03:25:51.007 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_items": 0, 2024-05-04T03:25:51.007 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_bytes": 0, 2024-05-04T03:25:51.007 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_items": 0, 2024-05-04T03:25:51.007 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_bytes": 0, 2024-05-04T03:25:51.007 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_items": 0, 2024-05-04T03:25:51.007 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_bytes": 0, 2024-05-04T03:25:51.007 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_items": 0, 2024-05-04T03:25:51.007 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_bytes": 0, 2024-05-04T03:25:51.007 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_items": 0, 2024-05-04T03:25:51.007 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_bytes": 0, 2024-05-04T03:25:51.007 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_items": 0, 2024-05-04T03:25:51.007 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_bytes": 928, 2024-05-04T03:25:51.007 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_items": 2, 2024-05-04T03:25:51.008 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_bytes": 0, 2024-05-04T03:25:51.008 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_items": 0, 2024-05-04T03:25:51.008 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_bytes": 0, 2024-05-04T03:25:51.008 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_items": 0 2024-05-04T03:25:51.008 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-04T03:25:51.008 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-04T03:25:51.008 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-04T03:25:51.008 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-05-04T03:25:51.008 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-04T03:25:51.008 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 182, 2024-05-04T03:25:51.008 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 776202, 2024-05-04T03:25:51.008 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:51.008 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 182, 2024-05-04T03:25:51.008 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-05-04T03:25:51.008 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-05-04T03:25:51.008 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 182, 2024-05-04T03:25:51.009 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 776202, 2024-05-04T03:25:51.009 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-04T03:25:51.009 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-04T03:25:51.009 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-04T03:25:51.009 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-04T03:25:51.009 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:51.009 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-04T03:25:51.009 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-05-04T03:25:51.724 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-05-04T03:25:51.724 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-05-04T03:25:51.724 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-05-04T03:25:51.724 DEBUG:tasks.check_counter:Getting stats from c 2024-05-04T03:25:51.725 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-04T03:25:51.975 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:25:52.678 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:52 smithi077 ceph-mon[33531]: pgmap v592: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:25:52.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:52 smithi001 ceph-mon[29731]: pgmap v592: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:25:52.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:52 smithi072 ceph-mon[33332]: pgmap v592: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:25:53.776 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-04T03:25:53.776 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-05-04T03:25:53.777 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 1455, 2024-05-04T03:25:53.777 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 1439, 2024-05-04T03:25:53.777 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 2521971, 2024-05-04T03:25:53.777 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 506878, 2024-05-04T03:25:53.777 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 5, 2024-05-04T03:25:53.777 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 1, 2024-05-04T03:25:53.777 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.151458750, 2024-05-04T03:25:53.777 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.049166674, 2024-05-04T03:25:53.777 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.052562108, 2024-05-04T03:25:53.777 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.042222608, 2024-05-04T03:25:53.777 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:53.777 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1438, 2024-05-04T03:25:53.778 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.052632427, 2024-05-04T03:25:53.778 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000036601 2024-05-04T03:25:53.778 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.778 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:53.778 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.778 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.778 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.778 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.778 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 862704, 2024-05-04T03:25:53.778 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 106512 2024-05-04T03:25:53.778 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.778 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-05-04T03:25:53.778 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 5, 2024-05-04T03:25:53.779 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 134, 2024-05-04T03:25:53.779 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 1816, 2024-05-04T03:25:53.779 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 301162, 2024-05-04T03:25:53.779 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 5, 2024-05-04T03:25:53.779 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 4, 2024-05-04T03:25:53.779 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.008508658, 2024-05-04T03:25:53.779 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.005059296, 2024-05-04T03:25:53.779 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.002063320, 2024-05-04T03:25:53.779 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000109032, 2024-05-04T03:25:53.779 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:53.779 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 132, 2024-05-04T03:25:53.779 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.010687428, 2024-05-04T03:25:53.779 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000080965 2024-05-04T03:25:53.779 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.780 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:53.780 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.780 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.780 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.780 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.780 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-04T03:25:53.780 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 299824 2024-05-04T03:25:53.780 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.780 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-05-04T03:25:53.780 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 588, 2024-05-04T03:25:53.780 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 588, 2024-05-04T03:25:53.780 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 146762, 2024-05-04T03:25:53.780 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 186167, 2024-05-04T03:25:53.780 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 4, 2024-05-04T03:25:53.781 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 3, 2024-05-04T03:25:53.781 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.057154302, 2024-05-04T03:25:53.781 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.019894910, 2024-05-04T03:25:53.781 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.012996752, 2024-05-04T03:25:53.781 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.019308147, 2024-05-04T03:25:53.781 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:53.781 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 587, 2024-05-04T03:25:53.781 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.023361831, 2024-05-04T03:25:53.781 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000039798 2024-05-04T03:25:53.781 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.781 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:53.781 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.781 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.782 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.782 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.782 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-04T03:25:53.782 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 448 2024-05-04T03:25:53.782 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.782 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-05-04T03:25:53.782 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 1, 2024-05-04T03:25:53.782 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-05-04T03:25:53.782 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.782 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-MDSRank": { 2024-05-04T03:25:53.782 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-05-04T03:25:53.782 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-05-04T03:25:53.783 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 3077, 2024-05-04T03:25:53.783 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.092858592, 2024-05-04T03:25:53.783 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000030178 2024-05-04T03:25:53.783 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:25:53.783 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.783 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-PurgeQueue": { 2024-05-04T03:25:53.783 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-05-04T03:25:53.783 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-05-04T03:25:53.783 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.783 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.783 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.783 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:25:53.783 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.784 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-05-04T03:25:53.784 INFO:teuthology.orchestra.run.smithi077.stdout: "request": 0, 2024-05-04T03:25:53.784 INFO:teuthology.orchestra.run.smithi077.stdout: "reply": 0, 2024-05-04T03:25:53.784 INFO:teuthology.orchestra.run.smithi077.stdout: "reply_latency": { 2024-05-04T03:25:53.784 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.784 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.784 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.784 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.784 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_reply": 0, 2024-05-04T03:25:53.784 INFO:teuthology.orchestra.run.smithi077.stdout: "forward": 0, 2024-05-04T03:25:53.784 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_complete": 0, 2024-05-04T03:25:53.784 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_keys": 0, 2024-05-04T03:25:53.784 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_commit": 0, 2024-05-04T03:25:53.784 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_split": 0, 2024-05-04T03:25:53.785 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_merge": 0, 2024-05-04T03:25:53.785 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes": 134, 2024-05-04T03:25:53.785 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_top": 13, 2024-05-04T03:25:53.785 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_bottom": 121, 2024-05-04T03:25:53.785 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pin_tail": 0, 2024-05-04T03:25:53.785 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pinned": 115, 2024-05-04T03:25:53.785 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_expired": 0, 2024-05-04T03:25:53.785 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_with_caps": 0, 2024-05-04T03:25:53.785 INFO:teuthology.orchestra.run.smithi077.stdout: "caps": 0, 2024-05-04T03:25:53.785 INFO:teuthology.orchestra.run.smithi077.stdout: "subtrees": 2, 2024-05-04T03:25:53.785 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse": 0, 2024-05-04T03:25:53.785 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_hit": 0, 2024-05-04T03:25:53.785 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_forward": 0, 2024-05-04T03:25:53.785 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_discover": 0, 2024-05-04T03:25:53.786 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_dir_fetch": 0, 2024-05-04T03:25:53.786 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_remote_ino": 0, 2024-05-04T03:25:53.786 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_lock": 0, 2024-05-04T03:25:53.786 INFO:teuthology.orchestra.run.smithi077.stdout: "load_cent": 0, 2024-05-04T03:25:53.786 INFO:teuthology.orchestra.run.smithi077.stdout: "q": 0, 2024-05-04T03:25:53.786 INFO:teuthology.orchestra.run.smithi077.stdout: "exported": 0, 2024-05-04T03:25:53.786 INFO:teuthology.orchestra.run.smithi077.stdout: "exported_inodes": 0, 2024-05-04T03:25:53.786 INFO:teuthology.orchestra.run.smithi077.stdout: "imported": 0, 2024-05-04T03:25:53.786 INFO:teuthology.orchestra.run.smithi077.stdout: "imported_inodes": 0, 2024-05-04T03:25:53.786 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_dir_fetch": 0, 2024-05-04T03:25:53.786 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_backtrace_fetch": 0, 2024-05-04T03:25:53.786 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_peer_discover": 0, 2024-05-04T03:25:53.786 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rfiles": 4, 2024-05-04T03:25:53.786 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rbytes": 234, 2024-05-04T03:25:53.786 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rsnaps": 0, 2024-05-04T03:25:53.787 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_fetch": 0, 2024-05-04T03:25:53.787 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_set_tag": 0, 2024-05-04T03:25:53.787 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_repaired": 0, 2024-05-04T03:25:53.787 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_inotable_repaired": 0, 2024-05-04T03:25:53.787 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_inodes": 0, 2024-05-04T03:25:53.787 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_base_inodes": 0, 2024-05-04T03:25:53.787 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dirfrag_rstats": 0, 2024-05-04T03:25:53.787 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_file_inodes": 0, 2024-05-04T03:25:53.787 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_inode_file_caps": 0, 2024-05-04T03:25:53.787 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_revoke": 0, 2024-05-04T03:25:53.787 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_grant": 0, 2024-05-04T03:25:53.787 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_trunc": 0, 2024-05-04T03:25:53.788 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-04T03:25:53.788 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-04T03:25:53.788 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps": 0, 2024-05-04T03:25:53.788 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps_dirty": 0, 2024-05-04T03:25:53.788 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_cap_release": 0, 2024-05-04T03:25:53.788 INFO:teuthology.orchestra.run.smithi077.stdout: "process_request_cap_release": 0 2024-05-04T03:25:53.788 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.788 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache": { 2024-05-04T03:25:53.788 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update": 0, 2024-05-04T03:25:53.788 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update_receipt": 0, 2024-05-04T03:25:53.788 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_try_discover": 0, 2024-05-04T03:25:53.788 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_send_discover": 0, 2024-05-04T03:25:53.788 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_handle_discover": 0, 2024-05-04T03:25:53.788 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays": 0, 2024-05-04T03:25:53.789 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_delayed": 0, 2024-05-04T03:25:53.789 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_enqueuing": 0, 2024-05-04T03:25:53.789 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_created": 0, 2024-05-04T03:25:53.789 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_enqueued": 0, 2024-05-04T03:25:53.789 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_reintegrated": 0, 2024-05-04T03:25:53.789 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_migrated": 0, 2024-05-04T03:25:53.789 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_processing": 0, 2024-05-04T03:25:53.789 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_enqueued": 0, 2024-05-04T03:25:53.789 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_prioritized": 0, 2024-05-04T03:25:53.789 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_started": 0, 2024-05-04T03:25:53.789 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_completed": 0, 2024-05-04T03:25:53.789 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_path": 0, 2024-05-04T03:25:53.789 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_inode": 0, 2024-05-04T03:25:53.790 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_enqueue_scrub": 0, 2024-05-04T03:25:53.790 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_exportdir": 0, 2024-05-04T03:25:53.790 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_flush": 0, 2024-05-04T03:25:53.790 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragmentdir": 0, 2024-05-04T03:25:53.790 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragstats": 0, 2024-05-04T03:25:53.790 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_inodestats": 0 2024-05-04T03:25:53.790 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.790 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log": { 2024-05-04T03:25:53.791 INFO:teuthology.orchestra.run.smithi077.stdout: "evlrg": 0, 2024-05-04T03:25:53.791 INFO:teuthology.orchestra.run.smithi077.stdout: "evadd": 0, 2024-05-04T03:25:53.791 INFO:teuthology.orchestra.run.smithi077.stdout: "evex": 0, 2024-05-04T03:25:53.791 INFO:teuthology.orchestra.run.smithi077.stdout: "evtrm": 0, 2024-05-04T03:25:53.791 INFO:teuthology.orchestra.run.smithi077.stdout: "ev": 630, 2024-05-04T03:25:53.791 INFO:teuthology.orchestra.run.smithi077.stdout: "evexg": 0, 2024-05-04T03:25:53.791 INFO:teuthology.orchestra.run.smithi077.stdout: "evexd": 0, 2024-05-04T03:25:53.791 INFO:teuthology.orchestra.run.smithi077.stdout: "segadd": 0, 2024-05-04T03:25:53.791 INFO:teuthology.orchestra.run.smithi077.stdout: "segex": 0, 2024-05-04T03:25:53.792 INFO:teuthology.orchestra.run.smithi077.stdout: "segtrm": 0, 2024-05-04T03:25:53.792 INFO:teuthology.orchestra.run.smithi077.stdout: "seg": 2, 2024-05-04T03:25:53.792 INFO:teuthology.orchestra.run.smithi077.stdout: "segmjr": 32, 2024-05-04T03:25:53.792 INFO:teuthology.orchestra.run.smithi077.stdout: "segexg": 0, 2024-05-04T03:25:53.792 INFO:teuthology.orchestra.run.smithi077.stdout: "segexd": 0, 2024-05-04T03:25:53.792 INFO:teuthology.orchestra.run.smithi077.stdout: "expos": 5398119, 2024-05-04T03:25:53.792 INFO:teuthology.orchestra.run.smithi077.stdout: "wrpos": 5480943, 2024-05-04T03:25:53.792 INFO:teuthology.orchestra.run.smithi077.stdout: "rdpos": 5478955, 2024-05-04T03:25:53.793 INFO:teuthology.orchestra.run.smithi077.stdout: "jlat": { 2024-05-04T03:25:53.793 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.793 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.793 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.793 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.793 INFO:teuthology.orchestra.run.smithi077.stdout: "replayed": 630 2024-05-04T03:25:53.793 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.793 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_mem": { 2024-05-04T03:25:53.793 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 15, 2024-05-04T03:25:53.794 INFO:teuthology.orchestra.run.smithi077.stdout: "ino+": 173, 2024-05-04T03:25:53.794 INFO:teuthology.orchestra.run.smithi077.stdout: "ino-": 158, 2024-05-04T03:25:53.794 INFO:teuthology.orchestra.run.smithi077.stdout: "dir": 10, 2024-05-04T03:25:53.794 INFO:teuthology.orchestra.run.smithi077.stdout: "dir+": 31, 2024-05-04T03:25:53.794 INFO:teuthology.orchestra.run.smithi077.stdout: "dir-": 21, 2024-05-04T03:25:53.794 INFO:teuthology.orchestra.run.smithi077.stdout: "dn": 134, 2024-05-04T03:25:53.794 INFO:teuthology.orchestra.run.smithi077.stdout: "dn+": 302, 2024-05-04T03:25:53.794 INFO:teuthology.orchestra.run.smithi077.stdout: "dn-": 168, 2024-05-04T03:25:53.795 INFO:teuthology.orchestra.run.smithi077.stdout: "cap": 0, 2024-05-04T03:25:53.795 INFO:teuthology.orchestra.run.smithi077.stdout: "cap+": 0, 2024-05-04T03:25:53.795 INFO:teuthology.orchestra.run.smithi077.stdout: "cap-": 0, 2024-05-04T03:25:53.795 INFO:teuthology.orchestra.run.smithi077.stdout: "rss": 120164, 2024-05-04T03:25:53.795 INFO:teuthology.orchestra.run.smithi077.stdout: "heap": 215440 2024-05-04T03:25:53.795 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.795 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_server": { 2024-05-04T03:25:53.795 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_client_request": 0, 2024-05-04T03:25:53.795 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_server_request": 0, 2024-05-04T03:25:53.795 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_request": 0, 2024-05-04T03:25:53.795 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_session": 0, 2024-05-04T03:25:53.795 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_peer_request": 0, 2024-05-04T03:25:53.795 INFO:teuthology.orchestra.run.smithi077.stdout: "req_create_latency": { 2024-05-04T03:25:53.795 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.796 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.796 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.796 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.796 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getattr_latency": { 2024-05-04T03:25:53.796 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.796 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.796 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.796 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.796 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getfilelock_latency": { 2024-05-04T03:25:53.796 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.796 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.796 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.797 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.797 INFO:teuthology.orchestra.run.smithi077.stdout: "req_link_latency": { 2024-05-04T03:25:53.797 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.797 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.797 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.797 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.797 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookup_latency": { 2024-05-04T03:25:53.797 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.797 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.797 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.797 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.797 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookuphash_latency": { 2024-05-04T03:25:53.797 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.798 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.798 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.798 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.798 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupino_latency": { 2024-05-04T03:25:53.798 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.798 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.798 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.798 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.798 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupname_latency": { 2024-05-04T03:25:53.798 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.798 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.798 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.798 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.799 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupparent_latency": { 2024-05-04T03:25:53.799 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.799 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.799 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.799 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.799 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupsnap_latency": { 2024-05-04T03:25:53.799 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.799 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.799 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.799 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.799 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lssnap_latency": { 2024-05-04T03:25:53.799 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.800 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.800 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.800 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.800 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mkdir_latency": { 2024-05-04T03:25:53.800 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.801 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.801 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.801 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.801 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mknod_latency": { 2024-05-04T03:25:53.801 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.801 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.801 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.801 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.801 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mksnap_latency": { 2024-05-04T03:25:53.801 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.801 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.801 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.801 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.802 INFO:teuthology.orchestra.run.smithi077.stdout: "req_open_latency": { 2024-05-04T03:25:53.802 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.802 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.802 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.802 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.802 INFO:teuthology.orchestra.run.smithi077.stdout: "req_readdir_latency": { 2024-05-04T03:25:53.802 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.802 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.802 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.802 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.802 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rename_latency": { 2024-05-04T03:25:53.802 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.802 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.803 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.803 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.803 INFO:teuthology.orchestra.run.smithi077.stdout: "req_renamesnap_latency": { 2024-05-04T03:25:53.803 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.803 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.803 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.803 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.803 INFO:teuthology.orchestra.run.smithi077.stdout: "req_snapdiff_latency": { 2024-05-04T03:25:53.803 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.803 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.803 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.803 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.803 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmdir_latency": { 2024-05-04T03:25:53.804 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.804 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.804 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.804 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.804 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmsnap_latency": { 2024-05-04T03:25:53.804 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.804 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.804 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.804 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.804 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmxattr_latency": { 2024-05-04T03:25:53.804 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.804 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.804 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.804 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.805 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setattr_latency": { 2024-05-04T03:25:53.805 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.805 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.805 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.805 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.805 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setdirlayout_latency": { 2024-05-04T03:25:53.805 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.805 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.805 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.805 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.805 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setfilelock_latency": { 2024-05-04T03:25:53.805 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.805 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.806 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.806 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.806 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setlayout_latency": { 2024-05-04T03:25:53.806 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.806 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.806 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.806 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.806 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setxattr_latency": { 2024-05-04T03:25:53.806 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.806 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.806 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.806 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.806 INFO:teuthology.orchestra.run.smithi077.stdout: "req_symlink_latency": { 2024-05-04T03:25:53.807 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.807 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.807 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.807 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.807 INFO:teuthology.orchestra.run.smithi077.stdout: "req_unlink_latency": { 2024-05-04T03:25:53.807 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.807 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.807 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.807 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.807 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_revoke_eviction": 0, 2024-05-04T03:25:53.807 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_acquisition_throttle": 0, 2024-05-04T03:25:53.807 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getvxattr_latency": { 2024-05-04T03:25:53.807 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.808 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.808 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.808 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:25:53.808 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.808 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_sessions": { 2024-05-04T03:25:53.808 INFO:teuthology.orchestra.run.smithi077.stdout: "session_count": 1, 2024-05-04T03:25:53.808 INFO:teuthology.orchestra.run.smithi077.stdout: "session_add": 3, 2024-05-04T03:25:53.808 INFO:teuthology.orchestra.run.smithi077.stdout: "session_remove": 2, 2024-05-04T03:25:53.808 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_open": 2, 2024-05-04T03:25:53.808 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_stale": 0, 2024-05-04T03:25:53.808 INFO:teuthology.orchestra.run.smithi077.stdout: "total_load": 0, 2024-05-04T03:25:53.808 INFO:teuthology.orchestra.run.smithi077.stdout: "average_load": 0, 2024-05-04T03:25:53.808 INFO:teuthology.orchestra.run.smithi077.stdout: "avg_session_uptime": 2948, 2024-05-04T03:25:53.808 INFO:teuthology.orchestra.run.smithi077.stdout: "mdthresh_evicted": 0 2024-05-04T03:25:53.809 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.809 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-05-04T03:25:53.809 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-05-04T03:25:53.809 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-05-04T03:25:53.809 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-05-04T03:25:53.809 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-05-04T03:25:53.809 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-05-04T03:25:53.809 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-05-04T03:25:53.809 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-04T03:25:53.809 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-05-04T03:25:53.809 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-04T03:25:53.809 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-05-04T03:25:53.809 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-05-04T03:25:53.809 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-05-04T03:25:53.809 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-04T03:25:53.810 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-05-04T03:25:53.810 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-05-04T03:25:53.810 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-05-04T03:25:53.810 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-05-04T03:25:53.810 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-05-04T03:25:53.810 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-04T03:25:53.810 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-05-04T03:25:53.810 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-04T03:25:53.810 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-05-04T03:25:53.810 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-05-04T03:25:53.810 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-05-04T03:25:53.810 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-05-04T03:25:53.810 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-05-04T03:25:53.810 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-04T03:25:53.811 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-05-04T03:25:53.811 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-05-04T03:25:53.811 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-05-04T03:25:53.811 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-05-04T03:25:53.811 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-05-04T03:25:53.811 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-05-04T03:25:53.811 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-05-04T03:25:53.811 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-05-04T03:25:53.811 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-05-04T03:25:53.811 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 189400, 2024-05-04T03:25:53.811 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 51, 2024-05-04T03:25:53.812 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-05-04T03:25:53.812 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-05-04T03:25:53.812 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-05-04T03:25:53.812 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-05-04T03:25:53.812 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-05-04T03:25:53.812 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-05-04T03:25:53.812 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-05-04T03:25:53.812 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-05-04T03:25:53.812 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 7240, 2024-05-04T03:25:53.812 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 152, 2024-05-04T03:25:53.812 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-05-04T03:25:53.813 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-05-04T03:25:53.813 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-05-04T03:25:53.813 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-05-04T03:25:53.813 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 140690, 2024-05-04T03:25:53.813 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 736, 2024-05-04T03:25:53.813 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-05-04T03:25:53.813 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-05-04T03:25:53.813 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-05-04T03:25:53.813 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-05-04T03:25:53.813 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.813 INFO:teuthology.orchestra.run.smithi077.stdout: "objecter": { 2024-05-04T03:25:53.813 INFO:teuthology.orchestra.run.smithi077.stdout: "op_active": 0, 2024-05-04T03:25:53.813 INFO:teuthology.orchestra.run.smithi077.stdout: "op_laggy": 0, 2024-05-04T03:25:53.813 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send": 1847, 2024-05-04T03:25:53.814 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send_bytes": 26, 2024-05-04T03:25:53.814 INFO:teuthology.orchestra.run.smithi077.stdout: "op_resend": 4, 2024-05-04T03:25:53.814 INFO:teuthology.orchestra.run.smithi077.stdout: "op_reply": 1847, 2024-05-04T03:25:53.814 INFO:teuthology.orchestra.run.smithi077.stdout: "op_latency": { 2024-05-04T03:25:53.814 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1847, 2024-05-04T03:25:53.814 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.812983998, 2024-05-04T03:25:53.814 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000440164 2024-05-04T03:25:53.814 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.814 INFO:teuthology.orchestra.run.smithi077.stdout: "op_inflight": 0, 2024-05-04T03:25:53.814 INFO:teuthology.orchestra.run.smithi077.stdout: "oplen_avg": { 2024-05-04T03:25:53.814 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1847, 2024-05-04T03:25:53.814 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 1848 2024-05-04T03:25:53.814 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.814 INFO:teuthology.orchestra.run.smithi077.stdout: "op": 1847, 2024-05-04T03:25:53.815 INFO:teuthology.orchestra.run.smithi077.stdout: "op_r": 1847, 2024-05-04T03:25:53.815 INFO:teuthology.orchestra.run.smithi077.stdout: "op_w": 0, 2024-05-04T03:25:53.815 INFO:teuthology.orchestra.run.smithi077.stdout: "op_rmw": 0, 2024-05-04T03:25:53.815 INFO:teuthology.orchestra.run.smithi077.stdout: "op_pg": 0, 2024-05-04T03:25:53.815 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_stat": 1169, 2024-05-04T03:25:53.815 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_create": 0, 2024-05-04T03:25:53.815 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_read": 675, 2024-05-04T03:25:53.815 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_write": 0, 2024-05-04T03:25:53.815 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writefull": 0, 2024-05-04T03:25:53.815 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writesame": 0, 2024-05-04T03:25:53.815 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_append": 0, 2024-05-04T03:25:53.815 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_zero": 0, 2024-05-04T03:25:53.816 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_truncate": 0, 2024-05-04T03:25:53.816 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_delete": 0, 2024-05-04T03:25:53.816 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_mapext": 0, 2024-05-04T03:25:53.816 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_sparse_read": 0, 2024-05-04T03:25:53.816 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_clonerange": 0, 2024-05-04T03:25:53.816 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_getxattr": 2, 2024-05-04T03:25:53.816 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_setxattr": 0, 2024-05-04T03:25:53.816 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_cmpxattr": 0, 2024-05-04T03:25:53.816 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_rmxattr": 0, 2024-05-04T03:25:53.816 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_resetxattrs": 0, 2024-05-04T03:25:53.816 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_call": 0, 2024-05-04T03:25:53.816 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_watch": 0, 2024-05-04T03:25:53.816 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_notify": 0, 2024-05-04T03:25:53.816 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_src_cmpxattr": 0, 2024-05-04T03:25:53.816 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls": 0, 2024-05-04T03:25:53.817 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls_filter": 0, 2024-05-04T03:25:53.817 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_other": 0, 2024-05-04T03:25:53.817 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_active": 0, 2024-05-04T03:25:53.817 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_send": 0, 2024-05-04T03:25:53.817 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_resend": 0, 2024-05-04T03:25:53.817 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_ping": 0, 2024-05-04T03:25:53.817 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_active": 0, 2024-05-04T03:25:53.817 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_send": 0, 2024-05-04T03:25:53.817 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_resend": 0, 2024-05-04T03:25:53.817 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_active": 0, 2024-05-04T03:25:53.817 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_send": 0, 2024-05-04T03:25:53.817 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_resend": 0, 2024-05-04T03:25:53.817 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_active": 0, 2024-05-04T03:25:53.817 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_send": 0, 2024-05-04T03:25:53.818 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_resend": 0, 2024-05-04T03:25:53.818 INFO:teuthology.orchestra.run.smithi077.stdout: "command_active": 0, 2024-05-04T03:25:53.818 INFO:teuthology.orchestra.run.smithi077.stdout: "command_send": 0, 2024-05-04T03:25:53.818 INFO:teuthology.orchestra.run.smithi077.stdout: "command_resend": 0, 2024-05-04T03:25:53.818 INFO:teuthology.orchestra.run.smithi077.stdout: "map_epoch": 83, 2024-05-04T03:25:53.818 INFO:teuthology.orchestra.run.smithi077.stdout: "map_full": 0, 2024-05-04T03:25:53.818 INFO:teuthology.orchestra.run.smithi077.stdout: "map_inc": 2, 2024-05-04T03:25:53.818 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_sessions": 5, 2024-05-04T03:25:53.818 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_open": 5, 2024-05-04T03:25:53.818 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_close": 0, 2024-05-04T03:25:53.818 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_laggy": 0, 2024-05-04T03:25:53.818 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_wr": 0, 2024-05-04T03:25:53.818 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_rd": 2, 2024-05-04T03:25:53.818 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_del": 0 2024-05-04T03:25:53.819 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.819 INFO:teuthology.orchestra.run.smithi077.stdout: "oft": { 2024-05-04T03:25:53.819 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_objs": 0, 2024-05-04T03:25:53.819 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_kv_pairs": 0, 2024-05-04T03:25:53.819 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_updates": 0, 2024-05-04T03:25:53.819 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_removes": 0 2024-05-04T03:25:53.819 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.819 INFO:teuthology.orchestra.run.smithi077.stdout: "osd-slow-ops": { 2024-05-04T03:25:53.819 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_ops_count": 0 2024-05-04T03:25:53.819 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.819 INFO:teuthology.orchestra.run.smithi077.stdout: "purge_queue": { 2024-05-04T03:25:53.819 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops": 0, 2024-05-04T03:25:53.819 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops_high_water": 0, 2024-05-04T03:25:53.819 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing": 0, 2024-05-04T03:25:53.819 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_high_water": 0, 2024-05-04T03:25:53.820 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed_ops": 0, 2024-05-04T03:25:53.820 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed": 0, 2024-05-04T03:25:53.820 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_item_in_journal": 0 2024-05-04T03:25:53.820 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.820 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-04T03:25:53.820 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-05-04T03:25:53.820 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-05-04T03:25:53.820 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-05-04T03:25:53.820 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 2044, 2024-05-04T03:25:53.820 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 2473549, 2024-05-04T03:25:53.820 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:53.820 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 2044, 2024-05-04T03:25:53.820 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-05-04T03:25:53.821 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-05-04T03:25:53.821 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 2044, 2024-05-04T03:25:53.821 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 2473549, 2024-05-04T03:25:53.821 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-05-04T03:25:53.821 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.821 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.821 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.821 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:25:53.821 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.821 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_bytes": { 2024-05-04T03:25:53.821 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-05-04T03:25:53.821 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-05-04T03:25:53.821 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-05-04T03:25:53.821 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-05-04T03:25:53.821 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-05-04T03:25:53.822 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:53.822 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-05-04T03:25:53.822 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 1847, 2024-05-04T03:25:53.822 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 1286649, 2024-05-04T03:25:53.822 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 1847, 2024-05-04T03:25:53.822 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 1286649, 2024-05-04T03:25:53.822 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-05-04T03:25:53.822 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.822 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.822 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.822 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:25:53.822 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.823 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_ops": { 2024-05-04T03:25:53.823 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-05-04T03:25:53.823 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 1024, 2024-05-04T03:25:53.823 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-05-04T03:25:53.823 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-05-04T03:25:53.823 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-05-04T03:25:53.823 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:53.823 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-05-04T03:25:53.823 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 1847, 2024-05-04T03:25:53.823 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 1847, 2024-05-04T03:25:53.823 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 1847, 2024-05-04T03:25:53.823 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 1847, 2024-05-04T03:25:53.824 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-05-04T03:25:53.824 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.824 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.824 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.824 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:25:53.824 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.824 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle": { 2024-05-04T03:25:53.824 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-05-04T03:25:53.824 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-05-04T03:25:53.824 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-05-04T03:25:53.824 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-05-04T03:25:53.824 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-05-04T03:25:53.825 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:53.825 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-05-04T03:25:53.825 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-05-04T03:25:53.825 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-05-04T03:25:53.825 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 0, 2024-05-04T03:25:53.825 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 0, 2024-05-04T03:25:53.825 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-05-04T03:25:53.825 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.825 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.825 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.825 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:25:53.825 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:53.825 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle-0x563dd7d048f0": { 2024-05-04T03:25:53.826 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-05-04T03:25:53.826 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-05-04T03:25:53.826 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-05-04T03:25:53.826 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-05-04T03:25:53.826 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-05-04T03:25:53.826 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:53.826 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-05-04T03:25:53.826 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-05-04T03:25:53.826 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-05-04T03:25:53.826 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 0, 2024-05-04T03:25:53.826 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 0, 2024-05-04T03:25:53.826 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-05-04T03:25:53.826 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:53.827 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:53.827 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:53.827 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:25:53.827 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:25:53.827 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-04T03:25:54.558 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-04T03:25:54.558 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-04T03:25:54.558 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-05-04T03:25:54.558 DEBUG:tasks.check_counter:Getting stats from f 2024-05-04T03:25:54.558 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-04T03:25:54.804 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:25:54.845 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:54 smithi077 ceph-mon[33531]: pgmap v593: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:25:54.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:54 smithi001 ceph-mon[29731]: pgmap v593: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:25:54.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:54 smithi072 ceph-mon[33332]: pgmap v593: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:25:56.622 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:56 smithi077 ceph-mon[33531]: pgmap v594: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-04T03:25:56.623 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-04T03:25:56.623 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-05-04T03:25:56.623 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 788, 2024-05-04T03:25:56.623 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 772, 2024-05-04T03:25:56.623 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 1061664, 2024-05-04T03:25:56.623 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 295570, 2024-05-04T03:25:56.623 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 5, 2024-05-04T03:25:56.624 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 1, 2024-05-04T03:25:56.624 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.106694363, 2024-05-04T03:25:56.624 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.032584933, 2024-05-04T03:25:56.624 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.045862135, 2024-05-04T03:25:56.624 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.027160206, 2024-05-04T03:25:56.624 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:56.624 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 771, 2024-05-04T03:25:56.624 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.039255151, 2024-05-04T03:25:56.624 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000050914 2024-05-04T03:25:56.624 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.624 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:56.624 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.625 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.625 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.625 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.625 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 863088, 2024-05-04T03:25:56.625 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 107632 2024-05-04T03:25:56.625 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.625 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-05-04T03:25:56.625 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 3, 2024-05-04T03:25:56.625 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 134, 2024-05-04T03:25:56.625 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 716, 2024-05-04T03:25:56.625 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 304106, 2024-05-04T03:25:56.625 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 4, 2024-05-04T03:25:56.625 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 3, 2024-05-04T03:25:56.625 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.008389179, 2024-05-04T03:25:56.626 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.005228575, 2024-05-04T03:25:56.626 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.001754591, 2024-05-04T03:25:56.626 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000080908, 2024-05-04T03:25:56.626 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:56.626 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 132, 2024-05-04T03:25:56.626 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.008066020, 2024-05-04T03:25:56.626 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000061106 2024-05-04T03:25:56.626 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.626 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:56.626 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.626 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.626 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.626 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.627 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-04T03:25:56.627 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 303408 2024-05-04T03:25:56.627 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.627 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-05-04T03:25:56.627 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 1046, 2024-05-04T03:25:56.627 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 1046, 2024-05-04T03:25:56.627 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 354872, 2024-05-04T03:25:56.627 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 331811, 2024-05-04T03:25:56.627 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 4, 2024-05-04T03:25:56.627 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 3, 2024-05-04T03:25:56.627 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.080179601, 2024-05-04T03:25:56.627 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.025811075, 2024-05-04T03:25:56.627 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.022424405, 2024-05-04T03:25:56.627 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.026619674, 2024-05-04T03:25:56.628 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:56.628 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1045, 2024-05-04T03:25:56.628 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.037580792, 2024-05-04T03:25:56.628 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000035962 2024-05-04T03:25:56.628 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.628 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:56.628 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.628 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.628 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.628 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.628 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-04T03:25:56.628 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 448 2024-05-04T03:25:56.628 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.628 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-05-04T03:25:56.629 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 1, 2024-05-04T03:25:56.629 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-05-04T03:25:56.629 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.629 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-MDSRank": { 2024-05-04T03:25:56.629 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-05-04T03:25:56.629 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-05-04T03:25:56.629 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2830, 2024-05-04T03:25:56.629 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.081231650, 2024-05-04T03:25:56.629 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000028703 2024-05-04T03:25:56.629 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:25:56.629 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.629 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-PurgeQueue": { 2024-05-04T03:25:56.629 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-05-04T03:25:56.629 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-05-04T03:25:56.629 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.630 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.630 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.630 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:25:56.630 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.630 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-05-04T03:25:56.630 INFO:teuthology.orchestra.run.smithi077.stdout: "request": 0, 2024-05-04T03:25:56.630 INFO:teuthology.orchestra.run.smithi077.stdout: "reply": 0, 2024-05-04T03:25:56.630 INFO:teuthology.orchestra.run.smithi077.stdout: "reply_latency": { 2024-05-04T03:25:56.630 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.630 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.630 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.630 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.630 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_reply": 0, 2024-05-04T03:25:56.631 INFO:teuthology.orchestra.run.smithi077.stdout: "forward": 0, 2024-05-04T03:25:56.631 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_complete": 0, 2024-05-04T03:25:56.631 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_keys": 0, 2024-05-04T03:25:56.631 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_commit": 0, 2024-05-04T03:25:56.631 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_split": 0, 2024-05-04T03:25:56.631 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_merge": 0, 2024-05-04T03:25:56.631 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes": 10, 2024-05-04T03:25:56.631 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_top": 0, 2024-05-04T03:25:56.631 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_bottom": 10, 2024-05-04T03:25:56.631 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pin_tail": 0, 2024-05-04T03:25:56.631 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pinned": 10, 2024-05-04T03:25:56.631 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_expired": 0, 2024-05-04T03:25:56.631 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_with_caps": 0, 2024-05-04T03:25:56.631 INFO:teuthology.orchestra.run.smithi077.stdout: "caps": 0, 2024-05-04T03:25:56.632 INFO:teuthology.orchestra.run.smithi077.stdout: "subtrees": 1, 2024-05-04T03:25:56.632 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse": 0, 2024-05-04T03:25:56.632 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_hit": 0, 2024-05-04T03:25:56.632 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_forward": 0, 2024-05-04T03:25:56.632 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_discover": 0, 2024-05-04T03:25:56.632 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_dir_fetch": 0, 2024-05-04T03:25:56.632 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_remote_ino": 0, 2024-05-04T03:25:56.632 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_lock": 0, 2024-05-04T03:25:56.632 INFO:teuthology.orchestra.run.smithi077.stdout: "load_cent": 0, 2024-05-04T03:25:56.632 INFO:teuthology.orchestra.run.smithi077.stdout: "q": 0, 2024-05-04T03:25:56.632 INFO:teuthology.orchestra.run.smithi077.stdout: "exported": 0, 2024-05-04T03:25:56.632 INFO:teuthology.orchestra.run.smithi077.stdout: "exported_inodes": 0, 2024-05-04T03:25:56.632 INFO:teuthology.orchestra.run.smithi077.stdout: "imported": 0, 2024-05-04T03:25:56.632 INFO:teuthology.orchestra.run.smithi077.stdout: "imported_inodes": 0, 2024-05-04T03:25:56.632 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_dir_fetch": 0, 2024-05-04T03:25:56.633 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_backtrace_fetch": 0, 2024-05-04T03:25:56.633 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_peer_discover": 0, 2024-05-04T03:25:56.633 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rfiles": 0, 2024-05-04T03:25:56.633 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rbytes": 0, 2024-05-04T03:25:56.633 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rsnaps": 0, 2024-05-04T03:25:56.633 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_fetch": 0, 2024-05-04T03:25:56.633 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_set_tag": 0, 2024-05-04T03:25:56.633 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_repaired": 0, 2024-05-04T03:25:56.633 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_inotable_repaired": 0, 2024-05-04T03:25:56.633 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_inodes": 0, 2024-05-04T03:25:56.633 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_base_inodes": 0, 2024-05-04T03:25:56.633 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dirfrag_rstats": 0, 2024-05-04T03:25:56.633 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_file_inodes": 0, 2024-05-04T03:25:56.633 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_inode_file_caps": 0, 2024-05-04T03:25:56.633 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_revoke": 0, 2024-05-04T03:25:56.634 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_grant": 0, 2024-05-04T03:25:56.634 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_trunc": 0, 2024-05-04T03:25:56.634 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-04T03:25:56.634 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-04T03:25:56.634 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps": 0, 2024-05-04T03:25:56.634 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps_dirty": 0, 2024-05-04T03:25:56.634 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_cap_release": 0, 2024-05-04T03:25:56.634 INFO:teuthology.orchestra.run.smithi077.stdout: "process_request_cap_release": 0 2024-05-04T03:25:56.634 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.634 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache": { 2024-05-04T03:25:56.634 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update": 0, 2024-05-04T03:25:56.634 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update_receipt": 0, 2024-05-04T03:25:56.634 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_try_discover": 0, 2024-05-04T03:25:56.634 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_send_discover": 0, 2024-05-04T03:25:56.635 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_handle_discover": 0, 2024-05-04T03:25:56.635 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays": 0, 2024-05-04T03:25:56.635 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_delayed": 0, 2024-05-04T03:25:56.635 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_enqueuing": 0, 2024-05-04T03:25:56.635 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_created": 0, 2024-05-04T03:25:56.635 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_enqueued": 0, 2024-05-04T03:25:56.635 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_reintegrated": 0, 2024-05-04T03:25:56.635 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_migrated": 0, 2024-05-04T03:25:56.635 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_processing": 0, 2024-05-04T03:25:56.635 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_enqueued": 0, 2024-05-04T03:25:56.635 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_prioritized": 0, 2024-05-04T03:25:56.635 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_started": 0, 2024-05-04T03:25:56.635 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_completed": 0, 2024-05-04T03:25:56.635 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_path": 0, 2024-05-04T03:25:56.635 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_inode": 0, 2024-05-04T03:25:56.636 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_enqueue_scrub": 0, 2024-05-04T03:25:56.636 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_exportdir": 0, 2024-05-04T03:25:56.636 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_flush": 0, 2024-05-04T03:25:56.636 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragmentdir": 0, 2024-05-04T03:25:56.636 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragstats": 0, 2024-05-04T03:25:56.636 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_inodestats": 0 2024-05-04T03:25:56.636 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.636 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log": { 2024-05-04T03:25:56.636 INFO:teuthology.orchestra.run.smithi077.stdout: "evlrg": 0, 2024-05-04T03:25:56.636 INFO:teuthology.orchestra.run.smithi077.stdout: "evadd": 0, 2024-05-04T03:25:56.636 INFO:teuthology.orchestra.run.smithi077.stdout: "evex": 0, 2024-05-04T03:25:56.636 INFO:teuthology.orchestra.run.smithi077.stdout: "evtrm": 0, 2024-05-04T03:25:56.636 INFO:teuthology.orchestra.run.smithi077.stdout: "ev": 54, 2024-05-04T03:25:56.636 INFO:teuthology.orchestra.run.smithi077.stdout: "evexg": 0, 2024-05-04T03:25:56.637 INFO:teuthology.orchestra.run.smithi077.stdout: "evexd": 0, 2024-05-04T03:25:56.637 INFO:teuthology.orchestra.run.smithi077.stdout: "segadd": 0, 2024-05-04T03:25:56.637 INFO:teuthology.orchestra.run.smithi077.stdout: "segex": 0, 2024-05-04T03:25:56.637 INFO:teuthology.orchestra.run.smithi077.stdout: "segtrm": 0, 2024-05-04T03:25:56.637 INFO:teuthology.orchestra.run.smithi077.stdout: "seg": 2, 2024-05-04T03:25:56.637 INFO:teuthology.orchestra.run.smithi077.stdout: "segmjr": 2, 2024-05-04T03:25:56.637 INFO:teuthology.orchestra.run.smithi077.stdout: "segexg": 0, 2024-05-04T03:25:56.637 INFO:teuthology.orchestra.run.smithi077.stdout: "segexd": 0, 2024-05-04T03:25:56.637 INFO:teuthology.orchestra.run.smithi077.stdout: "expos": 4194304, 2024-05-04T03:25:56.637 INFO:teuthology.orchestra.run.smithi077.stdout: "wrpos": 4279751, 2024-05-04T03:25:56.637 INFO:teuthology.orchestra.run.smithi077.stdout: "rdpos": 4278100, 2024-05-04T03:25:56.637 INFO:teuthology.orchestra.run.smithi077.stdout: "jlat": { 2024-05-04T03:25:56.637 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.637 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.637 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.638 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.638 INFO:teuthology.orchestra.run.smithi077.stdout: "replayed": 54 2024-05-04T03:25:56.638 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.638 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_mem": { 2024-05-04T03:25:56.638 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 4, 2024-05-04T03:25:56.638 INFO:teuthology.orchestra.run.smithi077.stdout: "ino+": 26, 2024-05-04T03:25:56.638 INFO:teuthology.orchestra.run.smithi077.stdout: "ino-": 22, 2024-05-04T03:25:56.638 INFO:teuthology.orchestra.run.smithi077.stdout: "dir": 2, 2024-05-04T03:25:56.638 INFO:teuthology.orchestra.run.smithi077.stdout: "dir+": 16, 2024-05-04T03:25:56.638 INFO:teuthology.orchestra.run.smithi077.stdout: "dir-": 14, 2024-05-04T03:25:56.638 INFO:teuthology.orchestra.run.smithi077.stdout: "dn": 10, 2024-05-04T03:25:56.638 INFO:teuthology.orchestra.run.smithi077.stdout: "dn+": 38, 2024-05-04T03:25:56.638 INFO:teuthology.orchestra.run.smithi077.stdout: "dn-": 28, 2024-05-04T03:25:56.638 INFO:teuthology.orchestra.run.smithi077.stdout: "cap": 0, 2024-05-04T03:25:56.639 INFO:teuthology.orchestra.run.smithi077.stdout: "cap+": 0, 2024-05-04T03:25:56.639 INFO:teuthology.orchestra.run.smithi077.stdout: "cap-": 0, 2024-05-04T03:25:56.639 INFO:teuthology.orchestra.run.smithi077.stdout: "rss": 119692, 2024-05-04T03:25:56.639 INFO:teuthology.orchestra.run.smithi077.stdout: "heap": 215440 2024-05-04T03:25:56.639 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.639 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_server": { 2024-05-04T03:25:56.639 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_client_request": 0, 2024-05-04T03:25:56.639 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_server_request": 0, 2024-05-04T03:25:56.639 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_request": 0, 2024-05-04T03:25:56.639 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_session": 0, 2024-05-04T03:25:56.639 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_peer_request": 0, 2024-05-04T03:25:56.639 INFO:teuthology.orchestra.run.smithi077.stdout: "req_create_latency": { 2024-05-04T03:25:56.639 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.639 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.639 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.640 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.640 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getattr_latency": { 2024-05-04T03:25:56.640 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.640 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.640 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.640 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.640 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getfilelock_latency": { 2024-05-04T03:25:56.640 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.640 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.640 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.640 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.640 INFO:teuthology.orchestra.run.smithi077.stdout: "req_link_latency": { 2024-05-04T03:25:56.640 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.640 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.640 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.641 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.641 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookup_latency": { 2024-05-04T03:25:56.641 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.641 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.641 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.641 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.641 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookuphash_latency": { 2024-05-04T03:25:56.641 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.641 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.641 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.641 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.641 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupino_latency": { 2024-05-04T03:25:56.641 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.641 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.642 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.642 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.642 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupname_latency": { 2024-05-04T03:25:56.642 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.642 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.642 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.642 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.642 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupparent_latency": { 2024-05-04T03:25:56.642 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.642 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.642 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.642 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.642 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupsnap_latency": { 2024-05-04T03:25:56.642 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.643 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.643 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.643 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.643 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lssnap_latency": { 2024-05-04T03:25:56.643 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.643 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.643 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.643 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.643 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mkdir_latency": { 2024-05-04T03:25:56.643 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.644 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.644 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.644 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.644 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mknod_latency": { 2024-05-04T03:25:56.644 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.644 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.644 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.644 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.644 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mksnap_latency": { 2024-05-04T03:25:56.645 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.645 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.645 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.645 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.645 INFO:teuthology.orchestra.run.smithi077.stdout: "req_open_latency": { 2024-05-04T03:25:56.645 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.645 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.645 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.645 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.645 INFO:teuthology.orchestra.run.smithi077.stdout: "req_readdir_latency": { 2024-05-04T03:25:56.645 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.645 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.645 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.645 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.646 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rename_latency": { 2024-05-04T03:25:56.646 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.646 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.646 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.646 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.646 INFO:teuthology.orchestra.run.smithi077.stdout: "req_renamesnap_latency": { 2024-05-04T03:25:56.646 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.646 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.646 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.646 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.646 INFO:teuthology.orchestra.run.smithi077.stdout: "req_snapdiff_latency": { 2024-05-04T03:25:56.646 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.646 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.647 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.647 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.647 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmdir_latency": { 2024-05-04T03:25:56.647 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.647 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.647 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.647 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.647 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmsnap_latency": { 2024-05-04T03:25:56.647 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.647 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.647 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.647 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.647 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmxattr_latency": { 2024-05-04T03:25:56.647 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.647 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.648 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.648 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.648 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setattr_latency": { 2024-05-04T03:25:56.648 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.648 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.648 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.648 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.648 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setdirlayout_latency": { 2024-05-04T03:25:56.648 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.648 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.648 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.648 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.648 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setfilelock_latency": { 2024-05-04T03:25:56.648 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.649 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.649 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.649 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.649 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setlayout_latency": { 2024-05-04T03:25:56.649 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.649 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.649 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.649 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.649 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setxattr_latency": { 2024-05-04T03:25:56.649 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.649 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.649 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.649 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.649 INFO:teuthology.orchestra.run.smithi077.stdout: "req_symlink_latency": { 2024-05-04T03:25:56.650 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.650 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.650 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.650 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.650 INFO:teuthology.orchestra.run.smithi077.stdout: "req_unlink_latency": { 2024-05-04T03:25:56.650 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.650 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.650 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.650 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.650 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_revoke_eviction": 0, 2024-05-04T03:25:56.650 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_acquisition_throttle": 0, 2024-05-04T03:25:56.650 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getvxattr_latency": { 2024-05-04T03:25:56.650 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.650 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.651 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.651 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:25:56.651 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.651 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_sessions": { 2024-05-04T03:25:56.651 INFO:teuthology.orchestra.run.smithi077.stdout: "session_count": 1, 2024-05-04T03:25:56.651 INFO:teuthology.orchestra.run.smithi077.stdout: "session_add": 1, 2024-05-04T03:25:56.651 INFO:teuthology.orchestra.run.smithi077.stdout: "session_remove": 0, 2024-05-04T03:25:56.651 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_open": 1, 2024-05-04T03:25:56.651 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_stale": 0, 2024-05-04T03:25:56.651 INFO:teuthology.orchestra.run.smithi077.stdout: "total_load": 0, 2024-05-04T03:25:56.651 INFO:teuthology.orchestra.run.smithi077.stdout: "average_load": 0, 2024-05-04T03:25:56.651 INFO:teuthology.orchestra.run.smithi077.stdout: "avg_session_uptime": 1690, 2024-05-04T03:25:56.651 INFO:teuthology.orchestra.run.smithi077.stdout: "mdthresh_evicted": 0 2024-05-04T03:25:56.651 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.652 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-05-04T03:25:56.652 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-05-04T03:25:56.652 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-05-04T03:25:56.652 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-05-04T03:25:56.652 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-05-04T03:25:56.652 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-05-04T03:25:56.652 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-05-04T03:25:56.652 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-04T03:25:56.652 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-05-04T03:25:56.652 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-04T03:25:56.652 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-05-04T03:25:56.652 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-05-04T03:25:56.652 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-05-04T03:25:56.652 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-04T03:25:56.653 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-05-04T03:25:56.653 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-05-04T03:25:56.653 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-05-04T03:25:56.653 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-05-04T03:25:56.653 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-05-04T03:25:56.653 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-04T03:25:56.653 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-05-04T03:25:56.653 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-04T03:25:56.653 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-05-04T03:25:56.653 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-05-04T03:25:56.653 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-05-04T03:25:56.653 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-05-04T03:25:56.653 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-05-04T03:25:56.653 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-04T03:25:56.653 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-05-04T03:25:56.654 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-05-04T03:25:56.654 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-05-04T03:25:56.654 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-05-04T03:25:56.654 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-05-04T03:25:56.654 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-05-04T03:25:56.654 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-05-04T03:25:56.654 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-05-04T03:25:56.654 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-05-04T03:25:56.654 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 182334, 2024-05-04T03:25:56.654 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 47, 2024-05-04T03:25:56.654 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-05-04T03:25:56.654 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-05-04T03:25:56.654 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-05-04T03:25:56.654 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-05-04T03:25:56.655 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-05-04T03:25:56.655 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-05-04T03:25:56.655 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-05-04T03:25:56.655 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-05-04T03:25:56.655 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 7240, 2024-05-04T03:25:56.655 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 152, 2024-05-04T03:25:56.655 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-05-04T03:25:56.655 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-05-04T03:25:56.655 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-05-04T03:25:56.655 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-05-04T03:25:56.655 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 19088, 2024-05-04T03:25:56.655 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 68, 2024-05-04T03:25:56.655 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-05-04T03:25:56.655 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-05-04T03:25:56.655 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-05-04T03:25:56.656 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-05-04T03:25:56.656 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.656 INFO:teuthology.orchestra.run.smithi077.stdout: "objecter": { 2024-05-04T03:25:56.656 INFO:teuthology.orchestra.run.smithi077.stdout: "op_active": 0, 2024-05-04T03:25:56.656 INFO:teuthology.orchestra.run.smithi077.stdout: "op_laggy": 0, 2024-05-04T03:25:56.656 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send": 1635, 2024-05-04T03:25:56.656 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send_bytes": 21, 2024-05-04T03:25:56.656 INFO:teuthology.orchestra.run.smithi077.stdout: "op_resend": 3, 2024-05-04T03:25:56.656 INFO:teuthology.orchestra.run.smithi077.stdout: "op_reply": 1635, 2024-05-04T03:25:56.656 INFO:teuthology.orchestra.run.smithi077.stdout: "op_latency": { 2024-05-04T03:25:56.656 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1635, 2024-05-04T03:25:56.656 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.831983620, 2024-05-04T03:25:56.656 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000508858 2024-05-04T03:25:56.656 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.657 INFO:teuthology.orchestra.run.smithi077.stdout: "op_inflight": 0, 2024-05-04T03:25:56.657 INFO:teuthology.orchestra.run.smithi077.stdout: "oplen_avg": { 2024-05-04T03:25:56.657 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1635, 2024-05-04T03:25:56.657 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 1636 2024-05-04T03:25:56.657 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.657 INFO:teuthology.orchestra.run.smithi077.stdout: "op": 1635, 2024-05-04T03:25:56.657 INFO:teuthology.orchestra.run.smithi077.stdout: "op_r": 1635, 2024-05-04T03:25:56.657 INFO:teuthology.orchestra.run.smithi077.stdout: "op_w": 0, 2024-05-04T03:25:56.657 INFO:teuthology.orchestra.run.smithi077.stdout: "op_rmw": 0, 2024-05-04T03:25:56.657 INFO:teuthology.orchestra.run.smithi077.stdout: "op_pg": 0, 2024-05-04T03:25:56.657 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_stat": 1023, 2024-05-04T03:25:56.657 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_create": 0, 2024-05-04T03:25:56.657 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_read": 610, 2024-05-04T03:25:56.657 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_write": 0, 2024-05-04T03:25:56.658 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writefull": 0, 2024-05-04T03:25:56.658 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writesame": 0, 2024-05-04T03:25:56.658 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_append": 0, 2024-05-04T03:25:56.658 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_zero": 0, 2024-05-04T03:25:56.658 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_truncate": 0, 2024-05-04T03:25:56.658 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_delete": 0, 2024-05-04T03:25:56.658 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_mapext": 0, 2024-05-04T03:25:56.658 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_sparse_read": 0, 2024-05-04T03:25:56.658 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_clonerange": 0, 2024-05-04T03:25:56.658 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_getxattr": 1, 2024-05-04T03:25:56.658 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_setxattr": 0, 2024-05-04T03:25:56.658 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_cmpxattr": 0, 2024-05-04T03:25:56.658 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_rmxattr": 0, 2024-05-04T03:25:56.658 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_resetxattrs": 0, 2024-05-04T03:25:56.659 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_call": 0, 2024-05-04T03:25:56.659 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_watch": 0, 2024-05-04T03:25:56.659 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_notify": 0, 2024-05-04T03:25:56.659 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_src_cmpxattr": 0, 2024-05-04T03:25:56.659 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls": 0, 2024-05-04T03:25:56.659 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls_filter": 0, 2024-05-04T03:25:56.659 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_other": 0, 2024-05-04T03:25:56.659 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_active": 0, 2024-05-04T03:25:56.659 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_send": 0, 2024-05-04T03:25:56.659 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_resend": 0, 2024-05-04T03:25:56.659 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_ping": 0, 2024-05-04T03:25:56.659 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_active": 0, 2024-05-04T03:25:56.659 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_send": 0, 2024-05-04T03:25:56.659 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_resend": 0, 2024-05-04T03:25:56.659 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_active": 0, 2024-05-04T03:25:56.660 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_send": 0, 2024-05-04T03:25:56.660 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_resend": 0, 2024-05-04T03:25:56.660 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_active": 0, 2024-05-04T03:25:56.660 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_send": 0, 2024-05-04T03:25:56.660 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_resend": 0, 2024-05-04T03:25:56.660 INFO:teuthology.orchestra.run.smithi077.stdout: "command_active": 0, 2024-05-04T03:25:56.660 INFO:teuthology.orchestra.run.smithi077.stdout: "command_send": 0, 2024-05-04T03:25:56.660 INFO:teuthology.orchestra.run.smithi077.stdout: "command_resend": 0, 2024-05-04T03:25:56.660 INFO:teuthology.orchestra.run.smithi077.stdout: "map_epoch": 83, 2024-05-04T03:25:56.660 INFO:teuthology.orchestra.run.smithi077.stdout: "map_full": 0, 2024-05-04T03:25:56.660 INFO:teuthology.orchestra.run.smithi077.stdout: "map_inc": 2, 2024-05-04T03:25:56.660 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_sessions": 4, 2024-05-04T03:25:56.660 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_open": 4, 2024-05-04T03:25:56.660 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_close": 0, 2024-05-04T03:25:56.661 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_laggy": 0, 2024-05-04T03:25:56.661 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_wr": 0, 2024-05-04T03:25:56.661 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_rd": 2, 2024-05-04T03:25:56.661 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_del": 0 2024-05-04T03:25:56.661 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.661 INFO:teuthology.orchestra.run.smithi077.stdout: "oft": { 2024-05-04T03:25:56.661 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_objs": 0, 2024-05-04T03:25:56.661 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_kv_pairs": 0, 2024-05-04T03:25:56.661 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_updates": 0, 2024-05-04T03:25:56.661 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_removes": 0 2024-05-04T03:25:56.661 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.661 INFO:teuthology.orchestra.run.smithi077.stdout: "osd-slow-ops": { 2024-05-04T03:25:56.661 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_ops_count": 0 2024-05-04T03:25:56.662 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.662 INFO:teuthology.orchestra.run.smithi077.stdout: "purge_queue": { 2024-05-04T03:25:56.662 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops": 0, 2024-05-04T03:25:56.662 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops_high_water": 0, 2024-05-04T03:25:56.662 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing": 0, 2024-05-04T03:25:56.662 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_high_water": 0, 2024-05-04T03:25:56.662 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed_ops": 0, 2024-05-04T03:25:56.662 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed": 0, 2024-05-04T03:25:56.662 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_item_in_journal": 0 2024-05-04T03:25:56.662 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.662 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-04T03:25:56.662 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-05-04T03:25:56.662 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-05-04T03:25:56.663 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-05-04T03:25:56.663 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 1833, 2024-05-04T03:25:56.663 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 1239184, 2024-05-04T03:25:56.663 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:56.663 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 1833, 2024-05-04T03:25:56.663 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-05-04T03:25:56.663 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-05-04T03:25:56.663 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 1833, 2024-05-04T03:25:56.663 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 1239184, 2024-05-04T03:25:56.663 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-05-04T03:25:56.663 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.663 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.663 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.663 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:25:56.664 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.664 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_bytes": { 2024-05-04T03:25:56.664 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-05-04T03:25:56.664 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-05-04T03:25:56.664 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-05-04T03:25:56.664 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-05-04T03:25:56.664 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-05-04T03:25:56.664 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:56.664 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-05-04T03:25:56.664 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 1635, 2024-05-04T03:25:56.664 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 85452, 2024-05-04T03:25:56.664 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 1635, 2024-05-04T03:25:56.664 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 85452, 2024-05-04T03:25:56.664 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-05-04T03:25:56.665 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.665 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.665 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.665 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:25:56.665 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.665 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_ops": { 2024-05-04T03:25:56.665 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-05-04T03:25:56.665 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 1024, 2024-05-04T03:25:56.665 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-05-04T03:25:56.665 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-05-04T03:25:56.665 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-05-04T03:25:56.665 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:56.665 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-05-04T03:25:56.666 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 1635, 2024-05-04T03:25:56.666 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 1635, 2024-05-04T03:25:56.666 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 1635, 2024-05-04T03:25:56.666 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 1635, 2024-05-04T03:25:56.666 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-05-04T03:25:56.666 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.666 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.666 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.666 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:25:56.666 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.666 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle": { 2024-05-04T03:25:56.666 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-05-04T03:25:56.666 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-05-04T03:25:56.666 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-05-04T03:25:56.666 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-05-04T03:25:56.667 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-05-04T03:25:56.667 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:56.667 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-05-04T03:25:56.667 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-05-04T03:25:56.667 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-05-04T03:25:56.667 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 0, 2024-05-04T03:25:56.667 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 0, 2024-05-04T03:25:56.667 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-05-04T03:25:56.667 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.667 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.667 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.667 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:25:56.667 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:56.667 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle-0x55e4e501a410": { 2024-05-04T03:25:56.668 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-05-04T03:25:56.668 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-05-04T03:25:56.668 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-05-04T03:25:56.668 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-05-04T03:25:56.668 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-05-04T03:25:56.668 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:56.668 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-05-04T03:25:56.668 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-05-04T03:25:56.668 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-05-04T03:25:56.668 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 0, 2024-05-04T03:25:56.668 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 0, 2024-05-04T03:25:56.668 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-05-04T03:25:56.668 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:56.668 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:56.669 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:56.669 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:25:56.669 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:25:56.669 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-04T03:25:56.746 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:56 smithi072 ceph-mon[33332]: pgmap v594: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-04T03:25:56.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:56 smithi001 ceph-mon[29731]: pgmap v594: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-04T03:25:57.407 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-04T03:25:57.408 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-04T03:25:57.408 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-05-04T03:25:57.408 DEBUG:tasks.check_counter:Getting stats from i 2024-05-04T03:25:57.408 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-04T03:25:57.650 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:25:58.770 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:25:58 smithi077 ceph-mon[33531]: pgmap v595: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-04T03:25:58.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:25:58 smithi001 ceph-mon[29731]: pgmap v595: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-04T03:25:58.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:25:58 smithi072 ceph-mon[33332]: pgmap v595: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-04T03:25:59.380 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-04T03:25:59.380 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-05-04T03:25:59.380 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 501, 2024-05-04T03:25:59.380 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 578, 2024-05-04T03:25:59.380 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 154323, 2024-05-04T03:25:59.380 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 295065, 2024-05-04T03:25:59.380 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 9, 2024-05-04T03:25:59.381 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 4, 2024-05-04T03:25:59.381 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.054935908, 2024-05-04T03:25:59.381 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.020799889, 2024-05-04T03:25:59.381 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.027283179, 2024-05-04T03:25:59.381 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.005516303, 2024-05-04T03:25:59.381 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:59.381 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 576, 2024-05-04T03:25:59.381 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.037588825, 2024-05-04T03:25:59.381 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000065258 2024-05-04T03:25:59.381 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.381 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:59.381 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 930, 2024-05-04T03:25:59.382 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000271487, 2024-05-04T03:25:59.382 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000291 2024-05-04T03:25:59.382 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.382 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-04T03:25:59.382 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 0 2024-05-04T03:25:59.382 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.382 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-05-04T03:25:59.382 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 310, 2024-05-04T03:25:59.382 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 318, 2024-05-04T03:25:59.382 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 902287, 2024-05-04T03:25:59.382 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 206794, 2024-05-04T03:25:59.382 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 10, 2024-05-04T03:25:59.382 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 6, 2024-05-04T03:25:59.383 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.046589321, 2024-05-04T03:25:59.383 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.015305530, 2024-05-04T03:25:59.383 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.032419063, 2024-05-04T03:25:59.383 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.004731907, 2024-05-04T03:25:59.383 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:59.383 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 318, 2024-05-04T03:25:59.383 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.027128208, 2024-05-04T03:25:59.383 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000085308 2024-05-04T03:25:59.383 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.383 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:59.383 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 83, 2024-05-04T03:25:59.383 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000036270, 2024-05-04T03:25:59.383 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000436 2024-05-04T03:25:59.383 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.383 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 881424, 2024-05-04T03:25:59.384 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 111456 2024-05-04T03:25:59.384 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.384 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-05-04T03:25:59.384 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 411, 2024-05-04T03:25:59.384 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 395, 2024-05-04T03:25:59.384 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 146998, 2024-05-04T03:25:59.384 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 415327, 2024-05-04T03:25:59.384 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 9, 2024-05-04T03:25:59.384 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 7, 2024-05-04T03:25:59.384 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.037235423, 2024-05-04T03:25:59.384 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.015101748, 2024-05-04T03:25:59.384 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.025745933, 2024-05-04T03:25:59.384 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.001352671, 2024-05-04T03:25:59.384 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:25:59.384 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 393, 2024-05-04T03:25:59.385 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.018975654, 2024-05-04T03:25:59.385 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000048284 2024-05-04T03:25:59.385 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.385 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:25:59.385 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 608, 2024-05-04T03:25:59.385 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000110735, 2024-05-04T03:25:59.385 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000182 2024-05-04T03:25:59.385 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.385 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-04T03:25:59.385 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 303056 2024-05-04T03:25:59.385 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.385 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-05-04T03:25:59.385 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 1, 2024-05-04T03:25:59.385 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-05-04T03:25:59.385 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.385 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-MDSRank": { 2024-05-04T03:25:59.386 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-05-04T03:25:59.386 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-05-04T03:25:59.386 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 98, 2024-05-04T03:25:59.386 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.013180751, 2024-05-04T03:25:59.386 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000134497 2024-05-04T03:25:59.386 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:25:59.386 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.386 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-PurgeQueue": { 2024-05-04T03:25:59.386 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-05-04T03:25:59.386 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-05-04T03:25:59.386 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 23, 2024-05-04T03:25:59.386 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000932986, 2024-05-04T03:25:59.386 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000040564 2024-05-04T03:25:59.386 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:25:59.387 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.387 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-05-04T03:25:59.387 INFO:teuthology.orchestra.run.smithi077.stdout: "request": 33, 2024-05-04T03:25:59.387 INFO:teuthology.orchestra.run.smithi077.stdout: "reply": 23, 2024-05-04T03:25:59.387 INFO:teuthology.orchestra.run.smithi077.stdout: "reply_latency": { 2024-05-04T03:25:59.387 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 23, 2024-05-04T03:25:59.387 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.026713767, 2024-05-04T03:25:59.387 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.001161468 2024-05-04T03:25:59.387 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.387 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_reply": 0, 2024-05-04T03:25:59.387 INFO:teuthology.orchestra.run.smithi077.stdout: "forward": 10, 2024-05-04T03:25:59.387 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_complete": 0, 2024-05-04T03:25:59.387 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_keys": 0, 2024-05-04T03:25:59.387 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_commit": 11, 2024-05-04T03:25:59.387 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_split": 0, 2024-05-04T03:25:59.388 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_merge": 0, 2024-05-04T03:25:59.388 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes": 18, 2024-05-04T03:25:59.388 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_top": 1, 2024-05-04T03:25:59.388 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_bottom": 17, 2024-05-04T03:25:59.388 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pin_tail": 0, 2024-05-04T03:25:59.388 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pinned": 14, 2024-05-04T03:25:59.388 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_expired": 12, 2024-05-04T03:25:59.388 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_with_caps": 2, 2024-05-04T03:25:59.388 INFO:teuthology.orchestra.run.smithi077.stdout: "caps": 2, 2024-05-04T03:25:59.388 INFO:teuthology.orchestra.run.smithi077.stdout: "subtrees": 4, 2024-05-04T03:25:59.388 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse": 75, 2024-05-04T03:25:59.388 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_hit": 41, 2024-05-04T03:25:59.388 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_forward": 9, 2024-05-04T03:25:59.388 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_discover": 8, 2024-05-04T03:25:59.388 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_dir_fetch": 0, 2024-05-04T03:25:59.388 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_remote_ino": 0, 2024-05-04T03:25:59.388 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_lock": 0, 2024-05-04T03:25:59.389 INFO:teuthology.orchestra.run.smithi077.stdout: "load_cent": 0, 2024-05-04T03:25:59.389 INFO:teuthology.orchestra.run.smithi077.stdout: "q": 0, 2024-05-04T03:25:59.389 INFO:teuthology.orchestra.run.smithi077.stdout: "exported": 2, 2024-05-04T03:25:59.389 INFO:teuthology.orchestra.run.smithi077.stdout: "exported_inodes": 0, 2024-05-04T03:25:59.389 INFO:teuthology.orchestra.run.smithi077.stdout: "imported": 2, 2024-05-04T03:25:59.389 INFO:teuthology.orchestra.run.smithi077.stdout: "imported_inodes": 13, 2024-05-04T03:25:59.389 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_dir_fetch": 0, 2024-05-04T03:25:59.389 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_backtrace_fetch": 0, 2024-05-04T03:25:59.389 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_peer_discover": 0, 2024-05-04T03:25:59.389 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rfiles": 4, 2024-05-04T03:25:59.389 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rbytes": 234, 2024-05-04T03:25:59.389 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rsnaps": 0, 2024-05-04T03:25:59.389 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_fetch": 0, 2024-05-04T03:25:59.389 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_set_tag": 0, 2024-05-04T03:25:59.389 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_repaired": 0, 2024-05-04T03:25:59.389 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_inotable_repaired": 0, 2024-05-04T03:25:59.390 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_inodes": 0, 2024-05-04T03:25:59.390 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_base_inodes": 0, 2024-05-04T03:25:59.390 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dirfrag_rstats": 0, 2024-05-04T03:25:59.390 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_file_inodes": 0, 2024-05-04T03:25:59.390 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_inode_file_caps": 0, 2024-05-04T03:25:59.390 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_revoke": 10, 2024-05-04T03:25:59.390 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_grant": 17, 2024-05-04T03:25:59.390 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_trunc": 0, 2024-05-04T03:25:59.390 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-04T03:25:59.390 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-04T03:25:59.390 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps": 19, 2024-05-04T03:25:59.390 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps_dirty": 9, 2024-05-04T03:25:59.390 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_cap_release": 4, 2024-05-04T03:25:59.390 INFO:teuthology.orchestra.run.smithi077.stdout: "process_request_cap_release": 21 2024-05-04T03:25:59.391 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.391 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache": { 2024-05-04T03:25:59.391 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update": 0, 2024-05-04T03:25:59.391 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update_receipt": 0, 2024-05-04T03:25:59.391 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_try_discover": 0, 2024-05-04T03:25:59.391 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_send_discover": 7, 2024-05-04T03:25:59.391 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_handle_discover": 0, 2024-05-04T03:25:59.391 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays": 0, 2024-05-04T03:25:59.391 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_delayed": 0, 2024-05-04T03:25:59.391 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_enqueuing": 0, 2024-05-04T03:25:59.391 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_created": 9, 2024-05-04T03:25:59.391 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_enqueued": 9, 2024-05-04T03:25:59.391 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_reintegrated": 0, 2024-05-04T03:25:59.391 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_migrated": 0, 2024-05-04T03:25:59.391 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_processing": 0, 2024-05-04T03:25:59.391 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_enqueued": 0, 2024-05-04T03:25:59.391 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_prioritized": 0, 2024-05-04T03:25:59.392 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_started": 0, 2024-05-04T03:25:59.392 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_completed": 0, 2024-05-04T03:25:59.392 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_path": 4, 2024-05-04T03:25:59.392 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_inode": 50, 2024-05-04T03:25:59.392 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_enqueue_scrub": 0, 2024-05-04T03:25:59.392 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_exportdir": 2, 2024-05-04T03:25:59.392 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_flush": 0, 2024-05-04T03:25:59.392 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragmentdir": 0, 2024-05-04T03:25:59.392 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragstats": 0, 2024-05-04T03:25:59.392 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_inodestats": 0 2024-05-04T03:25:59.392 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.392 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log": { 2024-05-04T03:25:59.392 INFO:teuthology.orchestra.run.smithi077.stdout: "evlrg": 0, 2024-05-04T03:25:59.392 INFO:teuthology.orchestra.run.smithi077.stdout: "evadd": 54, 2024-05-04T03:25:59.392 INFO:teuthology.orchestra.run.smithi077.stdout: "evex": 0, 2024-05-04T03:25:59.392 INFO:teuthology.orchestra.run.smithi077.stdout: "evtrm": 0, 2024-05-04T03:25:59.393 INFO:teuthology.orchestra.run.smithi077.stdout: "ev": 54, 2024-05-04T03:25:59.393 INFO:teuthology.orchestra.run.smithi077.stdout: "evexg": 0, 2024-05-04T03:25:59.393 INFO:teuthology.orchestra.run.smithi077.stdout: "evexd": 0, 2024-05-04T03:25:59.393 INFO:teuthology.orchestra.run.smithi077.stdout: "segadd": 2, 2024-05-04T03:25:59.393 INFO:teuthology.orchestra.run.smithi077.stdout: "segex": 0, 2024-05-04T03:25:59.393 INFO:teuthology.orchestra.run.smithi077.stdout: "segtrm": 0, 2024-05-04T03:25:59.393 INFO:teuthology.orchestra.run.smithi077.stdout: "seg": 2, 2024-05-04T03:25:59.393 INFO:teuthology.orchestra.run.smithi077.stdout: "segmjr": 2, 2024-05-04T03:25:59.393 INFO:teuthology.orchestra.run.smithi077.stdout: "segexg": 0, 2024-05-04T03:25:59.393 INFO:teuthology.orchestra.run.smithi077.stdout: "segexd": 0, 2024-05-04T03:25:59.393 INFO:teuthology.orchestra.run.smithi077.stdout: "expos": 4194304, 2024-05-04T03:25:59.393 INFO:teuthology.orchestra.run.smithi077.stdout: "wrpos": 4279751, 2024-05-04T03:25:59.393 INFO:teuthology.orchestra.run.smithi077.stdout: "rdpos": 0, 2024-05-04T03:25:59.393 INFO:teuthology.orchestra.run.smithi077.stdout: "jlat": { 2024-05-04T03:25:59.393 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 22, 2024-05-04T03:25:59.393 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.050871670, 2024-05-04T03:25:59.394 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.002312348 2024-05-04T03:25:59.394 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.394 INFO:teuthology.orchestra.run.smithi077.stdout: "replayed": 0 2024-05-04T03:25:59.394 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.394 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_mem": { 2024-05-04T03:25:59.394 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 23, 2024-05-04T03:25:59.394 INFO:teuthology.orchestra.run.smithi077.stdout: "ino+": 38, 2024-05-04T03:25:59.394 INFO:teuthology.orchestra.run.smithi077.stdout: "ino-": 15, 2024-05-04T03:25:59.394 INFO:teuthology.orchestra.run.smithi077.stdout: "dir": 22, 2024-05-04T03:25:59.394 INFO:teuthology.orchestra.run.smithi077.stdout: "dir+": 29, 2024-05-04T03:25:59.394 INFO:teuthology.orchestra.run.smithi077.stdout: "dir-": 7, 2024-05-04T03:25:59.394 INFO:teuthology.orchestra.run.smithi077.stdout: "dn": 18, 2024-05-04T03:25:59.394 INFO:teuthology.orchestra.run.smithi077.stdout: "dn+": 53, 2024-05-04T03:25:59.394 INFO:teuthology.orchestra.run.smithi077.stdout: "dn-": 35, 2024-05-04T03:25:59.394 INFO:teuthology.orchestra.run.smithi077.stdout: "cap": 2, 2024-05-04T03:25:59.395 INFO:teuthology.orchestra.run.smithi077.stdout: "cap+": 23, 2024-05-04T03:25:59.395 INFO:teuthology.orchestra.run.smithi077.stdout: "cap-": 21, 2024-05-04T03:25:59.395 INFO:teuthology.orchestra.run.smithi077.stdout: "rss": 126892, 2024-05-04T03:25:59.395 INFO:teuthology.orchestra.run.smithi077.stdout: "heap": 223632 2024-05-04T03:25:59.395 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.395 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_server": { 2024-05-04T03:25:59.395 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_client_request": 50, 2024-05-04T03:25:59.395 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_server_request": 3, 2024-05-04T03:25:59.395 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_request": 33, 2024-05-04T03:25:59.395 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_session": 6, 2024-05-04T03:25:59.395 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_peer_request": 50, 2024-05-04T03:25:59.395 INFO:teuthology.orchestra.run.smithi077.stdout: "req_create_latency": { 2024-05-04T03:25:59.395 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1, 2024-05-04T03:25:59.395 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.006141632, 2024-05-04T03:25:59.395 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.006141632 2024-05-04T03:25:59.395 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.396 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getattr_latency": { 2024-05-04T03:25:59.396 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1, 2024-05-04T03:25:59.396 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000927046, 2024-05-04T03:25:59.396 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000927046 2024-05-04T03:25:59.396 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.396 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getfilelock_latency": { 2024-05-04T03:25:59.396 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.396 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.396 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.396 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.396 INFO:teuthology.orchestra.run.smithi077.stdout: "req_link_latency": { 2024-05-04T03:25:59.396 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.396 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.396 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.396 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.397 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookup_latency": { 2024-05-04T03:25:59.397 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 8, 2024-05-04T03:25:59.397 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.002296340, 2024-05-04T03:25:59.397 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000287042 2024-05-04T03:25:59.397 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.397 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookuphash_latency": { 2024-05-04T03:25:59.397 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.397 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.397 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.397 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.397 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupino_latency": { 2024-05-04T03:25:59.397 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1, 2024-05-04T03:25:59.397 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000238087, 2024-05-04T03:25:59.397 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000238087 2024-05-04T03:25:59.397 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.397 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupname_latency": { 2024-05-04T03:25:59.397 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.398 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.398 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.398 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.398 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupparent_latency": { 2024-05-04T03:25:59.398 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.398 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.398 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.398 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.398 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupsnap_latency": { 2024-05-04T03:25:59.398 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.398 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.398 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.398 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.398 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lssnap_latency": { 2024-05-04T03:25:59.398 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.399 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.399 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.399 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.399 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mkdir_latency": { 2024-05-04T03:25:59.399 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.399 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.399 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.399 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.400 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mknod_latency": { 2024-05-04T03:25:59.400 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.400 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.400 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.400 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.400 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mksnap_latency": { 2024-05-04T03:25:59.400 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.400 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.400 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.400 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.400 INFO:teuthology.orchestra.run.smithi077.stdout: "req_open_latency": { 2024-05-04T03:25:59.400 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.400 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.400 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.400 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.401 INFO:teuthology.orchestra.run.smithi077.stdout: "req_readdir_latency": { 2024-05-04T03:25:59.401 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 3, 2024-05-04T03:25:59.401 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.001191030, 2024-05-04T03:25:59.401 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000397010 2024-05-04T03:25:59.401 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.401 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rename_latency": { 2024-05-04T03:25:59.401 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.401 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.401 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.401 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.401 INFO:teuthology.orchestra.run.smithi077.stdout: "req_renamesnap_latency": { 2024-05-04T03:25:59.401 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.401 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.401 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.401 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.402 INFO:teuthology.orchestra.run.smithi077.stdout: "req_snapdiff_latency": { 2024-05-04T03:25:59.402 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.402 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.402 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.402 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.402 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmdir_latency": { 2024-05-04T03:25:59.402 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.402 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.402 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.402 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.402 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmsnap_latency": { 2024-05-04T03:25:59.402 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.402 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.403 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.403 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.403 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmxattr_latency": { 2024-05-04T03:25:59.403 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.403 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.403 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.403 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.403 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setattr_latency": { 2024-05-04T03:25:59.403 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.403 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.403 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.403 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.403 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setdirlayout_latency": { 2024-05-04T03:25:59.403 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.403 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.404 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.404 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.404 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setfilelock_latency": { 2024-05-04T03:25:59.404 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.404 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.404 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.404 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.404 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setlayout_latency": { 2024-05-04T03:25:59.404 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.404 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.404 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.404 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.404 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setxattr_latency": { 2024-05-04T03:25:59.404 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.404 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.404 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.405 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.405 INFO:teuthology.orchestra.run.smithi077.stdout: "req_symlink_latency": { 2024-05-04T03:25:59.405 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.405 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.405 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.405 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.405 INFO:teuthology.orchestra.run.smithi077.stdout: "req_unlink_latency": { 2024-05-04T03:25:59.405 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 9, 2024-05-04T03:25:59.405 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.015919632, 2024-05-04T03:25:59.405 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.001768848 2024-05-04T03:25:59.405 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.405 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_revoke_eviction": 0, 2024-05-04T03:25:59.405 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_acquisition_throttle": 0, 2024-05-04T03:25:59.405 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getvxattr_latency": { 2024-05-04T03:25:59.405 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.405 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.406 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.406 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:25:59.406 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.406 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_sessions": { 2024-05-04T03:25:59.406 INFO:teuthology.orchestra.run.smithi077.stdout: "session_count": 1, 2024-05-04T03:25:59.406 INFO:teuthology.orchestra.run.smithi077.stdout: "session_add": 1, 2024-05-04T03:25:59.406 INFO:teuthology.orchestra.run.smithi077.stdout: "session_remove": 0, 2024-05-04T03:25:59.406 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_open": 1, 2024-05-04T03:25:59.406 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_stale": 0, 2024-05-04T03:25:59.406 INFO:teuthology.orchestra.run.smithi077.stdout: "total_load": 18, 2024-05-04T03:25:59.406 INFO:teuthology.orchestra.run.smithi077.stdout: "average_load": 18, 2024-05-04T03:25:59.406 INFO:teuthology.orchestra.run.smithi077.stdout: "avg_session_uptime": 434, 2024-05-04T03:25:59.406 INFO:teuthology.orchestra.run.smithi077.stdout: "mdthresh_evicted": 0 2024-05-04T03:25:59.406 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.407 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-05-04T03:25:59.407 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-05-04T03:25:59.407 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-05-04T03:25:59.407 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-05-04T03:25:59.407 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-05-04T03:25:59.407 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-05-04T03:25:59.407 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-05-04T03:25:59.407 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-04T03:25:59.407 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-05-04T03:25:59.407 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-04T03:25:59.407 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-05-04T03:25:59.407 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-05-04T03:25:59.407 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-05-04T03:25:59.407 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-04T03:25:59.407 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-05-04T03:25:59.408 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-05-04T03:25:59.408 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-05-04T03:25:59.408 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-05-04T03:25:59.408 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-05-04T03:25:59.408 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-04T03:25:59.408 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-05-04T03:25:59.408 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-04T03:25:59.408 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-05-04T03:25:59.408 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-05-04T03:25:59.408 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-05-04T03:25:59.408 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-05-04T03:25:59.408 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-05-04T03:25:59.408 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-04T03:25:59.408 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-05-04T03:25:59.408 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-05-04T03:25:59.408 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-05-04T03:25:59.409 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-05-04T03:25:59.409 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-05-04T03:25:59.409 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-05-04T03:25:59.409 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-05-04T03:25:59.409 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-05-04T03:25:59.409 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-05-04T03:25:59.409 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 270773, 2024-05-04T03:25:59.409 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 93, 2024-05-04T03:25:59.409 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-05-04T03:25:59.409 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-05-04T03:25:59.409 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-05-04T03:25:59.409 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-05-04T03:25:59.409 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-05-04T03:25:59.409 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-05-04T03:25:59.409 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-05-04T03:25:59.409 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-05-04T03:25:59.410 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 7240, 2024-05-04T03:25:59.410 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 152, 2024-05-04T03:25:59.410 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-05-04T03:25:59.410 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-05-04T03:25:59.410 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-05-04T03:25:59.410 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-05-04T03:25:59.410 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 93613, 2024-05-04T03:25:59.410 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 371, 2024-05-04T03:25:59.410 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-05-04T03:25:59.410 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-05-04T03:25:59.410 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-05-04T03:25:59.410 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-05-04T03:25:59.410 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.410 INFO:teuthology.orchestra.run.smithi077.stdout: "objecter": { 2024-05-04T03:25:59.410 INFO:teuthology.orchestra.run.smithi077.stdout: "op_active": 0, 2024-05-04T03:25:59.410 INFO:teuthology.orchestra.run.smithi077.stdout: "op_laggy": 0, 2024-05-04T03:25:59.411 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send": 94, 2024-05-04T03:25:59.411 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send_bytes": 100118, 2024-05-04T03:25:59.411 INFO:teuthology.orchestra.run.smithi077.stdout: "op_resend": 27, 2024-05-04T03:25:59.411 INFO:teuthology.orchestra.run.smithi077.stdout: "op_reply": 94, 2024-05-04T03:25:59.411 INFO:teuthology.orchestra.run.smithi077.stdout: "op_latency": { 2024-05-04T03:25:59.411 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 94, 2024-05-04T03:25:59.411 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.356992995, 2024-05-04T03:25:59.411 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.003797797 2024-05-04T03:25:59.411 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.411 INFO:teuthology.orchestra.run.smithi077.stdout: "op_inflight": 0, 2024-05-04T03:25:59.411 INFO:teuthology.orchestra.run.smithi077.stdout: "oplen_avg": { 2024-05-04T03:25:59.411 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 94, 2024-05-04T03:25:59.411 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 131 2024-05-04T03:25:59.411 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.411 INFO:teuthology.orchestra.run.smithi077.stdout: "op": 94, 2024-05-04T03:25:59.412 INFO:teuthology.orchestra.run.smithi077.stdout: "op_r": 1, 2024-05-04T03:25:59.412 INFO:teuthology.orchestra.run.smithi077.stdout: "op_w": 93, 2024-05-04T03:25:59.412 INFO:teuthology.orchestra.run.smithi077.stdout: "op_rmw": 0, 2024-05-04T03:25:59.412 INFO:teuthology.orchestra.run.smithi077.stdout: "op_pg": 0, 2024-05-04T03:25:59.412 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_stat": 0, 2024-05-04T03:25:59.412 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_create": 10, 2024-05-04T03:25:59.412 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_read": 1, 2024-05-04T03:25:59.412 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_write": 23, 2024-05-04T03:25:59.412 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writefull": 13, 2024-05-04T03:25:59.412 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writesame": 0, 2024-05-04T03:25:59.412 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_append": 0, 2024-05-04T03:25:59.412 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_zero": 0, 2024-05-04T03:25:59.412 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_truncate": 0, 2024-05-04T03:25:59.412 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_delete": 21, 2024-05-04T03:25:59.412 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_mapext": 0, 2024-05-04T03:25:59.412 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_sparse_read": 0, 2024-05-04T03:25:59.412 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_clonerange": 0, 2024-05-04T03:25:59.413 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_getxattr": 0, 2024-05-04T03:25:59.413 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_setxattr": 20, 2024-05-04T03:25:59.413 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_cmpxattr": 0, 2024-05-04T03:25:59.413 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_rmxattr": 0, 2024-05-04T03:25:59.413 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_resetxattrs": 0, 2024-05-04T03:25:59.413 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_call": 0, 2024-05-04T03:25:59.413 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_watch": 0, 2024-05-04T03:25:59.413 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_notify": 0, 2024-05-04T03:25:59.413 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_src_cmpxattr": 0, 2024-05-04T03:25:59.413 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls": 0, 2024-05-04T03:25:59.413 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls_filter": 0, 2024-05-04T03:25:59.413 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_other": 0, 2024-05-04T03:25:59.413 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_active": 0, 2024-05-04T03:25:59.413 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_send": 0, 2024-05-04T03:25:59.413 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_resend": 0, 2024-05-04T03:25:59.413 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_ping": 0, 2024-05-04T03:25:59.414 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_active": 0, 2024-05-04T03:25:59.414 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_send": 0, 2024-05-04T03:25:59.414 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_resend": 0, 2024-05-04T03:25:59.414 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_active": 0, 2024-05-04T03:25:59.414 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_send": 0, 2024-05-04T03:25:59.414 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_resend": 0, 2024-05-04T03:25:59.414 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_active": 0, 2024-05-04T03:25:59.414 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_send": 0, 2024-05-04T03:25:59.414 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_resend": 0, 2024-05-04T03:25:59.414 INFO:teuthology.orchestra.run.smithi077.stdout: "command_active": 0, 2024-05-04T03:25:59.414 INFO:teuthology.orchestra.run.smithi077.stdout: "command_send": 0, 2024-05-04T03:25:59.414 INFO:teuthology.orchestra.run.smithi077.stdout: "command_resend": 0, 2024-05-04T03:25:59.414 INFO:teuthology.orchestra.run.smithi077.stdout: "map_epoch": 83, 2024-05-04T03:25:59.414 INFO:teuthology.orchestra.run.smithi077.stdout: "map_full": 0, 2024-05-04T03:25:59.414 INFO:teuthology.orchestra.run.smithi077.stdout: "map_inc": 2, 2024-05-04T03:25:59.414 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_sessions": 10, 2024-05-04T03:25:59.415 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_open": 10, 2024-05-04T03:25:59.415 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_close": 0, 2024-05-04T03:25:59.415 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_laggy": 0, 2024-05-04T03:25:59.415 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_wr": 34, 2024-05-04T03:25:59.415 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_rd": 0, 2024-05-04T03:25:59.415 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_del": 9 2024-05-04T03:25:59.415 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.415 INFO:teuthology.orchestra.run.smithi077.stdout: "oft": { 2024-05-04T03:25:59.415 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_objs": 1, 2024-05-04T03:25:59.415 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_kv_pairs": 0, 2024-05-04T03:25:59.415 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_updates": 0, 2024-05-04T03:25:59.415 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_removes": 0 2024-05-04T03:25:59.415 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.415 INFO:teuthology.orchestra.run.smithi077.stdout: "osd-slow-ops": { 2024-05-04T03:25:59.415 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_ops_count": 54 2024-05-04T03:25:59.416 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.416 INFO:teuthology.orchestra.run.smithi077.stdout: "purge_queue": { 2024-05-04T03:25:59.416 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops": 0, 2024-05-04T03:25:59.416 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops_high_water": 9, 2024-05-04T03:25:59.416 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing": 0, 2024-05-04T03:25:59.416 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_high_water": 9, 2024-05-04T03:25:59.416 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed_ops": 9, 2024-05-04T03:25:59.416 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed": 9, 2024-05-04T03:25:59.416 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_item_in_journal": 0 2024-05-04T03:25:59.416 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.416 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-04T03:25:59.416 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-05-04T03:25:59.416 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-05-04T03:25:59.416 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-05-04T03:25:59.416 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 1218, 2024-05-04T03:25:59.417 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 1080608, 2024-05-04T03:25:59.417 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:59.417 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 1218, 2024-05-04T03:25:59.417 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-05-04T03:25:59.417 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-05-04T03:25:59.417 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 1218, 2024-05-04T03:25:59.417 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 1080608, 2024-05-04T03:25:59.417 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-05-04T03:25:59.417 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.417 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.417 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.417 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:25:59.417 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.417 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_bytes": { 2024-05-04T03:25:59.418 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-05-04T03:25:59.418 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-05-04T03:25:59.418 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-05-04T03:25:59.418 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-05-04T03:25:59.418 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-05-04T03:25:59.418 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:59.418 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-05-04T03:25:59.418 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 94, 2024-05-04T03:25:59.418 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 101027, 2024-05-04T03:25:59.418 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 94, 2024-05-04T03:25:59.418 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 101027, 2024-05-04T03:25:59.418 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-05-04T03:25:59.418 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.418 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.418 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.419 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:25:59.419 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.419 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_ops": { 2024-05-04T03:25:59.419 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-05-04T03:25:59.419 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 1024, 2024-05-04T03:25:59.419 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-05-04T03:25:59.419 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-05-04T03:25:59.419 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-05-04T03:25:59.419 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:59.419 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-05-04T03:25:59.419 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 94, 2024-05-04T03:25:59.419 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 94, 2024-05-04T03:25:59.419 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 94, 2024-05-04T03:25:59.419 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 94, 2024-05-04T03:25:59.419 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-05-04T03:25:59.419 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.420 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.420 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.420 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:25:59.420 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.420 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle": { 2024-05-04T03:25:59.420 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-05-04T03:25:59.420 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-05-04T03:25:59.420 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-05-04T03:25:59.420 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 9, 2024-05-04T03:25:59.420 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 909, 2024-05-04T03:25:59.420 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:59.420 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 9, 2024-05-04T03:25:59.420 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-05-04T03:25:59.420 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-05-04T03:25:59.420 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 1, 2024-05-04T03:25:59.421 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 909, 2024-05-04T03:25:59.421 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-05-04T03:25:59.421 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.421 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.421 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.421 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:25:59.421 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:25:59.421 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle-0x55b480fae4e0": { 2024-05-04T03:25:59.421 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-05-04T03:25:59.421 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-05-04T03:25:59.421 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-05-04T03:25:59.421 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 54, 2024-05-04T03:25:59.421 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 85447, 2024-05-04T03:25:59.421 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-05-04T03:25:59.421 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 54, 2024-05-04T03:25:59.422 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-05-04T03:25:59.422 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-05-04T03:25:59.422 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 22, 2024-05-04T03:25:59.422 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 85447, 2024-05-04T03:25:59.422 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-05-04T03:25:59.422 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:25:59.422 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:25:59.422 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:25:59.422 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:25:59.422 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:25:59.422 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-04T03:26:00.139 INFO:tasks.check_counter:Daemon mds.i mds.exported=2 2024-05-04T03:26:00.139 INFO:tasks.check_counter:Daemon mds.i mds.imported=2 2024-05-04T03:26:00.139 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-05-04T03:26:00.140 DEBUG:tasks.check_counter:Getting stats from l 2024-05-04T03:26:00.140 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-04T03:26:00.379 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.c/config 2024-05-04T03:26:00.756 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:00 smithi077 ceph-mon[33531]: pgmap v596: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-04T03:26:00.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:00 smithi001 ceph-mon[29731]: pgmap v596: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-04T03:26:00.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:00 smithi072 ceph-mon[33332]: pgmap v596: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-04T03:26:02.092 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-04T03:26:02.092 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-05-04T03:26:02.092 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 0, 2024-05-04T03:26:02.092 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 2, 2024-05-04T03:26:02.093 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 0, 2024-05-04T03:26:02.093 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 0, 2024-05-04T03:26:02.093 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 4, 2024-05-04T03:26:02.093 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 0, 2024-05-04T03:26:02.093 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.001020244, 2024-05-04T03:26:02.093 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.000000000, 2024-05-04T03:26:02.093 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.000219560, 2024-05-04T03:26:02.093 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-04T03:26:02.093 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:26:02.093 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:26:02.093 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:26:02.093 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:26:02.093 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:26:02.093 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:26:02.093 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:26:02.093 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:26:02.094 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:26:02.094 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:26:02.094 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-04T03:26:02.094 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 0 2024-05-04T03:26:02.094 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:26:02.094 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-05-04T03:26:02.094 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 5, 2024-05-04T03:26:02.094 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 138, 2024-05-04T03:26:02.094 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 3152, 2024-05-04T03:26:02.094 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 102976, 2024-05-04T03:26:02.094 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 3, 2024-05-04T03:26:02.094 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 2, 2024-05-04T03:26:02.094 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.009162704, 2024-05-04T03:26:02.094 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.006132695, 2024-05-04T03:26:02.095 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.001427665, 2024-05-04T03:26:02.095 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-04T03:26:02.095 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:26:02.095 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 137, 2024-05-04T03:26:02.095 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.009190937, 2024-05-04T03:26:02.095 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000067087 2024-05-04T03:26:02.095 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:26:02.095 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:26:02.095 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:26:02.095 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:26:02.095 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:26:02.095 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:26:02.095 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-05-04T03:26:02.095 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 102976 2024-05-04T03:26:02.095 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:26:02.095 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-05-04T03:26:02.095 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 184, 2024-05-04T03:26:02.096 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 179, 2024-05-04T03:26:02.096 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 802592, 2024-05-04T03:26:02.096 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 109648, 2024-05-04T03:26:02.096 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 2, 2024-05-04T03:26:02.096 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 1, 2024-05-04T03:26:02.096 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.034690531, 2024-05-04T03:26:02.096 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.010982660, 2024-05-04T03:26:02.096 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.023742716, 2024-05-04T03:26:02.096 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.004611934, 2024-05-04T03:26:02.096 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-05-04T03:26:02.096 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 178, 2024-05-04T03:26:02.096 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.009589988, 2024-05-04T03:26:02.096 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000053876 2024-05-04T03:26:02.096 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:26:02.096 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-05-04T03:26:02.096 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:26:02.097 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:26:02.097 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:26:02.097 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:26:02.097 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 802592, 2024-05-04T03:26:02.097 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 109648 2024-05-04T03:26:02.097 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:26:02.097 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-05-04T03:26:02.097 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 0, 2024-05-04T03:26:02.097 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-05-04T03:26:02.097 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:26:02.097 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-05-04T03:26:02.097 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-05-04T03:26:02.097 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-05-04T03:26:02.097 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-05-04T03:26:02.097 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-05-04T03:26:02.097 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-05-04T03:26:02.098 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-05-04T03:26:02.098 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-04T03:26:02.098 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-05-04T03:26:02.098 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-04T03:26:02.098 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-05-04T03:26:02.098 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-05-04T03:26:02.098 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-05-04T03:26:02.098 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-04T03:26:02.098 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-05-04T03:26:02.098 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-05-04T03:26:02.098 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-05-04T03:26:02.098 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-05-04T03:26:02.098 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-05-04T03:26:02.098 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-04T03:26:02.098 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-05-04T03:26:02.099 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-04T03:26:02.099 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-05-04T03:26:02.099 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-05-04T03:26:02.099 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-05-04T03:26:02.099 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-05-04T03:26:02.099 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-05-04T03:26:02.099 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-04T03:26:02.099 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-05-04T03:26:02.099 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-05-04T03:26:02.099 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-05-04T03:26:02.099 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-05-04T03:26:02.099 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-05-04T03:26:02.099 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-05-04T03:26:02.099 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-05-04T03:26:02.099 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-05-04T03:26:02.099 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-05-04T03:26:02.100 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 148967, 2024-05-04T03:26:02.100 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 29, 2024-05-04T03:26:02.100 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-05-04T03:26:02.100 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-05-04T03:26:02.100 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-05-04T03:26:02.100 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-05-04T03:26:02.100 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-05-04T03:26:02.100 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-05-04T03:26:02.100 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-05-04T03:26:02.100 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-05-04T03:26:02.100 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 0, 2024-05-04T03:26:02.100 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 0, 2024-05-04T03:26:02.100 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-05-04T03:26:02.100 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-05-04T03:26:02.100 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-05-04T03:26:02.100 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-05-04T03:26:02.100 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 928, 2024-05-04T03:26:02.101 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 2, 2024-05-04T03:26:02.101 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-05-04T03:26:02.101 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-05-04T03:26:02.101 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-05-04T03:26:02.101 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-05-04T03:26:02.101 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-04T03:26:02.101 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-04T03:26:02.101 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-05-04T03:26:02.101 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-05-04T03:26:02.101 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-05-04T03:26:02.101 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 185, 2024-05-04T03:26:02.101 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 776592, 2024-05-04T03:26:02.101 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-05-04T03:26:02.101 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 185, 2024-05-04T03:26:02.101 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-05-04T03:26:02.102 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-05-04T03:26:02.102 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 185, 2024-05-04T03:26:02.102 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 776592, 2024-05-04T03:26:02.102 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-05-04T03:26:02.102 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-04T03:26:02.102 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-04T03:26:02.102 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-04T03:26:02.102 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:26:02.102 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-04T03:26:02.102 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-04T03:26:02.762 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-05-04T03:26:02.762 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-05-04T03:26:02.762 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-05-04T03:26:02.763 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-04T03:26:02.773 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-04T03:26:02.774 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-04T03:26:02.774 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-04T03:26:02.795 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-05-04T03:26:02.868 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-04T03:26:02.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:02 smithi001 ceph-mon[29731]: pgmap v597: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:26:02.887 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-04T03:26:02.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:02 smithi077 ceph-mon[33531]: pgmap v597: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:26:02.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:02 smithi072 ceph-mon[33332]: pgmap v597: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-04T03:26:03.186 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:26:04.667 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:04 smithi001 ceph-mon[29731]: pgmap v598: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:26:04.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.871+0000 7f49c9f9c700 1 -- 172.21.15.1:0/18178087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c4108f10 msgr2=0x7f49c4109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:04.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.871+0000 7f49c9f9c700 1 --2- 172.21.15.1:0/18178087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c4108f10 0x7f49c4109320 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f49b80099c0 tx=0x7f49b8009cd0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.871+0000 7f49c9f9c700 1 -- 172.21.15.1:0/18178087 shutdown_connections 2024-05-04T03:26:04.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.871+0000 7f49c9f9c700 1 --2- 172.21.15.1:0/18178087 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49c4109860 0x7f49c41111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.871+0000 7f49c9f9c700 1 --2- 172.21.15.1:0/18178087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49c41045e0 0x7f49c41049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.871+0000 7f49c9f9c700 1 --2- 172.21.15.1:0/18178087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c4108f10 0x7f49c4109320 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.871+0000 7f49c9f9c700 1 -- 172.21.15.1:0/18178087 >> 172.21.15.1:0/18178087 conn(0x7f49c40ffed0 msgr2=0x7f49c4102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:04.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.871+0000 7f49c9f9c700 1 -- 172.21.15.1:0/18178087 shutdown_connections 2024-05-04T03:26:04.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.871+0000 7f49c9f9c700 1 -- 172.21.15.1:0/18178087 wait complete. 2024-05-04T03:26:04.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.872+0000 7f49c9f9c700 1 Processor -- start 2024-05-04T03:26:04.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.872+0000 7f49c9f9c700 1 -- start start 2024-05-04T03:26:04.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.873+0000 7f49c9f9c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49c41045e0 0x7f49c419aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:04.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.873+0000 7f49c9f9c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49c4108f10 0x7f49c419b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:04.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.873+0000 7f49c9f9c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c4109860 0x7f49c419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:04.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.873+0000 7f49c9f9c700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f49c41144e0 con 0x7f49c4109860 2024-05-04T03:26:04.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.873+0000 7f49c9f9c700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f49c4114360 con 0x7f49c4108f10 2024-05-04T03:26:04.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.873+0000 7f49c9f9c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f49c4114660 con 0x7f49c41045e0 2024-05-04T03:26:04.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.873+0000 7f49c3fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c4109860 0x7f49c419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:04.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.873+0000 7f49c3fff700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c4109860 0x7f49c419f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51612/0 (socket says 172.21.15.1:51612) 2024-05-04T03:26:04.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.873+0000 7f49c3fff700 1 -- 172.21.15.1:0/1564379831 learned_addr learned my addr 172.21.15.1:0/1564379831 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:26:04.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.873+0000 7f49c2ffd700 1 --2- 172.21.15.1:0/1564379831 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49c4108f10 0x7f49c419b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:04.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.873+0000 7f49c37fe700 1 --2- 172.21.15.1:0/1564379831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49c41045e0 0x7f49c419aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:04.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.873+0000 7f49c3fff700 1 -- 172.21.15.1:0/1564379831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49c41045e0 msgr2=0x7f49c419aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:04.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.873+0000 7f49c3fff700 1 --2- 172.21.15.1:0/1564379831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49c41045e0 0x7f49c419aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.873+0000 7f49c3fff700 1 -- 172.21.15.1:0/1564379831 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49c4108f10 msgr2=0x7f49c419b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:04.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.873+0000 7f49c3fff700 1 --2- 172.21.15.1:0/1564379831 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49c4108f10 0x7f49c419b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:04.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.873+0000 7f49c3fff700 1 -- 172.21.15.1:0/1564379831 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c419fd80 con 0x7f49c4109860 2024-05-04T03:26:04.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.873+0000 7f49c2ffd700 1 --2- 172.21.15.1:0/1564379831 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49c4108f10 0x7f49c419b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:26:04.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.873+0000 7f49c3fff700 1 --2- 172.21.15.1:0/1564379831 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c4109860 0x7f49c419f6e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f49bc007ee0 tx=0x7f49bc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:04.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.873+0000 7f49c0ff9700 1 -- 172.21.15.1:0/1564379831 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc00dbd0 con 0x7f49c4109860 2024-05-04T03:26:04.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.874+0000 7f49c9f9c700 1 -- 172.21.15.1:0/1564379831 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c41a0070 con 0x7f49c4109860 2024-05-04T03:26:04.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.874+0000 7f49c0ff9700 1 -- 172.21.15.1:0/1564379831 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49bc00dd70 con 0x7f49c4109860 2024-05-04T03:26:04.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.874+0000 7f49c0ff9700 1 -- 172.21.15.1:0/1564379831 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc005990 con 0x7f49c4109860 2024-05-04T03:26:04.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.874+0000 7f49c9f9c700 1 -- 172.21.15.1:0/1564379831 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c41a7990 con 0x7f49c4109860 2024-05-04T03:26:04.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.874+0000 7f49c9f9c700 1 -- 172.21.15.1:0/1564379831 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49c4105a90 con 0x7f49c4109860 2024-05-04T03:26:04.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.877+0000 7f49c0ff9700 1 -- 172.21.15.1:0/1564379831 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f49bc01a020 con 0x7f49c4109860 2024-05-04T03:26:04.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.878+0000 7f49c0ff9700 1 --2- 172.21.15.1:0/1564379831 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f49ac0b2880 0x7f49ac0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:04.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.878+0000 7f49c37fe700 1 --2- 172.21.15.1:0/1564379831 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f49ac0b2880 0x7f49ac0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:04.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.878+0000 7f49c0ff9700 1 -- 172.21.15.1:0/1564379831 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f49bc0e2120 con 0x7f49c4109860 2024-05-04T03:26:04.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.878+0000 7f49c37fe700 1 --2- 172.21.15.1:0/1564379831 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f49ac0b2880 0x7f49ac0b4d90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f49b4006fd0 tx=0x7f49b4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:04.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:04.881+0000 7f49c0ff9700 1 -- 172.21.15.1:0/1564379831 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f49bc010070 con 0x7f49c4109860 2024-05-04T03:26:04.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:04 smithi077 ceph-mon[33531]: pgmap v598: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:26:04.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:04 smithi072 ceph-mon[33332]: pgmap v598: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-04T03:26:05.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:05.109+0000 7f49c9f9c700 1 -- 172.21.15.1:0/1564379831 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f49c402e5f0 con 0x7f49c4109860 2024-05-04T03:26:05.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:05.110+0000 7f49c0ff9700 1 -- 172.21.15.1:0/1564379831 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v83) ==== 81+0+579 (secure 0 0 0) 0x7f49bc0a9e60 con 0x7f49c4109860 2024-05-04T03:26:05.112 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/2029897106 2024-05-05T03:08:33.045884+0000 2024-05-04T03:26:05.112 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/402488450 2024-05-05T03:08:33.045884+0000 2024-05-04T03:26:05.113 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/128793132 2024-05-05T03:08:33.045884+0000 2024-05-04T03:26:05.113 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/2233526606 2024-05-05T03:08:06.135783+0000 2024-05-04T03:26:05.113 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/2715541855 2024-05-05T03:08:33.045884+0000 2024-05-04T03:26:05.113 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/2415697325 2024-05-05T03:08:06.135783+0000 2024-05-04T03:26:05.113 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/2305298106 2024-05-05T03:08:06.135783+0000 2024-05-04T03:26:05.113 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/128793132 2024-05-05T03:08:33.045884+0000 2024-05-04T03:26:05.113 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/4200529863 2024-05-05T03:08:06.135783+0000 2024-05-04T03:26:05.113 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/4200529863 2024-05-05T03:08:06.135783+0000 2024-05-04T03:26:05.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:05.112+0000 7f49c9f9c700 1 -- 172.21.15.1:0/1564379831 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f49ac0b2880 msgr2=0x7f49ac0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:05.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:05.112+0000 7f49c9f9c700 1 --2- 172.21.15.1:0/1564379831 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f49ac0b2880 0x7f49ac0b4d90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f49b4006fd0 tx=0x7f49b4008040 comp rx=0 tx=0).stop 2024-05-04T03:26:05.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:05.112+0000 7f49c9f9c700 1 -- 172.21.15.1:0/1564379831 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c4109860 msgr2=0x7f49c419f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:05.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:05.112+0000 7f49c9f9c700 1 --2- 172.21.15.1:0/1564379831 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c4109860 0x7f49c419f6e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f49bc007ee0 tx=0x7f49bc00d3b0 comp rx=0 tx=0).stop 2024-05-04T03:26:05.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:05.112+0000 7f49c9f9c700 1 -- 172.21.15.1:0/1564379831 shutdown_connections 2024-05-04T03:26:05.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:05.112+0000 7f49c9f9c700 1 --2- 172.21.15.1:0/1564379831 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f49ac0b2880 0x7f49ac0b4d90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:05.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:05.112+0000 7f49c9f9c700 1 --2- 172.21.15.1:0/1564379831 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c4109860 0x7f49c419f6e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:05.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:05.112+0000 7f49c9f9c700 1 --2- 172.21.15.1:0/1564379831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49c41045e0 0x7f49c419aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:05.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:05.112+0000 7f49c9f9c700 1 --2- 172.21.15.1:0/1564379831 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49c4108f10 0x7f49c419b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:05.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:05.112+0000 7f49c9f9c700 1 -- 172.21.15.1:0/1564379831 >> 172.21.15.1:0/1564379831 conn(0x7f49c40ffed0 msgr2=0x7f49c410a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:05.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:05.113+0000 7f49c9f9c700 1 -- 172.21.15.1:0/1564379831 shutdown_connections 2024-05-04T03:26:05.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:05.113+0000 7f49c9f9c700 1 -- 172.21.15.1:0/1564379831 wait complete. 2024-05-04T03:26:05.116 INFO:teuthology.orchestra.run.smithi001.stderr:listed 10 entries 2024-05-04T03:26:05.728 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:05 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/1564379831' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-04T03:26:05.756 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-04T03:26:05.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:26:05.757 DEBUG:teuthology.orchestra.run.smithi001:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-04T03:26:05.826 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-05-04T03:26:05.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:26:05.826 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-04T03:26:05.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:26:05.863 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-05-04T03:26:05.892 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:05 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/1564379831' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-04T03:26:05.927 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-04T03:26:05.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:26:05.927 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-04T03:26:05.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-04T03:26:05.995 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link delete ceph-brx 2024-05-04T03:26:05.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:05 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/1564379831' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-04T03:26:06.080 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-04T03:26:06.091 INFO:tasks.cephadm:Teardown begin 2024-05-04T03:26:06.092 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-04T03:26:06.092 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:26:06.344 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:26:06.732 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:06 smithi001 ceph-mon[29731]: pgmap v599: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-04T03:26:06.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:06 smithi077 ceph-mon[33531]: pgmap v599: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-04T03:26:06.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:06 smithi072 ceph-mon[33332]: pgmap v599: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-04T03:26:08.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.139+0000 7fec8562f700 1 -- 172.21.15.1:0/2920733920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec801034c0 msgr2=0x7fec801038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:08.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.139+0000 7fec8562f700 1 --2- 172.21.15.1:0/2920733920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec801034c0 0x7fec801038d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fec700099c0 tx=0x7fec70009cd0 comp rx=0 tx=0).stop 2024-05-04T03:26:08.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.139+0000 7fec8562f700 1 -- 172.21.15.1:0/2920733920 shutdown_connections 2024-05-04T03:26:08.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.139+0000 7fec8562f700 1 --2- 172.21.15.1:0/2920733920 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fec80108740 0x7fec801100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:08.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.139+0000 7fec8562f700 1 --2- 172.21.15.1:0/2920733920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec801034c0 0x7fec801038d0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:08.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.139+0000 7fec8562f700 1 --2- 172.21.15.1:0/2920733920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec80107df0 0x7fec80108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:08.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.139+0000 7fec8562f700 1 -- 172.21.15.1:0/2920733920 >> 172.21.15.1:0/2920733920 conn(0x7fec800fedf0 msgr2=0x7fec80101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:08.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.139+0000 7fec8562f700 1 -- 172.21.15.1:0/2920733920 shutdown_connections 2024-05-04T03:26:08.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.139+0000 7fec8562f700 1 -- 172.21.15.1:0/2920733920 wait complete. 2024-05-04T03:26:08.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.140+0000 7fec8562f700 1 Processor -- start 2024-05-04T03:26:08.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.140+0000 7fec8562f700 1 -- start start 2024-05-04T03:26:08.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.140+0000 7fec8562f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec801034c0 0x7fec80113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:08.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.140+0000 7fec8562f700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec80107df0 0x7fec801140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:08.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.140+0000 7fec8562f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fec80108740 0x7fec80114610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:08.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.140+0000 7fec8562f700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fec80113370 con 0x7fec80107df0 2024-05-04T03:26:08.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.140+0000 7fec8562f700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fec801131f0 con 0x7fec80108740 2024-05-04T03:26:08.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.140+0000 7fec8562f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fec801134f0 con 0x7fec801034c0 2024-05-04T03:26:08.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.140+0000 7fec7e7fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec80107df0 0x7fec801140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:08.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.141+0000 7fec7e7fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec80107df0 0x7fec801140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51628/0 (socket says 172.21.15.1:51628) 2024-05-04T03:26:08.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.141+0000 7fec7e7fc700 1 -- 172.21.15.1:0/3669794414 learned_addr learned my addr 172.21.15.1:0/3669794414 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:26:08.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.141+0000 7fec7effd700 1 --2- 172.21.15.1:0/3669794414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec801034c0 0x7fec80113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:08.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.141+0000 7fec7f7fe700 1 --2- 172.21.15.1:0/3669794414 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fec80108740 0x7fec80114610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:08.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.141+0000 7fec7e7fc700 1 -- 172.21.15.1:0/3669794414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec801034c0 msgr2=0x7fec80113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:08.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.141+0000 7fec7e7fc700 1 --2- 172.21.15.1:0/3669794414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec801034c0 0x7fec80113b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:08.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.141+0000 7fec7e7fc700 1 -- 172.21.15.1:0/3669794414 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fec80108740 msgr2=0x7fec80114610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:08.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.141+0000 7fec7e7fc700 1 --2- 172.21.15.1:0/3669794414 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fec80108740 0x7fec80114610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:08.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.141+0000 7fec7e7fc700 1 -- 172.21.15.1:0/3669794414 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec801a9150 con 0x7fec80107df0 2024-05-04T03:26:08.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.141+0000 7fec7e7fc700 1 --2- 172.21.15.1:0/3669794414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec80107df0 0x7fec801140d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fec7400c9e0 tx=0x7fec7400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:08.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.141+0000 7fec67fff700 1 -- 172.21.15.1:0/3669794414 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec74007aa0 con 0x7fec80107df0 2024-05-04T03:26:08.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.141+0000 7fec67fff700 1 -- 172.21.15.1:0/3669794414 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec74007c40 con 0x7fec80107df0 2024-05-04T03:26:08.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.141+0000 7fec8562f700 1 -- 172.21.15.1:0/3669794414 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec801a9440 con 0x7fec80107df0 2024-05-04T03:26:08.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.141+0000 7fec67fff700 1 -- 172.21.15.1:0/3669794414 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec74018660 con 0x7fec80107df0 2024-05-04T03:26:08.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.141+0000 7fec8562f700 1 -- 172.21.15.1:0/3669794414 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec801a9980 con 0x7fec80107df0 2024-05-04T03:26:08.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.142+0000 7fec8562f700 1 -- 172.21.15.1:0/3669794414 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec800512c0 con 0x7fec80107df0 2024-05-04T03:26:08.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.144+0000 7fec67fff700 1 -- 172.21.15.1:0/3669794414 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7fec74018800 con 0x7fec80107df0 2024-05-04T03:26:08.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.145+0000 7fec67fff700 1 --2- 172.21.15.1:0/3669794414 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fec680b2880 0x7fec680b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:08.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.145+0000 7fec7effd700 1 --2- 172.21.15.1:0/3669794414 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fec680b2880 0x7fec680b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:08.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.145+0000 7fec67fff700 1 -- 172.21.15.1:0/3669794414 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fec740e23e0 con 0x7fec80107df0 2024-05-04T03:26:08.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.145+0000 7fec7effd700 1 --2- 172.21.15.1:0/3669794414 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fec680b2880 0x7fec680b4d90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fec70009990 tx=0x7fec70000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:08.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.149+0000 7fec67fff700 1 -- 172.21.15.1:0/3669794414 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fec74014030 con 0x7fec80107df0 2024-05-04T03:26:08.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.411+0000 7fec8562f700 1 -- 172.21.15.1:0/3669794414 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec8002e5f0 con 0x7fec80107df0 2024-05-04T03:26:08.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.412+0000 7fec67fff700 1 -- 172.21.15.1:0/3669794414 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11847 (secure 0 0 0) 0x7fec740aa250 con 0x7fec80107df0 2024-05-04T03:26:08.414 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:26:08.415 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2186681910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2186681910},{"type":"v1","addr":"172.21.15.1:6835","nonce":2186681910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2747191047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2747191047},{"type":"v1","addr":"172.21.15.1:6837","nonce":2747191047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4115120540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4115120540},{"type":"v1","addr":"172.21.15.1:6839","nonce":4115120540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3405709780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3405709780},{"type":"v1","addr":"172.21.15.1:6841","nonce":3405709780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/3816098761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":3816098761},{"type":"v1","addr":"172.21.15.72:6833","nonce":3816098761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/379026128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":379026128},{"type":"v1","addr":"172.21.15.77:6833","nonce":379026128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:15:49.823471+0000","modified":"2024-05-04T03:25:24.389587+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24472,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/3930984976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3930984976},{"type":"v1","addr":"172.21.15.72:6835","nonce":3930984976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/4279951305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4279951305},{"type":"v1","addr":"172.21.15.77:6835","nonce":4279951305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2003521417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2003521417},{"type":"v1","addr":"172.21.15.77:6837","nonce":2003521417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/1417614203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1417614203},{"type":"v1","addr":"172.21.15.72:6837","nonce":1417614203}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.77:6839/922723189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":922723189},{"type":"v1","addr":"172.21.15.77:6839","nonce":922723189}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2976050650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2976050650},{"type":"v1","addr":"172.21.15.72:6839","nonce":2976050650}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24467,24472,24494]},"id":1}]} 2024-05-04T03:26:08.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.414+0000 7fec8562f700 1 -- 172.21.15.1:0/3669794414 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fec680b2880 msgr2=0x7fec680b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:08.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.414+0000 7fec8562f700 1 --2- 172.21.15.1:0/3669794414 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fec680b2880 0x7fec680b4d90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fec70009990 tx=0x7fec70000bc0 comp rx=0 tx=0).stop 2024-05-04T03:26:08.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.414+0000 7fec8562f700 1 -- 172.21.15.1:0/3669794414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec80107df0 msgr2=0x7fec801140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:08.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.414+0000 7fec8562f700 1 --2- 172.21.15.1:0/3669794414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec80107df0 0x7fec801140d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fec7400c9e0 tx=0x7fec7400ccf0 comp rx=0 tx=0).stop 2024-05-04T03:26:08.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.414+0000 7fec8562f700 1 -- 172.21.15.1:0/3669794414 shutdown_connections 2024-05-04T03:26:08.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.414+0000 7fec8562f700 1 --2- 172.21.15.1:0/3669794414 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fec80108740 0x7fec80114610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:08.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.414+0000 7fec8562f700 1 --2- 172.21.15.1:0/3669794414 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fec680b2880 0x7fec680b4d90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:08.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.414+0000 7fec8562f700 1 --2- 172.21.15.1:0/3669794414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec801034c0 0x7fec80113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:08.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.414+0000 7fec8562f700 1 --2- 172.21.15.1:0/3669794414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fec80107df0 0x7fec801140d0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:08.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.414+0000 7fec8562f700 1 -- 172.21.15.1:0/3669794414 >> 172.21.15.1:0/3669794414 conn(0x7fec800fedf0 msgr2=0x7fec80076140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:08.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.415+0000 7fec8562f700 1 -- 172.21.15.1:0/3669794414 shutdown_connections 2024-05-04T03:26:08.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:08.415+0000 7fec8562f700 1 -- 172.21.15.1:0/3669794414 wait complete. 2024-05-04T03:26:08.418 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 35 2024-05-04T03:26:08.717 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:08 smithi001 ceph-mon[29731]: pgmap v600: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-04T03:26:08.717 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:08 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3669794414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:26:08.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:08 smithi077 ceph-mon[33531]: pgmap v600: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-04T03:26:08.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:08 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/3669794414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:26:08.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:08 smithi072 ceph-mon[33332]: pgmap v600: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-05-04T03:26:08.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:08 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/3669794414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:26:08.999 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-04T03:26:09.255 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:26:10.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:10 smithi001 ceph-mon[29731]: pgmap v601: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-04T03:26:10.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.918+0000 7f74e0987700 1 -- 172.21.15.1:0/3707552666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74dc1034c0 msgr2=0x7f74dc1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:10.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.918+0000 7f74e0987700 1 --2- 172.21.15.1:0/3707552666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74dc1034c0 0x7f74dc1038d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f74cc0099c0 tx=0x7f74cc009cd0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.918+0000 7f74e0987700 1 -- 172.21.15.1:0/3707552666 shutdown_connections 2024-05-04T03:26:10.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.918+0000 7f74e0987700 1 --2- 172.21.15.1:0/3707552666 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74dc108740 0x7f74dc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.918+0000 7f74e0987700 1 --2- 172.21.15.1:0/3707552666 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74dc1034c0 0x7f74dc1038d0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.918+0000 7f74e0987700 1 --2- 172.21.15.1:0/3707552666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74dc107df0 0x7f74dc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.918+0000 7f74e0987700 1 -- 172.21.15.1:0/3707552666 >> 172.21.15.1:0/3707552666 conn(0x7f74dc0fedf0 msgr2=0x7f74dc101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:10.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.918+0000 7f74e0987700 1 -- 172.21.15.1:0/3707552666 shutdown_connections 2024-05-04T03:26:10.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.918+0000 7f74e0987700 1 -- 172.21.15.1:0/3707552666 wait complete. 2024-05-04T03:26:10.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.919+0000 7f74e0987700 1 Processor -- start 2024-05-04T03:26:10.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.919+0000 7f74e0987700 1 -- start start 2024-05-04T03:26:10.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.919+0000 7f74e0987700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74dc1034c0 0x7f74dc19bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:10.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.919+0000 7f74e0987700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74dc107df0 0x7f74dc19c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:10.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.919+0000 7f74e0987700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74dc108740 0x7f74dc1a6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:10.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.919+0000 7f74e0987700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc113370 con 0x7f74dc1034c0 2024-05-04T03:26:10.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.919+0000 7f74e0987700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc1131f0 con 0x7f74dc107df0 2024-05-04T03:26:10.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.919+0000 7f74e0987700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc1134f0 con 0x7f74dc108740 2024-05-04T03:26:10.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.919+0000 7f74da59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74dc1034c0 0x7f74dc19bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:10.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.919+0000 7f74da59c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74dc1034c0 0x7f74dc19bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51644/0 (socket says 172.21.15.1:51644) 2024-05-04T03:26:10.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.919+0000 7f74da59c700 1 -- 172.21.15.1:0/1033301384 learned_addr learned my addr 172.21.15.1:0/1033301384 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:26:10.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.919+0000 7f74d3fff700 1 --2- 172.21.15.1:0/1033301384 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74dc107df0 0x7f74dc19c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:10.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.919+0000 7f74da59c700 1 -- 172.21.15.1:0/1033301384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74dc108740 msgr2=0x7f74dc1a6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:10.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.919+0000 7f74dad9d700 1 --2- 172.21.15.1:0/1033301384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74dc108740 0x7f74dc1a6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:10.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.920+0000 7f74da59c700 1 --2- 172.21.15.1:0/1033301384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74dc108740 0x7f74dc1a6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.920+0000 7f74da59c700 1 -- 172.21.15.1:0/1033301384 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74dc107df0 msgr2=0x7f74dc19c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:10.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.920+0000 7f74da59c700 1 --2- 172.21.15.1:0/1033301384 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74dc107df0 0x7f74dc19c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:10.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.920+0000 7f74da59c700 1 -- 172.21.15.1:0/1033301384 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74dc19ca00 con 0x7f74dc1034c0 2024-05-04T03:26:10.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.920+0000 7f74d3fff700 1 --2- 172.21.15.1:0/1033301384 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74dc107df0 0x7f74dc19c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:26:10.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.920+0000 7f74dad9d700 1 --2- 172.21.15.1:0/1033301384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74dc108740 0x7f74dc1a6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-04T03:26:10.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.920+0000 7f74da59c700 1 --2- 172.21.15.1:0/1033301384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74dc1034c0 0x7f74dc19bc10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f74cc00b580 tx=0x7f74cc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:10.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.920+0000 7f74d37fe700 1 -- 172.21.15.1:0/1033301384 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc027070 con 0x7f74dc1034c0 2024-05-04T03:26:10.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.920+0000 7f74d37fe700 1 -- 172.21.15.1:0/1033301384 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74cc004020 con 0x7f74dc1034c0 2024-05-04T03:26:10.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.920+0000 7f74e0987700 1 -- 172.21.15.1:0/1033301384 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74dc1a7290 con 0x7f74dc1034c0 2024-05-04T03:26:10.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.920+0000 7f74d37fe700 1 -- 172.21.15.1:0/1033301384 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc022e30 con 0x7f74dc1034c0 2024-05-04T03:26:10.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.920+0000 7f74e0987700 1 -- 172.21.15.1:0/1033301384 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74dc1a76e0 con 0x7f74dc1034c0 2024-05-04T03:26:10.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.921+0000 7f74e0987700 1 -- 172.21.15.1:0/1033301384 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74dc0512c0 con 0x7f74dc1034c0 2024-05-04T03:26:10.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.922+0000 7f74d37fe700 1 -- 172.21.15.1:0/1033301384 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f74cc022590 con 0x7f74dc1034c0 2024-05-04T03:26:10.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.923+0000 7f74d37fe700 1 --2- 172.21.15.1:0/1033301384 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f74c80b2880 0x7f74c80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:10.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.923+0000 7f74d3fff700 1 --2- 172.21.15.1:0/1033301384 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f74c80b2880 0x7f74c80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:10.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.923+0000 7f74d37fe700 1 -- 172.21.15.1:0/1033301384 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f74cc0e3300 con 0x7f74dc1034c0 2024-05-04T03:26:10.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.924+0000 7f74d3fff700 1 --2- 172.21.15.1:0/1033301384 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f74c80b2880 0x7f74c80b4d90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f74c4005950 tx=0x7f74c40058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:10.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:10 smithi077 ceph-mon[33531]: pgmap v601: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-04T03:26:10.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:10.927+0000 7f74d37fe700 1 -- 172.21.15.1:0/1033301384 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f74cc030080 con 0x7f74dc1034c0 2024-05-04T03:26:10.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:10 smithi072 ceph-mon[33332]: pgmap v601: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-04T03:26:11.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:11.188+0000 7f74e0987700 1 -- 172.21.15.1:0/1033301384 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74dc02e7b0 con 0x7f74dc1034c0 2024-05-04T03:26:11.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:11.188+0000 7f74d37fe700 1 -- 172.21.15.1:0/1033301384 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11847 (secure 0 0 0) 0x7f74cc013070 con 0x7f74dc1034c0 2024-05-04T03:26:11.191 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:26:11.191 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2186681910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2186681910},{"type":"v1","addr":"172.21.15.1:6835","nonce":2186681910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14517,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2747191047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2747191047},{"type":"v1","addr":"172.21.15.1:6837","nonce":2747191047}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/4115120540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":4115120540},{"type":"v1","addr":"172.21.15.1:6839","nonce":4115120540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14577,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/3405709780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":3405709780},{"type":"v1","addr":"172.21.15.1:6841","nonce":3405709780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/3816098761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":3816098761},{"type":"v1","addr":"172.21.15.72:6833","nonce":3816098761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/379026128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":379026128},{"type":"v1","addr":"172.21.15.77:6833","nonce":379026128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-04T03:15:49.823471+0000","modified":"2024-05-04T03:25:24.389587+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24472,"mds_2":24467},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/3930984976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":3930984976},{"type":"v1","addr":"172.21.15.72:6835","nonce":3930984976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/4279951305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":4279951305},{"type":"v1","addr":"172.21.15.77:6835","nonce":4279951305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6837/2003521417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":2003521417},{"type":"v1","addr":"172.21.15.77:6837","nonce":2003521417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/1417614203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":1417614203},{"type":"v1","addr":"172.21.15.72:6837","nonce":1417614203}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.77:6839/922723189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":922723189},{"type":"v1","addr":"172.21.15.77:6839","nonce":922723189}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2976050650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2976050650},{"type":"v1","addr":"172.21.15.72:6839","nonce":2976050650}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24467,24472,24494]},"id":1}]} 2024-05-04T03:26:11.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:11.191+0000 7f74e0987700 1 -- 172.21.15.1:0/1033301384 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f74c80b2880 msgr2=0x7f74c80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:11.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:11.191+0000 7f74e0987700 1 --2- 172.21.15.1:0/1033301384 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f74c80b2880 0x7f74c80b4d90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f74c4005950 tx=0x7f74c40058e0 comp rx=0 tx=0).stop 2024-05-04T03:26:11.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:11.191+0000 7f74e0987700 1 -- 172.21.15.1:0/1033301384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74dc1034c0 msgr2=0x7f74dc19bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:11.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:11.191+0000 7f74e0987700 1 --2- 172.21.15.1:0/1033301384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74dc1034c0 0x7f74dc19bc10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f74cc00b580 tx=0x7f74cc0047b0 comp rx=0 tx=0).stop 2024-05-04T03:26:11.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:11.191+0000 7f74e0987700 1 -- 172.21.15.1:0/1033301384 shutdown_connections 2024-05-04T03:26:11.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:11.191+0000 7f74e0987700 1 --2- 172.21.15.1:0/1033301384 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f74c80b2880 0x7f74c80b4d90 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:11.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:11.191+0000 7f74e0987700 1 --2- 172.21.15.1:0/1033301384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74dc108740 0x7f74dc1a6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:11.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:11.191+0000 7f74e0987700 1 --2- 172.21.15.1:0/1033301384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f74dc1034c0 0x7f74dc19bc10 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:11.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:11.191+0000 7f74e0987700 1 --2- 172.21.15.1:0/1033301384 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f74dc107df0 0x7f74dc19c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:11.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:11.191+0000 7f74e0987700 1 -- 172.21.15.1:0/1033301384 >> 172.21.15.1:0/1033301384 conn(0x7f74dc0fedf0 msgr2=0x7f74dc107070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:11.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:11.191+0000 7f74e0987700 1 -- 172.21.15.1:0/1033301384 shutdown_connections 2024-05-04T03:26:11.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:11.191+0000 7f74e0987700 1 -- 172.21.15.1:0/1033301384 wait complete. 2024-05-04T03:26:11.194 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 35 2024-05-04T03:26:11.865 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:11 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/1033301384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:26:11.898 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-04T03:26:11.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:11 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/1033301384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:26:11.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:11 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/1033301384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-04T03:26:12.153 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:26:12.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:12 smithi001 ceph-mon[29731]: pgmap v602: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-04T03:26:12.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:12 smithi077 ceph-mon[33531]: pgmap v602: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-04T03:26:12.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:12 smithi072 ceph-mon[33332]: pgmap v602: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-04T03:26:13.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.782+0000 7f9238029700 1 -- 172.21.15.1:0/3878237349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f923010e8d0 msgr2=0x7f9230110de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:13.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.782+0000 7f9238029700 1 --2- 172.21.15.1:0/3878237349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f923010e8d0 0x7f9230110de0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f922800b0a0 tx=0x7f922800b3b0 comp rx=0 tx=0).stop 2024-05-04T03:26:13.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.782+0000 7f9238029700 1 -- 172.21.15.1:0/3878237349 shutdown_connections 2024-05-04T03:26:13.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.782+0000 7f9238029700 1 --2- 172.21.15.1:0/3878237349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f923010e8d0 0x7f9230110de0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:13.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.782+0000 7f9238029700 1 --2- 172.21.15.1:0/3878237349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9230105460 0x7f9230105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:13.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.782+0000 7f9238029700 1 --2- 172.21.15.1:0/3878237349 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9230105db0 0x7f923010e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:13.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.782+0000 7f9238029700 1 -- 172.21.15.1:0/3878237349 >> 172.21.15.1:0/3878237349 conn(0x7f92300fedf0 msgr2=0x7f9230101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:13.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.782+0000 7f9238029700 1 -- 172.21.15.1:0/3878237349 shutdown_connections 2024-05-04T03:26:13.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.782+0000 7f9238029700 1 -- 172.21.15.1:0/3878237349 wait complete. 2024-05-04T03:26:13.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.783+0000 7f9238029700 1 Processor -- start 2024-05-04T03:26:13.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.783+0000 7f9238029700 1 -- start start 2024-05-04T03:26:13.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.783+0000 7f9238029700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9230105460 0x7f923019ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:13.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.783+0000 7f9238029700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9230105db0 0x7f923019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:13.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.783+0000 7f9238029700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f923010e8d0 0x7f923019f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:13.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.783+0000 7f9238029700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f9230114480 con 0x7f9230105460 2024-05-04T03:26:13.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.783+0000 7f9238029700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9230114300 con 0x7f9230105db0 2024-05-04T03:26:13.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.783+0000 7f9238029700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9230114600 con 0x7f923010e8d0 2024-05-04T03:26:13.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.783+0000 7f9235dc5700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9230105460 0x7f923019ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:13.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.784+0000 7f9235dc5700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9230105460 0x7f923019ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42320/0 (socket says 172.21.15.1:42320) 2024-05-04T03:26:13.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.784+0000 7f9235dc5700 1 -- 172.21.15.1:0/3377331779 learned_addr learned my addr 172.21.15.1:0/3377331779 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:26:13.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.784+0000 7f92355c4700 1 --2- 172.21.15.1:0/3377331779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9230105db0 0x7f923019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:13.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.784+0000 7f92365c6700 1 --2- 172.21.15.1:0/3377331779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f923010e8d0 0x7f923019f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:13.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.784+0000 7f9235dc5700 1 -- 172.21.15.1:0/3377331779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f923010e8d0 msgr2=0x7f923019f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:13.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.784+0000 7f9235dc5700 1 --2- 172.21.15.1:0/3377331779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f923010e8d0 0x7f923019f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:13.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.784+0000 7f9235dc5700 1 -- 172.21.15.1:0/3377331779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9230105db0 msgr2=0x7f923019b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:13.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.784+0000 7f9235dc5700 1 --2- 172.21.15.1:0/3377331779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9230105db0 0x7f923019b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:13.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.784+0000 7f9235dc5700 1 -- 172.21.15.1:0/3377331779 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f923019fbb0 con 0x7f9230105460 2024-05-04T03:26:13.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.784+0000 7f92355c4700 1 --2- 172.21.15.1:0/3377331779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9230105db0 0x7f923019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:26:13.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.784+0000 7f9235dc5700 1 --2- 172.21.15.1:0/3377331779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9230105460 0x7f923019ad90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f922c00c990 tx=0x7f922c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:13.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.784+0000 7f9226ffd700 1 -- 172.21.15.1:0/3377331779 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c007b00 con 0x7f9230105460 2024-05-04T03:26:13.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.784+0000 7f9238029700 1 -- 172.21.15.1:0/3377331779 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f923019fea0 con 0x7f9230105460 2024-05-04T03:26:13.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.784+0000 7f9226ffd700 1 -- 172.21.15.1:0/3377331779 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f922c007ca0 con 0x7f9230105460 2024-05-04T03:26:13.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.784+0000 7f9226ffd700 1 -- 172.21.15.1:0/3377331779 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922c018660 con 0x7f9230105460 2024-05-04T03:26:13.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.784+0000 7f9238029700 1 -- 172.21.15.1:0/3377331779 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92301a7870 con 0x7f9230105460 2024-05-04T03:26:13.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.786+0000 7f9238029700 1 -- 172.21.15.1:0/3377331779 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92300512c0 con 0x7f9230105460 2024-05-04T03:26:13.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.787+0000 7f9226ffd700 1 -- 172.21.15.1:0/3377331779 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f922c00f450 con 0x7f9230105460 2024-05-04T03:26:13.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.787+0000 7f9226ffd700 1 --2- 172.21.15.1:0/3377331779 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f921c0b2880 0x7f921c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:13.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.788+0000 7f92355c4700 1 --2- 172.21.15.1:0/3377331779 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f921c0b2880 0x7f921c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:13.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.788+0000 7f9226ffd700 1 -- 172.21.15.1:0/3377331779 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7f922c0e21b0 con 0x7f9230105460 2024-05-04T03:26:13.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.788+0000 7f92355c4700 1 --2- 172.21.15.1:0/3377331779 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f921c0b2880 0x7f921c0b4d90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9220005e10 tx=0x7f9220005d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:13.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:13.793+0000 7f9226ffd700 1 -- 172.21.15.1:0/3377331779 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f922c014030 con 0x7f9230105460 2024-05-04T03:26:14.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:14.013+0000 7f9238029700 1 -- 172.21.15.1:0/3377331779 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f923019c0f0 con 0x7f9230105460 2024-05-04T03:26:14.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:14.014+0000 7f9226ffd700 1 -- 172.21.15.1:0/3377331779 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22380 (secure 0 0 0) 0x7f922c0a9f90 con 0x7f9230105460 2024-05-04T03:26:14.017 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-05-04T03:26:14.017 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":83,"fsid":"4a572aa8-09c3-11ef-bc97-c7b262605968","created":"2024-05-04T03:07:32.845547+0000","modified":"2024-05-04T03:16:34.321580+0000","last_up_change":"2024-05-04T03:14:44.192571+0000","last_in_change":"2024-05-04T03:14:23.353342+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-04T03:11:01.139189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-04T03:15:43.151696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-04T03:15:46.753539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"100fe035-2454-4c95-90c7-759956ec36a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6803","nonce":3479011484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6805","nonce":3479011484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6809","nonce":3479011484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":3479011484},{"type":"v1","addr":"172.21.15.1:6807","nonce":3479011484}]},"public_addr":"172.21.15.1:6803/3479011484","cluster_addr":"172.21.15.1:6805/3479011484","heartbeat_back_addr":"172.21.15.1:6809/3479011484","heartbeat_front_addr":"172.21.15.1:6807/3479011484","state":["exists","up"]},{"osd":1,"uuid":"5826ac76-ef6b-4040-8ad1-305db355af79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6801","nonce":3725706149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6803","nonce":3725706149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6807","nonce":3725706149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":3725706149},{"type":"v1","addr":"172.21.15.72:6805","nonce":3725706149}]},"public_addr":"172.21.15.72:6801/3725706149","cluster_addr":"172.21.15.72:6803/3725706149","heartbeat_back_addr":"172.21.15.72:6807/3725706149","heartbeat_front_addr":"172.21.15.72:6805/3725706149","state":["exists","up"]},{"osd":2,"uuid":"923d76e5-b032-4190-9e35-546d27722a3c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6801","nonce":515038343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6803","nonce":515038343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6807","nonce":515038343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":515038343},{"type":"v1","addr":"172.21.15.77:6805","nonce":515038343}]},"public_addr":"172.21.15.77:6801/515038343","cluster_addr":"172.21.15.77:6803/515038343","heartbeat_back_addr":"172.21.15.77:6807/515038343","heartbeat_front_addr":"172.21.15.77:6805/515038343","state":["exists","up"]},{"osd":3,"uuid":"495cc0c5-6a99-481f-ad9e-38f05fb84356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6811","nonce":4050439365}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6813","nonce":4050439365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6817","nonce":4050439365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":4050439365},{"type":"v1","addr":"172.21.15.1:6815","nonce":4050439365}]},"public_addr":"172.21.15.1:6811/4050439365","cluster_addr":"172.21.15.1:6813/4050439365","heartbeat_back_addr":"172.21.15.1:6817/4050439365","heartbeat_front_addr":"172.21.15.1:6815/4050439365","state":["exists","up"]},{"osd":4,"uuid":"b86b8026-79c0-4056-9e67-a61f54b6ab5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6809","nonce":2501732391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6811","nonce":2501732391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6815","nonce":2501732391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":2501732391},{"type":"v1","addr":"172.21.15.72:6813","nonce":2501732391}]},"public_addr":"172.21.15.72:6809/2501732391","cluster_addr":"172.21.15.72:6811/2501732391","heartbeat_back_addr":"172.21.15.72:6815/2501732391","heartbeat_front_addr":"172.21.15.72:6813/2501732391","state":["exists","up"]},{"osd":5,"uuid":"80975cb9-39e0-46b8-a343-9ff69194a14e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6809","nonce":3582852165}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6811","nonce":3582852165}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6815","nonce":3582852165}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":3582852165},{"type":"v1","addr":"172.21.15.77:6813","nonce":3582852165}]},"public_addr":"172.21.15.77:6809/3582852165","cluster_addr":"172.21.15.77:6811/3582852165","heartbeat_back_addr":"172.21.15.77:6815/3582852165","heartbeat_front_addr":"172.21.15.77:6813/3582852165","state":["exists","up"]},{"osd":6,"uuid":"aca3fac5-fbb5-42de-bf0e-e4a0db7381e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6819","nonce":1799033649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6821","nonce":1799033649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6825","nonce":1799033649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1799033649},{"type":"v1","addr":"172.21.15.1:6823","nonce":1799033649}]},"public_addr":"172.21.15.1:6819/1799033649","cluster_addr":"172.21.15.1:6821/1799033649","heartbeat_back_addr":"172.21.15.1:6825/1799033649","heartbeat_front_addr":"172.21.15.1:6823/1799033649","state":["exists","up"]},{"osd":7,"uuid":"cbff6b8a-c463-4249-ae45-fdc6f99a5489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6817","nonce":3460835447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6819","nonce":3460835447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6823","nonce":3460835447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":3460835447},{"type":"v1","addr":"172.21.15.72:6821","nonce":3460835447}]},"public_addr":"172.21.15.72:6817/3460835447","cluster_addr":"172.21.15.72:6819/3460835447","heartbeat_back_addr":"172.21.15.72:6823/3460835447","heartbeat_front_addr":"172.21.15.72:6821/3460835447","state":["exists","up"]},{"osd":8,"uuid":"e140bf13-4887-4d68-9373-3d4fc1c4584a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6817","nonce":1124479323}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6819","nonce":1124479323}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6823","nonce":1124479323}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":1124479323},{"type":"v1","addr":"172.21.15.77:6821","nonce":1124479323}]},"public_addr":"172.21.15.77:6817/1124479323","cluster_addr":"172.21.15.77:6819/1124479323","heartbeat_back_addr":"172.21.15.77:6823/1124479323","heartbeat_front_addr":"172.21.15.77:6821/1124479323","state":["exists","up"]},{"osd":9,"uuid":"f1e8a232-cb11-4b9d-8789-e1637fa5aaa8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6827","nonce":2681322272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6829","nonce":2681322272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6833","nonce":2681322272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2681322272},{"type":"v1","addr":"172.21.15.1:6831","nonce":2681322272}]},"public_addr":"172.21.15.1:6827/2681322272","cluster_addr":"172.21.15.1:6829/2681322272","heartbeat_back_addr":"172.21.15.1:6833/2681322272","heartbeat_front_addr":"172.21.15.1:6831/2681322272","state":["exists","up"]},{"osd":10,"uuid":"79b570af-3d1e-439d-a328-9e6bc7d67839","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6825","nonce":3293703569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6827","nonce":3293703569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6831","nonce":3293703569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":3293703569},{"type":"v1","addr":"172.21.15.72:6829","nonce":3293703569}]},"public_addr":"172.21.15.72:6825/3293703569","cluster_addr":"172.21.15.72:6827/3293703569","heartbeat_back_addr":"172.21.15.72:6831/3293703569","heartbeat_front_addr":"172.21.15.72:6829/3293703569","state":["exists","up"]},{"osd":11,"uuid":"e35bf83d-9ee4-4745-96c2-1c35cc8eefba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6825","nonce":4155947548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6827","nonce":4155947548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6831","nonce":4155947548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":4155947548},{"type":"v1","addr":"172.21.15.77:6829","nonce":4155947548}]},"public_addr":"172.21.15.77:6825/4155947548","cluster_addr":"172.21.15.77:6827/4155947548","heartbeat_back_addr":"172.21.15.77:6831/4155947548","heartbeat_front_addr":"172.21.15.77:6829/4155947548","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:09.467126+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:33.861139+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:10:57.741962+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:11:21.746021+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:11:46.323252+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:12:10.640180+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:12:35.750591+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:00.306549+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:25.012495+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:13:51.185229+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:14:16.403850+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-04T03:14:41.653316+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/2029897106":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/402488450":"2024-05-05T03:08:33.045884+0000","172.21.15.1:6801/128793132":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/2233526606":"2024-05-05T03:08:06.135783+0000","172.21.15.1:0/2715541855":"2024-05-05T03:08:33.045884+0000","172.21.15.1:0/2415697325":"2024-05-05T03:08:06.135783+0000","172.21.15.1:0/2305298106":"2024-05-05T03:08:06.135783+0000","172.21.15.1:6800/128793132":"2024-05-05T03:08:33.045884+0000","172.21.15.1:6800/4200529863":"2024-05-05T03:08:06.135783+0000","172.21.15.1:6801/4200529863":"2024-05-05T03:08:06.135783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-04T03:26:14.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:14.017+0000 7f9238029700 1 -- 172.21.15.1:0/3377331779 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f921c0b2880 msgr2=0x7f921c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:14.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:14.017+0000 7f9238029700 1 --2- 172.21.15.1:0/3377331779 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f921c0b2880 0x7f921c0b4d90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9220005e10 tx=0x7f9220005d60 comp rx=0 tx=0).stop 2024-05-04T03:26:14.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:14.017+0000 7f9238029700 1 -- 172.21.15.1:0/3377331779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9230105460 msgr2=0x7f923019ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:14.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:14.017+0000 7f9238029700 1 --2- 172.21.15.1:0/3377331779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9230105460 0x7f923019ad90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f922c00c990 tx=0x7f922c00cd50 comp rx=0 tx=0).stop 2024-05-04T03:26:14.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:14.017+0000 7f9238029700 1 -- 172.21.15.1:0/3377331779 shutdown_connections 2024-05-04T03:26:14.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:14.017+0000 7f9238029700 1 --2- 172.21.15.1:0/3377331779 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f921c0b2880 0x7f921c0b4d90 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:14.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:14.017+0000 7f9238029700 1 --2- 172.21.15.1:0/3377331779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f923010e8d0 0x7f923019f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:14.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:14.017+0000 7f9238029700 1 --2- 172.21.15.1:0/3377331779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9230105460 0x7f923019ad90 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:14.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:14.017+0000 7f9238029700 1 --2- 172.21.15.1:0/3377331779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9230105db0 0x7f923019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:14.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:14.018+0000 7f9238029700 1 -- 172.21.15.1:0/3377331779 >> 172.21.15.1:0/3377331779 conn(0x7f92300fedf0 msgr2=0x7f92300ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:14.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:14.018+0000 7f9238029700 1 -- 172.21.15.1:0/3377331779 shutdown_connections 2024-05-04T03:26:14.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:14.018+0000 7f9238029700 1 -- 172.21.15.1:0/3377331779 wait complete. 2024-05-04T03:26:14.616 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs fail cephfs 2024-05-04T03:26:14.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:14 smithi001 ceph-mon[29731]: pgmap v603: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-04T03:26:14.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:14 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3377331779' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:26:14.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:14 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:26:14.882 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:26:14.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:14 smithi077 ceph-mon[33531]: pgmap v603: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-04T03:26:14.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:14 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/3377331779' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:26:14.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:14 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:26:14.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:14 smithi072 ceph-mon[33332]: pgmap v603: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-04T03:26:14.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:14 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/3377331779' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-04T03:26:14.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:14 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-04T03:26:15.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:15 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:26:15.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:15 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:26:15.630 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:15 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:26:15.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:15 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:26:15.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:15 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:26:15.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:15 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:26:15.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:15 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-04T03:26:15.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:15 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-04T03:26:15.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:15 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' 2024-05-04T03:26:16.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.530+0000 7fe22b2d1700 1 -- 172.21.15.1:0/2047787916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe224109860 msgr2=0x7fe2241111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:16.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.530+0000 7fe22b2d1700 1 --2- 172.21.15.1:0/2047787916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe224109860 0x7fe2241111c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00b0d0 tx=0x7fe21c00b3e0 comp rx=0 tx=0).stop 2024-05-04T03:26:16.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.530+0000 7fe22b2d1700 1 -- 172.21.15.1:0/2047787916 shutdown_connections 2024-05-04T03:26:16.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.530+0000 7fe22b2d1700 1 --2- 172.21.15.1:0/2047787916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe224109860 0x7fe2241111c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:16.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.530+0000 7fe22b2d1700 1 --2- 172.21.15.1:0/2047787916 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2241045e0 0x7fe2241049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:16.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.530+0000 7fe22b2d1700 1 --2- 172.21.15.1:0/2047787916 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe224108f10 0x7fe224109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:16.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.530+0000 7fe22b2d1700 1 -- 172.21.15.1:0/2047787916 >> 172.21.15.1:0/2047787916 conn(0x7fe2240ffe70 msgr2=0x7fe2241022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:16.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.530+0000 7fe22b2d1700 1 -- 172.21.15.1:0/2047787916 shutdown_connections 2024-05-04T03:26:16.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.530+0000 7fe22b2d1700 1 -- 172.21.15.1:0/2047787916 wait complete. 2024-05-04T03:26:16.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.531+0000 7fe22b2d1700 1 Processor -- start 2024-05-04T03:26:16.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.531+0000 7fe22b2d1700 1 -- start start 2024-05-04T03:26:16.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.531+0000 7fe22b2d1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2241045e0 0x7fe22419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:16.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.531+0000 7fe22b2d1700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe224108f10 0x7fe22419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:16.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.531+0000 7fe22b2d1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe224109860 0x7fe22419f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:16.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.531+0000 7fe22b2d1700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7fe224114490 con 0x7fe224108f10 2024-05-04T03:26:16.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.531+0000 7fe22b2d1700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe224114310 con 0x7fe2241045e0 2024-05-04T03:26:16.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.531+0000 7fe22b2d1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe224114610 con 0x7fe224109860 2024-05-04T03:26:16.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.531+0000 7fe22906d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2241045e0 0x7fe22419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:16.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.531+0000 7fe22886c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe224108f10 0x7fe22419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:16.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.531+0000 7fe22906d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2241045e0 0x7fe22419ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.1:38684/0 (socket says 172.21.15.1:38684) 2024-05-04T03:26:16.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.532+0000 7fe22906d700 1 -- 172.21.15.1:0/3746536686 learned_addr learned my addr 172.21.15.1:0/3746536686 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:26:16.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.532+0000 7fe22986e700 1 --2- 172.21.15.1:0/3746536686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe224109860 0x7fe22419f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:16.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.532+0000 7fe22906d700 1 -- 172.21.15.1:0/3746536686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe224109860 msgr2=0x7fe22419f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:16.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.532+0000 7fe22906d700 1 --2- 172.21.15.1:0/3746536686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe224109860 0x7fe22419f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:16.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.532+0000 7fe22906d700 1 -- 172.21.15.1:0/3746536686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe224108f10 msgr2=0x7fe22419b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:16.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.532+0000 7fe22906d700 1 --2- 172.21.15.1:0/3746536686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe224108f10 0x7fe22419b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:16.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.532+0000 7fe22906d700 1 -- 172.21.15.1:0/3746536686 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22419fcf0 con 0x7fe2241045e0 2024-05-04T03:26:16.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.532+0000 7fe22986e700 1 --2- 172.21.15.1:0/3746536686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe224109860 0x7fe22419f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-04T03:26:16.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.532+0000 7fe22906d700 1 --2- 172.21.15.1:0/3746536686 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2241045e0 0x7fe22419ae20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe22000c920 tx=0x7fe22000cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:16.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.532+0000 7fe2167fc700 1 -- 172.21.15.1:0/3746536686 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe220007b00 con 0x7fe2241045e0 2024-05-04T03:26:16.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.532+0000 7fe2167fc700 1 -- 172.21.15.1:0/3746536686 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe220004ce0 con 0x7fe2241045e0 2024-05-04T03:26:16.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.532+0000 7fe22b2d1700 1 -- 172.21.15.1:0/3746536686 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe22419ffe0 con 0x7fe2241045e0 2024-05-04T03:26:16.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.532+0000 7fe2167fc700 1 -- 172.21.15.1:0/3746536686 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe220005500 con 0x7fe2241045e0 2024-05-04T03:26:16.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.532+0000 7fe22b2d1700 1 -- 172.21.15.1:0/3746536686 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2240759f0 con 0x7fe2241045e0 2024-05-04T03:26:16.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.534+0000 7fe22b2d1700 1 -- 172.21.15.1:0/3746536686 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe224105a90 con 0x7fe2241045e0 2024-05-04T03:26:16.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.535+0000 7fe2167fc700 1 -- 172.21.15.1:0/3746536686 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7fe220007ca0 con 0x7fe2241045e0 2024-05-04T03:26:16.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.536+0000 7fe2167fc700 1 --2- 172.21.15.1:0/3746536686 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe2100b2880 0x7fe2100b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:16.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.536+0000 7fe22886c700 1 --2- 172.21.15.1:0/3746536686 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe2100b2880 0x7fe2100b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:16.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.536+0000 7fe2167fc700 1 -- 172.21.15.1:0/3746536686 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8124+0+0 (secure 0 0 0) 0x7fe2200e1d30 con 0x7fe2241045e0 2024-05-04T03:26:16.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.536+0000 7fe22886c700 1 --2- 172.21.15.1:0/3746536686 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe2100b2880 0x7fe2100b4d90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe224108780 tx=0x7fe218012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:16.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.541+0000 7fe2167fc700 1 -- 172.21.15.1:0/3746536686 <== mon.2 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe220014030 con 0x7fe2241045e0 2024-05-04T03:26:16.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.806+0000 7fe22b2d1700 1 -- 172.21.15.1:0/3746536686 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe22419c180 con 0x7fe2241045e0 2024-05-04T03:26:16.809 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:16 smithi001 ceph-mon[29731]: pgmap v604: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-04T03:26:16.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.816+0000 7fe2167fc700 1 -- 172.21.15.1:0/3746536686 <== mon.2 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) ==== 161+0+0 (secure 0 0 0) 0x7fe2200a9a70 con 0x7fe2241045e0 2024-05-04T03:26:16.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.819+0000 7fe22b2d1700 1 -- 172.21.15.1:0/3746536686 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe2100b2880 msgr2=0x7fe2100b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:16.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.819+0000 7fe22b2d1700 1 --2- 172.21.15.1:0/3746536686 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe2100b2880 0x7fe2100b4d90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe224108780 tx=0x7fe218012040 comp rx=0 tx=0).stop 2024-05-04T03:26:16.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.819+0000 7fe22b2d1700 1 -- 172.21.15.1:0/3746536686 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2241045e0 msgr2=0x7fe22419ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:16.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.819+0000 7fe22b2d1700 1 --2- 172.21.15.1:0/3746536686 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2241045e0 0x7fe22419ae20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe22000c920 tx=0x7fe22000cc30 comp rx=0 tx=0).stop 2024-05-04T03:26:16.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.819+0000 7fe22b2d1700 1 -- 172.21.15.1:0/3746536686 shutdown_connections 2024-05-04T03:26:16.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.819+0000 7fe22b2d1700 1 --2- 172.21.15.1:0/3746536686 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2241045e0 0x7fe22419ae20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:16.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.819+0000 7fe22b2d1700 1 --2- 172.21.15.1:0/3746536686 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7fe2100b2880 0x7fe2100b4d90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:16.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.819+0000 7fe22b2d1700 1 --2- 172.21.15.1:0/3746536686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe224109860 0x7fe22419f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:16.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.819+0000 7fe22b2d1700 1 --2- 172.21.15.1:0/3746536686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe224108f10 0x7fe22419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:16.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.819+0000 7fe22b2d1700 1 -- 172.21.15.1:0/3746536686 >> 172.21.15.1:0/3746536686 conn(0x7fe2240ffe70 msgr2=0x7fe224107040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:16.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.819+0000 7fe22b2d1700 1 -- 172.21.15.1:0/3746536686 shutdown_connections 2024-05-04T03:26:16.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:16.819+0000 7fe22b2d1700 1 -- 172.21.15.1:0/3746536686 wait complete. 2024-05-04T03:26:16.822 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-04T03:26:16.829 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:16 smithi077 ceph-mon[33531]: pgmap v604: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-04T03:26:16.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:16 smithi072 ceph-mon[33332]: pgmap v604: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-04T03:26:17.129 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:16 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mon-a[29727]: 2024-05-04T03:26:16.807+0000 7fa955bdd700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-04T03:26:17.179 INFO:journalctl@ceph.mds.c.smithi077.stdout:May 04 03:26:16 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-c[63817]: ignoring --setuser ceph since I am not root 2024-05-04T03:26:17.179 INFO:journalctl@ceph.mds.c.smithi077.stdout:May 04 03:26:16 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-c[63817]: ignoring --setgroup ceph since I am not root 2024-05-04T03:26:17.179 INFO:journalctl@ceph.mds.c.smithi077.stdout:May 04 03:26:16 smithi077 ceph-mds[63821]: -- 172.21.15.77:0/1880206220 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55630c25da40 con 0x55630c2a5a80 2024-05-04T03:26:17.179 INFO:journalctl@ceph.mds.c.smithi077.stdout:May 04 03:26:16 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-c[63817]: starting mds.c at 2024-05-04T03:26:17.179 INFO:journalctl@ceph.mds.f.smithi077.stdout:May 04 03:26:16 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-f[63504]: ignoring --setuser ceph since I am not root 2024-05-04T03:26:17.179 INFO:journalctl@ceph.mds.f.smithi077.stdout:May 04 03:26:16 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-f[63504]: ignoring --setgroup ceph since I am not root 2024-05-04T03:26:17.179 INFO:journalctl@ceph.mds.f.smithi077.stdout:May 04 03:26:16 smithi077 ceph-mds[63508]: -- 172.21.15.77:0/903629458 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b651631a40 con 0x55b651679180 2024-05-04T03:26:17.180 INFO:journalctl@ceph.mds.f.smithi077.stdout:May 04 03:26:16 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-f[63504]: starting mds.f at 2024-05-04T03:26:17.180 INFO:journalctl@ceph.mds.i.smithi077.stdout:May 04 03:26:16 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-i[64134]: ignoring --setuser ceph since I am not root 2024-05-04T03:26:17.180 INFO:journalctl@ceph.mds.i.smithi077.stdout:May 04 03:26:16 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-i[64134]: ignoring --setgroup ceph since I am not root 2024-05-04T03:26:17.180 INFO:journalctl@ceph.mds.i.smithi077.stdout:May 04 03:26:16 smithi077 ceph-mds[64138]: -- 172.21.15.77:0/2545714682 <== mon.2 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fa99c41a40 con 0x55fa99c89600 2024-05-04T03:26:17.180 INFO:journalctl@ceph.mds.i.smithi077.stdout:May 04 03:26:16 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-i[64134]: starting mds.i at 2024-05-04T03:26:17.247 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 04 03:26:16 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-e[62701]: ignoring --setuser ceph since I am not root 2024-05-04T03:26:17.247 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 04 03:26:16 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-e[62701]: ignoring --setgroup ceph since I am not root 2024-05-04T03:26:17.247 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 04 03:26:16 smithi072 ceph-mds[62705]: -- 172.21.15.72:0/684956380 <== mon.1 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ef77ad3a40 con 0x55ef77b1b600 2024-05-04T03:26:17.247 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 04 03:26:16 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-e[62701]: starting mds.e at 2024-05-04T03:26:17.247 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 04 03:26:16 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-b[63318]: ignoring --setuser ceph since I am not root 2024-05-04T03:26:17.248 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 04 03:26:16 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-b[63318]: ignoring --setgroup ceph since I am not root 2024-05-04T03:26:17.248 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 04 03:26:16 smithi072 ceph-mds[63322]: -- 172.21.15.72:0/1617590497 <== mon.2 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563ff6133a40 con 0x563ff617ba80 2024-05-04T03:26:17.248 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 04 03:26:16 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-b[63318]: starting mds.b at 2024-05-04T03:26:17.248 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 04 03:26:16 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-h[63012]: ignoring --setuser ceph since I am not root 2024-05-04T03:26:17.248 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 04 03:26:16 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-h[63012]: ignoring --setgroup ceph since I am not root 2024-05-04T03:26:17.248 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 04 03:26:16 smithi072 ceph-mds[63016]: -- 172.21.15.72:0/237252406 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564247a45a40 con 0x564247a8d180 2024-05-04T03:26:17.248 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 04 03:26:16 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-h[63012]: starting mds.h at 2024-05-04T03:26:17.438 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-04T03:26:17.688 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:26:17.810 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:17 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3746536686' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-04T03:26:17.810 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:17 smithi001 ceph-mon[29731]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-04T03:26:17.810 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:17 smithi001 ceph-mon[29731]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-04T03:26:17.810 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:17 smithi001 ceph-mon[29731]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-04T03:26:17.810 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:17 smithi001 ceph-mon[29731]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-04T03:26:17.810 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:17 smithi001 ceph-mon[29731]: osdmap e84: 12 total, 12 up, 12 in 2024-05-04T03:26:17.810 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:17 smithi001 ceph-mon[29731]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-04T03:26:17.845 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:17 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/3746536686' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-04T03:26:17.845 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:17 smithi077 ceph-mon[33531]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-04T03:26:17.845 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:17 smithi077 ceph-mon[33531]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-04T03:26:17.845 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:17 smithi077 ceph-mon[33531]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-04T03:26:17.845 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:17 smithi077 ceph-mon[33531]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-04T03:26:17.845 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:17 smithi077 ceph-mon[33531]: osdmap e84: 12 total, 12 up, 12 in 2024-05-04T03:26:17.845 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:17 smithi077 ceph-mon[33531]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-04T03:26:17.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:17 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/3746536686' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-04T03:26:17.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:17 smithi072 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-04T03:26:17.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:17 smithi072 ceph-mon[33332]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-04T03:26:17.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:17 smithi072 ceph-mon[33332]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-04T03:26:17.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:17 smithi072 ceph-mon[33332]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-04T03:26:17.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:17 smithi072 ceph-mon[33332]: osdmap e84: 12 total, 12 up, 12 in 2024-05-04T03:26:17.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:17 smithi072 ceph-mon[33332]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-04T03:26:18.838 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:18 smithi001 ceph-mon[29731]: pgmap v606: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 818 B/s wr, 7 op/s 2024-05-04T03:26:18.838 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:18 smithi001 ceph-mon[29731]: mds.? [v2:172.21.15.72:6834/1981057545,v1:172.21.15.72:6837/1981057545] up:boot 2024-05-04T03:26:18.839 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:18 smithi001 ceph-mon[29731]: mds.? [v2:172.21.15.72:6836/1477969060,v1:172.21.15.72:6839/1477969060] up:boot 2024-05-04T03:26:18.839 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:18 smithi001 ceph-mon[29731]: mds.? [v2:172.21.15.77:6834/4201692095,v1:172.21.15.77:6837/4201692095] up:boot 2024-05-04T03:26:18.839 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:18 smithi001 ceph-mon[29731]: mds.? [v2:172.21.15.72:6835/4101057281,v1:172.21.15.72:6838/4101057281] up:boot 2024-05-04T03:26:18.839 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:18 smithi001 ceph-mon[29731]: mds.? [v2:172.21.15.77:6836/2887392665,v1:172.21.15.77:6839/2887392665] up:boot 2024-05-04T03:26:18.839 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:18 smithi001 ceph-mon[29731]: mds.? [v2:172.21.15.77:6835/4194429268,v1:172.21.15.77:6838/4194429268] up:boot 2024-05-04T03:26:18.839 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:18 smithi001 ceph-mon[29731]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-04T03:26:18.839 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:18 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-04T03:26:18.839 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:18 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-04T03:26:18.839 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:18 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-04T03:26:18.839 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:18 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-04T03:26:18.839 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:18 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-04T03:26:18.839 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:18 smithi001 ceph-mon[29731]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-04T03:26:18.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:18 smithi077 ceph-mon[33531]: pgmap v606: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 818 B/s wr, 7 op/s 2024-05-04T03:26:18.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:18 smithi077 ceph-mon[33531]: mds.? [v2:172.21.15.72:6834/1981057545,v1:172.21.15.72:6837/1981057545] up:boot 2024-05-04T03:26:18.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:18 smithi077 ceph-mon[33531]: mds.? [v2:172.21.15.72:6836/1477969060,v1:172.21.15.72:6839/1477969060] up:boot 2024-05-04T03:26:18.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:18 smithi077 ceph-mon[33531]: mds.? [v2:172.21.15.77:6834/4201692095,v1:172.21.15.77:6837/4201692095] up:boot 2024-05-04T03:26:18.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:18 smithi077 ceph-mon[33531]: mds.? [v2:172.21.15.72:6835/4101057281,v1:172.21.15.72:6838/4101057281] up:boot 2024-05-04T03:26:18.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:18 smithi077 ceph-mon[33531]: mds.? [v2:172.21.15.77:6836/2887392665,v1:172.21.15.77:6839/2887392665] up:boot 2024-05-04T03:26:18.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:18 smithi077 ceph-mon[33531]: mds.? [v2:172.21.15.77:6835/4194429268,v1:172.21.15.77:6838/4194429268] up:boot 2024-05-04T03:26:18.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:18 smithi077 ceph-mon[33531]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-04T03:26:18.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:18 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-04T03:26:18.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:18 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-04T03:26:18.930 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:18 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-04T03:26:18.930 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:18 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-04T03:26:18.930 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:18 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-04T03:26:18.930 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:18 smithi077 ceph-mon[33531]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-04T03:26:18.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:18 smithi072 ceph-mon[33332]: pgmap v606: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 818 B/s wr, 7 op/s 2024-05-04T03:26:18.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:18 smithi072 ceph-mon[33332]: mds.? [v2:172.21.15.72:6834/1981057545,v1:172.21.15.72:6837/1981057545] up:boot 2024-05-04T03:26:18.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:18 smithi072 ceph-mon[33332]: mds.? [v2:172.21.15.72:6836/1477969060,v1:172.21.15.72:6839/1477969060] up:boot 2024-05-04T03:26:18.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:18 smithi072 ceph-mon[33332]: mds.? [v2:172.21.15.77:6834/4201692095,v1:172.21.15.77:6837/4201692095] up:boot 2024-05-04T03:26:18.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:18 smithi072 ceph-mon[33332]: mds.? [v2:172.21.15.72:6835/4101057281,v1:172.21.15.72:6838/4101057281] up:boot 2024-05-04T03:26:18.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:18 smithi072 ceph-mon[33332]: mds.? [v2:172.21.15.77:6836/2887392665,v1:172.21.15.77:6839/2887392665] up:boot 2024-05-04T03:26:18.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:18 smithi072 ceph-mon[33332]: mds.? [v2:172.21.15.77:6835/4194429268,v1:172.21.15.77:6838/4194429268] up:boot 2024-05-04T03:26:18.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:18 smithi072 ceph-mon[33332]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-04T03:26:18.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:18 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-04T03:26:18.998 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:18 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-04T03:26:18.998 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:18 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-04T03:26:18.998 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:18 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-04T03:26:18.998 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:18 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-04T03:26:18.998 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:18 smithi072 ceph-mon[33332]: from='mgr.14150 172.21.15.1:0/3402803078' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-04T03:26:19.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.234+0000 7f5be722c700 1 -- 172.21.15.1:0/943099345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5be0109300 msgr2=0x7f5be0109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:19.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.234+0000 7f5be722c700 1 --2- 172.21.15.1:0/943099345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5be0109300 0x7f5be0109710 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4009990 tx=0x7f5bd4009ca0 comp rx=0 tx=0).stop 2024-05-04T03:26:19.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.234+0000 7f5be722c700 1 -- 172.21.15.1:0/943099345 shutdown_connections 2024-05-04T03:26:19.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.234+0000 7f5be722c700 1 --2- 172.21.15.1:0/943099345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5be01030e0 0x7f5be0111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:19.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.234+0000 7f5be722c700 1 --2- 172.21.15.1:0/943099345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5be0109300 0x7f5be0109710 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:19.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.234+0000 7f5be722c700 1 --2- 172.21.15.1:0/943099345 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5be0102790 0x7f5be0102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:19.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.234+0000 7f5be722c700 1 -- 172.21.15.1:0/943099345 >> 172.21.15.1:0/943099345 conn(0x7f5be00fe520 msgr2=0x7f5be0100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:19.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.234+0000 7f5be722c700 1 -- 172.21.15.1:0/943099345 shutdown_connections 2024-05-04T03:26:19.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.234+0000 7f5be722c700 1 -- 172.21.15.1:0/943099345 wait complete. 2024-05-04T03:26:19.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.235+0000 7f5be722c700 1 Processor -- start 2024-05-04T03:26:19.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.235+0000 7f5be722c700 1 -- start start 2024-05-04T03:26:19.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.235+0000 7f5be722c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5be0102790 0x7f5be019af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:19.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.235+0000 7f5be722c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5be01030e0 0x7f5be019b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:19.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.235+0000 7f5be722c700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5be0109300 0x7f5be019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:19.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.235+0000 7f5be722c700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f5be01145b0 con 0x7f5be0109300 2024-05-04T03:26:19.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.235+0000 7f5be722c700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5be0114430 con 0x7f5be01030e0 2024-05-04T03:26:19.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.235+0000 7f5be722c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5be0114730 con 0x7f5be0102790 2024-05-04T03:26:19.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.235+0000 7f5be57c9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5be0109300 0x7f5be019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:19.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.235+0000 7f5be57c9700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5be0109300 0x7f5be019f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:42362/0 (socket says 172.21.15.1:42362) 2024-05-04T03:26:19.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.235+0000 7f5be57c9700 1 -- 172.21.15.1:0/954068712 learned_addr learned my addr 172.21.15.1:0/954068712 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:26:19.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.235+0000 7f5bdffff700 1 --2- 172.21.15.1:0/954068712 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5be01030e0 0x7f5be019b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:19.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.235+0000 7f5be4fc8700 1 --2- 172.21.15.1:0/954068712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5be0102790 0x7f5be019af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:19.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.236+0000 7f5be57c9700 1 -- 172.21.15.1:0/954068712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5be0102790 msgr2=0x7f5be019af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:19.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.236+0000 7f5be57c9700 1 --2- 172.21.15.1:0/954068712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5be0102790 0x7f5be019af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:19.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.236+0000 7f5be57c9700 1 -- 172.21.15.1:0/954068712 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5be01030e0 msgr2=0x7f5be019b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:19.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.236+0000 7f5be57c9700 1 --2- 172.21.15.1:0/954068712 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5be01030e0 0x7f5be019b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:19.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.236+0000 7f5be57c9700 1 -- 172.21.15.1:0/954068712 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be019fdc0 con 0x7f5be0109300 2024-05-04T03:26:19.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.236+0000 7f5be57c9700 1 --2- 172.21.15.1:0/954068712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5be0109300 0x7f5be019f720 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8007ee0 tx=0x7f5bd800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:19.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.236+0000 7f5bddffb700 1 -- 172.21.15.1:0/954068712 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8004bb0 con 0x7f5be0109300 2024-05-04T03:26:19.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.236+0000 7f5bddffb700 1 -- 172.21.15.1:0/954068712 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bd8004d50 con 0x7f5be0109300 2024-05-04T03:26:19.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.236+0000 7f5be722c700 1 -- 172.21.15.1:0/954068712 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be01a00b0 con 0x7f5be0109300 2024-05-04T03:26:19.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.236+0000 7f5bddffb700 1 -- 172.21.15.1:0/954068712 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8005920 con 0x7f5be0109300 2024-05-04T03:26:19.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.236+0000 7f5be722c700 1 -- 172.21.15.1:0/954068712 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be01a79d0 con 0x7f5be0109300 2024-05-04T03:26:19.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.237+0000 7f5be722c700 1 -- 172.21.15.1:0/954068712 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be00512c0 con 0x7f5be0109300 2024-05-04T03:26:19.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.240+0000 7f5bddffb700 1 -- 172.21.15.1:0/954068712 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f5bd801a020 con 0x7f5be0109300 2024-05-04T03:26:19.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.240+0000 7f5bddffb700 1 --2- 172.21.15.1:0/954068712 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5bd00b2880 0x7f5bd00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:19.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.240+0000 7f5be4fc8700 1 --2- 172.21.15.1:0/954068712 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5bd00b2880 0x7f5bd00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:19.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.240+0000 7f5bddffb700 1 -- 172.21.15.1:0/954068712 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8382+0+0 (secure 0 0 0) 0x7f5bd80e1f70 con 0x7f5be0109300 2024-05-04T03:26:19.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.241+0000 7f5be4fc8700 1 --2- 172.21.15.1:0/954068712 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5bd00b2880 0x7f5bd00b4d90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40059d0 tx=0x7f5bd4005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:19.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.244+0000 7f5bddffb700 1 -- 172.21.15.1:0/954068712 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5bd8010070 con 0x7f5be0109300 2024-05-04T03:26:19.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.508+0000 7f5be722c700 1 -- 172.21.15.1:0/954068712 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5be002e5f0 con 0x7f5be0109300 2024-05-04T03:26:19.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.574+0000 7f5bddffb700 1 -- 172.21.15.1:0/954068712 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) ==== 104+0+0 (secure 0 0 0) 0x7f5bd80a9c00 con 0x7f5be0109300 2024-05-04T03:26:19.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.577+0000 7f5be722c700 1 -- 172.21.15.1:0/954068712 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5bd00b2880 msgr2=0x7f5bd00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:19.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.578+0000 7f5be722c700 1 --2- 172.21.15.1:0/954068712 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5bd00b2880 0x7f5bd00b4d90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40059d0 tx=0x7f5bd4005960 comp rx=0 tx=0).stop 2024-05-04T03:26:19.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.578+0000 7f5be722c700 1 -- 172.21.15.1:0/954068712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5be0109300 msgr2=0x7f5be019f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:19.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.578+0000 7f5be722c700 1 --2- 172.21.15.1:0/954068712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5be0109300 0x7f5be019f720 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8007ee0 tx=0x7f5bd800d3b0 comp rx=0 tx=0).stop 2024-05-04T03:26:19.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.578+0000 7f5be722c700 1 -- 172.21.15.1:0/954068712 shutdown_connections 2024-05-04T03:26:19.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.578+0000 7f5be722c700 1 --2- 172.21.15.1:0/954068712 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f5bd00b2880 0x7f5bd00b4d90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:19.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.578+0000 7f5be722c700 1 --2- 172.21.15.1:0/954068712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5be0109300 0x7f5be019f720 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:19.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.578+0000 7f5be722c700 1 --2- 172.21.15.1:0/954068712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5be0102790 0x7f5be019af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:19.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.578+0000 7f5be722c700 1 --2- 172.21.15.1:0/954068712 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5be01030e0 0x7f5be019b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:19.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.578+0000 7f5be722c700 1 -- 172.21.15.1:0/954068712 >> 172.21.15.1:0/954068712 conn(0x7f5be00fe520 msgr2=0x7f5be01071e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:19.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.578+0000 7f5be722c700 1 -- 172.21.15.1:0/954068712 shutdown_connections 2024-05-04T03:26:19.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:19.578+0000 7f5be722c700 1 -- 172.21.15.1:0/954068712 wait complete. 2024-05-04T03:26:19.879 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:19 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/954068712' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-04T03:26:19.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:19 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/954068712' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-04T03:26:19.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:19 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/954068712' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-04T03:26:20.220 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-04T03:26:20.470 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:26:20.852 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:20 smithi001 ceph-mon[29731]: pgmap v607: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 102 B/s wr, 5 op/s 2024-05-04T03:26:20.852 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:20 smithi001 ceph-mon[29731]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-04T03:26:20.852 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:20 smithi001 ceph-mon[29731]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-04T03:26:20.852 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:20 smithi001 ceph-mon[29731]: Cluster is now healthy 2024-05-04T03:26:20.852 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:20 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/954068712' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-04T03:26:20.852 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:20 smithi001 ceph-mon[29731]: fsmap 12 up:standby 2024-05-04T03:26:20.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:20 smithi077 ceph-mon[33531]: pgmap v607: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 102 B/s wr, 5 op/s 2024-05-04T03:26:20.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:20 smithi077 ceph-mon[33531]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-04T03:26:20.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:20 smithi077 ceph-mon[33531]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-04T03:26:20.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:20 smithi077 ceph-mon[33531]: Cluster is now healthy 2024-05-04T03:26:20.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:20 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/954068712' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-04T03:26:20.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:20 smithi077 ceph-mon[33531]: fsmap 12 up:standby 2024-05-04T03:26:20.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:20 smithi072 ceph-mon[33332]: pgmap v607: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 102 B/s wr, 5 op/s 2024-05-04T03:26:20.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:20 smithi072 ceph-mon[33332]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-04T03:26:20.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:20 smithi072 ceph-mon[33332]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-04T03:26:20.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:20 smithi072 ceph-mon[33332]: Cluster is now healthy 2024-05-04T03:26:20.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:20 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/954068712' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-04T03:26:20.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:20 smithi072 ceph-mon[33332]: fsmap 12 up:standby 2024-05-04T03:26:22.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.063+0000 7f19d00e8700 1 -- 172.21.15.1:0/4941338 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19c81034e0 msgr2=0x7f19c81038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:22.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.063+0000 7f19d00e8700 1 --2- 172.21.15.1:0/4941338 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19c81034e0 0x7f19c81038f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f19c40099c0 tx=0x7f19c4009cd0 comp rx=0 tx=0).stop 2024-05-04T03:26:22.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.063+0000 7f19d00e8700 1 -- 172.21.15.1:0/4941338 shutdown_connections 2024-05-04T03:26:22.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.063+0000 7f19d00e8700 1 --2- 172.21.15.1:0/4941338 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f19c8108760 0x7f19c81100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:22.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.063+0000 7f19d00e8700 1 --2- 172.21.15.1:0/4941338 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19c81034e0 0x7f19c81038f0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:22.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.063+0000 7f19d00e8700 1 --2- 172.21.15.1:0/4941338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19c8107e10 0x7f19c8108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:22.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.063+0000 7f19d00e8700 1 -- 172.21.15.1:0/4941338 >> 172.21.15.1:0/4941338 conn(0x7f19c80fef30 msgr2=0x7f19c8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:22.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.063+0000 7f19d00e8700 1 -- 172.21.15.1:0/4941338 shutdown_connections 2024-05-04T03:26:22.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.063+0000 7f19d00e8700 1 -- 172.21.15.1:0/4941338 wait complete. 2024-05-04T03:26:22.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.064+0000 7f19d00e8700 1 Processor -- start 2024-05-04T03:26:22.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.064+0000 7f19d00e8700 1 -- start start 2024-05-04T03:26:22.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.064+0000 7f19d00e8700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19c81034e0 0x7f19c819d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:22.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.064+0000 7f19d00e8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19c8107e10 0x7f19c819d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:22.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.064+0000 7f19d00e8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f19c8108760 0x7f19c81a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:22.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.064+0000 7f19d00e8700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f19c8113390 con 0x7f19c81034e0 2024-05-04T03:26:22.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.064+0000 7f19d00e8700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f19c8113210 con 0x7f19c8108760 2024-05-04T03:26:22.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.064+0000 7f19d00e8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f19c8113510 con 0x7f19c8107e10 2024-05-04T03:26:22.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.064+0000 7f19cde84700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19c81034e0 0x7f19c819d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:22.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.064+0000 7f19cde84700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19c81034e0 0x7f19c819d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36796/0 (socket says 172.21.15.1:36796) 2024-05-04T03:26:22.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.064+0000 7f19cde84700 1 -- 172.21.15.1:0/3153999307 learned_addr learned my addr 172.21.15.1:0/3153999307 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:26:22.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.065+0000 7f19cd683700 1 --2- 172.21.15.1:0/3153999307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19c8107e10 0x7f19c819d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:22.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.065+0000 7f19ce685700 1 --2- 172.21.15.1:0/3153999307 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f19c8108760 0x7f19c81a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:22.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.065+0000 7f19cde84700 1 -- 172.21.15.1:0/3153999307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19c8107e10 msgr2=0x7f19c819d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:22.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.065+0000 7f19cde84700 1 --2- 172.21.15.1:0/3153999307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19c8107e10 0x7f19c819d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:22.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.065+0000 7f19cde84700 1 -- 172.21.15.1:0/3153999307 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f19c8108760 msgr2=0x7f19c81a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:22.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.065+0000 7f19cde84700 1 --2- 172.21.15.1:0/3153999307 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f19c8108760 0x7f19c81a1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:22.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.065+0000 7f19cde84700 1 -- 172.21.15.1:0/3153999307 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c81a1f00 con 0x7f19c81034e0 2024-05-04T03:26:22.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.065+0000 7f19cde84700 1 --2- 172.21.15.1:0/3153999307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19c81034e0 0x7f19c819d030 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f19c4006010 tx=0x7f19c40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:22.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.065+0000 7f19beffd700 1 -- 172.21.15.1:0/3153999307 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c4026070 con 0x7f19c81034e0 2024-05-04T03:26:22.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.065+0000 7f19d00e8700 1 -- 172.21.15.1:0/3153999307 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19c81a2190 con 0x7f19c81034e0 2024-05-04T03:26:22.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.065+0000 7f19beffd700 1 -- 172.21.15.1:0/3153999307 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19c400f460 con 0x7f19c81034e0 2024-05-04T03:26:22.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.065+0000 7f19beffd700 1 -- 172.21.15.1:0/3153999307 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c4009e60 con 0x7f19c81034e0 2024-05-04T03:26:22.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.066+0000 7f19d00e8700 1 -- 172.21.15.1:0/3153999307 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19c81a2670 con 0x7f19c81034e0 2024-05-04T03:26:22.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.067+0000 7f19d00e8700 1 -- 172.21.15.1:0/3153999307 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19c80512c0 con 0x7f19c81034e0 2024-05-04T03:26:22.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.068+0000 7f19beffd700 1 -- 172.21.15.1:0/3153999307 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f19c4029070 con 0x7f19c81034e0 2024-05-04T03:26:22.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.069+0000 7f19beffd700 1 --2- 172.21.15.1:0/3153999307 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f19b40b2880 0x7f19b40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:22.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.069+0000 7f19cd683700 1 --2- 172.21.15.1:0/3153999307 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f19b40b2880 0x7f19b40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:22.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.069+0000 7f19beffd700 1 -- 172.21.15.1:0/3153999307 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8382+0+0 (secure 0 0 0) 0x7f19c40e2e30 con 0x7f19c81034e0 2024-05-04T03:26:22.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.070+0000 7f19cd683700 1 --2- 172.21.15.1:0/3153999307 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f19b40b2880 0x7f19b40b4d90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f19c806cdf0 tx=0x7f19b800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:22.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.077+0000 7f19beffd700 1 -- 172.21.15.1:0/3153999307 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f19c405edf0 con 0x7f19c81034e0 2024-05-04T03:26:22.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.295+0000 7f19d00e8700 1 -- 172.21.15.1:0/3153999307 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f19c802e5f0 con 0x7f19c81034e0 2024-05-04T03:26:22.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.579+0000 7f19beffd700 1 -- 172.21.15.1:0/3153999307 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v85) ==== 181+0+0 (secure 0 0 0) 0x7f19c4013070 con 0x7f19c81034e0 2024-05-04T03:26:22.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.582+0000 7f19d00e8700 1 -- 172.21.15.1:0/3153999307 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f19b40b2880 msgr2=0x7f19b40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:22.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.582+0000 7f19d00e8700 1 --2- 172.21.15.1:0/3153999307 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f19b40b2880 0x7f19b40b4d90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f19c806cdf0 tx=0x7f19b800b410 comp rx=0 tx=0).stop 2024-05-04T03:26:22.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.582+0000 7f19d00e8700 1 -- 172.21.15.1:0/3153999307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19c81034e0 msgr2=0x7f19c819d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:22.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.582+0000 7f19d00e8700 1 --2- 172.21.15.1:0/3153999307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19c81034e0 0x7f19c819d030 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f19c4006010 tx=0x7f19c40045e0 comp rx=0 tx=0).stop 2024-05-04T03:26:22.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.582+0000 7f19d00e8700 1 -- 172.21.15.1:0/3153999307 shutdown_connections 2024-05-04T03:26:22.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.582+0000 7f19d00e8700 1 --2- 172.21.15.1:0/3153999307 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f19c8108760 0x7f19c81a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:22.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.582+0000 7f19d00e8700 1 --2- 172.21.15.1:0/3153999307 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f19b40b2880 0x7f19b40b4d90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:22.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.582+0000 7f19d00e8700 1 --2- 172.21.15.1:0/3153999307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19c81034e0 0x7f19c819d030 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:22.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.582+0000 7f19d00e8700 1 --2- 172.21.15.1:0/3153999307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19c8107e10 0x7f19c819d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:22.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.582+0000 7f19d00e8700 1 -- 172.21.15.1:0/3153999307 >> 172.21.15.1:0/3153999307 conn(0x7f19c80fef30 msgr2=0x7f19c81073f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:22.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.583+0000 7f19d00e8700 1 -- 172.21.15.1:0/3153999307 shutdown_connections 2024-05-04T03:26:22.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:22.583+0000 7f19d00e8700 1 -- 172.21.15.1:0/3153999307 wait complete. 2024-05-04T03:26:22.586 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' removed 2024-05-04T03:26:22.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:22 smithi001 ceph-mon[29731]: pgmap v608: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-04T03:26:22.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:22 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3153999307' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-04T03:26:22.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:22 smithi077 ceph-mon[33531]: pgmap v608: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-04T03:26:22.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:22 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/3153999307' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-04T03:26:22.996 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:22 smithi072 ceph-mon[33332]: pgmap v608: 97 pgs: 97 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-04T03:26:22.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:22 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/3153999307' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-04T03:26:23.222 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31 shell --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-04T03:26:23.484 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/mon.a/config 2024-05-04T03:26:23.848 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:23 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/3153999307' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-04T03:26:23.848 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:23 smithi001 ceph-mon[29731]: osdmap e85: 12 total, 12 up, 12 in 2024-05-04T03:26:23.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:23 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/3153999307' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-04T03:26:23.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:23 smithi077 ceph-mon[33531]: osdmap e85: 12 total, 12 up, 12 in 2024-05-04T03:26:23.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:23 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/3153999307' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-04T03:26:23.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:23 smithi072 ceph-mon[33332]: osdmap e85: 12 total, 12 up, 12 in 2024-05-04T03:26:24.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:24 smithi001 ceph-mon[29731]: pgmap v610: 65 pgs: 65 active+clean; 577 KiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:26:24.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:24 smithi077 ceph-mon[33531]: pgmap v610: 65 pgs: 65 active+clean; 577 KiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:26:24.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:24 smithi072 ceph-mon[33332]: pgmap v610: 65 pgs: 65 active+clean; 577 KiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:26:25.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.161+0000 7f0d5177d700 1 -- 172.21.15.1:0/2844972496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d4c107df0 msgr2=0x7f0d4c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:25.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.161+0000 7f0d5177d700 1 --2- 172.21.15.1:0/2844972496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d4c107df0 0x7f0d4c108200 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0d400099c0 tx=0x7f0d40009cd0 comp rx=0 tx=0).stop 2024-05-04T03:26:25.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.161+0000 7f0d5177d700 1 -- 172.21.15.1:0/2844972496 shutdown_connections 2024-05-04T03:26:25.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.161+0000 7f0d5177d700 1 --2- 172.21.15.1:0/2844972496 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0d4c1034c0 0x7f0d4c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:25.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.161+0000 7f0d5177d700 1 --2- 172.21.15.1:0/2844972496 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d4c108740 0x7f0d4c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:25.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.161+0000 7f0d5177d700 1 --2- 172.21.15.1:0/2844972496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d4c107df0 0x7f0d4c108200 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:25.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.161+0000 7f0d5177d700 1 -- 172.21.15.1:0/2844972496 >> 172.21.15.1:0/2844972496 conn(0x7f0d4c0fedf0 msgr2=0x7f0d4c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:25.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.161+0000 7f0d5177d700 1 -- 172.21.15.1:0/2844972496 shutdown_connections 2024-05-04T03:26:25.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.161+0000 7f0d5177d700 1 -- 172.21.15.1:0/2844972496 wait complete. 2024-05-04T03:26:25.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.162+0000 7f0d5177d700 1 Processor -- start 2024-05-04T03:26:25.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.162+0000 7f0d5177d700 1 -- start start 2024-05-04T03:26:25.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.162+0000 7f0d5177d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0d4c1034c0 0x7f0d4c19bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:25.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.162+0000 7f0d5177d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d4c107df0 0x7f0d4c19c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:25.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.162+0000 7f0d5177d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d4c108740 0x7f0d4c1a6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:25.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.162+0000 7f0d5177d700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c113370 con 0x7f0d4c107df0 2024-05-04T03:26:25.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.162+0000 7f0d5177d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c1131f0 con 0x7f0d4c1034c0 2024-05-04T03:26:25.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.162+0000 7f0d5177d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c1134f0 con 0x7f0d4c108740 2024-05-04T03:26:25.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.163+0000 7f0d4a7fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d4c107df0 0x7f0d4c19c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:25.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.163+0000 7f0d4a7fc700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d4c107df0 0x7f0d4c19c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36820/0 (socket says 172.21.15.1:36820) 2024-05-04T03:26:25.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.163+0000 7f0d4a7fc700 1 -- 172.21.15.1:0/2500592962 learned_addr learned my addr 172.21.15.1:0/2500592962 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-05-04T03:26:25.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.163+0000 7f0d4b7fe700 1 --2- 172.21.15.1:0/2500592962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d4c108740 0x7f0d4c1a6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:25.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.163+0000 7f0d4affd700 1 --2- 172.21.15.1:0/2500592962 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0d4c1034c0 0x7f0d4c19bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:25.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.163+0000 7f0d4a7fc700 1 -- 172.21.15.1:0/2500592962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d4c108740 msgr2=0x7f0d4c1a6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:25.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.163+0000 7f0d4a7fc700 1 --2- 172.21.15.1:0/2500592962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d4c108740 0x7f0d4c1a6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:25.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.163+0000 7f0d4a7fc700 1 -- 172.21.15.1:0/2500592962 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0d4c1034c0 msgr2=0x7f0d4c19bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:25.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.163+0000 7f0d4a7fc700 1 --2- 172.21.15.1:0/2500592962 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0d4c1034c0 0x7f0d4c19bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:25.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.163+0000 7f0d4a7fc700 1 -- 172.21.15.1:0/2500592962 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4c19ca20 con 0x7f0d4c107df0 2024-05-04T03:26:25.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.163+0000 7f0d4a7fc700 1 --2- 172.21.15.1:0/2500592962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d4c107df0 0x7f0d4c19c170 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0d40006010 tx=0x7f0d400045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:25.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.163+0000 7f0d33fff700 1 -- 172.21.15.1:0/2500592962 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40026070 con 0x7f0d4c107df0 2024-05-04T03:26:25.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.163+0000 7f0d5177d700 1 -- 172.21.15.1:0/2500592962 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4c1a72b0 con 0x7f0d4c107df0 2024-05-04T03:26:25.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.163+0000 7f0d33fff700 1 -- 172.21.15.1:0/2500592962 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d4000f460 con 0x7f0d4c107df0 2024-05-04T03:26:25.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.163+0000 7f0d33fff700 1 -- 172.21.15.1:0/2500592962 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40009e60 con 0x7f0d4c107df0 2024-05-04T03:26:25.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.163+0000 7f0d5177d700 1 -- 172.21.15.1:0/2500592962 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d4c1a7700 con 0x7f0d4c107df0 2024-05-04T03:26:25.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.165+0000 7f0d5177d700 1 -- 172.21.15.1:0/2500592962 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d4c0512c0 con 0x7f0d4c107df0 2024-05-04T03:26:25.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.166+0000 7f0d33fff700 1 -- 172.21.15.1:0/2500592962 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 20) ==== 149651+0+0 (secure 0 0 0) 0x7f0d40029070 con 0x7f0d4c107df0 2024-05-04T03:26:25.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.167+0000 7f0d33fff700 1 --2- 172.21.15.1:0/2500592962 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0d340b2880 0x7f0d340b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-04T03:26:25.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.167+0000 7f0d4affd700 1 --2- 172.21.15.1:0/2500592962 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0d340b2880 0x7f0d340b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-04T03:26:25.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.167+0000 7f0d33fff700 1 -- 172.21.15.1:0/2500592962 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7886+0+0 (secure 0 0 0) 0x7f0d400e2880 con 0x7f0d4c107df0 2024-05-04T03:26:25.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.167+0000 7f0d4affd700 1 --2- 172.21.15.1:0/2500592962 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0d340b2880 0x7f0d340b4d90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009c60 tx=0x7f0d3c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-04T03:26:25.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.172+0000 7f0d33fff700 1 -- 172.21.15.1:0/2500592962 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0d4005edf0 con 0x7f0d4c107df0 2024-05-04T03:26:25.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.384+0000 7f0d5177d700 1 -- 172.21.15.1:0/2500592962 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0d4c02e7b0 con 0x7f0d4c107df0 2024-05-04T03:26:25.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.591+0000 7f0d33fff700 1 -- 172.21.15.1:0/2500592962 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v86) ==== 169+0+0 (secure 0 0 0) 0x7f0d40013070 con 0x7f0d4c107df0 2024-05-04T03:26:25.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.595+0000 7f0d5177d700 1 -- 172.21.15.1:0/2500592962 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0d340b2880 msgr2=0x7f0d340b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:25.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.595+0000 7f0d5177d700 1 --2- 172.21.15.1:0/2500592962 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0d340b2880 0x7f0d340b4d90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009c60 tx=0x7f0d3c009260 comp rx=0 tx=0).stop 2024-05-04T03:26:25.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.595+0000 7f0d5177d700 1 -- 172.21.15.1:0/2500592962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d4c107df0 msgr2=0x7f0d4c19c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-04T03:26:25.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.595+0000 7f0d5177d700 1 --2- 172.21.15.1:0/2500592962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d4c107df0 0x7f0d4c19c170 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0d40006010 tx=0x7f0d400045e0 comp rx=0 tx=0).stop 2024-05-04T03:26:25.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.595+0000 7f0d5177d700 1 -- 172.21.15.1:0/2500592962 shutdown_connections 2024-05-04T03:26:25.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.595+0000 7f0d5177d700 1 --2- 172.21.15.1:0/2500592962 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0d4c1034c0 0x7f0d4c19bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:25.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.595+0000 7f0d5177d700 1 --2- 172.21.15.1:0/2500592962 >> [v2:172.21.15.1:6800/402647351,v1:172.21.15.1:6801/402647351] conn(0x7f0d340b2880 0x7f0d340b4d90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:25.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.595+0000 7f0d5177d700 1 --2- 172.21.15.1:0/2500592962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0d4c108740 0x7f0d4c1a6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-04T03:26:25.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.595+0000 7f0d5177d700 1 --2- 172.21.15.1:0/2500592962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d4c107df0 0x7f0d4c19c170 secure :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0d40006010 tx=0x7f0d400045e0 comp rx=0 tx=0).stop 2024-05-04T03:26:25.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.595+0000 7f0d5177d700 1 -- 172.21.15.1:0/2500592962 >> 172.21.15.1:0/2500592962 conn(0x7f0d4c0fedf0 msgr2=0x7f0d4c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-04T03:26:25.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.597+0000 7f0d5177d700 1 -- 172.21.15.1:0/2500592962 shutdown_connections 2024-05-04T03:26:25.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-05-04T03:26:25.597+0000 7f0d5177d700 1 -- 172.21.15.1:0/2500592962 wait complete. 2024-05-04T03:26:25.601 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' removed 2024-05-04T03:26:25.880 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:25 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2500592962' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-04T03:26:25.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:25 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/2500592962' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-04T03:26:25.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:25 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/2500592962' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-04T03:26:26.362 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-04T03:26:26.392 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-04T03:26:26.429 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-04T03:26:26.461 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-04T03:26:26.461 DEBUG:teuthology.orchestra.run.smithi001:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-04T03:26:26.479 DEBUG:teuthology.orchestra.run.smithi072:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-04T03:26:26.494 DEBUG:teuthology.orchestra.run.smithi077:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-04T03:26:26.517 INFO:tasks.cephadm:Stopping all daemons... 2024-05-04T03:26:26.518 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-04T03:26:26.518 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mon.a 2024-05-04T03:26:26.596 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:26 smithi001 systemd[1]: Stopping Ceph mon.a for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:26:26.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:26 smithi072 ceph-mon[33332]: pgmap v611: 65 pgs: 65 active+clean; 577 KiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:26:26.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:26 smithi072 ceph-mon[33332]: from='client.? 172.21.15.1:0/2500592962' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-04T03:26:26.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:26 smithi072 ceph-mon[33332]: osdmap e86: 12 total, 12 up, 12 in 2024-05-04T03:26:26.854 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:26 smithi001 ceph-mon[29731]: pgmap v611: 65 pgs: 65 active+clean; 577 KiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:26:26.854 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:26 smithi001 ceph-mon[29731]: from='client.? 172.21.15.1:0/2500592962' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-04T03:26:26.854 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:26 smithi001 ceph-mon[29731]: osdmap e86: 12 total, 12 up, 12 in 2024-05-04T03:26:26.854 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:26 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mon-a[29727]: 2024-05-04T03:26:26.820+0000 7fa95b3e8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:26:26.854 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:26 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mon-a[29727]: 2024-05-04T03:26:26.820+0000 7fa95b3e8700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-04T03:26:26.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:26 smithi077 ceph-mon[33531]: pgmap v611: 65 pgs: 65 active+clean; 577 KiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-04T03:26:26.928 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:26 smithi077 ceph-mon[33531]: from='client.? 172.21.15.1:0/2500592962' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-04T03:26:26.929 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:26 smithi077 ceph-mon[33531]: osdmap e86: 12 total, 12 up, 12 in 2024-05-04T03:26:27.123 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:26 smithi001 podman[112345]: 2024-05-04 03:26:26.852609643 +0000 UTC m=+0.140007317 container died f52e77ebe41e330bd75fe7f4e8036b1d7589c717670b79ca11530e843113e826 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mon-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-04T03:26:27.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:27 smithi001 podman[112345]: 2024-05-04 03:26:27.212349786 +0000 UTC m=+0.499747465 container cleanup f52e77ebe41e330bd75fe7f4e8036b1d7589c717670b79ca11530e843113e826 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-04T03:26:27.380 INFO:journalctl@ceph.mon.a.smithi001.stdout:May 04 03:26:27 smithi001 bash[112345]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mon-a 2024-05-04T03:26:27.658 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mon.a.service' 2024-05-04T03:26:27.716 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:26:27.716 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-04T03:26:27.716 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-04T03:26:27.716 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mon.b 2024-05-04T03:26:27.997 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:27 smithi072 systemd[1]: Stopping Ceph mon.b for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:26:28.442 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:28 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mon-b[33328]: 2024-05-04T03:26:28.028+0000 7f6c36783700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:26:28.442 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:28 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mon-b[33328]: 2024-05-04T03:26:28.029+0000 7f6c36783700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-04T03:26:28.442 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:28 smithi072 podman[67169]: 2024-05-04 03:26:28.152004306 +0000 UTC m=+0.237574087 container died 6bd75c9906c11351c7dc5f277a39c295870634082a7bc8dd70ca98d5b86e571b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mon-b, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:26:28.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:28 smithi072 podman[67169]: 2024-05-04 03:26:28.52483287 +0000 UTC m=+0.610402649 container cleanup 6bd75c9906c11351c7dc5f277a39c295870634082a7bc8dd70ca98d5b86e571b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-05-04T03:26:28.747 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 04 03:26:28 smithi072 bash[67169]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mon-b 2024-05-04T03:26:28.950 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mon.b.service' 2024-05-04T03:26:28.992 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:26:28.993 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-04T03:26:28.993 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-04T03:26:28.993 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mon.c 2024-05-04T03:26:29.314 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:29 smithi077 systemd[1]: Stopping Ceph mon.c for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:26:29.580 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:29 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mon-c[33525]: 2024-05-04T03:26:29.311+0000 7f23dce46700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:26:29.580 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:29 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mon-c[33525]: 2024-05-04T03:26:29.311+0000 7f23dce46700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-04T03:26:29.856 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:29 smithi077 podman[67894]: 2024-05-04 03:26:29.578527586 +0000 UTC m=+0.384154778 container died 2f48466bb79029e629d70e6e44bc66ee08540152011ced173d82717d37e56e14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mon-c, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-04T03:26:30.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:29 smithi077 podman[67894]: 2024-05-04 03:26:29.945971264 +0000 UTC m=+0.751598445 container cleanup 2f48466bb79029e629d70e6e44bc66ee08540152011ced173d82717d37e56e14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mon-c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS) 2024-05-04T03:26:30.178 INFO:journalctl@ceph.mon.c.smithi077.stdout:May 04 03:26:29 smithi077 bash[67894]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mon-c 2024-05-04T03:26:30.425 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mon.c.service' 2024-05-04T03:26:30.475 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:26:30.475 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-04T03:26:30.475 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-04T03:26:30.475 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mgr.x 2024-05-04T03:26:30.880 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:26:30 smithi001 systemd[1]: Stopping Ceph mgr.x for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:26:31.211 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:26:30 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x[29965]: 2024-05-04T03:26:30.899+0000 7f405d9ca700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:26:31.211 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:26:30 smithi001 podman[112465]: 2024-05-04 03:26:30.930738387 +0000 UTC m=+0.256963593 container died 5918334689e48eb373233fd1754c9eb2f79e4cb4d013f157ca7aa3272a080a18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-04T03:26:31.629 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:26:31 smithi001 podman[112465]: 2024-05-04 03:26:31.416598245 +0000 UTC m=+0.742823453 container cleanup 5918334689e48eb373233fd1754c9eb2f79e4cb4d013f157ca7aa3272a080a18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bd4e00a39d766328b458e4c953f4d722fed96a31, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-04T03:26:31.630 INFO:journalctl@ceph.mgr.x.smithi001.stdout:May 04 03:26:31 smithi001 bash[112465]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-x 2024-05-04T03:26:31.760 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mgr.x.service' 2024-05-04T03:26:31.794 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:26:31.795 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-04T03:26:31.795 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-04T03:26:31.795 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mgr.y 2024-05-04T03:26:32.121 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:26:31 smithi072 systemd[1]: Stopping Ceph mgr.y for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:26:32.404 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:26:32 smithi072 podman[67290]: 2024-05-04 03:26:32.119301289 +0000 UTC m=+0.124510845 container died d698f7723d6eb9b26e068da6cf3fbf4d25f3eeeab09509b2b4d053f745633356 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, ceph=True, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-05-04T03:26:32.746 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:26:32 smithi072 podman[67290]: 2024-05-04 03:26:32.494224461 +0000 UTC m=+0.499434024 container cleanup d698f7723d6eb9b26e068da6cf3fbf4d25f3eeeab09509b2b4d053f745633356 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-04T03:26:32.747 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 04 03:26:32 smithi072 bash[67290]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-y 2024-05-04T03:26:32.920 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mgr.y.service' 2024-05-04T03:26:32.955 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:26:32.955 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-04T03:26:32.955 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-04T03:26:32.955 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mgr.z 2024-05-04T03:26:33.298 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:26:33 smithi077 systemd[1]: Stopping Ceph mgr.z for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:26:33.595 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:26:33 smithi077 podman[68016]: 2024-05-04 03:26:33.297217732 +0000 UTC m=+0.145430831 container died 70f83db8ce86929d8bb99b7616ae62b55e9bccee110480fb28c2f80df496aee4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-04T03:26:33.912 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:26:33 smithi077 podman[68016]: 2024-05-04 03:26:33.685971707 +0000 UTC m=+0.534184805 container cleanup 70f83db8ce86929d8bb99b7616ae62b55e9bccee110480fb28c2f80df496aee4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:26:33.913 INFO:journalctl@ceph.mgr.z.smithi077.stdout:May 04 03:26:33 smithi077 bash[68016]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mgr-z 2024-05-04T03:26:34.031 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mgr.z.service' 2024-05-04T03:26:34.064 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:26:34.064 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-04T03:26:34.064 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-04T03:26:34.064 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.0 2024-05-04T03:26:34.347 INFO:journalctl@ceph.osd.0.smithi001.stdout:May 04 03:26:34 smithi001 systemd[1]: Stopping Ceph osd.0 for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:26:34.629 INFO:journalctl@ceph.osd.0.smithi001.stdout:May 04 03:26:34 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-0[41843]: 2024-05-04T03:26:34.344+0000 7f93e5969700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:26:34.629 INFO:journalctl@ceph.osd.0.smithi001.stdout:May 04 03:26:34 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-0[41843]: 2024-05-04T03:26:34.344+0000 7f93e5969700 -1 osd.0 86 *** Got signal Terminated *** 2024-05-04T03:26:34.630 INFO:journalctl@ceph.osd.0.smithi001.stdout:May 04 03:26:34 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-0[41843]: 2024-05-04T03:26:34.344+0000 7f93e5969700 -1 osd.0 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-04T03:26:39.991 INFO:journalctl@ceph.osd.0.smithi001.stdout:May 04 03:26:39 smithi001 podman[112587]: 2024-05-04 03:26:39.696705405 +0000 UTC m=+5.433395413 container died 0fbc01c85d8b5f5503c2bb3a7fba84a510a6cea47047fbc7d992b3ac8ed0ab07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-04T03:26:40.322 INFO:journalctl@ceph.osd.0.smithi001.stdout:May 04 03:26:40 smithi001 podman[112587]: 2024-05-04 03:26:40.087041562 +0000 UTC m=+5.823731559 container cleanup 0fbc01c85d8b5f5503c2bb3a7fba84a510a6cea47047fbc7d992b3ac8ed0ab07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-04T03:26:40.323 INFO:journalctl@ceph.osd.0.smithi001.stdout:May 04 03:26:40 smithi001 bash[112587]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-0 2024-05-04T03:26:40.629 INFO:journalctl@ceph.osd.0.smithi001.stdout:May 04 03:26:40 smithi001 podman[112605]: 2024-05-04 03:26:40.321313387 +0000 UTC m=+0.617731850 container remove 0fbc01c85d8b5f5503c2bb3a7fba84a510a6cea47047fbc7d992b3ac8ed0ab07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-0, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5) 2024-05-04T03:26:40.968 INFO:journalctl@ceph.osd.0.smithi001.stdout:May 04 03:26:40 smithi001 podman[112698]: 2024-05-04 03:26:40.565760928 +0000 UTC m=+0.017966046 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:26:40.968 INFO:journalctl@ceph.osd.0.smithi001.stdout:May 04 03:26:40 smithi001 podman[112698]: 2024-05-04 03:26:40.68079511 +0000 UTC m=+0.133000270 container create 4a695dd4fb05d45ef6fd1e4ce9449e4769743bd07f676540f492122b7738b25a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-04T03:26:40.968 INFO:journalctl@ceph.osd.0.smithi001.stdout:May 04 03:26:40 smithi001 podman[112698]: 2024-05-04 03:26:40.966829994 +0000 UTC m=+0.419035112 container init 4a695dd4fb05d45ef6fd1e4ce9449e4769743bd07f676540f492122b7738b25a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-04T03:26:41.289 INFO:journalctl@ceph.osd.0.smithi001.stdout:May 04 03:26:40 smithi001 podman[112698]: 2024-05-04 03:26:40.970473025 +0000 UTC m=+0.422678144 container start 4a695dd4fb05d45ef6fd1e4ce9449e4769743bd07f676540f492122b7738b25a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-0-deactivate, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-05-04T03:26:41.289 INFO:journalctl@ceph.osd.0.smithi001.stdout:May 04 03:26:41 smithi001 podman[112698]: 2024-05-04 03:26:41.066763775 +0000 UTC m=+0.518968918 container attach 4a695dd4fb05d45ef6fd1e4ce9449e4769743bd07f676540f492122b7738b25a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-05-04T03:26:41.580 INFO:journalctl@ceph.osd.0.smithi001.stdout:May 04 03:26:41 smithi001 podman[112698]: 2024-05-04 03:26:41.287343787 +0000 UTC m=+0.739548913 container died 4a695dd4fb05d45ef6fd1e4ce9449e4769743bd07f676540f492122b7738b25a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-04T03:26:41.872 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.0.service' 2024-05-04T03:26:41.912 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:26:41.912 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-04T03:26:41.912 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-04T03:26:41.912 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.1 2024-05-04T03:26:42.228 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 04 03:26:41 smithi072 systemd[1]: Stopping Ceph osd.1 for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:26:42.496 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 04 03:26:42 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-1[37925]: 2024-05-04T03:26:42.225+0000 7f609af27700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:26:42.497 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 04 03:26:42 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-1[37925]: 2024-05-04T03:26:42.225+0000 7f609af27700 -1 osd.1 86 *** Got signal Terminated *** 2024-05-04T03:26:42.497 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 04 03:26:42 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-1[37925]: 2024-05-04T03:26:42.225+0000 7f609af27700 -1 osd.1 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-04T03:26:47.941 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 04 03:26:47 smithi072 podman[67412]: 2024-05-04 03:26:47.667758592 +0000 UTC m=+5.564560985 container died 95e548501c954899b3a299bcf0d9c8b894912492233bded35501917d009f43d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430) 2024-05-04T03:26:48.247 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 04 03:26:48 smithi072 podman[67412]: 2024-05-04 03:26:48.056588441 +0000 UTC m=+5.953390824 container cleanup 95e548501c954899b3a299bcf0d9c8b894912492233bded35501917d009f43d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_BRANCH=HEAD) 2024-05-04T03:26:48.247 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 04 03:26:48 smithi072 bash[67412]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-1 2024-05-04T03:26:48.637 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 04 03:26:48 smithi072 podman[67431]: 2024-05-04 03:26:48.310730517 +0000 UTC m=+0.634876241 container remove 95e548501c954899b3a299bcf0d9c8b894912492233bded35501917d009f43d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-1, org.label-schema.build-date=20240430, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-04T03:26:48.901 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 04 03:26:48 smithi072 podman[67522]: 2024-05-04 03:26:48.538669353 +0000 UTC m=+0.017956459 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:26:48.901 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 04 03:26:48 smithi072 podman[67522]: 2024-05-04 03:26:48.670823169 +0000 UTC m=+0.150110300 container create c57e329798c953010601d8384d93faa442fa72e73686cacd0c05d4c2e0515643 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-1-deactivate, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-04T03:26:48.901 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 04 03:26:48 smithi072 podman[67522]: 2024-05-04 03:26:48.899864983 +0000 UTC m=+0.379152073 container init c57e329798c953010601d8384d93faa442fa72e73686cacd0c05d4c2e0515643 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-1-deactivate, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-04T03:26:49.202 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 04 03:26:48 smithi072 podman[67522]: 2024-05-04 03:26:48.902882234 +0000 UTC m=+0.382169326 container start c57e329798c953010601d8384d93faa442fa72e73686cacd0c05d4c2e0515643 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-04T03:26:49.203 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 04 03:26:49 smithi072 podman[67522]: 2024-05-04 03:26:49.000523129 +0000 UTC m=+0.479810233 container attach c57e329798c953010601d8384d93faa442fa72e73686cacd0c05d4c2e0515643 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430) 2024-05-04T03:26:49.481 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 04 03:26:49 smithi072 podman[67522]: 2024-05-04 03:26:49.202053956 +0000 UTC m=+0.681341060 container died c57e329798c953010601d8384d93faa442fa72e73686cacd0c05d4c2e0515643 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430) 2024-05-04T03:26:49.789 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.1.service' 2024-05-04T03:26:49.827 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:26:49.827 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-04T03:26:49.827 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-04T03:26:49.827 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.2 2024-05-04T03:26:50.116 INFO:journalctl@ceph.osd.2.smithi077.stdout:May 04 03:26:49 smithi077 systemd[1]: Stopping Ceph osd.2 for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:26:50.381 INFO:journalctl@ceph.osd.2.smithi077.stdout:May 04 03:26:50 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-2[38578]: 2024-05-04T03:26:50.113+0000 7f8b8dc20700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:26:50.381 INFO:journalctl@ceph.osd.2.smithi077.stdout:May 04 03:26:50 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-2[38578]: 2024-05-04T03:26:50.113+0000 7f8b8dc20700 -1 osd.2 86 *** Got signal Terminated *** 2024-05-04T03:26:50.381 INFO:journalctl@ceph.osd.2.smithi077.stdout:May 04 03:26:50 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-2[38578]: 2024-05-04T03:26:50.113+0000 7f8b8dc20700 -1 osd.2 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-04T03:26:55.929 INFO:journalctl@ceph.osd.2.smithi077.stdout:May 04 03:26:55 smithi077 podman[68139]: 2024-05-04 03:26:55.675138869 +0000 UTC m=+5.660434905 container died e3b02e68651d728a889c5cc45d14461196b16c5508dcac5d7b9faf1e89ce6b4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-2, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-04T03:26:56.333 INFO:journalctl@ceph.osd.2.smithi077.stdout:May 04 03:26:56 smithi077 podman[68139]: 2024-05-04 03:26:56.055901733 +0000 UTC m=+6.041197775 container cleanup e3b02e68651d728a889c5cc45d14461196b16c5508dcac5d7b9faf1e89ce6b4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-04T03:26:56.333 INFO:journalctl@ceph.osd.2.smithi077.stdout:May 04 03:26:56 smithi077 bash[68139]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-2 2024-05-04T03:26:56.678 INFO:journalctl@ceph.osd.2.smithi077.stdout:May 04 03:26:56 smithi077 podman[68168]: 2024-05-04 03:26:56.332000005 +0000 UTC m=+0.646715706 container remove e3b02e68651d728a889c5cc45d14461196b16c5508dcac5d7b9faf1e89ce6b4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-04T03:26:56.995 INFO:journalctl@ceph.osd.2.smithi077.stdout:May 04 03:26:56 smithi077 podman[68263]: 2024-05-04 03:26:56.581210734 +0000 UTC m=+0.018241428 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:26:56.995 INFO:journalctl@ceph.osd.2.smithi077.stdout:May 04 03:26:56 smithi077 podman[68263]: 2024-05-04 03:26:56.691376614 +0000 UTC m=+0.128407275 container create c5d05f340a8a883cb515991827ed88ec44682385d4b5b63b94a4e7dabee0b139 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-04T03:26:56.995 INFO:journalctl@ceph.osd.2.smithi077.stdout:May 04 03:26:56 smithi077 podman[68263]: 2024-05-04 03:26:56.919304607 +0000 UTC m=+0.356335278 container init c5d05f340a8a883cb515991827ed88ec44682385d4b5b63b94a4e7dabee0b139 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:26:56.995 INFO:journalctl@ceph.osd.2.smithi077.stdout:May 04 03:26:56 smithi077 podman[68263]: 2024-05-04 03:26:56.922765447 +0000 UTC m=+0.359796112 container start c5d05f340a8a883cb515991827ed88ec44682385d4b5b63b94a4e7dabee0b139 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-04T03:26:56.995 INFO:journalctl@ceph.osd.2.smithi077.stdout:May 04 03:26:56 smithi077 podman[68263]: 2024-05-04 03:26:56.993683032 +0000 UTC m=+0.430713706 container attach c5d05f340a8a883cb515991827ed88ec44682385d4b5b63b94a4e7dabee0b139 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True) 2024-05-04T03:26:57.428 INFO:journalctl@ceph.osd.2.smithi077.stdout:May 04 03:26:57 smithi077 podman[68263]: 2024-05-04 03:26:57.216020122 +0000 UTC m=+0.653050795 container died c5d05f340a8a883cb515991827ed88ec44682385d4b5b63b94a4e7dabee0b139 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-04T03:26:57.999 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.2.service' 2024-05-04T03:26:58.040 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:26:58.040 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-04T03:26:58.041 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-04T03:26:58.041 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.3 2024-05-04T03:26:58.367 INFO:journalctl@ceph.osd.3.smithi001.stdout:May 04 03:26:58 smithi001 systemd[1]: Stopping Ceph osd.3 for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:26:58.629 INFO:journalctl@ceph.osd.3.smithi001.stdout:May 04 03:26:58 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-3[48820]: 2024-05-04T03:26:58.363+0000 7f1a52f05700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:26:58.630 INFO:journalctl@ceph.osd.3.smithi001.stdout:May 04 03:26:58 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-3[48820]: 2024-05-04T03:26:58.363+0000 7f1a52f05700 -1 osd.3 86 *** Got signal Terminated *** 2024-05-04T03:26:58.630 INFO:journalctl@ceph.osd.3.smithi001.stdout:May 04 03:26:58 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-3[48820]: 2024-05-04T03:26:58.363+0000 7f1a52f05700 -1 osd.3 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-04T03:27:00.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:00 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:00.518+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:00.879 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:00 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:00.419+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:01.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:01 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:01.492+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:01.880 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:01 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:01.380+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:02.629 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:02 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:02.345+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:02.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:02 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:02.477+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:02.928 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:02 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:02.662+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:03.129 INFO:journalctl@ceph.osd.3.smithi001.stdout:May 04 03:27:02 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-3[48820]: 2024-05-04T03:27:02.727+0000 7f1a4b4b3700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:36.522404+0000 front 2024-05-04T03:26:36.522442+0000 (oldest deadline 2024-05-04T03:27:01.822545+0000) 2024-05-04T03:27:03.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:03 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:03.316+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:03.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:03 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:03.329+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.777831+0000 front 2024-05-04T03:26:37.777881+0000 (oldest deadline 2024-05-04T03:27:03.078064+0000) 2024-05-04T03:27:03.678 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:03 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5[45085]: 2024-05-04T03:27:03.345+0000 7f2bd5a31700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:38.778533+0000 front 2024-05-04T03:26:38.778367+0000 (oldest deadline 2024-05-04T03:27:02.878313+0000) 2024-05-04T03:27:03.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:03 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:03.460+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:04.130 INFO:journalctl@ceph.osd.3.smithi001.stdout:May 04 03:27:03 smithi001 podman[112832]: 2024-05-04 03:27:03.882411448 +0000 UTC m=+5.627401915 container died 994ea795d93284dd3ba4987ff3cce697c70d3587124f264094e52053b8b2cec9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-3, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-04T03:27:04.178 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:03 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:03.700+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:04.434 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:04 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:04.288+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.777831+0000 front 2024-05-04T03:26:37.777881+0000 (oldest deadline 2024-05-04T03:27:03.078064+0000) 2024-05-04T03:27:04.434 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:04 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:04.321+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:04.434 INFO:journalctl@ceph.osd.3.smithi001.stdout:May 04 03:27:04 smithi001 podman[112832]: 2024-05-04 03:27:04.222453324 +0000 UTC m=+5.967443790 container cleanup 994ea795d93284dd3ba4987ff3cce697c70d3587124f264094e52053b8b2cec9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-3, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-04T03:27:04.435 INFO:journalctl@ceph.osd.3.smithi001.stdout:May 04 03:27:04 smithi001 bash[112832]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-3 2024-05-04T03:27:04.653 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:04 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5[45085]: 2024-05-04T03:27:04.306+0000 7f2bd5a31700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:38.778533+0000 front 2024-05-04T03:26:38.778367+0000 (oldest deadline 2024-05-04T03:27:02.878313+0000) 2024-05-04T03:27:04.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:04 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:04.479+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:04.859 INFO:journalctl@ceph.osd.3.smithi001.stdout:May 04 03:27:04 smithi001 podman[112849]: 2024-05-04 03:27:04.432126571 +0000 UTC m=+0.546821942 container remove 994ea795d93284dd3ba4987ff3cce697c70d3587124f264094e52053b8b2cec9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-3, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-04T03:27:04.928 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:04 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:04.651+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:05.131 INFO:journalctl@ceph.osd.3.smithi001.stdout:May 04 03:27:04 smithi001 podman[112938]: 2024-05-04 03:27:04.858189356 +0000 UTC m=+0.113243097 container create 834c3cf61c1b869d76711b9f287c76b575c42f7429e3a64bf109a0f4ff8d4d5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-04T03:27:05.131 INFO:journalctl@ceph.osd.3.smithi001.stdout:May 04 03:27:04 smithi001 podman[112938]: 2024-05-04 03:27:04.76316054 +0000 UTC m=+0.018214282 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:27:05.131 INFO:journalctl@ceph.osd.3.smithi001.stdout:May 04 03:27:05 smithi001 podman[112938]: 2024-05-04 03:27:05.094459549 +0000 UTC m=+0.349513292 container init 834c3cf61c1b869d76711b9f287c76b575c42f7429e3a64bf109a0f4ff8d4d5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-04T03:27:05.131 INFO:journalctl@ceph.osd.3.smithi001.stdout:May 04 03:27:05 smithi001 podman[112938]: 2024-05-04 03:27:05.097867824 +0000 UTC m=+0.352921571 container start 834c3cf61c1b869d76711b9f287c76b575c42f7429e3a64bf109a0f4ff8d4d5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-05-04T03:27:05.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:05 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:05.317+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.777831+0000 front 2024-05-04T03:26:37.777881+0000 (oldest deadline 2024-05-04T03:27:03.078064+0000) 2024-05-04T03:27:05.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:05 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:05.300+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:05.630 INFO:journalctl@ceph.osd.3.smithi001.stdout:May 04 03:27:05 smithi001 podman[112938]: 2024-05-04 03:27:05.194429962 +0000 UTC m=+0.449483705 container attach 834c3cf61c1b869d76711b9f287c76b575c42f7429e3a64bf109a0f4ff8d4d5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-3-deactivate, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-04T03:27:05.630 INFO:journalctl@ceph.osd.3.smithi001.stdout:May 04 03:27:05 smithi001 podman[112938]: 2024-05-04 03:27:05.40686843 +0000 UTC m=+0.661922176 container died 834c3cf61c1b869d76711b9f287c76b575c42f7429e3a64bf109a0f4ff8d4d5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-04T03:27:05.647 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:05 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5[45085]: 2024-05-04T03:27:05.315+0000 7f2bd5a31700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:38.778533+0000 front 2024-05-04T03:26:38.778367+0000 (oldest deadline 2024-05-04T03:27:02.878313+0000) 2024-05-04T03:27:05.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:05 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:05.481+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:05.928 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:05 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:05.644+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:05.970 INFO:journalctl@ceph.osd.3.smithi001.stdout:May 04 03:27:05 smithi001 podman[112963]: 2024-05-04 03:27:05.947627007 +0000 UTC m=+0.531686658 container remove 834c3cf61c1b869d76711b9f287c76b575c42f7429e3a64bf109a0f4ff8d4d5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430) 2024-05-04T03:27:05.975 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.3.service' 2024-05-04T03:27:06.017 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:27:06.018 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-04T03:27:06.018 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-04T03:27:06.018 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.4 2024-05-04T03:27:06.324 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 04 03:27:06 smithi072 systemd[1]: Stopping Ceph osd.4 for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:27:06.616 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:06 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5[45085]: 2024-05-04T03:27:06.292+0000 7f2bd5a31700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:38.778533+0000 front 2024-05-04T03:26:38.778367+0000 (oldest deadline 2024-05-04T03:27:02.878313+0000) 2024-05-04T03:27:06.629 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:06 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:06.348+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.777831+0000 front 2024-05-04T03:26:37.777881+0000 (oldest deadline 2024-05-04T03:27:03.078064+0000) 2024-05-04T03:27:06.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:06 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:06.254+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:06.746 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:06 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:06.485+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:06.747 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 04 03:27:06 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-4[44447]: 2024-05-04T03:27:06.322+0000 7f3de55ed700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:27:06.747 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 04 03:27:06 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-4[44447]: 2024-05-04T03:27:06.322+0000 7f3de55ed700 -1 osd.4 86 *** Got signal Terminated *** 2024-05-04T03:27:06.747 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 04 03:27:06 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-4[44447]: 2024-05-04T03:27:06.322+0000 7f3de55ed700 -1 osd.4 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-04T03:27:06.928 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:06 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:06.613+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:07.596 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:07 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5[45085]: 2024-05-04T03:27:07.334+0000 7f2bd5a31700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:38.778533+0000 front 2024-05-04T03:26:38.778367+0000 (oldest deadline 2024-05-04T03:27:02.878313+0000) 2024-05-04T03:27:07.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:07 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:07.299+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.777831+0000 front 2024-05-04T03:26:37.777881+0000 (oldest deadline 2024-05-04T03:27:03.078064+0000) 2024-05-04T03:27:07.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:07 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:07.219+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:07.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:07 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:07.458+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:07.928 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:07 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:07.594+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:08.629 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:08 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:08.322+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.777831+0000 front 2024-05-04T03:26:37.777881+0000 (oldest deadline 2024-05-04T03:27:03.078064+0000) 2024-05-04T03:27:08.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:08 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:08.322+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.779163+0000 front 2024-05-04T03:26:44.779129+0000 (oldest deadline 2024-05-04T03:27:07.679296+0000) 2024-05-04T03:27:08.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:08 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:08.177+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:08.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:08 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:08.177+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:08.638 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:08 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5[45085]: 2024-05-04T03:27:08.347+0000 7f2bd5a31700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:38.778533+0000 front 2024-05-04T03:26:38.778367+0000 (oldest deadline 2024-05-04T03:27:02.878313+0000) 2024-05-04T03:27:08.928 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:08 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:08.864+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:08.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:08 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:08.636+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:08.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:08 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:08.497+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:08.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:08 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:08.497+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:09.629 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:09 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:09.342+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.777831+0000 front 2024-05-04T03:26:37.777881+0000 (oldest deadline 2024-05-04T03:27:03.078064+0000) 2024-05-04T03:27:09.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:09 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:09.342+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.779163+0000 front 2024-05-04T03:26:44.779129+0000 (oldest deadline 2024-05-04T03:27:07.679296+0000) 2024-05-04T03:27:09.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:09 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:09.189+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:09.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:09 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:09.189+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:09.664 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:09 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5[45085]: 2024-05-04T03:27:09.360+0000 7f2bd5a31700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:38.778533+0000 front 2024-05-04T03:26:38.778367+0000 (oldest deadline 2024-05-04T03:27:02.878313+0000) 2024-05-04T03:27:09.928 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:09 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:09.894+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:09.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:09 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:09.662+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:09.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:09 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:09.522+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:09.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:09 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:09.522+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:10.496 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:10 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:10.078+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.503707+0000 front 2024-05-04T03:26:43.503743+0000 (oldest deadline 2024-05-04T03:27:09.904588+0000) 2024-05-04T03:27:10.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:10 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:10.312+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.777831+0000 front 2024-05-04T03:26:37.777881+0000 (oldest deadline 2024-05-04T03:27:03.078064+0000) 2024-05-04T03:27:10.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:10 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:10.312+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.779163+0000 front 2024-05-04T03:26:44.779129+0000 (oldest deadline 2024-05-04T03:27:07.679296+0000) 2024-05-04T03:27:10.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:10 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:10.224+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:10.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:10 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:10.224+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:10.675 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:10 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5[45085]: 2024-05-04T03:27:10.358+0000 7f2bd5a31700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:38.778533+0000 front 2024-05-04T03:26:38.778367+0000 (oldest deadline 2024-05-04T03:27:02.878313+0000) 2024-05-04T03:27:10.928 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:10 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:10.673+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:10.997 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 04 03:27:10 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-4[44447]: 2024-05-04T03:27:10.695+0000 7f3de1ba3700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.321841+0000 front 2024-05-04T03:26:46.321873+0000 (oldest deadline 2024-05-04T03:27:10.322780+0000) 2024-05-04T03:27:10.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:10 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:10.569+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:10.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:10 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:10.569+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:11.393 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:10 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:10.927+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:11.496 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:11 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:11.128+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.503707+0000 front 2024-05-04T03:26:43.503743+0000 (oldest deadline 2024-05-04T03:27:09.904588+0000) 2024-05-04T03:27:11.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:11 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:11.355+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.777831+0000 front 2024-05-04T03:26:37.777881+0000 (oldest deadline 2024-05-04T03:27:03.078064+0000) 2024-05-04T03:27:11.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:11 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:11.355+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.779163+0000 front 2024-05-04T03:26:44.779129+0000 (oldest deadline 2024-05-04T03:27:07.679296+0000) 2024-05-04T03:27:11.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:11 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:11.181+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:11.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:11 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:11.181+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:11.664 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:11 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5[45085]: 2024-05-04T03:27:11.391+0000 7f2bd5a31700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:38.778533+0000 front 2024-05-04T03:26:38.778367+0000 (oldest deadline 2024-05-04T03:27:02.878313+0000) 2024-05-04T03:27:11.920 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 04 03:27:11 smithi072 podman[67640]: 2024-05-04 03:27:11.649040581 +0000 UTC m=+5.422937867 container died 492d1105c52e807c39db1966bd74e68fd1296c1cfaac66cf34d4353774579d04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:27:11.920 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:11 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:11.592+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:11.920 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:11 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:11.592+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:11.928 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:11 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:11.662+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:11.928 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:11 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:11.662+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:12.246 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:12 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:12.094+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.503707+0000 front 2024-05-04T03:26:43.503743+0000 (oldest deadline 2024-05-04T03:27:09.904588+0000) 2024-05-04T03:27:12.247 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 04 03:27:12 smithi072 podman[67640]: 2024-05-04 03:27:12.009016191 +0000 UTC m=+5.782913469 container cleanup 492d1105c52e807c39db1966bd74e68fd1296c1cfaac66cf34d4353774579d04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240430, io.buildah.version=1.33.5) 2024-05-04T03:27:12.247 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 04 03:27:12 smithi072 bash[67640]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-4 2024-05-04T03:27:12.417 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:11 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:11.928+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:12.549 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 04 03:27:12 smithi072 podman[67660]: 2024-05-04 03:27:12.288003681 +0000 UTC m=+0.630900260 container remove 492d1105c52e807c39db1966bd74e68fd1296c1cfaac66cf34d4353774579d04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-4, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:27:12.629 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:12 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:12.351+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.777831+0000 front 2024-05-04T03:26:37.777881+0000 (oldest deadline 2024-05-04T03:27:03.078064+0000) 2024-05-04T03:27:12.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:12 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:12.351+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.779163+0000 front 2024-05-04T03:26:44.779129+0000 (oldest deadline 2024-05-04T03:27:07.679296+0000) 2024-05-04T03:27:12.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:12 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:12.184+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:12.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:12 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:12.184+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:12.678 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:12 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5[45085]: 2024-05-04T03:27:12.414+0000 7f2bd5a31700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:38.778533+0000 front 2024-05-04T03:26:38.778367+0000 (oldest deadline 2024-05-04T03:27:02.878313+0000) 2024-05-04T03:27:12.678 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:12 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:12.652+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:12.679 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:12 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:12.652+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:12.879 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:12 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:12.546+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:12.879 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:12 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:12.546+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:12.879 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 04 03:27:12 smithi072 podman[67749]: 2024-05-04 03:27:12.523874874 +0000 UTC m=+0.017827615 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:27:12.879 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 04 03:27:12 smithi072 podman[67749]: 2024-05-04 03:27:12.640155776 +0000 UTC m=+0.134108502 container create 231c4c42a9024275d65220fcfbb7b18af3b3f9d06fe0f7f733cd8fbac4df18ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5) 2024-05-04T03:27:12.879 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 04 03:27:12 smithi072 podman[67749]: 2024-05-04 03:27:12.877474435 +0000 UTC m=+0.371427162 container init 231c4c42a9024275d65220fcfbb7b18af3b3f9d06fe0f7f733cd8fbac4df18ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-04T03:27:13.172 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:13 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:13.124+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.503707+0000 front 2024-05-04T03:26:43.503743+0000 (oldest deadline 2024-05-04T03:27:09.904588+0000) 2024-05-04T03:27:13.172 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 04 03:27:12 smithi072 podman[67749]: 2024-05-04 03:27:12.88085177 +0000 UTC m=+0.374804504 container start 231c4c42a9024275d65220fcfbb7b18af3b3f9d06fe0f7f733cd8fbac4df18ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-4-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-04T03:27:13.172 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 04 03:27:12 smithi072 podman[67749]: 2024-05-04 03:27:12.977994525 +0000 UTC m=+0.471947256 container attach 231c4c42a9024275d65220fcfbb7b18af3b3f9d06fe0f7f733cd8fbac4df18ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-04T03:27:13.428 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:12 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:12.945+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:13.468 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 04 03:27:13 smithi072 podman[67749]: 2024-05-04 03:27:13.170889444 +0000 UTC m=+0.664842172 container died 231c4c42a9024275d65220fcfbb7b18af3b3f9d06fe0f7f733cd8fbac4df18ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-04T03:27:13.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:13 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:13.390+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.777831+0000 front 2024-05-04T03:26:37.777881+0000 (oldest deadline 2024-05-04T03:27:03.078064+0000) 2024-05-04T03:27:13.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:13 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:13.391+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.779163+0000 front 2024-05-04T03:26:44.779129+0000 (oldest deadline 2024-05-04T03:27:07.679296+0000) 2024-05-04T03:27:13.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:13 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:13.217+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:13.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:13 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:13.217+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:13.698 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:13 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5[45085]: 2024-05-04T03:27:13.437+0000 7f2bd5a31700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:38.778533+0000 front 2024-05-04T03:26:38.778367+0000 (oldest deadline 2024-05-04T03:27:02.878313+0000) 2024-05-04T03:27:13.745 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:13 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:13.545+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:13.745 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:13 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:13.545+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:13.770 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.4.service' 2024-05-04T03:27:13.807 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:27:13.807 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-04T03:27:13.807 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-04T03:27:13.807 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.5 2024-05-04T03:27:14.094 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:13 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:13.933+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:14.094 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:13 smithi077 systemd[1]: Stopping Ceph osd.5 for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:27:14.094 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:14 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5[45085]: 2024-05-04T03:27:14.091+0000 7f2bdd483700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:27:14.094 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:14 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5[45085]: 2024-05-04T03:27:14.091+0000 7f2bdd483700 -1 osd.5 86 *** Got signal Terminated *** 2024-05-04T03:27:14.094 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:14 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5[45085]: 2024-05-04T03:27:14.091+0000 7f2bdd483700 -1 osd.5 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-04T03:27:14.095 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:13 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:13.696+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:14.095 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:13 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:13.696+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:14.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:14 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:14.155+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.503707+0000 front 2024-05-04T03:26:43.503743+0000 (oldest deadline 2024-05-04T03:27:09.904588+0000) 2024-05-04T03:27:14.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:14 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:14.395+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.777831+0000 front 2024-05-04T03:26:37.777881+0000 (oldest deadline 2024-05-04T03:27:03.078064+0000) 2024-05-04T03:27:14.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:14 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:14.395+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.779163+0000 front 2024-05-04T03:26:44.779129+0000 (oldest deadline 2024-05-04T03:27:07.679296+0000) 2024-05-04T03:27:14.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:14 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:14.170+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:14.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:14 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:14.170+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:14.678 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:14 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5[45085]: 2024-05-04T03:27:14.416+0000 7f2bd5a31700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:38.778533+0000 front 2024-05-04T03:26:38.778367+0000 (oldest deadline 2024-05-04T03:27:02.878313+0000) 2024-05-04T03:27:14.678 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:14 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5[45085]: 2024-05-04T03:27:14.416+0000 7f2bd5a31700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:42.878764+0000 front 2024-05-04T03:26:42.878826+0000 (oldest deadline 2024-05-04T03:27:13.479468+0000) 2024-05-04T03:27:14.979 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:14 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:14.686+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:14.979 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:14 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:14.686+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:14.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:14 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:14.582+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:14.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:14 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:14.582+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:15.428 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:14 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:14.976+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:15.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:15 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:15.113+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.503707+0000 front 2024-05-04T03:26:43.503743+0000 (oldest deadline 2024-05-04T03:27:09.904588+0000) 2024-05-04T03:27:15.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:15 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:15.378+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.777831+0000 front 2024-05-04T03:26:37.777881+0000 (oldest deadline 2024-05-04T03:27:03.078064+0000) 2024-05-04T03:27:15.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:15 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:15.378+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.779163+0000 front 2024-05-04T03:26:44.779129+0000 (oldest deadline 2024-05-04T03:27:07.679296+0000) 2024-05-04T03:27:15.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:15 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:15.220+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:15.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:15 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:15.220+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:15.928 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:15 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:15.649+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:15.928 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:15 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:15.649+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:15.929 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:15 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5[45085]: 2024-05-04T03:27:15.427+0000 7f2bd5a31700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:38.778533+0000 front 2024-05-04T03:26:38.778367+0000 (oldest deadline 2024-05-04T03:27:02.878313+0000) 2024-05-04T03:27:15.929 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:15 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5[45085]: 2024-05-04T03:27:15.427+0000 7f2bd5a31700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:42.878764+0000 front 2024-05-04T03:26:42.878826+0000 (oldest deadline 2024-05-04T03:27:13.479468+0000) 2024-05-04T03:27:15.996 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:15 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:15.580+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:15.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:15 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:15.580+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:16.385 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:15 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:15.988+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:16.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:16 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:16.076+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.503707+0000 front 2024-05-04T03:26:43.503743+0000 (oldest deadline 2024-05-04T03:27:09.904588+0000) 2024-05-04T03:27:16.629 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:16 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:16.402+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.777831+0000 front 2024-05-04T03:26:37.777881+0000 (oldest deadline 2024-05-04T03:27:03.078064+0000) 2024-05-04T03:27:16.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:16 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:16.402+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.779163+0000 front 2024-05-04T03:26:44.779129+0000 (oldest deadline 2024-05-04T03:27:07.679296+0000) 2024-05-04T03:27:16.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:16 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:16.221+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:16.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:16 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:16.221+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:16.636 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:16 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5[45085]: 2024-05-04T03:27:16.383+0000 7f2bd5a31700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:38.778533+0000 front 2024-05-04T03:26:38.778367+0000 (oldest deadline 2024-05-04T03:27:02.878313+0000) 2024-05-04T03:27:16.636 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:16 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5[45085]: 2024-05-04T03:27:16.383+0000 7f2bd5a31700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:42.878764+0000 front 2024-05-04T03:26:42.878826+0000 (oldest deadline 2024-05-04T03:27:13.479468+0000) 2024-05-04T03:27:16.928 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:16 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:16.634+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:16.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:16 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:16.634+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:16.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:16 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:16.558+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:16.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:16 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:16.558+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:17.341 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:16 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:16.959+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:17.341 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:16 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:16.959+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:55.040039+0000 front 2024-05-04T03:26:55.040081+0000 (oldest deadline 2024-05-04T03:27:16.140165+0000) 2024-05-04T03:27:17.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:17 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:17.091+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.503707+0000 front 2024-05-04T03:26:43.503743+0000 (oldest deadline 2024-05-04T03:27:09.904588+0000) 2024-05-04T03:27:17.629 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:17 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:17.380+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.777831+0000 front 2024-05-04T03:26:37.777881+0000 (oldest deadline 2024-05-04T03:27:03.078064+0000) 2024-05-04T03:27:17.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:17 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:17.380+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.779163+0000 front 2024-05-04T03:26:44.779129+0000 (oldest deadline 2024-05-04T03:27:07.679296+0000) 2024-05-04T03:27:17.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:17 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:17.253+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:17.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:17 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:17.253+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:17.678 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:17 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5[45085]: 2024-05-04T03:27:17.339+0000 7f2bd5a31700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:38.778533+0000 front 2024-05-04T03:26:38.778367+0000 (oldest deadline 2024-05-04T03:27:02.878313+0000) 2024-05-04T03:27:17.679 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:17 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5[45085]: 2024-05-04T03:27:17.339+0000 7f2bd5a31700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:42.878764+0000 front 2024-05-04T03:26:42.878826+0000 (oldest deadline 2024-05-04T03:27:13.479468+0000) 2024-05-04T03:27:17.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:17 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:17.554+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:17.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:17 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:17.554+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:18.001 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:17 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:17.678+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:18.001 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:17 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:17.678+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:18.372 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:18 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:17.999+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:18.372 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:18 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:17.999+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:55.040039+0000 front 2024-05-04T03:26:55.040081+0000 (oldest deadline 2024-05-04T03:27:16.140165+0000) 2024-05-04T03:27:18.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:18 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:18.076+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.503707+0000 front 2024-05-04T03:26:43.503743+0000 (oldest deadline 2024-05-04T03:27:09.904588+0000) 2024-05-04T03:27:18.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:18 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:18.076+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.506221+0000 front 2024-05-04T03:26:54.506290+0000 (oldest deadline 2024-05-04T03:27:18.006368+0000) 2024-05-04T03:27:18.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:18 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:18.383+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.777831+0000 front 2024-05-04T03:26:37.777881+0000 (oldest deadline 2024-05-04T03:27:03.078064+0000) 2024-05-04T03:27:18.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:18 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:18.383+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.779163+0000 front 2024-05-04T03:26:44.779129+0000 (oldest deadline 2024-05-04T03:27:07.679296+0000) 2024-05-04T03:27:18.631 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:18 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:18.223+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:18.631 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:18 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:18.223+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:18.631 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:18 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:18.223+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:18.678 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:18 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5[45085]: 2024-05-04T03:27:18.370+0000 7f2bd5a31700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:38.778533+0000 front 2024-05-04T03:26:38.778367+0000 (oldest deadline 2024-05-04T03:27:02.878313+0000) 2024-05-04T03:27:18.678 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:18 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5[45085]: 2024-05-04T03:27:18.370+0000 7f2bd5a31700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:42.878764+0000 front 2024-05-04T03:26:42.878826+0000 (oldest deadline 2024-05-04T03:27:13.479468+0000) 2024-05-04T03:27:18.678 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:18 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5[45085]: 2024-05-04T03:27:18.370+0000 7f2bd5a31700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:55.081180+0000 front 2024-05-04T03:26:55.081203+0000 (oldest deadline 2024-05-04T03:27:17.381366+0000) 2024-05-04T03:27:18.771 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:18 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:18.598+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:18.771 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:18 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:18.598+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:18.771 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:18 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:18.598+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:18.970 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:18 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:18.699+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:18.970 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:18 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:18.699+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:19.428 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:18 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:18.968+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:19.429 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:18 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:18.968+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:55.040039+0000 front 2024-05-04T03:26:55.040081+0000 (oldest deadline 2024-05-04T03:27:16.140165+0000) 2024-05-04T03:27:19.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:19 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:19.112+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.503707+0000 front 2024-05-04T03:26:43.503743+0000 (oldest deadline 2024-05-04T03:27:09.904588+0000) 2024-05-04T03:27:19.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:19 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:19.112+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.506221+0000 front 2024-05-04T03:26:54.506290+0000 (oldest deadline 2024-05-04T03:27:18.006368+0000) 2024-05-04T03:27:19.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:19 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:19.415+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.777831+0000 front 2024-05-04T03:26:37.777881+0000 (oldest deadline 2024-05-04T03:27:03.078064+0000) 2024-05-04T03:27:19.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:19 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:19.415+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.779163+0000 front 2024-05-04T03:26:44.779129+0000 (oldest deadline 2024-05-04T03:27:07.679296+0000) 2024-05-04T03:27:19.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:19 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:19.259+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:19.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:19 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:19.259+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:19.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:19 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:19.259+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:19.811 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:19 smithi077 podman[68369]: 2024-05-04 03:27:19.554907368 +0000 UTC m=+5.556830995 container died 309607a9285ef5587291a9c524c6fa7a1fea711166446f372c8478f7f2b665f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:27:19.812 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:19 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:19.664+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:19.812 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:19 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:19.664+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:19.812 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:19 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:19.664+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:19.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:19 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:19.648+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:19.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:19 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:19.648+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:19.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:19 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:19.648+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:20.178 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:19 smithi077 podman[68369]: 2024-05-04 03:27:19.909257243 +0000 UTC m=+5.911180867 container cleanup 309607a9285ef5587291a9c524c6fa7a1fea711166446f372c8478f7f2b665f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-04T03:27:20.179 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:19 smithi077 bash[68369]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5 2024-05-04T03:27:20.179 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:19 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:19.982+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:20.179 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:19 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:19.982+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:55.040039+0000 front 2024-05-04T03:26:55.040081+0000 (oldest deadline 2024-05-04T03:27:16.140165+0000) 2024-05-04T03:27:20.496 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:20 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:20.084+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.503707+0000 front 2024-05-04T03:26:43.503743+0000 (oldest deadline 2024-05-04T03:27:09.904588+0000) 2024-05-04T03:27:20.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:20 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:20.084+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.506221+0000 front 2024-05-04T03:26:54.506290+0000 (oldest deadline 2024-05-04T03:27:18.006368+0000) 2024-05-04T03:27:20.504 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:20 smithi077 podman[68386]: 2024-05-04 03:27:20.202430582 +0000 UTC m=+0.648815299 container remove 309607a9285ef5587291a9c524c6fa7a1fea711166446f372c8478f7f2b665f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-04T03:27:20.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:20 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:20.391+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.777831+0000 front 2024-05-04T03:26:37.777881+0000 (oldest deadline 2024-05-04T03:27:03.078064+0000) 2024-05-04T03:27:20.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:20 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:20.391+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.779163+0000 front 2024-05-04T03:26:44.779129+0000 (oldest deadline 2024-05-04T03:27:07.679296+0000) 2024-05-04T03:27:20.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:20 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:20.391+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.380790+0000 front 2024-05-04T03:26:52.380753+0000 (oldest deadline 2024-05-04T03:27:19.981402+0000) 2024-05-04T03:27:20.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:20 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:20.261+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:20.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:20 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:20.261+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:20.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:20 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:20.261+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:20.815 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:20 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:20.685+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:20.815 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:20 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:20.686+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:20.815 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:20 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:20.686+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:20.815 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:20 smithi077 podman[68475]: 2024-05-04 03:27:20.503228646 +0000 UTC m=+0.101186631 container create 421b2d551c6461df456d8a3ed7330ab6b73a6cec3936fb4715ad18501f8fe927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-04T03:27:20.815 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:20 smithi077 podman[68475]: 2024-05-04 03:27:20.420128297 +0000 UTC m=+0.018086303 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:27:20.815 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:20 smithi077 podman[68475]: 2024-05-04 03:27:20.738649139 +0000 UTC m=+0.336607120 container init 421b2d551c6461df456d8a3ed7330ab6b73a6cec3936fb4715ad18501f8fe927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:27:20.815 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:20 smithi077 podman[68475]: 2024-05-04 03:27:20.741708898 +0000 UTC m=+0.339666881 container start 421b2d551c6461df456d8a3ed7330ab6b73a6cec3936fb4715ad18501f8fe927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-04T03:27:20.815 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:20 smithi077 podman[68475]: 2024-05-04 03:27:20.813587016 +0000 UTC m=+0.411545013 container attach 421b2d551c6461df456d8a3ed7330ab6b73a6cec3936fb4715ad18501f8fe927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:27:20.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:20 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:20.617+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:20.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:20 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:20.617+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:20.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:20 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:20.617+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:21.178 INFO:journalctl@ceph.osd.5.smithi077.stdout:May 04 03:27:21 smithi077 podman[68475]: 2024-05-04 03:27:21.034231615 +0000 UTC m=+0.632189608 container died 421b2d551c6461df456d8a3ed7330ab6b73a6cec3936fb4715ad18501f8fe927 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-5-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-05-04T03:27:21.179 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:21 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:21.017+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:21.179 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:21 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:21.017+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:55.040039+0000 front 2024-05-04T03:26:55.040081+0000 (oldest deadline 2024-05-04T03:27:16.140165+0000) 2024-05-04T03:27:21.496 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:21 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:21.106+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.503707+0000 front 2024-05-04T03:26:43.503743+0000 (oldest deadline 2024-05-04T03:27:09.904588+0000) 2024-05-04T03:27:21.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:21 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:21.106+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.506221+0000 front 2024-05-04T03:26:54.506290+0000 (oldest deadline 2024-05-04T03:27:18.006368+0000) 2024-05-04T03:27:21.604 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.5.service' 2024-05-04T03:27:21.629 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:21 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:21.259+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:21.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:21 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:21.259+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:21.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:21 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:21.259+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:21.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:21 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:21.384+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.777831+0000 front 2024-05-04T03:26:37.777881+0000 (oldest deadline 2024-05-04T03:27:03.078064+0000) 2024-05-04T03:27:21.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:21 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:21.384+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.779163+0000 front 2024-05-04T03:26:44.779129+0000 (oldest deadline 2024-05-04T03:27:07.679296+0000) 2024-05-04T03:27:21.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:21 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:21.384+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.380790+0000 front 2024-05-04T03:26:52.380753+0000 (oldest deadline 2024-05-04T03:27:19.981402+0000) 2024-05-04T03:27:21.640 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:27:21.640 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-04T03:27:21.641 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-04T03:27:21.641 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.6 2024-05-04T03:27:21.928 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:21 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:21.670+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:21.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:21 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:21.670+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:21.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:21 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:21.670+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:21.951 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:21 smithi001 systemd[1]: Stopping Ceph osd.6 for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:27:21.951 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:21 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:21.949+0000 7faef0fa9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:27:21.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:21 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:21.609+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:21.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:21 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:21.609+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:21.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:21 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:21.609+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:22.237 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:21 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:21.949+0000 7faef0fa9700 -1 osd.6 86 *** Got signal Terminated *** 2024-05-04T03:27:22.237 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:21 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:21.949+0000 7faef0fa9700 -1 osd.6 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-04T03:27:22.428 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:22 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:22.026+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:22.428 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:22 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:22.026+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:55.040039+0000 front 2024-05-04T03:26:55.040081+0000 (oldest deadline 2024-05-04T03:27:16.140165+0000) 2024-05-04T03:27:22.496 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:22 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:22.123+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.503707+0000 front 2024-05-04T03:26:43.503743+0000 (oldest deadline 2024-05-04T03:27:09.904588+0000) 2024-05-04T03:27:22.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:22 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:22.123+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.506221+0000 front 2024-05-04T03:26:54.506290+0000 (oldest deadline 2024-05-04T03:27:18.006368+0000) 2024-05-04T03:27:22.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:22 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:22.367+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.777831+0000 front 2024-05-04T03:26:37.777881+0000 (oldest deadline 2024-05-04T03:27:03.078064+0000) 2024-05-04T03:27:22.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:22 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:22.367+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.779163+0000 front 2024-05-04T03:26:44.779129+0000 (oldest deadline 2024-05-04T03:27:07.679296+0000) 2024-05-04T03:27:22.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:22 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:22.367+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.380790+0000 front 2024-05-04T03:26:52.380753+0000 (oldest deadline 2024-05-04T03:27:19.981402+0000) 2024-05-04T03:27:22.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:22 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:22.234+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:22.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:22 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:22.234+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:22.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:22 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:22.234+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:22.928 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:22 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:22.653+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:22.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:22 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:22.653+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:22.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:22 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:22.653+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:22.996 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:22 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:22.624+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:22.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:22 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:22.624+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:22.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:22 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:22.624+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:23.428 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:22 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:22.984+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:23.428 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:22 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:22.984+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:55.040039+0000 front 2024-05-04T03:26:55.040081+0000 (oldest deadline 2024-05-04T03:27:16.140165+0000) 2024-05-04T03:27:23.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:23 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:23.171+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.503707+0000 front 2024-05-04T03:26:43.503743+0000 (oldest deadline 2024-05-04T03:27:09.904588+0000) 2024-05-04T03:27:23.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:23 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:23.171+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.506221+0000 front 2024-05-04T03:26:54.506290+0000 (oldest deadline 2024-05-04T03:27:18.006368+0000) 2024-05-04T03:27:23.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:23 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:23.378+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.777831+0000 front 2024-05-04T03:26:37.777881+0000 (oldest deadline 2024-05-04T03:27:03.078064+0000) 2024-05-04T03:27:23.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:23 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:23.378+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.779163+0000 front 2024-05-04T03:26:44.779129+0000 (oldest deadline 2024-05-04T03:27:07.679296+0000) 2024-05-04T03:27:23.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:23 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:23.378+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.380790+0000 front 2024-05-04T03:26:52.380753+0000 (oldest deadline 2024-05-04T03:27:19.981402+0000) 2024-05-04T03:27:23.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:23 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:23.188+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:23.631 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:23 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:23.188+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:23.631 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:23 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:23.188+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:23.953 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:23 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:23.677+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:23.954 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:23 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:23.677+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:23.954 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:23 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:23.677+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:23.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:23 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:23.633+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:23.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:23 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:23.633+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:23.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:23 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:23.633+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:24.428 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:23 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:23.951+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:24.428 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:23 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:23.951+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:55.040039+0000 front 2024-05-04T03:26:55.040081+0000 (oldest deadline 2024-05-04T03:27:16.140165+0000) 2024-05-04T03:27:24.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:24 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:24.199+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.503707+0000 front 2024-05-04T03:26:43.503743+0000 (oldest deadline 2024-05-04T03:27:09.904588+0000) 2024-05-04T03:27:24.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:24 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:24.199+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.506221+0000 front 2024-05-04T03:26:54.506290+0000 (oldest deadline 2024-05-04T03:27:18.006368+0000) 2024-05-04T03:27:24.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:24 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:24.334+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.777831+0000 front 2024-05-04T03:26:37.777881+0000 (oldest deadline 2024-05-04T03:27:03.078064+0000) 2024-05-04T03:27:24.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:24 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:24.334+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.779163+0000 front 2024-05-04T03:26:44.779129+0000 (oldest deadline 2024-05-04T03:27:07.679296+0000) 2024-05-04T03:27:24.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:24 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:24.334+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.380790+0000 front 2024-05-04T03:26:52.380753+0000 (oldest deadline 2024-05-04T03:27:19.981402+0000) 2024-05-04T03:27:24.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:24 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:24.195+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:24.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:24 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:24.195+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:24.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:24 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:24.195+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:24.982 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:24 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:24.686+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:24.982 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:24 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:24.686+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:24.983 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:24 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:24.686+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:24.996 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:24 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:24.619+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:24.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:24 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:24.619+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:24.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:24 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:24.619+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:25.428 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:24 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:24.980+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:25.428 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:24 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:24.980+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:55.040039+0000 front 2024-05-04T03:26:55.040081+0000 (oldest deadline 2024-05-04T03:27:16.140165+0000) 2024-05-04T03:27:25.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:25 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:25.185+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.503707+0000 front 2024-05-04T03:26:43.503743+0000 (oldest deadline 2024-05-04T03:27:09.904588+0000) 2024-05-04T03:27:25.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:25 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:25.185+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.506221+0000 front 2024-05-04T03:26:54.506290+0000 (oldest deadline 2024-05-04T03:27:18.006368+0000) 2024-05-04T03:27:25.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:25 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:25.185+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.107754+0000 front 2024-05-04T03:27:02.107724+0000 (oldest deadline 2024-05-04T03:27:25.007536+0000) 2024-05-04T03:27:25.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:25 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:25.377+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.777831+0000 front 2024-05-04T03:26:37.777881+0000 (oldest deadline 2024-05-04T03:27:03.078064+0000) 2024-05-04T03:27:25.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:25 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:25.377+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.779163+0000 front 2024-05-04T03:26:44.779129+0000 (oldest deadline 2024-05-04T03:27:07.679296+0000) 2024-05-04T03:27:25.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:25 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:25.377+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.380790+0000 front 2024-05-04T03:26:52.380753+0000 (oldest deadline 2024-05-04T03:27:19.981402+0000) 2024-05-04T03:27:25.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:25 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:25.151+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:25.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:25 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:25.151+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:25.631 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:25 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:25.151+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:25.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:25 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:25.658+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:25.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:25 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:25.658+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:25.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:25 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:25.658+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:26.026 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:25 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:25.723+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:26.026 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:25 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:25.723+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:26.026 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:25 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:25.723+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:26.428 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:26 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:26.024+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:26.428 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:26 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:26.024+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:55.040039+0000 front 2024-05-04T03:26:55.040081+0000 (oldest deadline 2024-05-04T03:27:16.140165+0000) 2024-05-04T03:27:26.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:26 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:26.155+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.503707+0000 front 2024-05-04T03:26:43.503743+0000 (oldest deadline 2024-05-04T03:27:09.904588+0000) 2024-05-04T03:27:26.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:26 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:26.155+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.506221+0000 front 2024-05-04T03:26:54.506290+0000 (oldest deadline 2024-05-04T03:27:18.006368+0000) 2024-05-04T03:27:26.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:26 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:26.155+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.107754+0000 front 2024-05-04T03:27:02.107724+0000 (oldest deadline 2024-05-04T03:27:25.007536+0000) 2024-05-04T03:27:26.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:26 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:26.392+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.777831+0000 front 2024-05-04T03:26:37.777881+0000 (oldest deadline 2024-05-04T03:27:03.078064+0000) 2024-05-04T03:27:26.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:26 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:26.392+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.779163+0000 front 2024-05-04T03:26:44.779129+0000 (oldest deadline 2024-05-04T03:27:07.679296+0000) 2024-05-04T03:27:26.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:26 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6[55397]: 2024-05-04T03:27:26.392+0000 7faee9557700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.380790+0000 front 2024-05-04T03:26:52.380753+0000 (oldest deadline 2024-05-04T03:27:19.981402+0000) 2024-05-04T03:27:26.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:26 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:26.194+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:26.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:26 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:26.194+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:26.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:26 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:26.194+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:26.996 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:26 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:26.629+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:26.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:26 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:26.629+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:26.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:26 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:26.629+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:27.009 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:26 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:26.743+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:27.009 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:26 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:26.743+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:27.010 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:26 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:26.743+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:27.428 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:27 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:27.007+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:27.428 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:27 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:27.007+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:55.040039+0000 front 2024-05-04T03:26:55.040081+0000 (oldest deadline 2024-05-04T03:27:16.140165+0000) 2024-05-04T03:27:27.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:27 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:27.129+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.503707+0000 front 2024-05-04T03:26:43.503743+0000 (oldest deadline 2024-05-04T03:27:09.904588+0000) 2024-05-04T03:27:27.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:27 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:27.129+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.506221+0000 front 2024-05-04T03:26:54.506290+0000 (oldest deadline 2024-05-04T03:27:18.006368+0000) 2024-05-04T03:27:27.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:27 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:27.129+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.107754+0000 front 2024-05-04T03:27:02.107724+0000 (oldest deadline 2024-05-04T03:27:25.007536+0000) 2024-05-04T03:27:27.630 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:27 smithi001 podman[113044]: 2024-05-04 03:27:27.383797323 +0000 UTC m=+5.545571824 container died e00742d9417ef518d379db5235349747780a0c55b7f265b4b381d9b3861002a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-04T03:27:27.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:27 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:27.196+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:27.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:27 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:27.196+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:27.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:27 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:27.196+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:27.985 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:27 smithi001 podman[113044]: 2024-05-04 03:27:27.741377592 +0000 UTC m=+5.903152083 container cleanup e00742d9417ef518d379db5235349747780a0c55b7f265b4b381d9b3861002a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-04T03:27:27.985 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:27 smithi001 bash[113044]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6 2024-05-04T03:27:27.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:27 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:27.668+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:27.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:27 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:27.668+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:27.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:27 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:27.668+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:27.998 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:27 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:27.705+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:27.998 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:27 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:27.705+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:27.998 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:27 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:27.705+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:28.304 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:27 smithi001 podman[113073]: 2024-05-04 03:27:27.98396246 +0000 UTC m=+0.600478592 container remove e00742d9417ef518d379db5235349747780a0c55b7f265b4b381d9b3861002a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-04T03:27:28.304 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:28 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:28.179+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:28.304 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:28 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:28.179+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:28.304 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:28 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:28.179+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:28.304 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:28 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:28.179+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:26:57.824941+0000 front 2024-05-04T03:26:57.824844+0000 (oldest deadline 2024-05-04T03:27:27.825442+0000) 2024-05-04T03:27:28.428 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:27 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:27.995+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:28.429 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:27 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:27.995+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:55.040039+0000 front 2024-05-04T03:26:55.040081+0000 (oldest deadline 2024-05-04T03:27:16.140165+0000) 2024-05-04T03:27:28.429 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:27 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:27.995+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:03.141932+0000 front 2024-05-04T03:27:03.142035+0000 (oldest deadline 2024-05-04T03:27:27.841830+0000) 2024-05-04T03:27:28.496 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:28 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:28.162+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.503707+0000 front 2024-05-04T03:26:43.503743+0000 (oldest deadline 2024-05-04T03:27:09.904588+0000) 2024-05-04T03:27:28.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:28 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:28.162+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.506221+0000 front 2024-05-04T03:26:54.506290+0000 (oldest deadline 2024-05-04T03:27:18.006368+0000) 2024-05-04T03:27:28.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:28 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:28.162+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.107754+0000 front 2024-05-04T03:27:02.107724+0000 (oldest deadline 2024-05-04T03:27:25.007536+0000) 2024-05-04T03:27:28.563 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:28 smithi001 podman[113163]: 2024-05-04 03:27:28.2056932 +0000 UTC m=+0.017871174 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:27:28.563 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:28 smithi001 podman[113163]: 2024-05-04 03:27:28.318290264 +0000 UTC m=+0.130468227 container create 1c3f2c5c16a801d5931e938abbec816536a7abbaeb55fb47f5056f4ca38a7825 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD) 2024-05-04T03:27:28.870 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:28 smithi001 podman[113163]: 2024-05-04 03:27:28.5623238 +0000 UTC m=+0.374501763 container init 1c3f2c5c16a801d5931e938abbec816536a7abbaeb55fb47f5056f4ca38a7825 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-04T03:27:28.870 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:28 smithi001 podman[113163]: 2024-05-04 03:27:28.56589102 +0000 UTC m=+0.378068980 container start 1c3f2c5c16a801d5931e938abbec816536a7abbaeb55fb47f5056f4ca38a7825 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6-deactivate, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-04T03:27:28.870 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:28 smithi001 podman[113163]: 2024-05-04 03:27:28.654012992 +0000 UTC m=+0.466190983 container attach 1c3f2c5c16a801d5931e938abbec816536a7abbaeb55fb47f5056f4ca38a7825 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6-deactivate, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-04T03:27:28.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:28 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:28.625+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:28.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:28 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:28.625+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:28.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:28 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:28.625+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:28.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:28 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:28.625+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:29.129 INFO:journalctl@ceph.osd.6.smithi001.stdout:May 04 03:27:28 smithi001 podman[113190]: 2024-05-04 03:27:28.897378908 +0000 UTC m=+0.018996320 container died 1c3f2c5c16a801d5931e938abbec816536a7abbaeb55fb47f5056f4ca38a7825 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-6-deactivate, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-04T03:27:29.178 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:28 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:28.700+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:29.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:28 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:28.700+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:29.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:28 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:28.700+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:29.179 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:28 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:28.948+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:29.179 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:28 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:28.948+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:55.040039+0000 front 2024-05-04T03:26:55.040081+0000 (oldest deadline 2024-05-04T03:27:16.140165+0000) 2024-05-04T03:27:29.179 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:28 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:28.948+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:03.141932+0000 front 2024-05-04T03:27:03.142035+0000 (oldest deadline 2024-05-04T03:27:27.841830+0000) 2024-05-04T03:27:29.409 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:29 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:29.129+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:29.409 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:29 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:29.130+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:29.409 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:29 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:29.130+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:29.409 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:29 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:29.130+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:26:57.824941+0000 front 2024-05-04T03:26:57.824844+0000 (oldest deadline 2024-05-04T03:27:27.825442+0000) 2024-05-04T03:27:29.431 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.6.service' 2024-05-04T03:27:29.478 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:27:29.478 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-04T03:27:29.478 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-04T03:27:29.478 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.7 2024-05-04T03:27:29.496 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:29 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:29.190+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.503707+0000 front 2024-05-04T03:26:43.503743+0000 (oldest deadline 2024-05-04T03:27:09.904588+0000) 2024-05-04T03:27:29.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:29 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:29.190+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.506221+0000 front 2024-05-04T03:26:54.506290+0000 (oldest deadline 2024-05-04T03:27:18.006368+0000) 2024-05-04T03:27:29.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:29 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:29.190+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.107754+0000 front 2024-05-04T03:27:02.107724+0000 (oldest deadline 2024-05-04T03:27:25.007536+0000) 2024-05-04T03:27:29.762 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:29 smithi072 systemd[1]: Stopping Ceph osd.7 for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:27:29.762 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:29 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:29.596+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:29.762 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:29 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:29.596+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:29.762 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:29 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:29.596+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:29.762 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:29 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:29.596+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:29.988 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:29 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:29.727+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:29.988 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:29 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:29.727+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:29.988 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:29 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:29.727+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:30.235 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:29 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:29.783+0000 7f20a5bdd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:27:30.236 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:29 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:29.783+0000 7f20a5bdd700 -1 osd.7 86 *** Got signal Terminated *** 2024-05-04T03:27:30.236 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:29 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:29.783+0000 7f20a5bdd700 -1 osd.7 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-04T03:27:30.428 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:29 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:29.986+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:30.429 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:29 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:29.986+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:55.040039+0000 front 2024-05-04T03:26:55.040081+0000 (oldest deadline 2024-05-04T03:27:16.140165+0000) 2024-05-04T03:27:30.429 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:29 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:29.986+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:03.141932+0000 front 2024-05-04T03:27:03.142035+0000 (oldest deadline 2024-05-04T03:27:27.841830+0000) 2024-05-04T03:27:30.496 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:30 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:30.233+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.503707+0000 front 2024-05-04T03:26:43.503743+0000 (oldest deadline 2024-05-04T03:27:09.904588+0000) 2024-05-04T03:27:30.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:30 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:30.233+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.506221+0000 front 2024-05-04T03:26:54.506290+0000 (oldest deadline 2024-05-04T03:27:18.006368+0000) 2024-05-04T03:27:30.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:30 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:30.233+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.107754+0000 front 2024-05-04T03:27:02.107724+0000 (oldest deadline 2024-05-04T03:27:25.007536+0000) 2024-05-04T03:27:30.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:30 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:30.169+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:30.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:30 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:30.169+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:30.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:30 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:30.169+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:30.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:30 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:30.169+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:26:57.824941+0000 front 2024-05-04T03:26:57.824844+0000 (oldest deadline 2024-05-04T03:27:27.825442+0000) 2024-05-04T03:27:30.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:30 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:30.592+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:30.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:30 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:30.592+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:30.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:30 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:30.592+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:30.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:30 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:30.592+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:31.178 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:30 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:30.727+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:31.178 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:30 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:30.727+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:31.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:30 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:30.727+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:31.179 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:30 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:30.951+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:31.179 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:30 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:30.951+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:55.040039+0000 front 2024-05-04T03:26:55.040081+0000 (oldest deadline 2024-05-04T03:27:16.140165+0000) 2024-05-04T03:27:31.179 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:30 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:30.951+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:03.141932+0000 front 2024-05-04T03:27:03.142035+0000 (oldest deadline 2024-05-04T03:27:27.841830+0000) 2024-05-04T03:27:31.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:31 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:31.128+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:31.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:31 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:31.128+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:31.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:31 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:31.128+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:31.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:31 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:31.128+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:26:57.824941+0000 front 2024-05-04T03:26:57.824844+0000 (oldest deadline 2024-05-04T03:27:27.825442+0000) 2024-05-04T03:27:31.636 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:31 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:31.275+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.503707+0000 front 2024-05-04T03:26:43.503743+0000 (oldest deadline 2024-05-04T03:27:09.904588+0000) 2024-05-04T03:27:31.636 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:31 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:31.275+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.506221+0000 front 2024-05-04T03:26:54.506290+0000 (oldest deadline 2024-05-04T03:27:18.006368+0000) 2024-05-04T03:27:31.636 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:31 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:31.275+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.107754+0000 front 2024-05-04T03:27:02.107724+0000 (oldest deadline 2024-05-04T03:27:25.007536+0000) 2024-05-04T03:27:31.991 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:31 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:31.735+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:31.991 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:31 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:31.735+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:31.991 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:31 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:31.735+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:31.996 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:31 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:31.634+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:31.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:31 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:31.634+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:31.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:31 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:31.634+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:31.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:31 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:31.634+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:32.313 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:31 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:31.989+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:32.313 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:31 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:31.989+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:55.040039+0000 front 2024-05-04T03:26:55.040081+0000 (oldest deadline 2024-05-04T03:27:16.140165+0000) 2024-05-04T03:27:32.313 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:31 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:31.989+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:03.141932+0000 front 2024-05-04T03:27:03.142035+0000 (oldest deadline 2024-05-04T03:27:27.841830+0000) 2024-05-04T03:27:32.589 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:32 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:32.302+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.503707+0000 front 2024-05-04T03:26:43.503743+0000 (oldest deadline 2024-05-04T03:27:09.904588+0000) 2024-05-04T03:27:32.590 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:32 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:32.302+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.506221+0000 front 2024-05-04T03:26:54.506290+0000 (oldest deadline 2024-05-04T03:27:18.006368+0000) 2024-05-04T03:27:32.590 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:32 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:32.302+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.107754+0000 front 2024-05-04T03:27:02.107724+0000 (oldest deadline 2024-05-04T03:27:25.007536+0000) 2024-05-04T03:27:32.629 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:32 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:32.153+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:32.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:32 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:32.153+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:32.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:32 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:32.153+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:32.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:32 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:32.153+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:26:57.824941+0000 front 2024-05-04T03:26:57.824844+0000 (oldest deadline 2024-05-04T03:27:27.825442+0000) 2024-05-04T03:27:32.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:32 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:32.587+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:32.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:32 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:32.587+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:32.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:32 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:32.587+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:32.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:32 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:32.587+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:33.178 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:32 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:32.982+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:33.179 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:32 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:32.982+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:55.040039+0000 front 2024-05-04T03:26:55.040081+0000 (oldest deadline 2024-05-04T03:27:16.140165+0000) 2024-05-04T03:27:33.179 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:32 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:32.982+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:03.141932+0000 front 2024-05-04T03:27:03.142035+0000 (oldest deadline 2024-05-04T03:27:27.841830+0000) 2024-05-04T03:27:33.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:32 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:32.745+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:33.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:32 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:32.745+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:33.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:32 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:32.745+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:33.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:32 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:32.745+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:27:33.584 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:33 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:33.294+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.503707+0000 front 2024-05-04T03:26:43.503743+0000 (oldest deadline 2024-05-04T03:27:09.904588+0000) 2024-05-04T03:27:33.584 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:33 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:33.294+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.506221+0000 front 2024-05-04T03:26:54.506290+0000 (oldest deadline 2024-05-04T03:27:18.006368+0000) 2024-05-04T03:27:33.584 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:33 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:33.294+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.107754+0000 front 2024-05-04T03:27:02.107724+0000 (oldest deadline 2024-05-04T03:27:25.007536+0000) 2024-05-04T03:27:33.584 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:33 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:33.294+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:07.308361+0000 front 2024-05-04T03:27:07.308416+0000 (oldest deadline 2024-05-04T03:27:33.208472+0000) 2024-05-04T03:27:33.629 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:33 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:33.189+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:33.629 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:33 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:33.189+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:33.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:33 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:33.189+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:33.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:33 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:33.189+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:26:57.824941+0000 front 2024-05-04T03:26:57.824844+0000 (oldest deadline 2024-05-04T03:27:27.825442+0000) 2024-05-04T03:27:33.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:33 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:33.189+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:07.826283+0000 front 2024-05-04T03:27:07.826320+0000 (oldest deadline 2024-05-04T03:27:32.526031+0000) 2024-05-04T03:27:33.996 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:33 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:33.744+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:33.996 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:33 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:33.744+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:33.996 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:33 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:33.744+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:33.996 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:33 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:33.744+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:27:33.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:33 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:33.582+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:33.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:33 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:33.582+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:33.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:33 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:33.582+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:33.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:33 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:33.582+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:34.428 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:33 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:33.994+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:34.428 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:33 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:33.994+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:55.040039+0000 front 2024-05-04T03:26:55.040081+0000 (oldest deadline 2024-05-04T03:27:16.140165+0000) 2024-05-04T03:27:34.429 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:33 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:33.994+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:03.141932+0000 front 2024-05-04T03:27:03.142035+0000 (oldest deadline 2024-05-04T03:27:27.841830+0000) 2024-05-04T03:27:34.595 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:34 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:34.323+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.503707+0000 front 2024-05-04T03:26:43.503743+0000 (oldest deadline 2024-05-04T03:27:09.904588+0000) 2024-05-04T03:27:34.595 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:34 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:34.323+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.506221+0000 front 2024-05-04T03:26:54.506290+0000 (oldest deadline 2024-05-04T03:27:18.006368+0000) 2024-05-04T03:27:34.595 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:34 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:34.323+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.107754+0000 front 2024-05-04T03:27:02.107724+0000 (oldest deadline 2024-05-04T03:27:25.007536+0000) 2024-05-04T03:27:34.595 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:34 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7[51002]: 2024-05-04T03:27:34.323+0000 7f209e18b700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:07.308361+0000 front 2024-05-04T03:27:07.308416+0000 (oldest deadline 2024-05-04T03:27:33.208472+0000) 2024-05-04T03:27:34.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:34 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:34.221+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:34.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:34 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:34.221+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:34.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:34 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:34.221+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:34.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:34 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:34.221+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:26:57.824941+0000 front 2024-05-04T03:26:57.824844+0000 (oldest deadline 2024-05-04T03:27:27.825442+0000) 2024-05-04T03:27:34.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:34 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:34.221+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:07.826283+0000 front 2024-05-04T03:27:07.826320+0000 (oldest deadline 2024-05-04T03:27:32.526031+0000) 2024-05-04T03:27:34.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:34 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:34.593+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:34.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:34 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:34.593+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:34.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:34 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:34.593+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:34.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:34 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:34.593+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:34.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:34 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:34.593+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:11.025148+0000 front 2024-05-04T03:27:11.025194+0000 (oldest deadline 2024-05-04T03:27:34.525456+0000) 2024-05-04T03:27:35.178 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:34 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:34.948+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:35.179 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:34 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:34.948+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:55.040039+0000 front 2024-05-04T03:26:55.040081+0000 (oldest deadline 2024-05-04T03:27:16.140165+0000) 2024-05-04T03:27:35.179 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:34 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:34.948+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:03.141932+0000 front 2024-05-04T03:27:03.142035+0000 (oldest deadline 2024-05-04T03:27:27.841830+0000) 2024-05-04T03:27:35.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:34 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:34.754+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:35.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:34 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:34.754+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:35.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:34 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:34.754+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:35.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:34 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:34.754+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:27:35.497 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:35 smithi072 podman[67868]: 2024-05-04 03:27:35.222335107 +0000 UTC m=+5.553265004 container died 06eb9b19c757427dfb7968538b66982bc3fda83fac61251426343681e2091261 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0) 2024-05-04T03:27:35.629 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:35 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:35.181+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:35.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:35 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:35.181+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:35.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:35 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:35.181+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:35.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:35 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:35.181+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:26:57.824941+0000 front 2024-05-04T03:26:57.824844+0000 (oldest deadline 2024-05-04T03:27:27.825442+0000) 2024-05-04T03:27:35.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:35 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:35.181+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:07.826283+0000 front 2024-05-04T03:27:07.826320+0000 (oldest deadline 2024-05-04T03:27:32.526031+0000) 2024-05-04T03:27:35.842 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:35 smithi072 podman[67868]: 2024-05-04 03:27:35.586600415 +0000 UTC m=+5.917530300 container cleanup 06eb9b19c757427dfb7968538b66982bc3fda83fac61251426343681e2091261 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:27:35.842 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:35 smithi072 bash[67868]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7 2024-05-04T03:27:35.842 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:35 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:35.555+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:35.843 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:35 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:35.555+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:35.843 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:35 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:35.555+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:35.843 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:35 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:35.555+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:35.843 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:35 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:35.555+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:11.025148+0000 front 2024-05-04T03:27:11.025194+0000 (oldest deadline 2024-05-04T03:27:34.525456+0000) 2024-05-04T03:27:36.156 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:35 smithi072 podman[67885]: 2024-05-04 03:27:35.840487951 +0000 UTC m=+0.618080951 container remove 06eb9b19c757427dfb7968538b66982bc3fda83fac61251426343681e2091261 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-04T03:27:36.178 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:35 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:35.971+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:36.179 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:35 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:35.971+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:55.040039+0000 front 2024-05-04T03:26:55.040081+0000 (oldest deadline 2024-05-04T03:27:16.140165+0000) 2024-05-04T03:27:36.179 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:35 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:35.971+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:03.141932+0000 front 2024-05-04T03:27:03.142035+0000 (oldest deadline 2024-05-04T03:27:27.841830+0000) 2024-05-04T03:27:36.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:35 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:35.755+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:36.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:35 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:35.755+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:36.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:35 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:35.755+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:36.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:35 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:35.755+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:27:36.409 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:36 smithi072 podman[67971]: 2024-05-04 03:27:36.058203411 +0000 UTC m=+0.018873796 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:27:36.409 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:36 smithi072 podman[67971]: 2024-05-04 03:27:36.175540703 +0000 UTC m=+0.136211070 container create 1287964de45c8df868955e5842ca3fb8787277b01de2de64335292c27b7ce6b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430) 2024-05-04T03:27:36.409 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:36 smithi072 podman[67971]: 2024-05-04 03:27:36.404594086 +0000 UTC m=+0.365264454 container init 1287964de45c8df868955e5842ca3fb8787277b01de2de64335292c27b7ce6b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True) 2024-05-04T03:27:36.409 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:36 smithi072 podman[67971]: 2024-05-04 03:27:36.407573946 +0000 UTC m=+0.368244315 container start 1287964de45c8df868955e5842ca3fb8787277b01de2de64335292c27b7ce6b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-04T03:27:36.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:36 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:36.221+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:36.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:36 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:36.221+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:36.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:36 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:36.221+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:36.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:36 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:36.221+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:26:57.824941+0000 front 2024-05-04T03:26:57.824844+0000 (oldest deadline 2024-05-04T03:27:27.825442+0000) 2024-05-04T03:27:36.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:36 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:36.221+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:07.826283+0000 front 2024-05-04T03:27:07.826320+0000 (oldest deadline 2024-05-04T03:27:32.526031+0000) 2024-05-04T03:27:36.704 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:36 smithi072 podman[67971]: 2024-05-04 03:27:36.521828112 +0000 UTC m=+0.482498498 container attach 1287964de45c8df868955e5842ca3fb8787277b01de2de64335292c27b7ce6b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-04T03:27:36.704 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:36 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:36.599+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:36.704 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:36 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:36.599+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:36.704 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:36 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:36.599+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:36.704 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:36 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:36.599+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:36.704 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:36 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:36.599+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:11.025148+0000 front 2024-05-04T03:27:11.025194+0000 (oldest deadline 2024-05-04T03:27:34.525456+0000) 2024-05-04T03:27:36.994 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:36 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:36.738+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:36.994 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:36 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:36.738+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:36.994 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:36 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:36.738+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:36.994 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:36 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:36.738+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:27:36.996 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:36 smithi072 podman[67971]: 2024-05-04 03:27:36.702596948 +0000 UTC m=+0.663267309 container died 1287964de45c8df868955e5842ca3fb8787277b01de2de64335292c27b7ce6b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5) 2024-05-04T03:27:37.345 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.7.service' 2024-05-04T03:27:37.348 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:37 smithi072 podman[67997]: 2024-05-04 03:27:37.322066674 +0000 UTC m=+0.613214995 container remove 1287964de45c8df868955e5842ca3fb8787277b01de2de64335292c27b7ce6b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-04T03:27:37.348 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:37 smithi072 systemd[1]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.7.service: Deactivated successfully. 2024-05-04T03:27:37.348 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:37 smithi072 systemd[1]: Stopped Ceph osd.7 for 4a572aa8-09c3-11ef-bc97-c7b262605968. 2024-05-04T03:27:37.348 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 04 03:27:37 smithi072 systemd[1]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.7.service: Consumed 42.377s CPU time. 2024-05-04T03:27:37.385 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:27:37.385 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-04T03:27:37.386 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-04T03:27:37.386 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.8 2024-05-04T03:27:37.412 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:36 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:36.992+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:37.412 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:36 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:36.992+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:55.040039+0000 front 2024-05-04T03:26:55.040081+0000 (oldest deadline 2024-05-04T03:27:16.140165+0000) 2024-05-04T03:27:37.412 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:36 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:36.992+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:03.141932+0000 front 2024-05-04T03:27:03.142035+0000 (oldest deadline 2024-05-04T03:27:27.841830+0000) 2024-05-04T03:27:37.412 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:36 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:36.992+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:07.842582+0000 front 2024-05-04T03:27:07.842498+0000 (oldest deadline 2024-05-04T03:27:36.642940+0000) 2024-05-04T03:27:37.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:37 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:37.250+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:37.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:37 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:37.250+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:37.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:37 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:37.250+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:37.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:37 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:37.250+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:26:57.824941+0000 front 2024-05-04T03:26:57.824844+0000 (oldest deadline 2024-05-04T03:27:27.825442+0000) 2024-05-04T03:27:37.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:37 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:37.250+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:07.826283+0000 front 2024-05-04T03:27:07.826320+0000 (oldest deadline 2024-05-04T03:27:32.526031+0000) 2024-05-04T03:27:37.678 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:37 smithi077 systemd[1]: Stopping Ceph osd.8 for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:27:37.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:37 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:37.640+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:37.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:37 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:37.640+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:37.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:37 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:37.640+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:37.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:37 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:37.640+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:37.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:37 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:37.640+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:11.025148+0000 front 2024-05-04T03:27:11.025194+0000 (oldest deadline 2024-05-04T03:27:34.525456+0000) 2024-05-04T03:27:38.011 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:37 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:37.678+0000 7feeee5b7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:27:38.011 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:37 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:37.678+0000 7feeee5b7700 -1 osd.8 86 *** Got signal Terminated *** 2024-05-04T03:27:38.011 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:37 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:37.678+0000 7feeee5b7700 -1 osd.8 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-04T03:27:38.011 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:37 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:37.696+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:38.012 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:37 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:37.696+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:38.012 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:37 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:37.696+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:38.012 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:37 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:37.696+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:27:38.428 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:38 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:38.009+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:38.429 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:38 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:38.009+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:55.040039+0000 front 2024-05-04T03:26:55.040081+0000 (oldest deadline 2024-05-04T03:27:16.140165+0000) 2024-05-04T03:27:38.429 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:38 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:38.009+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:03.141932+0000 front 2024-05-04T03:27:03.142035+0000 (oldest deadline 2024-05-04T03:27:27.841830+0000) 2024-05-04T03:27:38.429 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:38 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:38.009+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:07.842582+0000 front 2024-05-04T03:27:07.842498+0000 (oldest deadline 2024-05-04T03:27:36.642940+0000) 2024-05-04T03:27:38.629 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:38 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:38.209+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:38.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:38 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:38.209+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:38.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:38 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:38.209+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:38.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:38 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:38.209+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:26:57.824941+0000 front 2024-05-04T03:26:57.824844+0000 (oldest deadline 2024-05-04T03:27:27.825442+0000) 2024-05-04T03:27:38.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:38 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:38.209+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:07.826283+0000 front 2024-05-04T03:27:07.826320+0000 (oldest deadline 2024-05-04T03:27:32.526031+0000) 2024-05-04T03:27:38.928 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:38 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:38.656+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:38.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:38 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:38.656+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:38.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:38 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:38.656+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:38.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:38 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:38.656+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:27:38.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:38 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:38.644+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:38.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:38 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:38.644+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:38.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:38 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:38.644+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:38.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:38 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:38.644+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:38.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:38 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:38.644+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:11.025148+0000 front 2024-05-04T03:27:11.025194+0000 (oldest deadline 2024-05-04T03:27:34.525456+0000) 2024-05-04T03:27:39.428 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:39 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:39.038+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:39.429 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:39 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:39.038+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:55.040039+0000 front 2024-05-04T03:26:55.040081+0000 (oldest deadline 2024-05-04T03:27:16.140165+0000) 2024-05-04T03:27:39.429 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:39 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:39.038+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:03.141932+0000 front 2024-05-04T03:27:03.142035+0000 (oldest deadline 2024-05-04T03:27:27.841830+0000) 2024-05-04T03:27:39.429 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:39 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:39.038+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:07.842582+0000 front 2024-05-04T03:27:07.842498+0000 (oldest deadline 2024-05-04T03:27:36.642940+0000) 2024-05-04T03:27:39.483 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:39 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:39.171+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:39.483 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:39 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:39.171+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:39.483 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:39 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:39.171+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:39.483 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:39 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:39.171+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:26:57.824941+0000 front 2024-05-04T03:26:57.824844+0000 (oldest deadline 2024-05-04T03:27:27.825442+0000) 2024-05-04T03:27:39.483 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:39 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:39.171+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:07.826283+0000 front 2024-05-04T03:27:07.826320+0000 (oldest deadline 2024-05-04T03:27:32.526031+0000) 2024-05-04T03:27:39.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:39 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:39.603+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:39.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:39 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:39.603+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:39.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:39 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:39.603+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:39.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:39 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:39.603+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:39.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:39 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:39.603+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:11.025148+0000 front 2024-05-04T03:27:11.025194+0000 (oldest deadline 2024-05-04T03:27:34.525456+0000) 2024-05-04T03:27:40.063 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:39 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:39.695+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:40.063 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:39 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:39.695+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:40.063 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:39 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:39.695+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:40.063 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:39 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:39.695+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:27:40.428 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:40 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:40.061+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:40.429 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:40 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:40.061+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:55.040039+0000 front 2024-05-04T03:26:55.040081+0000 (oldest deadline 2024-05-04T03:27:16.140165+0000) 2024-05-04T03:27:40.429 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:40 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:40.061+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:03.141932+0000 front 2024-05-04T03:27:03.142035+0000 (oldest deadline 2024-05-04T03:27:27.841830+0000) 2024-05-04T03:27:40.429 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:40 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:40.061+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:07.842582+0000 front 2024-05-04T03:27:07.842498+0000 (oldest deadline 2024-05-04T03:27:36.642940+0000) 2024-05-04T03:27:40.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:40 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:40.144+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:40.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:40 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:40.144+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:40.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:40 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:40.144+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:40.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:40 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:40.144+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:26:57.824941+0000 front 2024-05-04T03:26:57.824844+0000 (oldest deadline 2024-05-04T03:27:27.825442+0000) 2024-05-04T03:27:40.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:40 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:40.144+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:07.826283+0000 front 2024-05-04T03:27:07.826320+0000 (oldest deadline 2024-05-04T03:27:32.526031+0000) 2024-05-04T03:27:40.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:40 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:40.144+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.227196+0000 front 2024-05-04T03:27:17.227228+0000 (oldest deadline 2024-05-04T03:27:40.127259+0000) 2024-05-04T03:27:40.928 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:40 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:40.665+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:40.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:40 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:40.665+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:40.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:40 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:40.665+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:40.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:40 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:40.665+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:27:40.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:40 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:40.665+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.736972+0000 front 2024-05-04T03:27:17.737072+0000 (oldest deadline 2024-05-04T03:27:40.037084+0000) 2024-05-04T03:27:40.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:40 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:40.637+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:40.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:40 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:40.637+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:40.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:40 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:40.637+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:40.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:40 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:40.637+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:40.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:40 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:40.637+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:11.025148+0000 front 2024-05-04T03:27:11.025194+0000 (oldest deadline 2024-05-04T03:27:34.525456+0000) 2024-05-04T03:27:40.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:40 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:40.637+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:16.826676+0000 front 2024-05-04T03:27:16.826817+0000 (oldest deadline 2024-05-04T03:27:40.326638+0000) 2024-05-04T03:27:41.428 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:41 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:41.014+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:41.429 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:41 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:41.014+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:55.040039+0000 front 2024-05-04T03:26:55.040081+0000 (oldest deadline 2024-05-04T03:27:16.140165+0000) 2024-05-04T03:27:41.429 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:41 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:41.014+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:03.141932+0000 front 2024-05-04T03:27:03.142035+0000 (oldest deadline 2024-05-04T03:27:27.841830+0000) 2024-05-04T03:27:41.429 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:41 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:41.014+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:07.842582+0000 front 2024-05-04T03:27:07.842498+0000 (oldest deadline 2024-05-04T03:27:36.642940+0000) 2024-05-04T03:27:41.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:41 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:41.137+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:41.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:41 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:41.137+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:41.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:41 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:41.137+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:41.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:41 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:41.137+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:26:57.824941+0000 front 2024-05-04T03:26:57.824844+0000 (oldest deadline 2024-05-04T03:27:27.825442+0000) 2024-05-04T03:27:41.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:41 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:41.137+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:07.826283+0000 front 2024-05-04T03:27:07.826320+0000 (oldest deadline 2024-05-04T03:27:32.526031+0000) 2024-05-04T03:27:41.630 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:41 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:41.137+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.227196+0000 front 2024-05-04T03:27:17.227228+0000 (oldest deadline 2024-05-04T03:27:40.127259+0000) 2024-05-04T03:27:41.987 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:41 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:41.685+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:41.988 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:41 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:41.685+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:41.988 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:41 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:41.685+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:41.988 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:41 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:41.685+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:27:41.988 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:41 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:41.685+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.736972+0000 front 2024-05-04T03:27:17.737072+0000 (oldest deadline 2024-05-04T03:27:40.037084+0000) 2024-05-04T03:27:41.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:41 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:41.660+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:41.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:41 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:41.660+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:41.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:41 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:41.660+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:41.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:41 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:41.660+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:41.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:41 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:41.660+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:11.025148+0000 front 2024-05-04T03:27:11.025194+0000 (oldest deadline 2024-05-04T03:27:34.525456+0000) 2024-05-04T03:27:41.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:41 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:41.660+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:16.826676+0000 front 2024-05-04T03:27:16.826817+0000 (oldest deadline 2024-05-04T03:27:40.326638+0000) 2024-05-04T03:27:42.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:42 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:42.127+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:42.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:42 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:42.127+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:42.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:42 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:42.127+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:42.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:42 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:42.127+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:26:57.824941+0000 front 2024-05-04T03:26:57.824844+0000 (oldest deadline 2024-05-04T03:27:27.825442+0000) 2024-05-04T03:27:42.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:42 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:42.127+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:07.826283+0000 front 2024-05-04T03:27:07.826320+0000 (oldest deadline 2024-05-04T03:27:32.526031+0000) 2024-05-04T03:27:42.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:42 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:42.127+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.227196+0000 front 2024-05-04T03:27:17.227228+0000 (oldest deadline 2024-05-04T03:27:40.127259+0000) 2024-05-04T03:27:42.428 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:41 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:41.986+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:47.037873+0000 front 2024-05-04T03:26:47.037899+0000 (oldest deadline 2024-05-04T03:27:08.737891+0000) 2024-05-04T03:27:42.428 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:41 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:41.986+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:55.040039+0000 front 2024-05-04T03:26:55.040081+0000 (oldest deadline 2024-05-04T03:27:16.140165+0000) 2024-05-04T03:27:42.428 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:41 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:41.986+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:03.141932+0000 front 2024-05-04T03:27:03.142035+0000 (oldest deadline 2024-05-04T03:27:27.841830+0000) 2024-05-04T03:27:42.429 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:41 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8[51636]: 2024-05-04T03:27:41.986+0000 7feee6b65700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:07.842582+0000 front 2024-05-04T03:27:07.842498+0000 (oldest deadline 2024-05-04T03:27:36.642940+0000) 2024-05-04T03:27:42.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:42 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:42.682+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:42.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:42 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:42.682+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:42.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:42 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:42.682+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:42.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:42 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:42.682+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:42.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:42 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:42.682+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:11.025148+0000 front 2024-05-04T03:27:11.025194+0000 (oldest deadline 2024-05-04T03:27:34.525456+0000) 2024-05-04T03:27:42.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:42 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:42.682+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:16.826676+0000 front 2024-05-04T03:27:16.826817+0000 (oldest deadline 2024-05-04T03:27:40.326638+0000) 2024-05-04T03:27:43.178 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:42 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:42.690+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:43.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:42 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:42.690+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:43.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:42 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:42.690+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:43.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:42 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:42.690+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:27:43.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:42 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:42.690+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.736972+0000 front 2024-05-04T03:27:17.737072+0000 (oldest deadline 2024-05-04T03:27:40.037084+0000) 2024-05-04T03:27:43.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:43 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:43.096+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:43.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:43 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:43.096+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:43.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:43 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:43.096+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:43.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:43 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:43.096+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:26:57.824941+0000 front 2024-05-04T03:26:57.824844+0000 (oldest deadline 2024-05-04T03:27:27.825442+0000) 2024-05-04T03:27:43.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:43 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:43.096+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:07.826283+0000 front 2024-05-04T03:27:07.826320+0000 (oldest deadline 2024-05-04T03:27:32.526031+0000) 2024-05-04T03:27:43.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:43 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:43.096+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.227196+0000 front 2024-05-04T03:27:17.227228+0000 (oldest deadline 2024-05-04T03:27:40.127259+0000) 2024-05-04T03:27:43.505 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:43 smithi077 podman[68594]: 2024-05-04 03:27:43.230656802 +0000 UTC m=+5.655265437 container died d0855f59db561eb8335d82899cb342b8d662a135494a4491c5facd48ab438c9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430) 2024-05-04T03:27:43.847 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:43 smithi077 podman[68594]: 2024-05-04 03:27:43.602914321 +0000 UTC m=+6.027522963 container cleanup d0855f59db561eb8335d82899cb342b8d662a135494a4491c5facd48ab438c9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-04T03:27:43.847 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:43 smithi077 bash[68594]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8 2024-05-04T03:27:43.847 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:43 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:43.656+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:43.847 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:43 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:43.656+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:43.847 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:43 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:43.656+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:43.847 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:43 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:43.656+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:27:43.847 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:43 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:43.656+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.736972+0000 front 2024-05-04T03:27:17.737072+0000 (oldest deadline 2024-05-04T03:27:40.037084+0000) 2024-05-04T03:27:43.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:43 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:43.639+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:43.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:43 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:43.639+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:43.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:43 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:43.639+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:43.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:43 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:43.639+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:43.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:43 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:43.639+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:11.025148+0000 front 2024-05-04T03:27:11.025194+0000 (oldest deadline 2024-05-04T03:27:34.525456+0000) 2024-05-04T03:27:43.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:43 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:43.639+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:16.826676+0000 front 2024-05-04T03:27:16.826817+0000 (oldest deadline 2024-05-04T03:27:40.326638+0000) 2024-05-04T03:27:44.172 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:43 smithi077 podman[68611]: 2024-05-04 03:27:43.845438997 +0000 UTC m=+0.613125426 container remove d0855f59db561eb8335d82899cb342b8d662a135494a4491c5facd48ab438c9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-04T03:27:44.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:44 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:44.081+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:44.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:44 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:44.081+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:44.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:44 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:44.081+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:44.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:44 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:44.081+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:26:57.824941+0000 front 2024-05-04T03:26:57.824844+0000 (oldest deadline 2024-05-04T03:27:27.825442+0000) 2024-05-04T03:27:44.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:44 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:44.081+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:07.826283+0000 front 2024-05-04T03:27:07.826320+0000 (oldest deadline 2024-05-04T03:27:32.526031+0000) 2024-05-04T03:27:44.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:44 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:44.081+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.227196+0000 front 2024-05-04T03:27:17.227228+0000 (oldest deadline 2024-05-04T03:27:40.127259+0000) 2024-05-04T03:27:44.428 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:44 smithi077 podman[68705]: 2024-05-04 03:27:44.171512277 +0000 UTC m=+0.114224311 container create 2e0795a73f8a875e9694180185285557a4b901cacf721f8b2dff062f6d7696db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-05-04T03:27:44.429 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:44 smithi077 podman[68705]: 2024-05-04 03:27:44.075366253 +0000 UTC m=+0.018078258 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:27:44.429 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:44 smithi077 podman[68705]: 2024-05-04 03:27:44.382195299 +0000 UTC m=+0.324907288 container init 2e0795a73f8a875e9694180185285557a4b901cacf721f8b2dff062f6d7696db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-04T03:27:44.429 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:44 smithi077 podman[68705]: 2024-05-04 03:27:44.385788733 +0000 UTC m=+0.328500729 container start 2e0795a73f8a875e9694180185285557a4b901cacf721f8b2dff062f6d7696db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430) 2024-05-04T03:27:44.690 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:44 smithi077 podman[68705]: 2024-05-04 03:27:44.465536506 +0000 UTC m=+0.408248513 container attach 2e0795a73f8a875e9694180185285557a4b901cacf721f8b2dff062f6d7696db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-04T03:27:44.690 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:44 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:44.671+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:44.690 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:44 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:44.671+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:44.690 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:44 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:44.671+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:44.690 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:44 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:44.671+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:27:44.690 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:44 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:44.671+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.736972+0000 front 2024-05-04T03:27:17.737072+0000 (oldest deadline 2024-05-04T03:27:40.037084+0000) 2024-05-04T03:27:44.961 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:44 smithi077 podman[68705]: 2024-05-04 03:27:44.688859929 +0000 UTC m=+0.631571931 container died 2e0795a73f8a875e9694180185285557a4b901cacf721f8b2dff062f6d7696db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-04T03:27:44.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:44 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:44.626+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:44.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:44 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:44.626+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:44.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:44 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:44.626+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:44.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:44 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:44.626+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:44.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:44 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:44.626+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:11.025148+0000 front 2024-05-04T03:27:11.025194+0000 (oldest deadline 2024-05-04T03:27:34.525456+0000) 2024-05-04T03:27:44.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:44 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:44.626+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:16.826676+0000 front 2024-05-04T03:27:16.826817+0000 (oldest deadline 2024-05-04T03:27:40.326638+0000) 2024-05-04T03:27:45.196 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.8.service' 2024-05-04T03:27:45.259 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:45 smithi077 podman[68731]: 2024-05-04 03:27:45.16898099 +0000 UTC m=+0.473235067 container remove 2e0795a73f8a875e9694180185285557a4b901cacf721f8b2dff062f6d7696db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-04T03:27:45.260 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:45 smithi077 systemd[1]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.8.service: Deactivated successfully. 2024-05-04T03:27:45.260 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:45 smithi077 systemd[1]: Stopped Ceph osd.8 for 4a572aa8-09c3-11ef-bc97-c7b262605968. 2024-05-04T03:27:45.260 INFO:journalctl@ceph.osd.8.smithi077.stdout:May 04 03:27:45 smithi077 systemd[1]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.8.service: Consumed 58.950s CPU time. 2024-05-04T03:27:45.275 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:27:45.275 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-04T03:27:45.275 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-04T03:27:45.276 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.9 2024-05-04T03:27:45.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:45 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:45.074+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:45.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:45 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:45.074+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:45.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:45 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:45.074+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:45.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:45 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:45.074+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:26:57.824941+0000 front 2024-05-04T03:26:57.824844+0000 (oldest deadline 2024-05-04T03:27:27.825442+0000) 2024-05-04T03:27:45.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:45 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:45.074+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:07.826283+0000 front 2024-05-04T03:27:07.826320+0000 (oldest deadline 2024-05-04T03:27:32.526031+0000) 2024-05-04T03:27:45.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:45 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:45.074+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.227196+0000 front 2024-05-04T03:27:17.227228+0000 (oldest deadline 2024-05-04T03:27:40.127259+0000) 2024-05-04T03:27:45.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:45 smithi001 systemd[1]: Stopping Ceph osd.9 for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:27:45.879 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:45 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:45.585+0000 7fc0d8ae6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:27:45.880 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:45 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:45.585+0000 7fc0d8ae6700 -1 osd.9 86 *** Got signal Terminated *** 2024-05-04T03:27:45.880 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:45 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:45.585+0000 7fc0d8ae6700 -1 osd.9 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-04T03:27:45.928 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:45 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:45.652+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:45.928 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:45 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:45.652+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:45.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:45 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:45.652+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:45.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:45 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:45.652+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:27:45.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:45 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:45.652+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.736972+0000 front 2024-05-04T03:27:17.737072+0000 (oldest deadline 2024-05-04T03:27:40.037084+0000) 2024-05-04T03:27:45.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:45 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:45.601+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:45.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:45 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:45.601+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:45.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:45 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:45.601+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:45.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:45 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:45.601+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:45.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:45 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:45.601+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:11.025148+0000 front 2024-05-04T03:27:11.025194+0000 (oldest deadline 2024-05-04T03:27:34.525456+0000) 2024-05-04T03:27:45.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:45 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:45.601+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:16.826676+0000 front 2024-05-04T03:27:16.826817+0000 (oldest deadline 2024-05-04T03:27:40.326638+0000) 2024-05-04T03:27:46.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:46 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:46.028+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:46.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:46 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:46.028+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:46.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:46 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:46.028+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:46.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:46 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:46.028+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:26:57.824941+0000 front 2024-05-04T03:26:57.824844+0000 (oldest deadline 2024-05-04T03:27:27.825442+0000) 2024-05-04T03:27:46.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:46 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:46.028+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:07.826283+0000 front 2024-05-04T03:27:07.826320+0000 (oldest deadline 2024-05-04T03:27:32.526031+0000) 2024-05-04T03:27:46.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:46 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:46.028+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.227196+0000 front 2024-05-04T03:27:17.227228+0000 (oldest deadline 2024-05-04T03:27:40.127259+0000) 2024-05-04T03:27:46.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:46 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:46.571+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:46.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:46 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:46.571+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:46.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:46 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:46.571+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:46.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:46 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:46.571+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:46.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:46 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:46.571+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:11.025148+0000 front 2024-05-04T03:27:11.025194+0000 (oldest deadline 2024-05-04T03:27:34.525456+0000) 2024-05-04T03:27:46.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:46 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:46.571+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:16.826676+0000 front 2024-05-04T03:27:16.826817+0000 (oldest deadline 2024-05-04T03:27:40.326638+0000) 2024-05-04T03:27:47.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:46 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:46.698+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:47.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:46 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:46.698+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:47.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:46 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:46.698+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:47.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:46 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:46.698+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:27:47.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:46 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:46.698+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.736972+0000 front 2024-05-04T03:27:17.737072+0000 (oldest deadline 2024-05-04T03:27:40.037084+0000) 2024-05-04T03:27:47.379 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:47 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:47.004+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:47.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:47 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:47.004+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:47.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:47 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:47.004+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:47.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:47 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:47.004+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:26:57.824941+0000 front 2024-05-04T03:26:57.824844+0000 (oldest deadline 2024-05-04T03:27:27.825442+0000) 2024-05-04T03:27:47.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:47 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:47.004+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:07.826283+0000 front 2024-05-04T03:27:07.826320+0000 (oldest deadline 2024-05-04T03:27:32.526031+0000) 2024-05-04T03:27:47.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:47 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:47.004+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.227196+0000 front 2024-05-04T03:27:17.227228+0000 (oldest deadline 2024-05-04T03:27:40.127259+0000) 2024-05-04T03:27:47.898 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:47 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:47.577+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:47.898 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:47 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:47.577+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:47.898 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:47 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:47.577+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:47.898 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:47 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:47.577+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:47.899 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:47 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:47.577+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:11.025148+0000 front 2024-05-04T03:27:11.025194+0000 (oldest deadline 2024-05-04T03:27:34.525456+0000) 2024-05-04T03:27:47.899 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:47 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:47.577+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:16.826676+0000 front 2024-05-04T03:27:16.826817+0000 (oldest deadline 2024-05-04T03:27:40.326638+0000) 2024-05-04T03:27:47.928 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:47 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:47.665+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:47.928 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:47 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:47.665+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:47.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:47 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:47.665+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:47.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:47 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:47.665+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:27:47.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:47 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:47.665+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.736972+0000 front 2024-05-04T03:27:17.737072+0000 (oldest deadline 2024-05-04T03:27:40.037084+0000) 2024-05-04T03:27:47.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:47 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:47.665+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:25.338187+0000 front 2024-05-04T03:27:25.338202+0000 (oldest deadline 2024-05-04T03:27:47.638115+0000) 2024-05-04T03:27:48.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:48 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:48.017+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:48.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:48 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:48.017+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:48.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:48 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:48.017+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:48.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:48 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:48.017+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:26:57.824941+0000 front 2024-05-04T03:26:57.824844+0000 (oldest deadline 2024-05-04T03:27:27.825442+0000) 2024-05-04T03:27:48.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:48 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:48.017+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:07.826283+0000 front 2024-05-04T03:27:07.826320+0000 (oldest deadline 2024-05-04T03:27:32.526031+0000) 2024-05-04T03:27:48.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:48 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:48.017+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.227196+0000 front 2024-05-04T03:27:17.227228+0000 (oldest deadline 2024-05-04T03:27:40.127259+0000) 2024-05-04T03:27:48.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:48 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:48.673+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:48.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:48 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:48.673+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:48.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:48 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:48.673+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:48.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:48 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:48.673+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:27:48.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:48 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:48.673+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.736972+0000 front 2024-05-04T03:27:17.737072+0000 (oldest deadline 2024-05-04T03:27:40.037084+0000) 2024-05-04T03:27:48.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:48 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:48.673+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:25.338187+0000 front 2024-05-04T03:27:25.338202+0000 (oldest deadline 2024-05-04T03:27:47.638115+0000) 2024-05-04T03:27:48.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:48 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:48.543+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:48.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:48 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:48.543+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:48.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:48 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:48.543+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:48.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:48 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:48.543+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:48.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:48 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:48.543+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:11.025148+0000 front 2024-05-04T03:27:11.025194+0000 (oldest deadline 2024-05-04T03:27:34.525456+0000) 2024-05-04T03:27:48.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:48 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:48.543+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:16.826676+0000 front 2024-05-04T03:27:16.826817+0000 (oldest deadline 2024-05-04T03:27:40.326638+0000) 2024-05-04T03:27:49.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:49 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:49.037+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:49.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:49 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:49.037+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:49.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:49 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:49.037+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:49.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:49 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:49.037+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:26:57.824941+0000 front 2024-05-04T03:26:57.824844+0000 (oldest deadline 2024-05-04T03:27:27.825442+0000) 2024-05-04T03:27:49.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:49 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:49.037+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:07.826283+0000 front 2024-05-04T03:27:07.826320+0000 (oldest deadline 2024-05-04T03:27:32.526031+0000) 2024-05-04T03:27:49.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:49 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:49.037+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.227196+0000 front 2024-05-04T03:27:17.227228+0000 (oldest deadline 2024-05-04T03:27:40.127259+0000) 2024-05-04T03:27:49.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:49 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:49.037+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:25.328582+0000 front 2024-05-04T03:27:25.328603+0000 (oldest deadline 2024-05-04T03:27:48.228542+0000) 2024-05-04T03:27:49.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:49 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:49.661+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:49.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:49 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:49.661+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:49.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:49 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:49.661+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:49.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:49 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:49.661+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:27:49.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:49 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:49.661+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.736972+0000 front 2024-05-04T03:27:17.737072+0000 (oldest deadline 2024-05-04T03:27:40.037084+0000) 2024-05-04T03:27:49.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:49 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:49.661+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:25.338187+0000 front 2024-05-04T03:27:25.338202+0000 (oldest deadline 2024-05-04T03:27:47.638115+0000) 2024-05-04T03:27:49.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:49 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:49.550+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:49.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:49 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:49.550+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:49.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:49 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:49.550+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:49.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:49 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:49.550+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:49.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:49 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:49.550+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:11.025148+0000 front 2024-05-04T03:27:11.025194+0000 (oldest deadline 2024-05-04T03:27:34.525456+0000) 2024-05-04T03:27:49.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:49 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:49.550+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:16.826676+0000 front 2024-05-04T03:27:16.826817+0000 (oldest deadline 2024-05-04T03:27:40.326638+0000) 2024-05-04T03:27:49.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:49 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:49.550+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:23.827631+0000 front 2024-05-04T03:27:23.827663+0000 (oldest deadline 2024-05-04T03:27:49.127592+0000) 2024-05-04T03:27:50.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:50 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:50.055+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.921870+0000 front 2024-05-04T03:26:37.922082+0000 (oldest deadline 2024-05-04T03:27:00.222044+0000) 2024-05-04T03:27:50.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:50 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:50.055+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:44.923191+0000 front 2024-05-04T03:26:44.923279+0000 (oldest deadline 2024-05-04T03:27:07.823329+0000) 2024-05-04T03:27:50.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:50 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:50.055+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:52.524650+0000 front 2024-05-04T03:26:52.524505+0000 (oldest deadline 2024-05-04T03:27:17.824371+0000) 2024-05-04T03:27:50.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:50 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:50.055+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:26:57.824941+0000 front 2024-05-04T03:26:57.824844+0000 (oldest deadline 2024-05-04T03:27:27.825442+0000) 2024-05-04T03:27:50.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:50 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:50.055+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:07.826283+0000 front 2024-05-04T03:27:07.826320+0000 (oldest deadline 2024-05-04T03:27:32.526031+0000) 2024-05-04T03:27:50.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:50 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:50.055+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.227196+0000 front 2024-05-04T03:27:17.227228+0000 (oldest deadline 2024-05-04T03:27:40.127259+0000) 2024-05-04T03:27:50.381 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:50 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9[62293]: 2024-05-04T03:27:50.055+0000 7fc0d509c700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:25.328582+0000 front 2024-05-04T03:27:25.328603+0000 (oldest deadline 2024-05-04T03:27:48.228542+0000) 2024-05-04T03:27:50.928 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:50 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:50.642+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:50.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:50 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:50.642+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:50.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:50 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:50.642+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:50.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:50 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:50.642+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:27:50.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:50 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:50.642+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.736972+0000 front 2024-05-04T03:27:17.737072+0000 (oldest deadline 2024-05-04T03:27:40.037084+0000) 2024-05-04T03:27:50.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:50 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:50.642+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:25.338187+0000 front 2024-05-04T03:27:25.338202+0000 (oldest deadline 2024-05-04T03:27:47.638115+0000) 2024-05-04T03:27:50.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:50 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:50.526+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:50.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:50 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:50.526+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:50.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:50 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:50.526+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:50.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:50 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:50.526+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:50.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:50 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:50.526+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:11.025148+0000 front 2024-05-04T03:27:11.025194+0000 (oldest deadline 2024-05-04T03:27:34.525456+0000) 2024-05-04T03:27:50.998 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:50 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:50.526+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:16.826676+0000 front 2024-05-04T03:27:16.826817+0000 (oldest deadline 2024-05-04T03:27:40.326638+0000) 2024-05-04T03:27:50.998 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:50 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:50.526+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:23.827631+0000 front 2024-05-04T03:27:23.827663+0000 (oldest deadline 2024-05-04T03:27:49.127592+0000) 2024-05-04T03:27:51.413 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:51 smithi001 podman[113269]: 2024-05-04 03:27:51.138336909 +0000 UTC m=+5.664227175 container died a1fa31ce26873895fc41addadd9fcb8289ae329e89b2b82768dd8e78cb116457 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-04T03:27:51.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:51 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:51.485+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:51.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:51 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:51.485+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:51.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:51 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:51.485+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:51.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:51 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:51.485+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:51.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:51 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:51.485+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:11.025148+0000 front 2024-05-04T03:27:11.025194+0000 (oldest deadline 2024-05-04T03:27:34.525456+0000) 2024-05-04T03:27:51.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:51 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:51.485+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:16.826676+0000 front 2024-05-04T03:27:16.826817+0000 (oldest deadline 2024-05-04T03:27:40.326638+0000) 2024-05-04T03:27:51.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:51 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:51.485+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:23.827631+0000 front 2024-05-04T03:27:23.827663+0000 (oldest deadline 2024-05-04T03:27:49.127592+0000) 2024-05-04T03:27:52.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:51 smithi001 podman[113269]: 2024-05-04 03:27:51.684748293 +0000 UTC m=+6.210638558 container cleanup a1fa31ce26873895fc41addadd9fcb8289ae329e89b2b82768dd8e78cb116457 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-04T03:27:52.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:51 smithi001 bash[113269]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9 2024-05-04T03:27:52.130 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:51 smithi001 podman[113286]: 2024-05-04 03:27:51.869638249 +0000 UTC m=+0.725328189 container remove a1fa31ce26873895fc41addadd9fcb8289ae329e89b2b82768dd8e78cb116457 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-04T03:27:52.178 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:51 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:51.688+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:52.178 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:51 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:51.688+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:52.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:51 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:51.688+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:52.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:51 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:51.688+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:27:52.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:51 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:51.688+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.736972+0000 front 2024-05-04T03:27:17.737072+0000 (oldest deadline 2024-05-04T03:27:40.037084+0000) 2024-05-04T03:27:52.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:51 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:51.688+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:25.338187+0000 front 2024-05-04T03:27:25.338202+0000 (oldest deadline 2024-05-04T03:27:47.638115+0000) 2024-05-04T03:27:52.404 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:52 smithi001 podman[113376]: 2024-05-04 03:27:52.29543922 +0000 UTC m=+0.104954694 container create 4059e3abafdbf3ef77f21b5e7dca264eac2b496aaf15993c04a4781d73203668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-04T03:27:52.405 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:52 smithi001 podman[113376]: 2024-05-04 03:27:52.208739705 +0000 UTC m=+0.018255200 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:27:52.837 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:52 smithi001 podman[113376]: 2024-05-04 03:27:52.532066887 +0000 UTC m=+0.341582376 container init 4059e3abafdbf3ef77f21b5e7dca264eac2b496aaf15993c04a4781d73203668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:27:52.837 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:52 smithi001 podman[113376]: 2024-05-04 03:27:52.535547912 +0000 UTC m=+0.345063396 container start 4059e3abafdbf3ef77f21b5e7dca264eac2b496aaf15993c04a4781d73203668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-04T03:27:52.837 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:52 smithi001 podman[113376]: 2024-05-04 03:27:52.589356283 +0000 UTC m=+0.398871759 container attach 4059e3abafdbf3ef77f21b5e7dca264eac2b496aaf15993c04a4781d73203668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.schema-version=1.0) 2024-05-04T03:27:52.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:52 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:52.495+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:52.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:52 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:52.495+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:52.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:52 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:52.495+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:52.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:52 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:52.495+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:52.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:52 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:52.495+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:11.025148+0000 front 2024-05-04T03:27:11.025194+0000 (oldest deadline 2024-05-04T03:27:34.525456+0000) 2024-05-04T03:27:52.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:52 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:52.495+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:16.826676+0000 front 2024-05-04T03:27:16.826817+0000 (oldest deadline 2024-05-04T03:27:40.326638+0000) 2024-05-04T03:27:52.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:52 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:52.495+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:23.827631+0000 front 2024-05-04T03:27:23.827663+0000 (oldest deadline 2024-05-04T03:27:49.127592+0000) 2024-05-04T03:27:53.127 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:52 smithi001 podman[113376]: 2024-05-04 03:27:52.835843193 +0000 UTC m=+0.645358681 container died 4059e3abafdbf3ef77f21b5e7dca264eac2b496aaf15993c04a4781d73203668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS) 2024-05-04T03:27:53.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:52 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:52.697+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:53.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:52 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:52.697+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:53.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:52 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:52.697+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:53.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:52 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:52.697+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:27:53.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:52 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:52.697+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.736972+0000 front 2024-05-04T03:27:17.737072+0000 (oldest deadline 2024-05-04T03:27:40.037084+0000) 2024-05-04T03:27:53.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:52 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:52.697+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:25.338187+0000 front 2024-05-04T03:27:25.338202+0000 (oldest deadline 2024-05-04T03:27:47.638115+0000) 2024-05-04T03:27:53.380 INFO:journalctl@ceph.osd.9.smithi001.stdout:May 04 03:27:53 smithi001 podman[113402]: 2024-05-04 03:27:53.367522009 +0000 UTC m=+0.520889286 container remove 4059e3abafdbf3ef77f21b5e7dca264eac2b496aaf15993c04a4781d73203668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-04T03:27:53.386 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.9.service' 2024-05-04T03:27:53.425 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:27:53.425 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-04T03:27:53.425 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-04T03:27:53.425 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.10 2024-05-04T03:27:53.738 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:53 smithi072 systemd[1]: Stopping Ceph osd.10 for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:27:53.738 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:53 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:53.476+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:53.738 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:53 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:53.476+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:53.738 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:53 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:53.476+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:53.739 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:53 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:53.476+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:53.739 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:53 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:53.476+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:11.025148+0000 front 2024-05-04T03:27:11.025194+0000 (oldest deadline 2024-05-04T03:27:34.525456+0000) 2024-05-04T03:27:53.739 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:53 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:53.476+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:16.826676+0000 front 2024-05-04T03:27:16.826817+0000 (oldest deadline 2024-05-04T03:27:40.326638+0000) 2024-05-04T03:27:53.739 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:53 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:53.476+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:23.827631+0000 front 2024-05-04T03:27:23.827663+0000 (oldest deadline 2024-05-04T03:27:49.127592+0000) 2024-05-04T03:27:53.996 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:53 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:53.735+0000 7fe553ba9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:27:53.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:53 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:53.735+0000 7fe553ba9700 -1 osd.10 86 *** Got signal Terminated *** 2024-05-04T03:27:53.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:53 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:53.735+0000 7fe553ba9700 -1 osd.10 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-04T03:27:54.178 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:53 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:53.729+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:54.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:53 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:53.729+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:54.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:53 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:53.729+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:54.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:53 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:53.729+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:27:54.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:53 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:53.729+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.736972+0000 front 2024-05-04T03:27:17.737072+0000 (oldest deadline 2024-05-04T03:27:40.037084+0000) 2024-05-04T03:27:54.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:53 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:53.729+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:25.338187+0000 front 2024-05-04T03:27:25.338202+0000 (oldest deadline 2024-05-04T03:27:47.638115+0000) 2024-05-04T03:27:54.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:54 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:54.434+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:54.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:54 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:54.434+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:54.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:54 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:54.434+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:54.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:54 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:54.434+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:54.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:54 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:54.434+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:11.025148+0000 front 2024-05-04T03:27:11.025194+0000 (oldest deadline 2024-05-04T03:27:34.525456+0000) 2024-05-04T03:27:54.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:54 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:54.434+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:16.826676+0000 front 2024-05-04T03:27:16.826817+0000 (oldest deadline 2024-05-04T03:27:40.326638+0000) 2024-05-04T03:27:54.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:54 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:54.434+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:23.827631+0000 front 2024-05-04T03:27:23.827663+0000 (oldest deadline 2024-05-04T03:27:49.127592+0000) 2024-05-04T03:27:55.178 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:54 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:54.711+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:55.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:54 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:54.711+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:55.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:54 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:54.711+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:55.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:54 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:54.711+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:27:55.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:54 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:54.711+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.736972+0000 front 2024-05-04T03:27:17.737072+0000 (oldest deadline 2024-05-04T03:27:40.037084+0000) 2024-05-04T03:27:55.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:54 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:54.711+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:25.338187+0000 front 2024-05-04T03:27:25.338202+0000 (oldest deadline 2024-05-04T03:27:47.638115+0000) 2024-05-04T03:27:55.746 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:55 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:55.451+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:55.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:55 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:55.451+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:55.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:55 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:55.451+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:55.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:55 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:55.451+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:55.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:55 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:55.451+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:11.025148+0000 front 2024-05-04T03:27:11.025194+0000 (oldest deadline 2024-05-04T03:27:34.525456+0000) 2024-05-04T03:27:55.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:55 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:55.451+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:16.826676+0000 front 2024-05-04T03:27:16.826817+0000 (oldest deadline 2024-05-04T03:27:40.326638+0000) 2024-05-04T03:27:55.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:55 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:55.451+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:23.827631+0000 front 2024-05-04T03:27:23.827663+0000 (oldest deadline 2024-05-04T03:27:49.127592+0000) 2024-05-04T03:27:56.178 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:55 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:55.754+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:56.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:55 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:55.754+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:56.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:55 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:55.754+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:56.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:55 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:55.754+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:27:56.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:55 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:55.754+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.736972+0000 front 2024-05-04T03:27:17.737072+0000 (oldest deadline 2024-05-04T03:27:40.037084+0000) 2024-05-04T03:27:56.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:55 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:55.754+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:25.338187+0000 front 2024-05-04T03:27:25.338202+0000 (oldest deadline 2024-05-04T03:27:47.638115+0000) 2024-05-04T03:27:56.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:56 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:56.456+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:56.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:56 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:56.456+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:56.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:56 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:56.456+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:56.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:56 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:56.456+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:56.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:56 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:56.456+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:11.025148+0000 front 2024-05-04T03:27:11.025194+0000 (oldest deadline 2024-05-04T03:27:34.525456+0000) 2024-05-04T03:27:56.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:56 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:56.456+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:16.826676+0000 front 2024-05-04T03:27:16.826817+0000 (oldest deadline 2024-05-04T03:27:40.326638+0000) 2024-05-04T03:27:56.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:56 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:56.456+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:23.827631+0000 front 2024-05-04T03:27:23.827663+0000 (oldest deadline 2024-05-04T03:27:49.127592+0000) 2024-05-04T03:27:57.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:56 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:56.758+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:57.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:56 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:56.758+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:57.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:56 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:56.758+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:57.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:56 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:56.758+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:27:57.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:56 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:56.758+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.736972+0000 front 2024-05-04T03:27:17.737072+0000 (oldest deadline 2024-05-04T03:27:40.037084+0000) 2024-05-04T03:27:57.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:56 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:56.758+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:25.338187+0000 front 2024-05-04T03:27:25.338202+0000 (oldest deadline 2024-05-04T03:27:47.638115+0000) 2024-05-04T03:27:57.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:57 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:57.438+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:57.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:57 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:57.438+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:57.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:57 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:57.438+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:57.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:57 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:57.438+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:57.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:57 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:57.438+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:11.025148+0000 front 2024-05-04T03:27:11.025194+0000 (oldest deadline 2024-05-04T03:27:34.525456+0000) 2024-05-04T03:27:57.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:57 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:57.438+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:16.826676+0000 front 2024-05-04T03:27:16.826817+0000 (oldest deadline 2024-05-04T03:27:40.326638+0000) 2024-05-04T03:27:57.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:57 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:57.438+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:23.827631+0000 front 2024-05-04T03:27:23.827663+0000 (oldest deadline 2024-05-04T03:27:49.127592+0000) 2024-05-04T03:27:58.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:57 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:57.801+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:58.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:57 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:57.801+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:58.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:57 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:57.801+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:58.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:57 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:57.801+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:27:58.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:57 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:57.801+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.736972+0000 front 2024-05-04T03:27:17.737072+0000 (oldest deadline 2024-05-04T03:27:40.037084+0000) 2024-05-04T03:27:58.180 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:57 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:57.801+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:25.338187+0000 front 2024-05-04T03:27:25.338202+0000 (oldest deadline 2024-05-04T03:27:47.638115+0000) 2024-05-04T03:27:58.180 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:57 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:57.801+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6820 osd.7 since back 2024-05-04T03:27:33.539125+0000 front 2024-05-04T03:27:33.539045+0000 (oldest deadline 2024-05-04T03:27:57.638814+0000) 2024-05-04T03:27:58.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:58 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:58.480+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:37.418505+0000 front 2024-05-04T03:26:37.418577+0000 (oldest deadline 2024-05-04T03:27:00.318510+0000) 2024-05-04T03:27:58.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:58 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:58.480+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:43.620903+0000 front 2024-05-04T03:26:43.620875+0000 (oldest deadline 2024-05-04T03:27:08.320981+0000) 2024-05-04T03:27:58.747 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:58 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:58.480+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:53.022263+0000 front 2024-05-04T03:26:53.022222+0000 (oldest deadline 2024-05-04T03:27:17.722075+0000) 2024-05-04T03:27:58.748 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:58 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:58.480+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-05-04T03:27:02.224215+0000 front 2024-05-04T03:27:02.224275+0000 (oldest deadline 2024-05-04T03:27:28.124095+0000) 2024-05-04T03:27:58.748 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:58 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:58.480+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:11.025148+0000 front 2024-05-04T03:27:11.025194+0000 (oldest deadline 2024-05-04T03:27:34.525456+0000) 2024-05-04T03:27:58.748 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:58 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:58.480+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:16.826676+0000 front 2024-05-04T03:27:16.826817+0000 (oldest deadline 2024-05-04T03:27:40.326638+0000) 2024-05-04T03:27:58.748 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:58 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10[58070]: 2024-05-04T03:27:58.480+0000 7fe55015f700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:23.827631+0000 front 2024-05-04T03:27:23.827663+0000 (oldest deadline 2024-05-04T03:27:49.127592+0000) 2024-05-04T03:27:59.178 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:58 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:58.844+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:27:59.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:58 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:58.844+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:27:59.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:58 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:58.844+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:27:59.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:58 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:58.844+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:27:59.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:58 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:58.844+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.736972+0000 front 2024-05-04T03:27:17.737072+0000 (oldest deadline 2024-05-04T03:27:40.037084+0000) 2024-05-04T03:27:59.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:58 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:58.844+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:25.338187+0000 front 2024-05-04T03:27:25.338202+0000 (oldest deadline 2024-05-04T03:27:47.638115+0000) 2024-05-04T03:27:59.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:58 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:58.844+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6820 osd.7 since back 2024-05-04T03:27:33.539125+0000 front 2024-05-04T03:27:33.539045+0000 (oldest deadline 2024-05-04T03:27:57.638814+0000) 2024-05-04T03:27:59.506 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:59 smithi072 podman[68087]: 2024-05-04 03:27:59.252560005 +0000 UTC m=+5.638474096 container died 96b02b0a9cd62137c57dadb03705d9ad6b9ddb06069e66f46f49100b28b8eb85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0) 2024-05-04T03:27:59.802 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:59 smithi072 podman[68087]: 2024-05-04 03:27:59.596378667 +0000 UTC m=+5.982292760 container cleanup 96b02b0a9cd62137c57dadb03705d9ad6b9ddb06069e66f46f49100b28b8eb85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-04T03:27:59.802 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:59 smithi072 bash[68087]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10 2024-05-04T03:28:00.178 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:59 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:59.874+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:28:00.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:59 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:59.874+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:28:00.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:59 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:59.874+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:28:00.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:59 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:59.874+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:28:00.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:59 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:59.874+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.736972+0000 front 2024-05-04T03:27:17.737072+0000 (oldest deadline 2024-05-04T03:27:40.037084+0000) 2024-05-04T03:28:00.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:59 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:59.874+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:25.338187+0000 front 2024-05-04T03:27:25.338202+0000 (oldest deadline 2024-05-04T03:27:47.638115+0000) 2024-05-04T03:28:00.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:27:59 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:27:59.874+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6820 osd.7 since back 2024-05-04T03:27:33.539125+0000 front 2024-05-04T03:27:33.539045+0000 (oldest deadline 2024-05-04T03:27:57.638814+0000) 2024-05-04T03:28:00.211 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:27:59 smithi072 podman[68105]: 2024-05-04 03:27:59.800124931 +0000 UTC m=+0.545408009 container remove 96b02b0a9cd62137c57dadb03705d9ad6b9ddb06069e66f46f49100b28b8eb85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-05-04T03:28:00.497 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:28:00 smithi072 podman[68197]: 2024-05-04 03:28:00.210121699 +0000 UTC m=+0.114286536 container create c1cf21d03f0dd3fb6791ab26ec4f67de0a6c77d0f6ea2749224ee0c18741c55e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-04T03:28:00.497 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:28:00 smithi072 podman[68197]: 2024-05-04 03:28:00.113897751 +0000 UTC m=+0.018062605 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:28:00.497 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:28:00 smithi072 podman[68197]: 2024-05-04 03:28:00.43881666 +0000 UTC m=+0.342981498 container init c1cf21d03f0dd3fb6791ab26ec4f67de0a6c77d0f6ea2749224ee0c18741c55e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:28:00.497 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:28:00 smithi072 podman[68197]: 2024-05-04 03:28:00.441814858 +0000 UTC m=+0.345979696 container start c1cf21d03f0dd3fb6791ab26ec4f67de0a6c77d0f6ea2749224ee0c18741c55e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-04T03:28:00.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:28:00 smithi072 podman[68197]: 2024-05-04 03:28:00.531365559 +0000 UTC m=+0.435530423 container attach c1cf21d03f0dd3fb6791ab26ec4f67de0a6c77d0f6ea2749224ee0c18741c55e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-04T03:28:00.997 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 04 03:28:00 smithi072 podman[68197]: 2024-05-04 03:28:00.731503678 +0000 UTC m=+0.635668516 container died c1cf21d03f0dd3fb6791ab26ec4f67de0a6c77d0f6ea2749224ee0c18741c55e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True) 2024-05-04T03:28:01.178 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:00 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:00.871+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:28:01.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:00 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:00.871+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:28:01.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:00 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:00.871+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:28:01.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:00 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:00.871+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:28:01.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:00 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:00.871+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.736972+0000 front 2024-05-04T03:27:17.737072+0000 (oldest deadline 2024-05-04T03:27:40.037084+0000) 2024-05-04T03:28:01.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:00 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:00.871+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:25.338187+0000 front 2024-05-04T03:27:25.338202+0000 (oldest deadline 2024-05-04T03:27:47.638115+0000) 2024-05-04T03:28:01.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:00 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:00.871+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6820 osd.7 since back 2024-05-04T03:27:33.539125+0000 front 2024-05-04T03:27:33.539045+0000 (oldest deadline 2024-05-04T03:27:57.638814+0000) 2024-05-04T03:28:01.340 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.10.service' 2024-05-04T03:28:01.378 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:28:01.379 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-04T03:28:01.379 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-04T03:28:01.379 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.11 2024-05-04T03:28:01.678 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:01 smithi077 systemd[1]: Stopping Ceph osd.11 for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:28:02.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:01 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:01.796+0000 7fa0b6b6a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:28:02.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:01 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:01.796+0000 7fa0b6b6a700 -1 osd.11 86 *** Got signal Terminated *** 2024-05-04T03:28:02.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:01 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:01.796+0000 7fa0b6b6a700 -1 osd.11 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-04T03:28:02.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:01 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:01.837+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:28:02.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:01 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:01.837+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:28:02.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:01 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:01.837+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:28:02.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:01 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:01.837+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:28:02.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:01 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:01.837+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.736972+0000 front 2024-05-04T03:27:17.737072+0000 (oldest deadline 2024-05-04T03:27:40.037084+0000) 2024-05-04T03:28:02.180 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:01 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:01.837+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:25.338187+0000 front 2024-05-04T03:27:25.338202+0000 (oldest deadline 2024-05-04T03:27:47.638115+0000) 2024-05-04T03:28:02.180 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:01 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:01.837+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6820 osd.7 since back 2024-05-04T03:27:33.539125+0000 front 2024-05-04T03:27:33.539045+0000 (oldest deadline 2024-05-04T03:27:57.638814+0000) 2024-05-04T03:28:03.178 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:02 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:02.884+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:28:03.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:02 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:02.884+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:28:03.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:02 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:02.884+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:28:03.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:02 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:02.884+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:28:03.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:02 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:02.884+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.736972+0000 front 2024-05-04T03:27:17.737072+0000 (oldest deadline 2024-05-04T03:27:40.037084+0000) 2024-05-04T03:28:03.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:02 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:02.884+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:25.338187+0000 front 2024-05-04T03:27:25.338202+0000 (oldest deadline 2024-05-04T03:27:47.638115+0000) 2024-05-04T03:28:03.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:02 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:02.884+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6820 osd.7 since back 2024-05-04T03:27:33.539125+0000 front 2024-05-04T03:27:33.539045+0000 (oldest deadline 2024-05-04T03:27:57.638814+0000) 2024-05-04T03:28:04.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:03 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:03.862+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:28:04.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:03 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:03.862+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:28:04.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:03 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:03.862+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:28:04.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:03 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:03.862+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:28:04.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:03 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:03.862+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.736972+0000 front 2024-05-04T03:27:17.737072+0000 (oldest deadline 2024-05-04T03:27:40.037084+0000) 2024-05-04T03:28:04.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:03 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:03.862+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:25.338187+0000 front 2024-05-04T03:27:25.338202+0000 (oldest deadline 2024-05-04T03:27:47.638115+0000) 2024-05-04T03:28:04.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:03 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:03.862+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6820 osd.7 since back 2024-05-04T03:27:33.539125+0000 front 2024-05-04T03:27:33.539045+0000 (oldest deadline 2024-05-04T03:27:57.638814+0000) 2024-05-04T03:28:05.178 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:04 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:04.851+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:28:05.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:04 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:04.851+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:28:05.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:04 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:04.851+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:28:05.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:04 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:04.851+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:28:05.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:04 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:04.851+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.736972+0000 front 2024-05-04T03:27:17.737072+0000 (oldest deadline 2024-05-04T03:27:40.037084+0000) 2024-05-04T03:28:05.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:04 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:04.851+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:25.338187+0000 front 2024-05-04T03:27:25.338202+0000 (oldest deadline 2024-05-04T03:27:47.638115+0000) 2024-05-04T03:28:05.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:04 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:04.851+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6820 osd.7 since back 2024-05-04T03:27:33.539125+0000 front 2024-05-04T03:27:33.539045+0000 (oldest deadline 2024-05-04T03:27:57.638814+0000) 2024-05-04T03:28:06.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:05 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:05.853+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-05-04T03:26:39.330239+0000 front 2024-05-04T03:26:39.330113+0000 (oldest deadline 2024-05-04T03:27:01.630009+0000) 2024-05-04T03:28:06.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:05 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:05.853+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-04T03:26:46.132707+0000 front 2024-05-04T03:26:46.132566+0000 (oldest deadline 2024-05-04T03:27:10.832399+0000) 2024-05-04T03:28:06.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:05 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:05.853+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6804 osd.2 since back 2024-05-04T03:26:54.333459+0000 front 2024-05-04T03:26:54.333426+0000 (oldest deadline 2024-05-04T03:27:19.534178+0000) 2024-05-04T03:28:06.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:05 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:05.853+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-04T03:27:05.936106+0000 front 2024-05-04T03:27:05.936027+0000 (oldest deadline 2024-05-04T03:27:31.835932+0000) 2024-05-04T03:28:06.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:05 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:05.853+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6812 osd.5 since back 2024-05-04T03:27:17.736972+0000 front 2024-05-04T03:27:17.737072+0000 (oldest deadline 2024-05-04T03:27:40.037084+0000) 2024-05-04T03:28:06.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:05 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:05.853+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-05-04T03:27:25.338187+0000 front 2024-05-04T03:27:25.338202+0000 (oldest deadline 2024-05-04T03:27:47.638115+0000) 2024-05-04T03:28:06.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:05 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:05.853+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.72:6820 osd.7 since back 2024-05-04T03:27:33.539125+0000 front 2024-05-04T03:27:33.539045+0000 (oldest deadline 2024-05-04T03:27:57.638814+0000) 2024-05-04T03:28:06.179 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:05 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11[58372]: 2024-05-04T03:28:05.853+0000 7fa0af118700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.77:6820 osd.8 since back 2024-05-04T03:27:41.540464+0000 front 2024-05-04T03:27:41.540554+0000 (oldest deadline 2024-05-04T03:28:05.040427+0000) 2024-05-04T03:28:07.482 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:07 smithi077 podman[68822]: 2024-05-04 03:28:07.209248378 +0000 UTC m=+5.636533361 container died 0973570496ae5a767f7f3729b9d036c9ae2b546060421fda7776e89b9f568e32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-04T03:28:07.831 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:07 smithi077 podman[68822]: 2024-05-04 03:28:07.579311717 +0000 UTC m=+6.006596711 container cleanup 0973570496ae5a767f7f3729b9d036c9ae2b546060421fda7776e89b9f568e32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-04T03:28:07.832 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:07 smithi077 bash[68822]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11 2024-05-04T03:28:08.165 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:07 smithi077 podman[68839]: 2024-05-04 03:28:07.830371487 +0000 UTC m=+0.612879770 container remove 0973570496ae5a767f7f3729b9d036c9ae2b546060421fda7776e89b9f568e32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:28:08.429 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:08 smithi077 podman[68928]: 2024-05-04 03:28:08.164354838 +0000 UTC m=+0.125763493 container create 88f89bc3b463633600e81f38447c0ef24475b5361bd62776a4203b4200ec7f7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-04T03:28:08.429 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:08 smithi077 podman[68928]: 2024-05-04 03:28:08.068203171 +0000 UTC m=+0.029611858 image pull afca8dfc9ff09c154a440b34faa874b1d754e8131601e9345ee004bbce766f62 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2 2024-05-04T03:28:08.429 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:08 smithi077 podman[68928]: 2024-05-04 03:28:08.366656227 +0000 UTC m=+0.328064882 container init 88f89bc3b463633600e81f38447c0ef24475b5361bd62776a4203b4200ec7f7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-04T03:28:08.429 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:08 smithi077 podman[68928]: 2024-05-04 03:28:08.369653909 +0000 UTC m=+0.331062565 container start 88f89bc3b463633600e81f38447c0ef24475b5361bd62776a4203b4200ec7f7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True) 2024-05-04T03:28:08.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:08 smithi077 podman[68928]: 2024-05-04 03:28:08.452511976 +0000 UTC m=+0.413920644 container attach 88f89bc3b463633600e81f38447c0ef24475b5361bd62776a4203b4200ec7f7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-04T03:28:08.929 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:08 smithi077 podman[68928]: 2024-05-04 03:28:08.656996522 +0000 UTC m=+0.618405192 container died 88f89bc3b463633600e81f38447c0ef24475b5361bd62776a4203b4200ec7f7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-04T03:28:09.229 INFO:journalctl@ceph.osd.11.smithi077.stdout:May 04 03:28:09 smithi077 podman[68954]: 2024-05-04 03:28:09.202885678 +0000 UTC m=+0.540818074 container remove 88f89bc3b463633600e81f38447c0ef24475b5361bd62776a4203b4200ec7f7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS) 2024-05-04T03:28:09.234 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@osd.11.service' 2024-05-04T03:28:09.271 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:28:09.271 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-04T03:28:09.271 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-04T03:28:09.271 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.a 2024-05-04T03:28:09.562 INFO:journalctl@ceph.mds.a.smithi001.stdout:May 04 03:28:09 smithi001 systemd[1]: Stopping Ceph mds.a for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:28:09.879 INFO:journalctl@ceph.mds.a.smithi001.stdout:May 04 03:28:09 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-a[69570]: 2024-05-04T03:28:09.560+0000 7f3745666700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:28:09.880 INFO:journalctl@ceph.mds.a.smithi001.stdout:May 04 03:28:09 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-a[69570]: 2024-05-04T03:28:09.560+0000 7f3745666700 -1 mds.a *** got signal Terminated *** 2024-05-04T03:28:11.630 INFO:journalctl@ceph.mds.a.smithi001.stdout:May 04 03:28:11 smithi001 podman[113480]: 2024-05-04 03:28:11.337119597 +0000 UTC m=+1.876475291 container died 70e62fe9401746ab8675e241af98a0791b4babee5ab6389f2d28b0dc63c135c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:28:11.980 INFO:journalctl@ceph.mds.a.smithi001.stdout:May 04 03:28:11 smithi001 podman[113480]: 2024-05-04 03:28:11.727232605 +0000 UTC m=+2.266588298 container cleanup 70e62fe9401746ab8675e241af98a0791b4babee5ab6389f2d28b0dc63c135c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_CLEAN=True) 2024-05-04T03:28:11.980 INFO:journalctl@ceph.mds.a.smithi001.stdout:May 04 03:28:11 smithi001 bash[113480]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-a 2024-05-04T03:28:12.123 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.a.service' 2024-05-04T03:28:12.159 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:28:12.159 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-04T03:28:12.159 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-04T03:28:12.159 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.d 2024-05-04T03:28:12.630 INFO:journalctl@ceph.mds.d.smithi001.stdout:May 04 03:28:12 smithi001 systemd[1]: Stopping Ceph mds.d for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:28:12.630 INFO:journalctl@ceph.mds.d.smithi001.stdout:May 04 03:28:12 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-d[68296]: 2024-05-04T03:28:12.489+0000 7f00dcb9b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:28:12.630 INFO:journalctl@ceph.mds.d.smithi001.stdout:May 04 03:28:12 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-d[68296]: 2024-05-04T03:28:12.489+0000 7f00dcb9b700 -1 mds.d *** got signal Terminated *** 2024-05-04T03:28:14.758 INFO:journalctl@ceph.mds.d.smithi001.stdout:May 04 03:28:14 smithi001 podman[113599]: 2024-05-04 03:28:14.48187151 +0000 UTC m=+2.100907889 container died efad207677c1b861078dbc90370ca477a7475957600b32031493eb4462f16d19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-d, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2) 2024-05-04T03:28:15.108 INFO:journalctl@ceph.mds.d.smithi001.stdout:May 04 03:28:14 smithi001 podman[113599]: 2024-05-04 03:28:14.856091373 +0000 UTC m=+2.475127760 container cleanup efad207677c1b861078dbc90370ca477a7475957600b32031493eb4462f16d19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux ) 2024-05-04T03:28:15.109 INFO:journalctl@ceph.mds.d.smithi001.stdout:May 04 03:28:14 smithi001 bash[113599]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-d 2024-05-04T03:28:15.255 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.d.service' 2024-05-04T03:28:15.287 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:28:15.287 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-04T03:28:15.287 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-04T03:28:15.287 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.g 2024-05-04T03:28:15.619 INFO:journalctl@ceph.mds.g.smithi001.stdout:May 04 03:28:15 smithi001 systemd[1]: Stopping Ceph mds.g for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:28:15.880 INFO:journalctl@ceph.mds.g.smithi001.stdout:May 04 03:28:15 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-g[69127]: 2024-05-04T03:28:15.617+0000 7fec861b9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:28:15.880 INFO:journalctl@ceph.mds.g.smithi001.stdout:May 04 03:28:15 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-g[69127]: 2024-05-04T03:28:15.617+0000 7fec861b9700 -1 mds.g *** got signal Terminated *** 2024-05-04T03:28:19.630 INFO:journalctl@ceph.mds.g.smithi001.stdout:May 04 03:28:19 smithi001 podman[113720]: 2024-05-04 03:28:19.33552388 +0000 UTC m=+3.825700834 container died 2a50cbfb89a0762921211768ca2a0c162da67bfb985b94e00102ec2c1df7cbf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-g, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-04T03:28:20.073 INFO:journalctl@ceph.mds.g.smithi001.stdout:May 04 03:28:19 smithi001 podman[113720]: 2024-05-04 03:28:19.651280059 +0000 UTC m=+4.141457009 container cleanup 2a50cbfb89a0762921211768ca2a0c162da67bfb985b94e00102ec2c1df7cbf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-g, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-04T03:28:20.073 INFO:journalctl@ceph.mds.g.smithi001.stdout:May 04 03:28:19 smithi001 bash[113720]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-g 2024-05-04T03:28:20.073 INFO:journalctl@ceph.mds.g.smithi001.stdout:May 04 03:28:19 smithi001 podman[113734]: 2024-05-04 03:28:19.852398773 +0000 UTC m=+0.512414859 container remove 2a50cbfb89a0762921211768ca2a0c162da67bfb985b94e00102ec2c1df7cbf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-g, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430) 2024-05-04T03:28:20.092 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.g.service' 2024-05-04T03:28:20.123 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:28:20.124 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-04T03:28:20.124 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-04T03:28:20.124 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.j 2024-05-04T03:28:20.379 INFO:journalctl@ceph.mds.j.smithi001.stdout:May 04 03:28:20 smithi001 systemd[1]: Stopping Ceph mds.j for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:28:20.879 INFO:journalctl@ceph.mds.j.smithi001.stdout:May 04 03:28:20 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-j[67877]: 2024-05-04T03:28:20.470+0000 7fd763dcb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:28:20.880 INFO:journalctl@ceph.mds.j.smithi001.stdout:May 04 03:28:20 smithi001 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-j[67877]: 2024-05-04T03:28:20.470+0000 7fd763dcb700 -1 mds.j *** got signal Terminated *** 2024-05-04T03:28:22.565 INFO:journalctl@ceph.mds.j.smithi001.stdout:May 04 03:28:22 smithi001 podman[113842]: 2024-05-04 03:28:22.176144118 +0000 UTC m=+1.825127136 container died fe7d3006db2c9062f758434814b5cf789c2a4fc87af5902b4500facf362095e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-j, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-04T03:28:22.565 INFO:journalctl@ceph.mds.j.smithi001.stdout:May 04 03:28:22 smithi001 podman[113842]: 2024-05-04 03:28:22.563528334 +0000 UTC m=+2.212511363 container cleanup fe7d3006db2c9062f758434814b5cf789c2a4fc87af5902b4500facf362095e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-j, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-04T03:28:22.879 INFO:journalctl@ceph.mds.j.smithi001.stdout:May 04 03:28:22 smithi001 bash[113842]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-j 2024-05-04T03:28:22.880 INFO:journalctl@ceph.mds.j.smithi001.stdout:May 04 03:28:22 smithi001 podman[113856]: 2024-05-04 03:28:22.78120835 +0000 UTC m=+0.601606151 container remove fe7d3006db2c9062f758434814b5cf789c2a4fc87af5902b4500facf362095e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430) 2024-05-04T03:28:22.934 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.j.service' 2024-05-04T03:28:22.965 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:28:22.966 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-04T03:28:22.966 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-04T03:28:22.966 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.b 2024-05-04T03:28:23.247 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 04 03:28:23 smithi072 systemd[1]: Stopping Ceph mds.b for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:28:23.505 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 04 03:28:23 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-b[63318]: 2024-05-04T03:28:23.288+0000 7f6ae8e49700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:28:23.506 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 04 03:28:23 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-b[63318]: 2024-05-04T03:28:23.288+0000 7f6ae8e49700 -1 mds.b *** got signal Terminated *** 2024-05-04T03:28:25.193 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 04 03:28:24 smithi072 podman[68301]: 2024-05-04 03:28:24.90652981 +0000 UTC m=+1.755734487 container died 1e23e5da397c3c694217040d2f1c98a9cfd1822899f0085f6eb81d58ac79050a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-b, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS) 2024-05-04T03:28:25.497 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 04 03:28:25 smithi072 podman[68301]: 2024-05-04 03:28:25.284182639 +0000 UTC m=+2.133387308 container cleanup 1e23e5da397c3c694217040d2f1c98a9cfd1822899f0085f6eb81d58ac79050a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-04T03:28:25.497 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 04 03:28:25 smithi072 bash[68301]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-b 2024-05-04T03:28:25.667 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.b.service' 2024-05-04T03:28:25.702 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:28:25.702 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-04T03:28:25.702 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-04T03:28:25.703 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.e 2024-05-04T03:28:25.997 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 04 03:28:25 smithi072 systemd[1]: Stopping Ceph mds.e for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:28:26.497 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 04 03:28:26 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-e[62701]: 2024-05-04T03:28:26.030+0000 7f3539de9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:28:26.497 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 04 03:28:26 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-e[62701]: 2024-05-04T03:28:26.030+0000 7f3539de9700 -1 mds.e *** got signal Terminated *** 2024-05-04T03:28:29.237 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 04 03:28:28 smithi072 podman[68434]: 2024-05-04 03:28:28.901793195 +0000 UTC m=+2.982856685 container died f59a9e96a9bc80a1aacb27b33e8390798d9bda5ee7fec1f985c9b825f720f870 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-e, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-04T03:28:29.237 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 04 03:28:29 smithi072 podman[68434]: 2024-05-04 03:28:29.235438801 +0000 UTC m=+3.316502349 container cleanup f59a9e96a9bc80a1aacb27b33e8390798d9bda5ee7fec1f985c9b825f720f870 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-e, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-05-04T03:28:29.490 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 04 03:28:29 smithi072 bash[68434]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-e 2024-05-04T03:28:29.718 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.e.service' 2024-05-04T03:28:29.740 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 04 03:28:29 smithi072 podman[68448]: 2024-05-04 03:28:29.48835601 +0000 UTC m=+0.580085656 container remove f59a9e96a9bc80a1aacb27b33e8390798d9bda5ee7fec1f985c9b825f720f870 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-e, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-04T03:28:29.740 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 04 03:28:29 smithi072 systemd[1]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.e.service: Deactivated successfully. 2024-05-04T03:28:29.741 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 04 03:28:29 smithi072 systemd[1]: Stopped Ceph mds.e for 4a572aa8-09c3-11ef-bc97-c7b262605968. 2024-05-04T03:28:29.741 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 04 03:28:29 smithi072 systemd[1]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.e.service: Consumed 9.552s CPU time. 2024-05-04T03:28:29.751 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:28:29.751 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-04T03:28:29.751 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-04T03:28:29.752 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.h 2024-05-04T03:28:29.997 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 04 03:28:29 smithi072 systemd[1]: Stopping Ceph mds.h for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:28:30.497 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 04 03:28:30 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-h[63012]: 2024-05-04T03:28:30.082+0000 7f79a4a8f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:28:30.497 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 04 03:28:30 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-h[63012]: 2024-05-04T03:28:30.082+0000 7f79a4a8f700 -1 mds.h *** got signal Terminated *** 2024-05-04T03:28:33.170 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 04 03:28:32 smithi072 podman[68554]: 2024-05-04 03:28:32.900458247 +0000 UTC m=+2.937854993 container died 0cc596835e9bdda9a0da128f7a87a740e10713c235308f608bd6e7cb776ef3a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-h, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-04T03:28:33.497 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 04 03:28:33 smithi072 podman[68554]: 2024-05-04 03:28:33.261330732 +0000 UTC m=+3.298727482 container cleanup 0cc596835e9bdda9a0da128f7a87a740e10713c235308f608bd6e7cb776ef3a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-h, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, ceph=True) 2024-05-04T03:28:33.497 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 04 03:28:33 smithi072 bash[68554]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-h 2024-05-04T03:28:33.690 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.h.service' 2024-05-04T03:28:33.721 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:28:33.721 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-04T03:28:33.722 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-04T03:28:33.722 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.k 2024-05-04T03:28:34.152 INFO:journalctl@ceph.mds.k.smithi072.stdout:May 04 03:28:33 smithi072 systemd[1]: Stopping Ceph mds.k for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:28:34.497 INFO:journalctl@ceph.mds.k.smithi072.stdout:May 04 03:28:34 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-k[62381]: 2024-05-04T03:28:34.149+0000 7f6a28e64700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:28:34.497 INFO:journalctl@ceph.mds.k.smithi072.stdout:May 04 03:28:34 smithi072 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-k[62381]: 2024-05-04T03:28:34.149+0000 7f6a28e64700 -1 mds.k *** got signal Terminated *** 2024-05-04T03:28:38.446 INFO:journalctl@ceph.mds.k.smithi072.stdout:May 04 03:28:38 smithi072 podman[68676]: 2024-05-04 03:28:38.165127847 +0000 UTC m=+4.226634033 container died 040b9c4f8036c656a3a1bc8f4da9832d8d74ccdd7c61c4bc5d6c2ed6d38231b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a) 2024-05-04T03:28:38.747 INFO:journalctl@ceph.mds.k.smithi072.stdout:May 04 03:28:38 smithi072 podman[68676]: 2024-05-04 03:28:38.546510285 +0000 UTC m=+4.608016473 container cleanup 040b9c4f8036c656a3a1bc8f4da9832d8d74ccdd7c61c4bc5d6c2ed6d38231b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-k, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-04T03:28:38.747 INFO:journalctl@ceph.mds.k.smithi072.stdout:May 04 03:28:38 smithi072 bash[68676]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-k 2024-05-04T03:28:38.916 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.k.service' 2024-05-04T03:28:38.947 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:28:38.948 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-04T03:28:38.948 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-04T03:28:38.948 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.c 2024-05-04T03:28:39.247 INFO:journalctl@ceph.mds.c.smithi077.stdout:May 04 03:28:38 smithi077 systemd[1]: Stopping Ceph mds.c for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:28:39.510 INFO:journalctl@ceph.mds.c.smithi077.stdout:May 04 03:28:39 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-c[63817]: 2024-05-04T03:28:39.245+0000 7f6b62df5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:28:39.511 INFO:journalctl@ceph.mds.c.smithi077.stdout:May 04 03:28:39 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-c[63817]: 2024-05-04T03:28:39.245+0000 7f6b62df5700 -1 mds.c *** got signal Terminated *** 2024-05-04T03:28:41.172 INFO:journalctl@ceph.mds.c.smithi077.stdout:May 04 03:28:40 smithi077 podman[69032]: 2024-05-04 03:28:40.90454904 +0000 UTC m=+1.759375630 container died 8c5f0f199e56162e53c6da217a95e25f14e30c1a31b260d486cc23176c097f6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-c, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430) 2024-05-04T03:28:41.428 INFO:journalctl@ceph.mds.c.smithi077.stdout:May 04 03:28:41 smithi077 podman[69032]: 2024-05-04 03:28:41.269994333 +0000 UTC m=+2.124820921 container cleanup 8c5f0f199e56162e53c6da217a95e25f14e30c1a31b260d486cc23176c097f6f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-04T03:28:41.429 INFO:journalctl@ceph.mds.c.smithi077.stdout:May 04 03:28:41 smithi077 bash[69032]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-c 2024-05-04T03:28:41.705 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.c.service' 2024-05-04T03:28:41.739 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:28:41.739 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-04T03:28:41.739 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-04T03:28:41.740 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.f 2024-05-04T03:28:42.178 INFO:journalctl@ceph.mds.f.smithi077.stdout:May 04 03:28:41 smithi077 systemd[1]: Stopping Ceph mds.f for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:28:42.179 INFO:journalctl@ceph.mds.f.smithi077.stdout:May 04 03:28:42 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-f[63504]: 2024-05-04T03:28:42.057+0000 7f85563e5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:28:42.179 INFO:journalctl@ceph.mds.f.smithi077.stdout:May 04 03:28:42 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-f[63504]: 2024-05-04T03:28:42.057+0000 7f85563e5700 -1 mds.f *** got signal Terminated *** 2024-05-04T03:28:45.159 INFO:journalctl@ceph.mds.f.smithi077.stdout:May 04 03:28:44 smithi077 podman[69168]: 2024-05-04 03:28:44.904355221 +0000 UTC m=+2.949516507 container died f7ab088fc14c4d6ec13e9419e16cc280ef396e76da45868154a0f7ded879d349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-f, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-05-04T03:28:45.428 INFO:journalctl@ceph.mds.f.smithi077.stdout:May 04 03:28:45 smithi077 podman[69168]: 2024-05-04 03:28:45.265194499 +0000 UTC m=+3.310355785 container cleanup f7ab088fc14c4d6ec13e9419e16cc280ef396e76da45868154a0f7ded879d349 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2) 2024-05-04T03:28:45.429 INFO:journalctl@ceph.mds.f.smithi077.stdout:May 04 03:28:45 smithi077 bash[69168]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-f 2024-05-04T03:28:45.702 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.f.service' 2024-05-04T03:28:45.734 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:28:45.735 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-04T03:28:45.735 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-04T03:28:45.735 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.i 2024-05-04T03:28:45.928 INFO:journalctl@ceph.mds.i.smithi077.stdout:May 04 03:28:45 smithi077 systemd[1]: Stopping Ceph mds.i for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:28:46.428 INFO:journalctl@ceph.mds.i.smithi077.stdout:May 04 03:28:46 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-i[64134]: 2024-05-04T03:28:46.034+0000 7f0a8c8b9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:28:46.429 INFO:journalctl@ceph.mds.i.smithi077.stdout:May 04 03:28:46 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-i[64134]: 2024-05-04T03:28:46.034+0000 7f0a8c8b9700 -1 mds.i *** got signal Terminated *** 2024-05-04T03:28:49.163 INFO:journalctl@ceph.mds.i.smithi077.stdout:May 04 03:28:48 smithi077 podman[69289]: 2024-05-04 03:28:48.905539812 +0000 UTC m=+2.957629520 container died 33378ccac362362ec807e5664c01d9a9f09a3eeab97185ba74f81a6c806fc1cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-i, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-04T03:28:49.430 INFO:journalctl@ceph.mds.i.smithi077.stdout:May 04 03:28:49 smithi077 podman[69289]: 2024-05-04 03:28:49.235639705 +0000 UTC m=+3.287729458 container cleanup 33378ccac362362ec807e5664c01d9a9f09a3eeab97185ba74f81a6c806fc1cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-04T03:28:49.430 INFO:journalctl@ceph.mds.i.smithi077.stdout:May 04 03:28:49 smithi077 bash[69289]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-i 2024-05-04T03:28:49.663 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.i.service' 2024-05-04T03:28:49.694 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:28:49.695 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-04T03:28:49.695 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-04T03:28:49.695 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.l 2024-05-04T03:28:50.032 INFO:journalctl@ceph.mds.l.smithi077.stdout:May 04 03:28:49 smithi077 systemd[1]: Stopping Ceph mds.l for 4a572aa8-09c3-11ef-bc97-c7b262605968... 2024-05-04T03:28:50.428 INFO:journalctl@ceph.mds.l.smithi077.stdout:May 04 03:28:50 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-l[63196]: 2024-05-04T03:28:50.029+0000 7ff0c8e5d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-04T03:28:50.428 INFO:journalctl@ceph.mds.l.smithi077.stdout:May 04 03:28:50 smithi077 ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-l[63196]: 2024-05-04T03:28:50.029+0000 7ff0c8e5d700 -1 mds.l *** got signal Terminated *** 2024-05-04T03:28:51.733 INFO:journalctl@ceph.mds.l.smithi077.stdout:May 04 03:28:51 smithi077 podman[69409]: 2024-05-04 03:28:51.473206363 +0000 UTC m=+1.566086206 container died f9f7deb8f457e83202ea260ae0c04567e3c60859f1ea218ac25a00b120f0510f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-l, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-04T03:28:51.983 INFO:journalctl@ceph.mds.l.smithi077.stdout:May 04 03:28:51 smithi077 podman[69409]: 2024-05-04 03:28:51.822567492 +0000 UTC m=+1.915447334 container cleanup f9f7deb8f457e83202ea260ae0c04567e3c60859f1ea218ac25a00b120f0510f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:d5b18277c2b57720d1088cbe1b013aeecc01ebb3b317f5a7d068e714a67844c2, name=ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-l, RELEASE=wip-pdonnell-testing-20240503.163550-debug-bd4e00a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-04T03:28:51.983 INFO:journalctl@ceph.mds.l.smithi077.stdout:May 04 03:28:51 smithi077 bash[69409]: ceph-4a572aa8-09c3-11ef-bc97-c7b262605968-mds-l 2024-05-04T03:28:52.188 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4a572aa8-09c3-11ef-bc97-c7b262605968@mds.l.service' 2024-05-04T03:28:52.218 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-04T03:28:52.218 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-04T03:28:52.218 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-04T03:28:52.419 INFO:teuthology.orchestra.run.smithi001.stdout:Deleting cluster with fsid: 4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:28:54.954 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-04T03:28:55.143 INFO:teuthology.orchestra.run.smithi072.stdout:Deleting cluster with fsid: 4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:28:57.603 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-04T03:28:57.791 INFO:teuthology.orchestra.run.smithi077.stdout:Deleting cluster with fsid: 4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:29:00.125 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-04T03:29:00.157 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-04T03:29:00.188 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-04T03:29:00.214 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-04T03:29:00.216 DEBUG:teuthology.misc:Transferring archived files from smithi001:/var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688625/remote/smithi001/crash 2024-05-04T03:29:00.217 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/crash -- . 2024-05-04T03:29:00.284 INFO:teuthology.orchestra.run.smithi001.stderr:tar: /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-04T03:29:00.285 INFO:teuthology.orchestra.run.smithi001.stderr:tar: Error is not recoverable: exiting now 2024-05-04T03:29:00.287 DEBUG:teuthology.misc:Transferring archived files from smithi072:/var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688625/remote/smithi072/crash 2024-05-04T03:29:00.287 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/crash -- . 2024-05-04T03:29:00.336 INFO:teuthology.orchestra.run.smithi072.stderr:tar: /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-04T03:29:00.336 INFO:teuthology.orchestra.run.smithi072.stderr:tar: Error is not recoverable: exiting now 2024-05-04T03:29:00.338 DEBUG:teuthology.misc:Transferring archived files from smithi077:/var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688625/remote/smithi077/crash 2024-05-04T03:29:00.339 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/crash -- . 2024-05-04T03:29:00.374 INFO:teuthology.orchestra.run.smithi077.stderr:tar: /var/lib/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-04T03:29:00.374 INFO:teuthology.orchestra.run.smithi077.stderr:tar: Error is not recoverable: exiting now 2024-05-04T03:29:00.375 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-04T03:29:00.376 DEBUG:teuthology.orchestra.run.smithi001:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-05-04T03:29:00.413 INFO:tasks.cephadm:Compressing logs... 2024-05-04T03:29:00.413 DEBUG:teuthology.orchestra.run.smithi001:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-04T03:29:00.456 DEBUG:teuthology.orchestra.run.smithi072:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-04T03:29:00.459 DEBUG:teuthology.orchestra.run.smithi077:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-04T03:29:00.480 INFO:teuthology.orchestra.run.smithi001.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log: No such file or directory 2024-05-04T03:29:00.480 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-04T03:29:00.480 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph.log 2024-05-04T03:29:00.480 INFO:teuthology.orchestra.run.smithi001.stderr:gzip/var/log/ceph/cephadm.log: -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-04T03:29:00.481 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.g.log 2024-05-04T03:29:00.481 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.j.log 2024-05-04T03:29:00.481 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.3.log 2024-05-04T03:29:00.481 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.g.log: /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.0.log 2024-05-04T03:29:00.482 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mgr.x.log 2024-05-04T03:29:00.482 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mon.a.log 2024-05-04T03:29:00.488 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mgr.x.log: 86.8% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-04T03:29:00.488 INFO:teuthology.orchestra.run.smithi001.stderr: 90.2%gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.6.log 2024-05-04T03:29:00.488 INFO:teuthology.orchestra.run.smithi001.stderr: -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-04T03:29:00.488 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-04T03:29:00.489 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.6.log: 94.6% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.g.log.gz 2024-05-04T03:29:00.489 INFO:teuthology.orchestra.run.smithi001.stderr:gzip 94.6% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.j.log.gz 2024-05-04T03:29:00.489 INFO:teuthology.orchestra.run.smithi001.stderr: -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.a.log 2024-05-04T03:29:00.489 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-04T03:29:00.489 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.a.log: 83.0% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-04T03:29:00.489 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.d.log 2024-05-04T03:29:00.495 INFO:teuthology.orchestra.run.smithi077.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-04T03:29:00.495 INFO:teuthology.orchestra.run.smithi077.stderr:: No such file or directory 2024-05-04T03:29:00.495 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph.log 2024-05-04T03:29:00.496 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.2.log 2024-05-04T03:29:00.496 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.f.log 2024-05-04T03:29:00.496 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-04T03:29:00.496 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.5.log 2024-05-04T03:29:00.496 INFO:teuthology.orchestra.run.smithi077.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-04T03:29:00.496 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.c.log 2024-05-04T03:29:00.496 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mon.c.log 2024-05-04T03:29:00.496 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mgr.z.log 2024-05-04T03:29:00.496 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.i.log 2024-05-04T03:29:00.496 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mgr.z.log: 86.8% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-04T03:29:00.496 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-04T03:29:00.496 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.i.log: 90.4% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-04T03:29:00.496 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.11.log 2024-05-04T03:29:00.496 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph.cephadm.log: 82.6% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-04T03:29:00.497 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.8.log 2024-05-04T03:29:00.497 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.11.log: 92.7% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mgr.z.log.gz 2024-05-04T03:29:00.497 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-04T03:29:00.498 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-volume.log: 93.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-04T03:29:00.498 INFO:teuthology.orchestra.run.smithi001.stderr:gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.9.log 2024-05-04T03:29:00.498 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.d.log: 94.5% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.a.log.gz 2024-05-04T03:29:00.499 INFO:teuthology.orchestra.run.smithi001.stderr: 94.5% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.d.log.gz 2024-05-04T03:29:00.501 INFO:teuthology.orchestra.run.smithi072.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-04T03:29:00.501 INFO:teuthology.orchestra.run.smithi072.stderr:: No such file or directory 2024-05-04T03:29:00.501 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph.log 2024-05-04T03:29:00.501 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-04T03:29:00.501 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.7.log 2024-05-04T03:29:00.501 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.b.log 2024-05-04T03:29:00.501 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.4.log 2024-05-04T03:29:00.501 INFO:teuthology.orchestra.run.smithi072.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-04T03:29:00.501 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.1.log 2024-05-04T03:29:00.501 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mgr.y.log 2024-05-04T03:29:00.501 INFO:teuthology.orchestra.run.smithi072.stderr: 86.8% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-04T03:29:00.501 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.h.log 2024-05-04T03:29:00.501 INFO:teuthology.orchestra.run.smithi072.stderr:gzip/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mgr.y.log: -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.10.log 2024-05-04T03:29:00.502 INFO:teuthology.orchestra.run.smithi072.stderr: 90.4% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-04T03:29:00.502 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-04T03:29:00.502 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.e.log 2024-05-04T03:29:00.502 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph.cephadm.log: /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.10.log: 82.7% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-04T03:29:00.502 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-04T03:29:00.502 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.e.log: 92.7% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mgr.y.log.gz 2024-05-04T03:29:00.502 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mon.b.log 2024-05-04T03:29:00.502 INFO:teuthology.orchestra.run.smithi001.stderr: 91.0% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-04T03:29:00.505 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.l.log 2024-05-04T03:29:00.506 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.k.log 2024-05-04T03:29:00.508 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-04T03:29:00.527 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.k.log: 94.6% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.k.log.gz 2024-05-04T03:29:00.529 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-volume.log: /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.l.log: 94.6% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.l.log.gz 2024-05-04T03:29:00.532 INFO:teuthology.orchestra.run.smithi077.stderr: 91.0% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-04T03:29:00.546 INFO:teuthology.orchestra.run.smithi077.stderr: 93.9% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.f.log.gz 2024-05-04T03:29:00.585 INFO:teuthology.orchestra.run.smithi077.stderr: 93.7% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.c.log.gz 2024-05-04T03:29:00.619 INFO:teuthology.orchestra.run.smithi077.stderr: 93.3% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.i.log.gz 2024-05-04T03:29:00.743 INFO:teuthology.orchestra.run.smithi001.stderr:/var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.9.log: 90.3% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mgr.x.log.gz 2024-05-04T03:29:00.820 INFO:teuthology.orchestra.run.smithi072.stderr: 92.0% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mon.b.log.gz 2024-05-04T03:29:00.843 INFO:teuthology.orchestra.run.smithi077.stderr: 91.8% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mon.c.log.gz 2024-05-04T03:29:01.198 INFO:teuthology.orchestra.run.smithi001.stderr: 90.4% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mon.a.log.gz 2024-05-04T03:29:01.485 INFO:teuthology.orchestra.run.smithi072.stderr: 94.1% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.b.log.gz 2024-05-04T03:29:02.658 INFO:teuthology.orchestra.run.smithi001.stderr: 94.5% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.9.log.gz 2024-05-04T03:29:02.720 INFO:teuthology.orchestra.run.smithi077.stderr: 94.3% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.11.log.gz 2024-05-04T03:29:03.278 INFO:teuthology.orchestra.run.smithi001.stderr: 93.9% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.0.log.gz 2024-05-04T03:29:03.355 INFO:teuthology.orchestra.run.smithi072.stderr: 96.2% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.e.log.gz 2024-05-04T03:29:03.394 INFO:teuthology.orchestra.run.smithi072.stderr: 93.8% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.1.log.gz 2024-05-04T03:29:03.762 INFO:teuthology.orchestra.run.smithi072.stderr: 94.0% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.10.log.gz 2024-05-04T03:29:03.823 INFO:teuthology.orchestra.run.smithi001.stderr: 94.1% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.6.log.gz 2024-05-04T03:29:03.996 INFO:teuthology.orchestra.run.smithi072.stderr: 94.0% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.4.log.gz 2024-05-04T03:29:04.083 INFO:teuthology.orchestra.run.smithi077.stderr: 94.0% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.2.log.gz 2024-05-04T03:29:04.515 INFO:teuthology.orchestra.run.smithi077.stderr: 94.2% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.5.log.gz 2024-05-04T03:29:04.982 INFO:teuthology.orchestra.run.smithi072.stderr: 93.9% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.7.log.gz 2024-05-04T03:29:05.780 INFO:teuthology.orchestra.run.smithi077.stderr: 94.2% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.8.log.gz 2024-05-04T03:29:05.782 INFO:teuthology.orchestra.run.smithi077.stderr: 2024-05-04T03:29:05.782 INFO:teuthology.orchestra.run.smithi077.stderr:real 0m5.309s 2024-05-04T03:29:05.782 INFO:teuthology.orchestra.run.smithi077.stderr:user 0m15.273s 2024-05-04T03:29:05.782 INFO:teuthology.orchestra.run.smithi077.stderr:sys 0m0.541s 2024-05-04T03:29:07.755 INFO:teuthology.orchestra.run.smithi001.stderr: 94.1% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-osd.3.log.gz 2024-05-04T03:29:07.757 INFO:teuthology.orchestra.run.smithi001.stderr: 2024-05-04T03:29:07.757 INFO:teuthology.orchestra.run.smithi001.stderr:real 0m7.288s 2024-05-04T03:29:07.757 INFO:teuthology.orchestra.run.smithi001.stderr:user 0m15.997s 2024-05-04T03:29:07.757 INFO:teuthology.orchestra.run.smithi001.stderr:sys 0m0.589s 2024-05-04T03:29:28.311 INFO:teuthology.orchestra.run.smithi072.stderr: 93.7% -- replaced with /var/log/ceph/4a572aa8-09c3-11ef-bc97-c7b262605968/ceph-mds.h.log.gz 2024-05-04T03:29:28.312 INFO:teuthology.orchestra.run.smithi072.stderr: 2024-05-04T03:29:28.312 INFO:teuthology.orchestra.run.smithi072.stderr:real 0m27.839s 2024-05-04T03:29:28.313 INFO:teuthology.orchestra.run.smithi072.stderr:user 0m44.651s 2024-05-04T03:29:28.313 INFO:teuthology.orchestra.run.smithi072.stderr:sys 0m1.462s 2024-05-04T03:29:28.313 INFO:tasks.cephadm:Archiving logs... 2024-05-04T03:29:28.313 DEBUG:teuthology.misc:Transferring archived files from smithi001:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688625/remote/smithi001/log 2024-05-04T03:29:28.314 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-04T03:29:30.566 DEBUG:teuthology.misc:Transferring archived files from smithi072:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688625/remote/smithi072/log 2024-05-04T03:29:30.567 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-04T03:29:36.780 DEBUG:teuthology.misc:Transferring archived files from smithi077:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688625/remote/smithi077/log 2024-05-04T03:29:36.781 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-04T03:29:38.992 INFO:tasks.cephadm:Removing cluster... 2024-05-04T03:29:38.993 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 --force 2024-05-04T03:29:39.194 INFO:teuthology.orchestra.run.smithi001.stdout:Deleting cluster with fsid: 4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:29:39.473 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 --force 2024-05-04T03:29:39.659 INFO:teuthology.orchestra.run.smithi072.stdout:Deleting cluster with fsid: 4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:29:39.971 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 4a572aa8-09c3-11ef-bc97-c7b262605968 --force 2024-05-04T03:29:40.162 INFO:teuthology.orchestra.run.smithi077.stdout:Deleting cluster with fsid: 4a572aa8-09c3-11ef-bc97-c7b262605968 2024-05-04T03:29:40.431 INFO:tasks.cephadm:Removing cephadm ... 2024-05-04T03:29:40.431 DEBUG:teuthology.orchestra.run.smithi001:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-04T03:29:40.450 DEBUG:teuthology.orchestra.run.smithi072:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-04T03:29:40.466 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-04T03:29:40.482 INFO:tasks.cephadm:Teardown complete 2024-05-04T03:29:40.482 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-04T03:29:40.493 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-04T03:29:40.494 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-04T03:29:40.496 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-04T03:29:40.508 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-04T03:29:40.610 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-04T03:29:40.620 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-04T03:29:40.621 DEBUG:teuthology.orchestra.run.smithi001:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-04T03:29:40.623 DEBUG:teuthology.orchestra.run.smithi072:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-04T03:29:40.625 DEBUG:teuthology.orchestra.run.smithi077:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-04T03:29:40.637 INFO:teuthology.orchestra.run.smithi001.stderr:bash: line 1: ntpq: command not found 2024-05-04T03:29:40.637 INFO:teuthology.orchestra.run.smithi072.stderr:bash: line 1: ntpq: command not found 2024-05-04T03:29:40.639 INFO:teuthology.orchestra.run.smithi077.stderr:bash: line 1: ntpq: command not found 2024-05-04T03:29:40.642 INFO:teuthology.orchestra.run.smithi072.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-04T03:29:40.642 INFO:teuthology.orchestra.run.smithi072.stdout:=============================================================================== 2024-05-04T03:29:40.642 INFO:teuthology.orchestra.run.smithi072.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 39 +578us[ +569us] +/- 46ms 2024-05-04T03:29:40.643 INFO:teuthology.orchestra.run.smithi072.stdout:^* hv02.front.sepia.ceph.com 3 6 377 37 -1643us[-1652us] +/- 51ms 2024-05-04T03:29:40.643 INFO:teuthology.orchestra.run.smithi072.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 +1300us[+1300us] +/- 70ms 2024-05-04T03:29:40.643 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T03:29:40.643 INFO:teuthology.orchestra.run.smithi001.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-04T03:29:40.643 INFO:teuthology.orchestra.run.smithi001.stdout:=============================================================================== 2024-05-04T03:29:40.643 INFO:teuthology.orchestra.run.smithi001.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 36 +585us[ +585us] +/- 46ms 2024-05-04T03:29:40.643 INFO:teuthology.orchestra.run.smithi001.stdout:^* hv02.front.sepia.ceph.com 3 6 377 39 -1626us[-1627us] +/- 51ms 2024-05-04T03:29:40.643 INFO:teuthology.orchestra.run.smithi001.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 +1319us[+1319us] +/- 70ms 2024-05-04T03:29:40.643 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T03:29:40.644 INFO:teuthology.orchestra.run.smithi077.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-04T03:29:40.644 INFO:teuthology.orchestra.run.smithi077.stdout:=============================================================================== 2024-05-04T03:29:40.644 INFO:teuthology.orchestra.run.smithi077.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 38 +542us[ +542us] +/- 46ms 2024-05-04T03:29:40.644 INFO:teuthology.orchestra.run.smithi077.stdout:^* hv02.front.sepia.ceph.com 3 7 377 103 -1660us[-1693us] +/- 50ms 2024-05-04T03:29:40.644 INFO:teuthology.orchestra.run.smithi077.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 +1272us[+1272us] +/- 70ms 2024-05-04T03:29:40.644 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-04T03:29:40.645 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-04T03:29:40.668 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-04T03:29:40.669 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-04T03:29:40.772 DEBUG:teuthology.orchestra.run.smithi001:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-04T03:29:40.854 DEBUG:teuthology.orchestra.run.smithi072:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-04T03:29:40.917 DEBUG:teuthology.orchestra.run.smithi077:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-04T03:29:40.980 DEBUG:teuthology.orchestra.run.smithi001:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-04T03:29:41.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T03:29:41.009 DEBUG:teuthology.orchestra.run.smithi072:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-04T03:29:41.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T03:29:41.035 DEBUG:teuthology.orchestra.run.smithi077:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-04T03:29:41.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T03:29:41.061 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-04T03:29:41.072 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-04T03:29:41.148 INFO:teuthology.task.internal:Duration was 1805.361703 seconds 2024-05-04T03:29:41.149 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-04T03:29:41.229 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-04T03:29:41.230 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-04T03:29:41.232 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-04T03:29:41.234 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-04T03:29:41.268 INFO:teuthology.orchestra.run.smithi001.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-04T03:29:41.269 INFO:teuthology.orchestra.run.smithi072.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-04T03:29:41.271 INFO:teuthology.orchestra.run.smithi077.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-04T03:29:41.780 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-04T03:29:41.780 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi001.front.sepia.ceph.com 2024-05-04T03:29:41.781 DEBUG:teuthology.orchestra.run.smithi001:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-04T03:29:41.802 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi072.front.sepia.ceph.com 2024-05-04T03:29:41.802 DEBUG:teuthology.orchestra.run.smithi072:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-04T03:29:41.848 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi077.front.sepia.ceph.com 2024-05-04T03:29:41.849 DEBUG:teuthology.orchestra.run.smithi077:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-04T03:29:41.876 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-04T03:29:41.876 DEBUG:teuthology.orchestra.run.smithi001:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-04T03:29:41.880 DEBUG:teuthology.orchestra.run.smithi072:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-04T03:29:41.891 DEBUG:teuthology.orchestra.run.smithi077:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-04T03:29:41.942 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-04T03:29:41.943 DEBUG:teuthology.orchestra.run.smithi001:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-04T03:29:41.946 DEBUG:teuthology.orchestra.run.smithi072:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-04T03:29:41.959 DEBUG:teuthology.orchestra.run.smithi077:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-04T03:29:42.562 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-04T03:29:42.576 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-04T03:29:42.576 DEBUG:teuthology.orchestra.run.smithi001:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-04T03:29:42.600 DEBUG:teuthology.orchestra.run.smithi072:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-04T03:29:42.626 DEBUG:teuthology.orchestra.run.smithi077:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-04T03:29:42.651 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-04T03:29:42.698 DEBUG:teuthology.orchestra.run.smithi001:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-04T03:29:42.701 DEBUG:teuthology.orchestra.run.smithi072:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-04T03:29:42.703 DEBUG:teuthology.orchestra.run.smithi077:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-04T03:29:42.721 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern = core 2024-05-04T03:29:42.723 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern = core 2024-05-04T03:29:42.726 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern = core 2024-05-04T03:29:42.785 DEBUG:teuthology.orchestra.run.smithi001:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-04T03:29:42.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T03:29:42.800 DEBUG:teuthology.orchestra.run.smithi072:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-04T03:29:42.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T03:29:42.815 DEBUG:teuthology.orchestra.run.smithi077:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-04T03:29:42.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-04T03:29:42.840 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-04T03:29:42.852 INFO:teuthology.task.internal:Transferring archived files... 2024-05-04T03:29:42.852 DEBUG:teuthology.misc:Transferring archived files from smithi001:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688625/remote/smithi001 2024-05-04T03:29:42.852 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-04T03:29:42.900 DEBUG:teuthology.misc:Transferring archived files from smithi072:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688625/remote/smithi072 2024-05-04T03:29:42.901 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-04T03:29:42.936 DEBUG:teuthology.misc:Transferring archived files from smithi077:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-03_22:48:16-fs-wip-pdonnell-testing-20240503.163550-debug-distro-default-smithi/7688625/remote/smithi077 2024-05-04T03:29:42.936 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-04T03:29:42.971 INFO:teuthology.task.internal:Removing archive directory... 2024-05-04T03:29:42.971 DEBUG:teuthology.orchestra.run.smithi001:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-04T03:29:42.974 DEBUG:teuthology.orchestra.run.smithi072:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-04T03:29:42.976 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-04T03:29:43.019 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-04T03:29:43.032 INFO:teuthology.task.internal:Not uploading archives. 2024-05-04T03:29:43.032 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-04T03:29:43.100 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-04T03:29:43.100 DEBUG:teuthology.orchestra.run.smithi001:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-04T03:29:43.103 DEBUG:teuthology.orchestra.run.smithi072:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-04T03:29:43.105 DEBUG:teuthology.orchestra.run.smithi077:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-04T03:29:43.115 INFO:teuthology.orchestra.run.smithi001.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 4 03:29 /home/ubuntu/cephtest 2024-05-04T03:29:43.117 INFO:teuthology.orchestra.run.smithi072.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 4 03:29 /home/ubuntu/cephtest 2024-05-04T03:29:43.119 INFO:teuthology.orchestra.run.smithi077.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 4 03:29 /home/ubuntu/cephtest 2024-05-04T03:29:43.120 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-04T03:29:43.149 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-04T03:29:43.268 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} duration: 1805.3617026805878 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-04T03:29:43.268 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-04T03:29:43.351 INFO:teuthology.run:pass